Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus

Brain activation patterns in patients with post-stroke cognitive impairment during working memory task: a functional near-infrared spectroscopy study

Objective

To explore the activation patterns in the frontal cortex of patients with post-stroke cognitive impairment during the execution of working memory tasks.

Methods

15 patients with post-stroke cognitive impairment, 17 patients without cognitive impairment, and 15 healthy controls of similar age and sex were included. All participants under-went immediate recall task testing and near-infrared spectroscopy imaging to measure frontal cortex activation during the task.

Results

The healthy control group performed the best in the immediate recall task, followed by the post-stroke non-cognitive impairment group. The post-stroke cognitive impairment group had the poorest performance. The near-infrared spectroscopy results revealed that during the immediate recall task, the healthy control group primarily activated the left frontal lobe region. In contrast, post-stroke patients exhibited reduced activation in the left frontal lobe and increased activation in the right frontal cortex, particularly in the right frontopolar and orbitofrontal regions, with the post-stroke cognitive impairment group displaying the most pronounced changes.

Conclusion

Patients with post-stroke cognitive impairment exhibit reduced activation in the left prefrontal cortex during the working memory tasks. They rely on compensatory activation in the right prefrontal cortex, particularly in the frontopolar and orbitofrontal cortex, to successfully complete the task.

Encoding of Predictive Associations in Human Prefrontal and Medial Temporal Neurons During Pavlovian Appetitive Conditioning

Pavlovian conditioning is thought to involve the formation of learned associations between stimuli and values, and between stimuli and specific features of outcomes. Here, we leveraged human single neuron recordings in ventromedial prefrontal, dorsomedial frontal, hippocampus, and amygdala while patients of both sexes performed an appetitive Pavlovian conditioning task probing both stimulus-value and stimulus-stimulus associations. Ventromedial prefrontal cortex encoded predictive value along with the amygdala, and also encoded predictions about the identity of stimuli that would subsequently be presented, suggesting a role for neurons in this region in encoding predictive information beyond value. Unsigned error signals were found in dorsomedial frontal areas and hippocampus, potentially supporting learning of non-value related outcome features. Our findings implicate distinct human prefrontal and medial temporal neuronal populations in mediating predictive associations which could partially support model-based mechanisms during Pavlovian conditioning.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons

Dopamine neurons respond to reward-predicting cues but also modulate information processing in the prefrontal cortex essential for cognitive control. Whether dopamine controls reward expectation signals in prefrontal cortex that motivate cognitive control is unknown. We trained two male macaques on a working memory task while varying the reward size earned for successful task completion. We recorded neurons in lateral prefrontal cortex while simultaneously stimulating dopamine D1 receptor (D1R) or D2 receptor (D2R) families using micro-iontophoresis. We show that many neurons predict reward size throughout the trial. D1R stimulation showed mixed effects following reward cues but decreased reward expectancy coding during the memory delay. By contrast, D2R stimulation increased reward expectancy coding in multiple task periods, including cueing and memory periods. Stimulation of either dopamine receptors increased the neurons' selective responses to reward size upon reward delivery. The differential modulation of reward expectancy by dopamine receptors suggests that dopamine regulates reward expectancy necessary for successful cognitive control.

Connecting Circuits with Networks in Addiction Neuroscience: A Salience Network Perspective

Human neuroimaging has demonstrated the existence of large-scale functional networks in the cerebral cortex consisting of topographically distant brain regions with functionally correlated activity. The salience network (SN), which is involved in detecting salient stimuli and mediating inter-network communication, is a crucial functional network that is disrupted in addiction. Individuals with addiction display dysfunctional structural and functional connectivity of the SN. Furthermore, while there is a growing body of evidence regarding the SN, addiction, and the relationship between the two, there are still many unknowns, and there are fundamental limitations to human neuroimaging studies. At the same time, advances in molecular and systems neuroscience techniques allow researchers to manipulate neural circuits in nonhuman animals with increasing precision. Here, we describe attempts to translate human functional networks to nonhuman animals to uncover circuit-level mechanisms. To do this, we review the structural and functional connections of the salience network and its homology across species. We then describe the existing literature in which circuit-specific perturbation of the SN sheds light on how functional cortical networks operate, both within and outside the context of addiction. Finally, we highlight key outstanding opportunities for mechanistic studies of the SN.

The architecture of abnormal reward behaviour in dementia: multimodal hedonic phenotypes and brain substrate

Abnormal reward processing is a hallmark of neurodegenerative diseases, most strikingly in frontotemporal dementia. However, the phenotypic repertoire and neuroanatomical substrates of abnormal reward behaviour in these diseases remain incompletely characterized and poorly understood. Here we addressed these issues in a large, intensively phenotyped patient cohort representing all major syndromes of sporadic frontotemporal dementia and Alzheimer's disease. We studied 27 patients with behavioural variant frontotemporal dementia, 58 with primary progressive aphasia (22 semantic variant, 24 non-fluent/agrammatic variant and 12 logopenic) and 34 with typical amnestic Alzheimer's disease, in relation to 42 healthy older individuals. Changes in behavioural responsiveness were assessed for canonical primary rewards (appetite, sweet tooth, sexual activity) and non-primary rewards (music, religion, art, colours), using a semi-structured survey completed by patients' primary caregivers. Changes in more general socio-emotional behaviours were also recorded. We applied multiple correspondence analysis and k-means clustering to map relationships between hedonic domains and extract core factors defining aberrant hedonic phenotypes. Neuroanatomical associations were assessed using voxel-based morphometry of brain MRI images across the combined patient cohort. Altered (increased and/or decreased) reward responsiveness was exhibited by most patients in the behavioural and semantic variants of frontotemporal dementia and around two-thirds of patients in other dementia groups, significantly (P < 0.05) more frequently than in healthy controls. While food-directed changes were most prevalent across the patient cohort, behavioural changes directed toward non-primary rewards occurred significantly more frequently (P < 0.05) in the behavioural and semantic variants of frontotemporal dementia than in other patient groups. Hedonic behavioural changes across the patient cohort were underpinned by two principal factors: a 'gating' factor determining the emergence of altered reward behaviour and a 'modulatory' factor determining how that behaviour is directed. These factors were expressed jointly in a set of four core, trans-diagnostic and multimodal hedonic phenotypes: 'reward-seeking', 'reward-restricted', 'eating-predominant' and 'control-like'-variably represented across the cohort and associated with more pervasive socio-emotional behavioural abnormalities. The principal gating factor was associated (P < 0.05 after correction for multiple voxel-wise comparisons over the whole brain) with a common profile of grey matter atrophy in anterior cingulate, bilateral temporal poles, right middle frontal and fusiform gyri: the cortical circuitry that mediates behavioural salience and semantic and affective appraisal of sensory stimuli. Our findings define a multi-domain phenotypic architecture for aberrant reward behaviours in major dementias, with novel implications for the neurobiological understanding and clinical management of these diseases.

Reinforcement learning detuned in addiction: integrative and translational approaches

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Anterior Cingulate Cortex Ablation Disrupts Affective Vigor and Vigilance

Despite many observations of anterior cingulate cortex (ACC) activity related to cognition and affect in humans and nonhuman animals, little is known about the causal role of the ACC in psychological processes. Here, we investigate the causal role of the ACC in affective responding to threat in rhesus monkeys (Macaca mulatta), a species with an ACC largely homologous to humans in structure and connectivity. Male adult monkeys received bilateral ibotenate axon-sparing lesions to the ACC (sulcus and gyrus of areas 24, 32, and 25) and were tested in two classic tasks of monkey threat processing: the human intruder and object responsiveness tasks. Monkeys with ACC lesions did not significantly differ from controls in their overall mean reactivity toward threatening or novel stimuli. However, while control monkeys maintained their reactivity across test days, monkeys with ACC lesions reduced their reactivity toward stimuli as days advanced. Critically, this attenuated reactivity was found even when the stimuli presented each day were novel, suggesting that ACC lesions did not simply cause accelerated adaptation to stimuli as they became less novel over repeated presentations. Rather, these results imply that the primate ACC is necessary for maintaining appropriate affective responses toward potentially harmful and/or novel stimuli. These findings therefore have implications for mood disorders in which responding to threat and novelty is disrupted.SIGNIFICANCE STATEMENT Decades of research in humans and nonhuman animals have investigated the role of the anterior cingulate cortex in a huge number and variety of psychological processes spanning cognition and affect, as well as in psychological and neurologic diseases. The structure is broadly implicated in psychological processes and mental and neurologic health, yet its causal role in these processes has largely gone untested, particularly in primates. Here we demonstrate that when anterior cingulate cortex is completely eliminated, rhesus monkeys are initially responsive to threats, but these responses attenuate rather than persist, resembling a pattern of behavior commonly seen in patients diagnosed with mood disorders.

Interaction between decision-making and interoceptive representations of bodily arousal in frontal cortex

How bodily arousal states influence decision-making has been a central question in psychology, but the neural mechanisms are unclear. We recorded heart rate (HR), a measure of bodily arousal, while simultaneously monitoring neural activity in orbitofrontal cortex (OFC) and dorsal anterior cingulate cortex (dACC) of macaques making reward-guided decisions. In intact macaques, higher HR was associated with shorter reaction times. Concurrently, the activity of a set of neurons in OFC and dACC selectively encoded HR. Following amygdala lesions, HR generally increased, and now the relationship between HR and reaction times was altered. At the neural level, the balance of encoding in dACC shifted toward signaling HR, suggesting a specific mechanism through which bodily arousal influences decision-making.

Decision-making and representations of arousal are intimately linked. Behavioral investigations have classically shown that either too little or too much bodily arousal is detrimental to decision-making, indicating that there is an inverted “U” relationship between bodily arousal and performance. How these processes interact at the level of single neurons as well as the neural circuits involved are unclear. Here we recorded neural activity from orbitofrontal cortex (OFC) and dorsal anterior cingulate cortex (dACC) of macaque monkeys while they made reward-guided decisions. Heart rate (HR) was also recorded and used as a proxy for bodily arousal. Recordings were made both before and after subjects received excitotoxic lesions of the bilateral amygdala. In intact monkeys, higher HR facilitated reaction times (RTs). Concurrently, a set of neurons in OFC and dACC selectively encoded trial-by-trial variations in HR independent of reward value. After amygdala lesions, HR increased, and the relationship between HR and RTs was altered. Concurrent with this change, there was an increase in the proportion of dACC neurons encoding HR. Applying a population-coding analysis, we show that after bilateral amygdala lesions, the balance of encoding in dACC is skewed away from signaling either reward value or choice direction toward HR coding around the time that choices are made. Taken together, the present results provide insight into how bodily arousal and decision-making are signaled in frontal cortex.

Valence processing in the PFC: Reconciling circuit-level and systems-level views

An essential component in animal behavior is the ability to process emotion and dissociate among positive and negative valence in response to a rewarding or aversive stimulus. The medial prefrontal cortex (mPFC)-responsible for higher order executive functions that include cognition, learning, and working memory; and is also involved in sociability-plays a major role in emotional processing and control. Although the amygdala is widely regarded as the "emotional hub," the mPFC encodes for context-specific salience and elicits top-down control over limbic circuitry. The mPFC can then conduct behavioral responses, via cortico-striatal and cortico-brainstem pathways, that correspond to emotional stimuli. Evidence shows that abnormalities within the mPFC lead to sociability deficits, working memory impairments, and drug-seeking behavior that include addiction and compulsive disorders; as well as conditions such as anhedonia. Recent studies investigate the effects of aberrant salience processing on cortical circuitry and neuronal populations associated with these behaviors. In this chapter, we discuss mPFC valence processing, neuroanatomical connections, and physiological substrates involved in mPFC-associated behavior. We review neurocomputational and theoretical models such as "mixed selectivity," that describe cognitive control, attentiveness, and motivational drives. Using this knowledge, we describe the effects of valence imbalances and its influence on mPFC neural pathways that contribute to deficits in social cognition, while understanding the effects in addiction/compulsive behaviors and anhedonia.

The anterior cingulate cortex and event-based modulation of autonomic states

In spite of being an intensive area of research focus, the anterior cingulate cortex (ACC) remains somewhat of an enigma. Many theories have focused on its role in various aspects of cognition yet surgically precise lesions of the ACC, used to treat severe emotional disorders in human patients, typically have no lasting effects on cognition. An alternative view is that the ACC has a prominent role in regulating autonomic states. This view is consistent with anatomical data showing that a main target of the ACC are regions involved in autonomic control and with the observation that stimulation of the ACC evokes changes in autonomic states in both animals and humans. From an electrophysiological perspective, ACC neurons appear able to represent virtually any event or internal state, even though there is not always a strong link between these representations and behavior. Ensembles of neurons form robust contextual representations that strongly influence how specific events are encoded. The activity patterns associated with these contextually-based event representations presumably impact activity in downstream regions that control autonomic state. As a result, the ACC may regulate the autonomic and perhaps emotional reactions to events it is representing. This event-based control of autonomic tone by the ACC would likely arise during all types of cognitive and affective processes, without necessarily being critical for any of them.

Reinforcement Learning during Adolescence in Rats

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Stable and dynamic representations of value in the prefrontal cortex

Optimal decision-making requires that stimulus-value associations are kept up to date by constantly comparing the expected value of a stimulus with its experienced outcome. To do this, value information must be held in mind when a stimulus and outcome are separated in time. However, little is known about the neural mechanisms of working memory (WM) for value. Contradicting theories have suggested WM requires either persistent or transient neuronal activity, with stable or dynamic representations, respectively. To test these hypotheses, we recorded neuronal activity in the orbitofrontal and anterior cingulate cortex of two monkeys performing a valuation task. We found that features of all hypotheses were simultaneously present in prefrontal activity, and no single hypothesis was exclusively supported. Instead, mixed dynamics supported robust, time invariant value representations while also encoding the information in a temporally specific manner. We suggest that this hybrid coding is a critical mechanism supporting flexible cognitive abilities.

Reorganization of theta phase-locking in the orbitofrontal cortex drives cocaine choice under the influence

Cortical theta oscillations of neuronal activity are a fundamental mechanism driving goal-directed behavior. We previously identified in the rat orbitofrontal cortex (OFC) a neuronal correlate of individual preferences between cocaine use and an alternative nondrug reward (i.e. saccharin). Whether theta oscillations are also associated with choice behavior between a drug and a nondrug reward remains unknown. Here we investigated the temporal structure between single unit activity and theta band oscillations (4-12 Hz) in the OFC of rats choosing between cocaine and saccharin. First, we found that the relative amplitude of theta oscillations is associated with subjective value and preference between two rewards. Second, OFC phase-locked neurons fired on opposite phase of the theta oscillation during saccharin and cocaine rewards, suggesting the existence of two separable neuronal assemblies. Finally, the pharmacological influence of cocaine at the moment of choice altered both theta band power and theta phase-locking in the OFC. That is, this drug influence shifted spike-phase relative to theta cycle and decreased the synchronization of OFC neurons relative to the theta oscillation. Overall, this study indicates that the reorganization of theta phase-locking under the influence of cocaine biases OFC neuronal assemblies in favor of cocaine choice and at the expense of a normally preferred alternative, a neuronal change that may contribute to drug preference in cocaine addiction.

Orbitofrontal cortex is selectively activated in a primate model of attentional bias to cocaine cues

Attentional bias to drug-associated cues correlates with extent of current use, and risk of relapse among those attempting abstinence. Electroencephalogram (EEG) and functional imaging measures in clinical studies have previously investigated the neural basis of attentional bias, but the lack of animal models precluded investigation at the single-unit level. To complement results obtained from clinical studies, we have employed a non-human primate model of attentional bias to cocaine cues while simultaneously recording single-unit activity in cortical and striatal regions implicated in reward processing. Rhesus macaques conditioned to associate particular colors with cocaine or water reward performed an attentional bias task, in which those colors served as irrelevant distractors. Concurrently, multiple electrode arrays for recording single-unit activity were acutely implanted into the orbitofrontal cortex, anterior cingulate cortex, dorsal anterior striatum, and ventral striatum. As in clinical studies, attentional bias was indicated by elongated response times on trials with cocaine-associated distractors compared with trials with water-associated, or control unconditioned distractors. In both animals studied, across an unbiased sample of neurons, the orbitofrontal cortex differentiated distractor condition by the proportion of single-units activated, as well as by population response. In one of the two, the anterior cingulate cortex did as well, but neither striatal region did in either animal. These direct measures of single-unit activity in a primate model complement clinical imaging observations suggesting that cortical mechanisms, especially in orbitofrontal cortex, are likely involved in attentional bias to cocaine-associated environmental cues.

Reward

Neurons throughout frontal cortex show robust responses to rewards, but a challenge is determining the specific function served by these different reward signals. Most neuropsychiatric disorders involve dysfunction of circuits between frontal cortex and subcortical structures, such as the striatum. There are multiple frontostriatal loops, and different neuropsychiatric disorders involve different loops to greater or lesser extents. Understanding the role of reward in each of these different circuits is a necessary step in developing novel treatments for these disorders. This chapter summarizes the recent literature that has identified the role of reward in different subregions of the frontal cortex. Orbitofrontal cortex integrates information about multiple aspects of expected rewards in order to derive their value, which can then be used to decide between alternative potential rewards. Neurons in anterior cingulate cortex encode the difference between the expected reward and the actual outcome. This information is useful for learning, since it can ensure that behavior changes when the outcome was not anticipated. Reward also affects signals in lateral prefrontal cortex related to attention and response selection, ensuring that behaviors are optimally prioritized. Finally, the chapter discusses how reward signals contribute to social processing and autonomic control.

Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes

Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.

Preference for Cocaine is Represented in the Orbitofrontal Cortex by an Increased Proportion of Cocaine Use-Coding Neurons

Cocaine addiction is a harmful preference for drug use over and at the expense of other nondrug-related activities. Here we identify in the rat orbitofrontal cortex (OFC) a mechanism that explains individual preferences between cocaine use and an alternative, nondrug action. OFC neuronal activity was recorded while rats performed each of these 2 actions separately or while they chose between them. First, we found that these actions are encoded by 2 nonoverlapping neuronal populations and that the relative size of the cocaine population represented individual preferences. A larger relative size was only observed in cocaine-preferring individuals. Second, OFC neurons encoding a given individual's preferred action progressively fired more than other action-coding neurons few seconds before the preferred action was actually chosen, suggesting a prechoice neuronal competition for action selection. In cocaine-preferring rats, this manifested by a prechoice ramping-up activity in favor of the cocaine population. Finally, pharmacological manipulation of prechoice activity in favor of the cocaine population caused nondrug-preferring rats to shift their choice to cocaine. Overall, this study suggests that an individual preference for cocaine is represented in the OFC by a population size bias that systematically advantages cocaine use-coding neurons during prechoice competition for action selection.

Anterior cingulate cortex encoding of effortful behavior

An animal’s ability to assess the value of their behaviors to minimize energy use while maximizing goal achievement is critical to its survival. The anterior cingulate cortex (ACC) has been previously shown to play a critical role in this behavioral optimization process, especially when animals are faced with effortful behaviors. In the present study, we designed a novel task to investigate the role of the ACC in evaluating behaviors that varied in effort but all resulted in the same outcome. We recorded single unit activity from the ACC as rats ran back and forth in a shuttle box that could be tilted to different tilt angles (0, 15, and 25°) to manipulate effort. Overall, a majority of ACC neurons showed selective firing to specific effort conditions. During effort expenditure, ACC units showed a consistent firing rate bias toward the downhill route compared with the more difficult uphill route, regardless of the tilt angle of the apparatus. Once rats completed a run and received their fixed reward, ACC units also showed a clear firing rate preference for the single condition with the highest relative value (25° downhill). To assess effort preferences, we used a choice version of our task and confirmed that rats prefer downhill routes to uphill routes when given the choice. Overall, these results help to elucidate the functional role of the ACC in monitoring and evaluating effortful behaviors that may then bias decision-making toward behaviors with the highest utility.

NEW & NOTEWORTHY We developed a novel effort paradigm to investigate how the anterior cingulate cortex (ACC) responds to behaviors with varied degrees of physical effort and how changes in effort influence the ACC’s evaluation of behavioral outcomes. Our results provide evidence for a wider role of the ACC in its ability to motivate effortful behaviors and evaluate the outcome of multiple behaviors within an environment.

Sucrose intensity coding and decision-making in rat gustatory cortices

Sucrose's sweet intensity is one attribute contributing to the overconsumption of high-energy palatable foods. However, it is not known how sucrose intensity is encoded and used to make perceptual decisions by neurons in taste-sensitive cortices. We trained rats in a sucrose intensity discrimination task and found that sucrose evoked a widespread response in neurons recorded in posterior-Insula (pIC), anterior-Insula (aIC), and Orbitofrontal cortex (OFC). Remarkably, only a few Intensity-selective neurons conveyed the most information about sucrose's intensity, indicating that for sweetness the gustatory system uses a compact and distributed code. Sucrose intensity was encoded in both firing-rates and spike-timing. The pIC, aIC, and OFC neurons tracked movement direction, with OFC neurons yielding the most robust response. aIC and OFC neurons encoded the subject's choices, whereas all three regions tracked reward omission. Overall, these multimodal areas provide a neural representation of perceived sucrose intensity, and of task-related information underlying perceptual decision-making.

Orbitofrontal Lesion Alters Brain Dynamics of Emotion-Attention and Emotion-Cognitive Control Interaction in Humans

Patients with lesion to the orbitofrontal cortex (OFC) experience challenges in emotional control and emotion-guided behaviors. The OFC is known to participate in executive functions and attentional control of emotion and our previous research suggests OFC lesion alters the balance between voluntary and involuntary attention and cognitive control within the context of emotion. To better understand how OFC lesion affects the dynamics and interaction of these functions, we studied EEG and performance of 12 patients with lesion to the OFC and 11 control subjects with intact OFC in a Go/NoGo visual reaction time (RT) task with neutral targets and intervening threat-related emotional distractors (Executive RT Test). Event-related potentials (ERPs), specifically N2P3 peak-to-peak amplitude and the following late positive potential (LPP), were used to measure allocation of attention and cognitive control to emotional distractors. Task performance and Behavior Rating Inventory of Executive Functions—Adult version (BRIEF-A) scores were used to assess executive functions. As expected, the Control group showed increased N2P3 amplitude in the context of threat-related distractors, particularly over the right hemisphere, while LPP was not modulated by these distractors. In contrast, patients with OFC lesion showed no such impact of threat-related distractors on N2P3 amplitude but exhibited increased and prolonged left-lateralized impact of threat on LPP in the Go-condition. In NoGo-condition, the N2P3 amplitude was increased in both groups due to threat, but the impact was seen earlier, i.e., at the N2 peak in the OFC group and later at the P3 peak in Controls. The OFC group committed more errors in the Executive RT Test and reported more problems in BRIEF-A, thus both objective and subjective evidence for challenges in executive functions was obtained in patients with orbitofrontal lesion. Furthermore, the time-course of attention allocation and cognitive control towards task-irrelevant emotional stimuli was altered as evidenced by ERPs. We conclude that orbitofrontal lesion is associated with altered neural dynamics underlying the interaction of involuntary attention to emotion and cognitive control. These alterations in brain dynamics may underlie some of the challenges patients encounter in everyday life when emotional events interact with cognitive demands.

Prefrontal expectancy and reinforcement-driven antidepressant placebo effects

Placebo responses in depression exemplify how expectancies and appraisals impact mood. Cognitive and neural mechanisms underlying these responses are still poorly understood, partly due to the difficulty of simulating antidepressant effects and manipulating mood experimentally. To address these challenges, we developed an acute antidepressant placebo experiment involving the intravenous administration of a "fast-acting antidepressant" and a trial-by-trial sham fMRI "neurofeedback" manipulation, purporting to reveal mood-relevant neural responses. Twenty volunteers with major depression underwent this experiment while rating their expected and actual mood improvement. Mixed-effects analyses of trial-by-trial ratings revealed that the "drug" infusion cues induced higher expectancies of mood improvement, while both the "drug" infusion cue and the sham neurofeedback induced a reported mood improvement. Neurofeedback of greater magnitude, compared to lower magnitude, recruited the lateral prefrontal cortex (lPFC). Individuals with greater lPFC responses to neurofeedback displayed: (1) greater effect of previous mood improvement on expectancy ratings and (2) greater effect of sham neurofeedback on mood improvement. Behavioral antidepressant placebo effects were additionally moderated by changes in peripheral β-endorphin plasma levels and depressive symptomatology. These data demonstrate the feasibility of trial-by-trial manipulation of antidepressant placebo-associated expectancies and their reinforcement. We provide initial insights into the role of the lPFC in the interplay between placebo-induced expectancies and mood, as well as preliminary evidence for the role of the opioid system in antidepressant placebo effects.

Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex

Competing accounts propose that working memory (WM) is subserved either by persistent activity in single neurons or by dynamic (time-varying) activity across a neural population. Here, we compare these hypotheses across four regions of prefrontal cortex (PFC) in an oculomotor-delayed-response task, where an intervening cue indicated the reward available for a correct saccade. WM representations were strongest in ventrolateral PFC neurons with higher intrinsic temporal stability (time-constant). At the population-level, although a stable mnemonic state was reached during the delay, this tuning geometry was reversed relative to cue-period selectivity, and was disrupted by the reward cue. Single-neuron analysis revealed many neurons switched to coding reward, rather than maintaining task-relevant spatial selectivity until saccade. These results imply WM is fulfilled by dynamic, population-level activity within high time-constant neurons. Rather than persistent activity supporting stable mnemonic representations that bridge subsequent salient stimuli, PFC neurons may stabilise a dynamic population-level process supporting WM.

Economic Choice as an Untangling of Options into Actions

We propose that economic choice can be understood as a gradual transformation from a domain of options to one of the actions. We draw an analogy with the idea of untangling information in the form vision system and propose that form vision and economic choice may be two aspects of a larger process that sculpts actions based on sensory inputs. From this viewpoint, choice results from the accumulated effect of repetitions of simple computations. These may consist primarily of relative valuations (evaluations relative to the value of rejection, perhaps in a manner akin to divisive normalization) applied to individual offers. With regard to economic choice, cortical brain regions differ primarily in their position and in what information they prioritize, and do not-with a few exceptions-have categorically distinct roles. Each region's specific contribution is determined largely by its inputs; thus, understanding connectivity is crucial for understanding choice. This view suggests that there is no single site of choice, that there is no meaningful distinction between pre- and post-decisionality, and that there is no explicit representation of value in the brain.

Corticoinsular circuits encode subjective value expectation and violation for effortful goal-directed behavior

The ability to form value estimates is crucial for optimal decision making, especially when not all features of a choice option are known. To date, however, the neural mechanisms for expectation processes under conditions of incomplete information are unknown. Using computational fMRI, we show that ventromedial prefrontal cortex encodes the expected value of a trial. We also observe a distinct network composed of dorsal anterior cingulate, anterior insula, and dorsomedial caudate that encodes an expectation violation or prediction error signal, based on previous trial history. These findings highlight how the brain computes and monitors value-based predictions during effortful goal-directed behavior when choice-relevant information is not fully available.

We are presented with choices each day about how to invest our effort to achieve our goals. Critically, these decisions must frequently be made under conditions of incomplete information, where either the effort required or possible reward to be gained is uncertain. Such choices therefore require the development of potential value estimates to guide effortful goal-directed behavior. To date, however, the neural mechanisms for this expectation process are unknown. Here, we used computational fMRI during an effort-based decision-making task where trial-wise information about effort costs and reward magnitudes was presented separately over time, thereby allowing us to model distinct effort/reward computations as choice-relevant information unfolded. We found that ventromedial prefrontal cortex (vmPFC) encoded expected subjective value. Further, activity in dorsal anterior cingulate (dACC) and anterior insula (aI) reflected both effort discounting as well as a subjective value prediction error signal derived from trial history. While prior studies have identified these regions as being involved in effort-based decision making, these data demonstrate their specific role in the formation and maintenance of subjective value estimates as relevant information becomes available.

Robust Encoding of Spatial Information in Orbitofrontal Cortex and Striatum

Knowing whether core reward regions carry information about the positions of relevant objects is crucial for adjudicating between choice models. One limitation of previous studies, including our own, is that spatial positions can be consistently differentially associated with rewards, and thus position can be confounded with attention, motor plans, or target identity. We circumvented these problems by using a task in which value-and thus choices-was determined solely by a frequently changing rule, which was randomized relative to spatial position on each trial. We presented offers asynchronously, which allowed us to control for reward expectation, spatial attention, and motor plans in our analyses. We find robust encoding of the spatial position of both offers and choices in two core reward regions, orbitofrontal Area 13 and ventral striatum, as well as in dorsal striatum of macaques. The trial-by-trial correlation in noise in encoding of position was associated with variation in choice, an effect known as choice probability correlation, suggesting that the spatial encoding is associated with choice and is not incidental to it. Spatial information and reward information are not carried by separate sets of neurons, although the two forms of information are temporally dissociable. These results highlight the ubiquity of multiplexed information in association cortex and argue against the idea that these ostensible reward regions serve as part of a pure value domain.

Covert shift of attention modulates the value encoding in the orbitofrontal cortex

During value-based decision making, we often evaluate the value of each option sequentially by shifting our attention, even when the options are presented simultaneously. The orbitofrontal cortex (OFC) has been suggested to encode value during value-based decision making. Yet it is not known how its activity is modulated by attention shifts. We investigated this question by employing a passive viewing task that allowed us to disentangle effects of attention, value, choice and eye movement. We found that the attention modulated OFC activity through a winner-take-all mechanism. When we attracted the monkeys’ attention covertly, the OFC neuronal activity reflected the reward value of the newly attended cue. The shift of attention could be explained by a normalization model. Our results strongly argue for the hypothesis that the OFC neuronal activity represents the value of the attended item. They provide important insights toward understanding the OFC’s role in value-based decision making.

Orbitofrontal Cortex: A Neural Circuit for Economic Decisions

Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.

Brain activation for spontaneous and explicit false belief tasks overlaps: new fMRI evidence on belief processing and violation of expectation

There is extensive discussion on whether spontaneous and explicit forms of ToM are based on the same cognitive/neural mechanisms or rather reflect qualitatively different processes. For the first time, we analyzed the BOLD signal for false belief processing by directly comparing spontaneous and explicit ToM task versions. In both versions, participants watched videos of a scene including an agent who acquires a true or false belief about the location of an object (belief formation phase). At the end of the movies (outcome phase), participants had to react to the presence of the object. During the belief formation phase, greater activity was found for false vs true belief trials in the right posterior parietal cortex. The ROI analysis of the right temporo-parietal junction (TPJ), confirmed this observation. Moreover, the anterior medial prefrontal cortex (aMPFC) was active during the outcome phase, being sensitive to violation of both the participant’s and agent’s expectations about the location of the object. Activity in the TPJ and aMPFC was not modulated by the spontaneous/explicit task. Overall, these data show that neural mechanisms for spontaneous and explicit ToM overlap. Interestingly, a dissociation between TPJ and aMPFC for belief tracking and outcome evaluation, respectively, was also found.

Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes

Advantageous foraging choices benefit from an estimation of two aspects of a resource's value: its current desirability and availability. Both orbitofrontal and ventrolateral prefrontal areas contribute to updating these valuations, but their precise roles remain unclear. To explore their specializations, we trained macaque monkeys on two tasks: one required updating representations of a predicted outcome's desirability, as adjusted by selective satiation, and the other required updating representations of an outcome's availability, as indexed by its probability. We evaluated performance on both tasks in three groups of monkeys: unoperated controls and those with selective, fiber-sparing lesions of either the OFC or VLPFC. Representations that depend on the VLPFC but not the OFC play a necessary role in choices based on outcome availability; in contrast, representations that depend on the OFC but not the VLPFC play a necessary role in choices based on outcome desirability.

Linking ADHD to the Neural Circuitry of Attention

Attention deficit hyperactivity disorder (ADHD) is a complex condition with a heterogeneous presentation. Current diagnosis is primarily based on subjective experience and observer reports of behavioral symptoms - an approach that has significant limitations. Many studies show that individuals with ADHD exhibit poorer performance on cognitive tasks than neurotypical controls, and at least seven main functional domains appear to be implicated in ADHD. We discuss the underlying neural mechanisms of cognitive functions associated with ADHD, with emphasis on the neural basis of selective attention, demonstrating the feasibility of basic research approaches for further understanding cognitive behavioral processes as they relate to human psychopathology. The study of circuit-level mechanisms underlying executive functions in nonhuman primates holds promise for advancing our understanding, and ultimately the treatment, of ADHD.

Sustained Activity Encoding Working Memories: Not Fully Distributed

Working memory (WM) is the ability to remember and manipulate information for short time intervals. Recent studies have proposed that sustained firing encoding the contents of WM is ubiquitous across cortical neurons. We review here the collective evidence supporting this claim. A variety of studies report that neurons in prefrontal, parietal, and inferotemporal association cortices show robust sustained activity encoding the location and features of memoranda during WM tasks. However, reports of WM-related sustained activity in early sensory areas are rare, and typically lack stimulus specificity. We propose that robust sustained activity that can support WM coding arises as a property of association cortices downstream from the early stages of sensory processing.

Fidelity of the representation of value in decision-making

The ability to make optimal decisions depends on evaluating the expected rewards associated with different potential actions. This process is critically dependent on the fidelity with which reward value information can be maintained in the nervous system. Here we directly probe the fidelity of value representation following a standard reinforcement learning task. The results demonstrate a previously-unrecognized bias in the representation of value: extreme reward values, both low and high, are stored significantly more accurately and precisely than intermediate rewards. The symmetry between low and high rewards pertained despite substantially higher frequency of exposure to high rewards, resulting from preferential exploitation of more rewarding options. The observed variation in fidelity of value representation retrospectively predicted performance on the reinforcement learning task, demonstrating that the bias in representation has an impact on decision-making. A second experiment in which one or other extreme-valued option was omitted from the learning sequence showed that representational fidelity is primarily determined by the relative position of an encoded value on the scale of rewards experienced during learning. Both variability and guessing decreased with the reduction in the number of options, consistent with allocation of a limited representational resource. These findings have implications for existing models of reward-based learning, which typically assume defectless representation of reward value.

Many models of learning and decision-making assume that experienced rewards are stored without error. We examined this assumption experimentally: participants first learned an association between different options and rewards in a simple two-alternative choice task. We then asked them to report what reward they expected to receive for each of the options they had experienced. We checked that the reports we collected matched performance on the choice task, meaning that the values participants reported were the same as those they used to decide between options. The results showed that participants were both less precise (greater variability) and less accurate (greater bias) in their reports of middling reward values compared to either high- or low-valued options. Reports of high and low values were similar in quality even though participants had experienced the rewards associated with high-value options considerably more often. Whether an option’s value was stored well or poorly was not fixed, but instead depended on how the value compared to other options the participant had experienced. These results should lead to better models of how decisions are made based on experiences of reward.

Orbitofrontal Cortex Signals Expected Outcomes with Predictive Codes When Stable Contingencies Promote the Integration of Reward History

Memory can inform goal-directed behavior by linking current opportunities to past outcomes. The orbitofrontal cortex (OFC) may guide value-based responses by integrating the history of stimulus-reward associations into expected outcomes, representations of predicted hedonic value and quality. Alternatively, the OFC may rapidly compute flexible "online" reward predictions by associating stimuli with the latest outcome. OFC neurons develop predictive codes when rats learn to associate arbitrary stimuli with outcomes, but the extent to which predictive coding depends on most recent events and the integrated history of rewards is unclear. To investigate how reward history modulates OFC activity, we recorded OFC ensembles as rats performed spatial discriminations that differed only in the number of rewarded trials between goal reversals. The firing rate of single OFC neurons distinguished identical behaviors guided by different goals. When >20 rewarded trials separated goal switches, OFC ensembles developed stable and anticorrelated population vectors that predicted overall choice accuracy and the goal selected in single trials. When <10 rewarded trials separated goal switches, OFC population vectors decorrelated rapidly after each switch, but did not develop anticorrelated firing patterns or predict choice accuracy. The results show that, whereas OFC signals respond rapidly to contingency changes, they predict choices only when reward history is relatively stable, suggesting that consecutive rewarded episodes are needed for OFC computations that integrate reward history into expected outcomes.SIGNIFICANCE STATEMENT Adapting to changing contingencies and making decisions engages the orbitofrontal cortex (OFC). Previous work shows that OFC function can either improve or impair learning depending on reward stability, suggesting that OFC guides behavior optimally when contingencies apply consistently. The mechanisms that link reward history to OFC computations remain obscure. Here, we examined OFC unit activity as rodents performed tasks controlled by contingencies that varied reward history. When contingencies were stable, OFC neurons signaled past, present, and pending events; when contingencies were unstable, past and present coding persisted, but predictive coding diminished. The results suggest that OFC mechanisms require stable contingencies across consecutive episodes to integrate reward history, represent predicted outcomes, and inform goal-directed choices.

Back to basics: Making predictions in the orbitofrontal-amygdala circuit

Underlying many complex behaviors are simple learned associations that allow humans and animals to anticipate the consequences of their actions. The orbitofrontal cortex and basolateral amygdala are two regions which are crucial to this process. In this review, we go back to basics and discuss the literature implicating both these regions in simple paradigms requiring the development of associations between stimuli and the motivationally-significant outcomes they predict. Much of the functional research surrounding this ability has suggested that the orbitofrontal cortex and basolateral amygdala play very similar roles in making these predictions. However, electrophysiological data demonstrates critical differences in the way neurons in these regions respond to predictive cues, revealing a difference in their functional role. On the basis of these data and theories that have come before, we propose that the basolateral amygdala is integral to updating information about cue-outcome contingencies whereas the orbitofrontal cortex is critical to forming a wider network of past and present associations that are called upon by the basolateral amygdala to benefit future learning episodes. The tendency for orbitofrontal neurons to encode past and present contingencies in distinct neuronal populations may facilitate its role in the formation of complex, high-dimensional state-specific associations.

Dorsal Anterior Cingulate Cortex: A Bottom-Up View

The dorsal anterior cingulate cortex (dACC) has attracted great interest from neuroscientists because it is associated with so many important cognitive functions. Despite, or perhaps because of, its rich functional repertoire, we lack a single comprehensive view of its function. Most research has approached this puzzle from the top down, using aggregate measures such as neuroimaging. We provide a view from the bottom up, with a focus on single-unit responses and anatomy. We summarize the strengths and weaknesses of the three major approaches to characterizing the dACC: as a monitor, as a controller, and as an economic structure. We argue that neurons in the dACC are specialized for representing contexts, or task-state variables relevant for behavior, and strategies, or aspects of future plans. We propose that dACC neurons link contexts with strategies by integrating diverse task-relevant information to create a rich representation of task space and exert high-level and abstract control over decision and action.

Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.

Individualized real-time fMRI neurofeedback to attenuate craving in nicotine-dependent smokers

BACKGROUND

Cue-induced craving plays an important role in relapse, and the neural correlates of cue-induced craving have been elucidated using fMRI. This study examined the utility of real-time fMRI (rtfMRI) neurofeedback to strengthen self-regulation of craving-related neural activation and cue-reactivity in cigarette smokers.

METHODS

Nicotine-dependent smokers were randomized to rtfMRI neurofeedback or to a no-feedback control group. Participants completed 3 neuroimaging visits. Within each visit, an initial run during which smoking-related cues were used to provoke craving, an individualized craving-related region of interest (ROI) in the prefrontal cortex or anterior cingulate cortex was identified. In the rtfMRI group, activity from the ROI was fed back via a visual display during 3 subsequent runs while participants were instructed to reduce craving during cue exposure. The control group had an identical experience with no feedback provided.

RESULTS

Forty-four nicotine-dependent smokers were recruited to participate in our study; data from the 33 participants who completed a 1-week follow-up visit were included in the analysis. Subjective craving ratings and cue-induced brain activation were lower in the rtfMRI group than in the control group.

LIMITATIONS

As participants were not seeking treatment, clinical outcomes are lacking.

CONCLUSION

Nicotine-dependent smokers receiving rtfMRI feedback from an individualized ROI attenuated smoking cue-elicited neural activation and craving, relative to a control group. Further studies are needed in treatment-seeking smokers to determine if this intervention can translate into a clinically meaningful treatment modality.

Slow-γ Rhythms Coordinate Cingulate Cortical Responses to Hippocampal Sharp-Wave Ripples during Wakefulness

Behavioral changes in response to reward require monitoring past behavior relative to present outcomes. This is thought to involve a fine coordination between the hippocampus (HIPP), which stores and replays memories of past events, and cortical regions such as cingulate cortex, responsible for behavioral planning. Sharp-wave ripple (SWR)-mediated memory replay in the HIPP of awake rodents contributes to learning, but cortical responses to hippocampal SWR during wakefulness are not known. We now show that in rats, during quiet-wakefulness, cingulate neurons exhibit significant responses to SWR, as well as increased modulation by the accompanying hippocampal local field potential slow-γ oscillation, a rhythm associated with intra-hippocampal information processing. The magnitude of cingulate neurons' responses to SWR is significantly correlated with the degree of their modulation by HIPP slow-γ. We hypothesize that during pauses cingulate neurons transiently access episodic information concerning previous choices, replayed by HIPP SWR, to evaluate past trajectories in light of their outcome.

"Heart Shots": a classroom activity to instigate active learning

The present study aimed to provide undergraduate medical students at Melaka Manipal Medical College (Manipal Campus), Manipal University, in Karnataka, India, an opportunity to apply their knowledge in cardiovascular concepts to real-life situations. A group activity named "Heart Shots" was implemented for a batch of first-year undergraduate students (n = 105) at the end of a block (teaching unit). Students were divided into 10 groups each having 10-11 students. They were requested to make a video/PowerPoint presentation about the application of cardiovascular principles to real-life situations. The presentation was required to be of only pictures/photos and no text material, with a maximum duration of 7 min. More than 95% of students considered that the activity helped them to apply their knowledge in cardiovascular concepts to real-life situations and understand the relevance of physiology in medicine and to revise the topic. More than 90% of students agreed that the activity helped them to apply their creativity in improving their knowledge and to establish a link between concepts rather than learning them as isolated facts. Based on the feedback, we conclude that the activity was student centered and that it facilitated learning.

Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning

For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible.

Feedback associated with expectation for larger-reward improves visuospatial working memory performances in children with ADHD

•

Children with ADHD and normal controls were tested in working memory tasks.

•

Availability of feedback and expectation for monetary reward were manipulated.

•

ADHDs showed improved working memory when feedback was associated with larger-reward.

•

Performance improvement in ADHD was associated with brain activity normalization.

We tested the interactive effect of feedback and reward on visuospatial working memory in children with ADHD. Seventeen boys with ADHD and 17 Normal Control (NC) boys underwent functional magnetic resonance imaging (fMRI) while performing four visuospatial 2-back tasks that required monitoring the spatial location of letters presented on a display. Tasks varied in reward size (large; small) and feedback availability (no-feedback; feedback). While the performance of NC boys was high in all conditions, boys with ADHD exhibited higher performance (similar to those of NC boys) only when they received feedback associated with large-reward. Performance pattern in both groups was mirrored by neural activity in an executive function neural network comprised of few distinct frontal brain regions. Specifically, neural activity in the left and right middle frontal gyri of boys with ADHD became normal-like only when feedback was available, mainly when feedback was associated with large-reward. When feedback was associated with small-reward, or when large-reward was expected but feedback was not available, boys with ADHD exhibited altered neural activity in the medial orbitofrontal cortex and anterior insula. This suggests that contextual support normalizes activity in executive brain regions in children with ADHD, which results in improved working memory.

The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes

The orbitofrontal cortex (OFC) has long been associated with the flexible control of behavior and concepts such as behavioral inhibition, self-control, and emotional regulation. These ideas emphasize the suppression of behaviors and emotions, but OFC's affirmative functions have remained enigmatic. Here we review recent work that has advanced our understanding of this prefrontal area and how its functions are shaped through interaction with subcortical structures such as the amygdala. Recent findings have overturned theories emphasizing behavioral inhibition as OFC's fundamental function. Instead, new findings indicate that OFC provides predictions about specific outcomes associated with stimuli, choices, and actions, especially their moment-to-moment value based on current internal states. OFC function thereby encompasses a broad representation or model of an individual's sensory milieu and potential actions, along with their relationship to likely behavioral outcomes.

Reward Acts on the pFC to Enhance Distractor Resistance of Working Memory Representations

Working memory and reward processing are often thought to be separate, unrelated processes. However, most daily activities involve integrating these two types of information, and the two processes rarely, if ever, occur in isolation. Here, we show that working memory and reward interact in a task-dependent manner and that this task-dependent interaction involves modulation of the pFC by the ventral striatum. Specifically, BOLD signal during gains relative to losses in the ventral striatum and pFC was associated not only with enhanced distractor resistance but also with impairment in the ability to update working memory representations. Furthermore, the effect of reward on working memory was accompanied by differential coupling between the ventral striatum and ignore-related regions in the pFC. Together, these data demonstrate that reward-related signals modulate the balance between cognitive stability and cognitive flexibility by altering functional coupling between the ventral striatum and the pFC.

Absence of spatial tuning in the orbitofrontal cortex

There is limited data in the literature to explicitly support the notion that neurons in OFC are truly action-independent in their coding. We set out to specifically test the hypothesis that OFC value-related neurons in area 13 m of the monkey do not carry information about the action required to obtain that reward – that activity in this area represents reward values in an abstract and action-independent manner. To accomplish that goal we had two monkeys select and execute saccadic eye movements to 81 locations in the visual field for three different kinds of juice rewards. Our detailed analysis of the response fields indicates that these neurons are insensitive to the amplitude or direction of the saccade required to obtain these rewards. Our data thus validate earlier proposals that neurons of 13 m in the OFC encode subjective value independent of the saccadic action required to obtain that reward.

Orbitofrontal cortical neurons encode expectation-driven initiation of reward-seeking

Adaptive execution and inhibition of behavior are guided by the activity of neuronal populations across multiple frontal cortical areas. The rodent medial prefrontal cortex has been well studied with respect to these behaviors, influencing behavioral execution/inhibition based on context. Other frontal regions, in particular the orbitofrontal cortex (OFC), are critical in directing behavior to obtain rewards, but the relationship between OFC neuronal activity and response execution or inhibition has been poorly characterized. In particular, little is known about OFC with respect to extinction learning, an important example of context-guided response inhibition. Here, we recorded the activity of OFC neurons while rats performed a discriminative-stimulus (DS)-driven sucrose-seeking task followed by multiple days of extinction of the DS. OFC neuronal activity was maximally responsive (1) to reward-predicting stimuli (RS) that triggered a lever press (i.e., lever-response initiation) and (2) during reward-well approach in pursuit of sucrose (i.e., well-response initiation). RS presentation that was not followed by a lever press or RS presentation during extinction produced weak activation, as did nonrewarded stimulus (NS) presentation regardless of response (press or withhold) or session (DS-sucrose or extinction). Activity related to nonrewarded well entry was minor, and activity was significantly inhibited during reward consumption. Finally, OFC neuronal activity switched selectivity to track rewarded behaviors when the RS/NS contingencies were reversed. Thus, rather than signaling variables related to extinction or response inhibition, activity in OFC was strongest at the initiation of multiple components of reward-seeking behavior, most prominently when valid reward-predicting cues drove these behaviors.