Mechanisms of reinforcement learning and decision making in the primate dorsolateral prefrontal cortex

Decisional brain of lawyers at the workplace. A neurolaw pilot study

This pilot study investigated legal and non-legal professionals' decision process during a typical working day. During self-evaluated highly relevant decisions (rated through a daily diary), the two groups were asked to wear the Muse™ Headband to record their electrophysiological (EEG) activity in terms of frequency bands (delta, theta, alpha and beta). EEG cognitive findings displayed a generally increased beta power in the anterior frontal region (mainly in the right than left) for both groups during highly relevant decisions. Significantly results were also found for the legal professionals' group, for which a decrease of alpha power was found in the left compared to right frontal cortex. Furthermore, a decreased alpha power and increased delta and theta power in the right compared to left Temporo-Parietal Junction was observed in the legal professionals when taking highly relevant decisions. This pilot study suggested a specific EEG pattern for legal professionals while taking highly relevant decisions.

The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning

Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.

Dynamic Foraging Behavior Performance Is Not Affected by Scn2a Haploinsufficiency

Dysfunction in the gene SCN2A, which encodes the voltage-gated sodium channel Nav1.2, is strongly associated with neurodevelopmental disorders including autism spectrum disorder and intellectual disability (ASD/ID). This dysfunction typically manifests in these disorders as a haploinsufficiency, where loss of one copy of a gene cannot be compensated for by the other allele. Scn2a haploinsufficiency affects a range of cells and circuits across the brain, including associative neocortical circuits that are important for cognitive flexibility and decision-making behaviors. Here, we tested whether Scn2a haploinsufficiency has any effect on a dynamic foraging task that engages such circuits. Scn2a +/- mice and wild-type (WT) littermates were trained on a choice behavior where the probability of reward between two options varied dynamically across trials and where the location of the high reward underwent uncued reversals. Despite impairments in Scn2a-related neuronal excitability, we found that both male and female Scn2a +/- mice performed these tasks as well as wild-type littermates, with no behavioral difference across genotypes in learning or performance parameters. Varying the number of trials between reversals or probabilities of receiving reward did not result in an observable behavioral difference, either. These data suggest that, despite heterozygous loss of Scn2a, mice can perform relatively complex foraging tasks that make use of higher-order neuronal circuits.

Instinct to insight: Neural correlates of ethological strategy learning

In ethological behaviors like parenting, animals innately follow stereotyped patterns of choices to decide between uncertain outcomes but can learn to modify their strategies to incorporate new information. For example, female mice in a T-maze instinctively use spatial-memory to search for pups where they last found them but can learn more efficient strategies employing pup-associated acoustic cues. We uncovered neural correlates for transitioning between these innate and learned strategies. Auditory cortex (ACx) was required during learning. ACx firing at the nest increased with learning and correlated with subsequent search speed but not outcome. Surprisingly, ACx suppression rather than facilitation during search was more prognostic of correct sound-cued outcomes - even before adopting a sound-cued strategy. Meanwhile medial prefrontal cortex encoded the last pup location, but this decayed as the spatial-memory strategy declined. Our results suggest a neural competition between a weakening spatial-memory and strengthening sound-cued neural representation to mediate strategy switches.

Neuroeconomics of decision-making during COVID-19 pandemic

The coronavirus disease 2019 (COVID-19) pandemic reveals the decision-making challenges faced by communities, governments, and international organizations, globally. Policymakers are much concerned about protecting the population from the deadly virus while lacking reliable information on the virus and its spread mechanisms and the effectiveness of possible measures and their (direct and indirect) health and socioeconomic costs. This review aims to highlight the various balanced policy decision that would combine the best obtainable scientific evidence characteristically provided by expert opinions and modeling studies. This article's main goal is to summarize the main significant progress in the understanding of neuroeconomics of decision-making and discuss the anatomy of decision making in the light of COVID-19 pandemic.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

The Effect of Repetitive Transcranial Magnetic Stimulation (rTMS) on Cognition in Patients With Traumatic Brain Injury: A Protocol for a Randomized Controlled Trial

Cognitive impairment, defined as a decline in memory and executive function, is one of the most severe complications of traumatic brain injury (TBI). Patients with TBI are often unable to return to work due to cognitive impairment and their overall quality of life is reduced. TBI can bring a serious economic burden to patient's families and to society. Reported findings on the efficacy of repetitive transcranial magnetic stimulation (rTMS) in improving cognitive impairment following TBI are inconsistent. The purpose of the proposed study is to investigate whether rTMS can improve memory and executive function in patients with TBI. Herein, we propose a prospective randomized placebo-controlled (rTMS, sham rTMS, cognitive training), parallel-group, single-center trial. 36 participants with a TBI occurring at least 6 months prior will be recruited from an inpatient rehabilitation center. Participants will be randomly assigned to the real rTMS, sham rTMS, or cognitive training groups with a ratio of 1:1:1. A 20-session transcranial magnetic stimulation protocol will be applied to the left and right dorsolateral prefrontal cortices (DLPFC) at frequencies of 10 Hz and 1 Hz, respectively. Neuropsychological assessments will be performed at four time points: baseline, after the 10th rTMS session, after the 20th rTMS session, and 30 days post-intervention. The primary outcome is change in executive function assessed using the Shape Trail Test (STT). The secondary outcome measures are measures from neuropsychological tests: the Hopkins Verbal Learning Test (HVLT), the Brief Visuospatial Memory Test (BVMT), the Digit Span Test (DST). We report on positive preliminary results in terms of improving memory and executive function as well as beneficial changes in brain connectivity among TBI patients undergoing rTMS and hypothesize that we will obtain similar results in the proposed study.

Impulsivity and Emotional Dysregulation Predict Choice Behavior During a Mixed-Strategy Game in Adolescents With Borderline Personality Disorder

Impulsivity and emotional dysregulation are two core features of borderline personality disorder (BPD), and the neural mechanisms recruited during mixed-strategy interactions overlap with frontolimbic networks that have been implicated in BPD. We investigated strategic choice patterns during the classic two-player game, Matching Pennies, where the most efficient strategy is to choose each option randomly from trial-to-trial to avoid exploitation by one’s opponent. Twenty-seven female adolescents with BPD (mean age: 16 years) and twenty-seven age-matched female controls (mean age: 16 years) participated in an experiment that explored the relationship between strategic choice behavior and impulsivity in both groups and emotional dysregulation in BPD. Relative to controls, BPD participants showed marginally fewer reinforcement learning biases, particularly decreased lose-shift biases, increased variability in reaction times (coefficient of variation; CV), and a greater percentage of anticipatory decisions. A subset of BPD participants with high levels of impulsivity showed higher overall reward rates, and greater modulation of reaction times by outcome, particularly following loss trials, relative to control and BPD participants with lower levels of impulsivity. Additionally, BPD participants with higher levels of emotional dysregulation showed marginally increased reward rate and increased entropy in choice patterns. Together, our preliminary results suggest that impulsivity and emotional dysregulation may contribute to variability in mixed-strategy decision-making in female adolescents with BPD.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

Brain activations associated with anticipation and delivery of monetary reward: A systematic review and meta-analysis of fMRI studies

BACKGROUND

While multiple studies have examined the brain functional correlates of reward, meta-analyses have either focused on studies using the monetary incentive delay (MID) task, or have adopted a broad strategy, combining data from studies using both monetary and non-monetary reward, as probed using a wide range of tasks.

OBJECTIVE

To meta-analyze fMRI studies that used monetary reward and in which there was a definable cue-reward contingency. Studies were limited to those using monetary reward in order to avoid potential heterogeneity from use of other rewards, especially social rewards. Studies using gambling or delay discounting tasks were excluded on the grounds that reward anticipation is not easily quantifiable.

STUDY ELIGIBILITY

English-language fMRI studies (i) that reported fMRI findings on healthy adults; (ii) that used monetary reward; and (iii) in which a cue that was predictive of reward was compared to a no win (or lesser win) condition. Only voxel-based studies were included; those where brain coverage was incomplete were excluded.

DATA SOURCES

Ovid, Medline and PsycInfo, from 2000 to 2020, plus checking of review articles and meta-analyses.

DATA SYNTHESIS

Data were pooled using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI). Heterogeneity among studies was examined using the I2 statistic. Publication bias was examined using funnel plots and statistical examination of asymmetries. Moderator variables including whether the task was pre-learnt, sex distribution, amount of money won and width of smoothing kernel were examined.

RESULTS

Pooled data from 45 studies of reward anticipation revealed activations in the ventral striatum, the middle cingulate cortex/supplementary motor area and the insula. Pooled data from 28 studies of reward delivery again revealed ventral striatal activation, plus cortical activations in the anterior and posterior cingulate cortex. There was relatively little evidence of publication bias. Among moderating variables, only whether the task was pre-learnt exerted an influence.

CONCLUSIONS

According to this meta-analysis monetary reward anticipation and delivery both activate the ventral but not the dorsal striatum, and are associated with different patterns of cortical activation.

Social processing by the primate medial frontal cortex

The primate medial frontal cortex is comprised of several brain regions that are consistently implicated in regulating complex social behaviors. The medial frontal cortex is also critically involved in many non-social behaviors, such as those involved in reward, affective, and decision-making processes, broadly implicating the fundamental role of the medial frontal cortex in internally guided cognition. An essential question therefore is what unique contributions, if any, does the medial frontal cortex make to social behaviors? In this chapter, we outline several neural algorithms necessary for mediating adaptive social interactions and discuss selected evidence from behavioral neurophysiology experiments supporting the role of the medial frontal cortex in implementing these algorithms. By doing so, we primarily focus on research in nonhuman primates and examine several key attributes of the medial frontal cortex. Specifically, we review neuronal substrates in the medial frontal cortex uniquely suitable for enabling social monitoring, observational and vicarious learning, as well as predicting the behaviors of social partners. Moreover, by utilizing the three levels of organization in information processing systems proposed by Marr (1982) and recently adapted by Lockwood, Apps, and Chang (2020) for social information processing, we survey selected social functions of the medial frontal cortex through the lens of socially relevant algorithms and implementations. Overall, this chapter provides a broad overview of the behavioral neurophysiology literature endorsing the importance of socially relevant neural algorithms implemented by the primate medial frontal cortex for regulating social interactions.

Functional Alterations and Cerebral Variations in Humans Exposed to Early Life Stress

Early life stress can be caused by acute or chronic exposure to childhood events, such as emotional, physical, sexual abuse, and neglect. Early stress is associated with subsequent alterations in physical and mental health, which can extend into adolescence, adulthood, and even old age. The effects of early stress exposure include alterations in cognitive, neuropsychological, and behavioral functions, and can even lead to the development of psychiatric disorders and changes in brain anatomy. The present manuscript provides a review of the main findings on these effects reported in the scientific literature in recent decades. Early life stress is associated with the presence of psychiatric disorders, mainly mood disorders such as depression and risk of suicide, as well as with the presence of post-traumatic stress disorder. At the neuropsychological level, the involvement of different mental processes such as executive functions, abstract reasoning, certain memory modalities, and poor school-skill performance has been reported. In addition, we identified reports of alterations of different subdomains of each of these processes. Regarding neuroanatomical effects, the involvement of cortical regions, subcortical nuclei, and the subcortical white matter has been documented. Among the telencephalic regions most affected and studied are the prefrontal cortex, the hippocampus, the amygdala, and the anterior cingulate cortex. Understanding the impact of early life stress on postnatal brain development is very important for the orientation of therapeutic intervention programs and could help in the formulation and implementation of preventive measures as well as in the reorientation of research targets.

Spaceflight and brain plasticity: Spaceflight effects on regional expression of neurotransmitter systems and neurotrophic factors encoding genes

The critical problem of space exploration is the effect of long-term space travel on brain functioning. Current information concerning the effects of actual spaceflight on the brain was obtained on rats and mice flown on five missions of Soviet/Russian biosatellites, NASA Neurolab Mission STS90, and International Space Station (ISS). The review provides converging lines of evidence that: 1) long-term spaceflight affects both principle regulators of brain neuroplasticity - neurotransmitters (5-HT and DA) and neurotrophic factors (CDNF, GDNF but not BDNF); 2) 5-HT- (5-HT_2A receptor and MAO A) and especially DA-related genes (TH, MAO A, COMT, D1 receptor, CDNF and GDNF) belong to the risk neurogenes; 3) brain response to spaceflight is region-specific. Substantia nigra, striatum and hypothalamus are highly sensitive to the long-term spaceflight: in these brain areas spaceflight decreased the expression of both DA-related and neurotrophic factors genes. Since DA system is involved in the regulation of movement and cognition the data discussed in the review could explain dysfunction of locomotion and behavior of astronauts and direct further investigations to the DA system.

Multi-voxel pattern analysis of amygdala functional connectivity at rest predicts variability in posttraumatic stress severity

INTRODUCTION

Resting state functional magnetic resonance imaging (rsfMRI) studies demonstrate that individuals with posttraumatic stress disorder (PTSD) exhibit atypical functional connectivity (FC) between the amygdala, involved in the generation of emotion, and regions responsible for emotional appraisal (e.g., insula, orbitofrontal cortex [OFC]) and regulation (prefrontal cortex [PFC], anterior cingulate cortex). Consequently, atypical amygdala FC within an emotional processing and regulation network may be a defining feature of PTSD, although altered FC does not seem constrained to one brain region. Instead, altered amygdala FC involves a large, distributed brain network in those with PTSD. The present study used a machine-learning data-driven approach, multi-voxel pattern analysis (MVPA), to predict PTSD severity based on whole-brain patterns of amygdala FC.

METHODS

Trauma-exposed adults (N = 90) completed the PTSD Checklist-Civilian Version to assess symptoms and a 5-min rsfMRI. Whole-brain FC values to bilateral amygdala were extracted and used in a relevance vector regression analysis with a leave-one-out approach for cross-validation with permutation testing (1,000) to obtain significance values.

RESULTS

Results demonstrated that amygdala FC predicted PCL-C scores with statistically significant accuracy (r = .46, p = .001; mean sum of squares = 130.46, p = .001; R²  = 0.21, p = .001). Prediction was based on whole-brain amygdala FC, although regions that informed prediction (top 10%) included the OFC, amygdala, and dorsolateral PFC.

CONCLUSION

Findings demonstrate the utility of MVPA based on amygdala FC to predict individual severity of PTSD symptoms and that amygdala FC within a fear acquisition and regulation network contributed to accurate prediction.

A high-resolution fMRI approach to characterize functionally distinct neural pathways within dopaminergic midbrain and nucleus accumbens during reward and salience processing

Processing of reward and salience without reward association are known to critically rely on the dopamine system. A growing body of evidence from animal studies suggests that both functions may be subserved by distinct subregions in midbrain and ventral striatum, specifically nucleus accumbens (NAcc). Yet in vivo investigation of these brain structures in humans has been rare. Here we examined blood oxygen level dependent signals in response to frequently presented rewarding events and infrequently presented neutral events in 20 healthy subjects using high-resolution functional magnetic resonance imaging (fMRI) for imaging the human midbrain and NAcc. The present findings revealed distinct activation patterns in brain regions of interest, namely increased activation in substantia nigra pars compacta (SNc) and dorsolateral NAcc in response to neutral events, while the VTA and both the ventromedial and dorsolateral NAcc were significantly activated due to rewarding events. Moreover, psychophysiological interaction analyses demonstrated regionally specialized processing pathways, such as a dorsolateral pathway when processing salience per se, i.e. increased functional interactions between SNc, dorsolateral NAcc and dorsolateral and medial prefrontal cortex (PFC); and a ventromedial pathway during reward processing, i.e. increased functional coupling between VTA and ventromedial NAcc. Thus, these findings may not only accelerate the integration of animal models of brain function with human neuroscience but may also improve diagnosis and treatment in patients with neuropsychiatric disorders such as schizophrenia and depression in which dopaminergic dysfunction and aberrant attribution of salience have been implicated.

Short-Term Influence of Recent Trial History on Perceptual Choice Changes with Stimulus Strength

Perceptual decisions, especially for difficult stimuli, can be influenced by choices and outcomes in previous trials. However, it is not well understood how stimulus strength modulates the temporal characteristics as well as the magnitude of trial history influence. We addressed this question using a contrast detection task in freely moving mice. We found that, at lower as compared to higher stimulus contrast, the current choice of the mice was more influenced by choices and outcomes in the past trials and the influence emerged from a longer history. To examine the neural basis of stimulus strength-dependent history influence, we recorded from the secondary motor cortex (M2), a prefrontal region that plays an important role in cue-guided actions and memory-guided behaviors. We found that more M2 neurons conveyed information about choices on the past two trials at lower than at higher contrast. Furthermore, history-trial activity in M2 was important for decoding upcoming choice at low contrast. Thus, trial history influence of perceptual choice is adaptive to the strength of sensory evidence, which may be important for action selection in a dynamic environment.

A Motivational Model of BCI-Controlled Heuristic Search

Several researchers have proposed a new application for human augmentation, which is to provide human supervision to autonomous artificial intelligence (AI) systems. In this paper, we introduce a framework to implement this proposal, which consists of using Brain⁻Computer Interfaces (BCI) to influence AI computation via some of their core algorithmic components, such as heuristic search. Our framework is based on a joint analysis of philosophical proposals characterising the behaviour of autonomous AI systems and recent research in cognitive neuroscience that support the design of appropriate BCI. Our framework is defined as a motivational approach, which, on the AI side, influences the shape of the solution produced by heuristic search using a BCI motivational signal reflecting the user's disposition towards the anticipated result. The actual mapping is based on a measure of prefrontal asymmetry, which is translated into a non-admissible variant of the heuristic function. Finally, we discuss results from a proof-of-concept experiment using functional near-infrared spectroscopy (fNIRS) to capture prefrontal asymmetry and control the progression of AI computation of traditional heuristic search problems.

Altered gray matter volume and disrupted functional connectivity of dorsolateral prefrontal cortex in men with heroin dependence

AIM

Chronic heroin use can cause various neuropathological characteristics that may compromise brain function. The present study evaluated the alteration of gray matter volume (GMV) and its resting-state functional connectivity (rsFC) over the dorsolateral prefrontal cortex (DLPFC) among male heroin users.

METHODS

Thirty heroin-dependent men undergoing methadone maintenance therapy and 30 educational-level- and age-matched male controls were recruited for this study. To assess their GMV and rsFC, the participants were evaluated using spoiled gradient echo and gradient-recalled echo planar imaging sequences with a 3-Tesla General Electric MR scanner under resting state.

RESULTS

The heroin-dependent men showed lower GMV over the right DLPFC in comparison with the controls. Further evaluation of the rsFC of the right DLPFC revealed a marked decrease in interhemispheric DLPFC connectivity among those with heroin dependence under control of head movement and GMV of the right DLPFC.

CONCLUSION

Although the mechanism remains unclear, the present study shows that chronic heroin use is associated with alteration of morphology as well as rsFC over the right DLPFC. As the DLPFC plays an imperative role in various domains of cognitive function, service providers for heroin users should consider the impacts of possible DLPFC-related cognitive deficits on treatment effectiveness.

Effects of Alcohol Dependence Severity on Neural Correlates of Delay Discounting

Aims

The current study examines the relationship between alcohol dependence severity and delay discounting neural activation.

Methods

Participants (N = 17; 6 female) completed measures of alcohol use and severity and a functional magnetic resonance imaging version of a delay discounting task.

Results

Alcohol dependence severity was negatively associated with activation in superior frontal gyrus during impulsive relative to delayed decisions, and positively associated with activation in paracingulate gyrus and frontal pole in delayed relative to impulsive decisions.

Conclusions

These results indicate that alcohol dependence severity tracks closely with dysregulations in cognitive control and reward evaluation areas during impulsive and delayed decisions, respectively. Delay discounting may be a useful construct in capturing these cognitive dysregulations as alcohol use disorders become more severe.

Short summary

Among alcohol-dependent individuals, alcohol dependence severity is associated with overactivation of ventromedial prefrontal areas during delayed and underactivation of dorsolateral prefrontal regions during impulsive reward decisions.

Functional Neuroanatomy of Emotion and Its Regulation in PTSD

Posttraumatic stress disorder (PTSD) is a devastating disorder, linked to profound mental, physical, occupational, and functional impairment. In addition, it is a highly complex disorder, characterized by symptom heterogeneity across multiple domains. Nevertheless, emotion dysregulation arising from the exaggerated response to threat or from the inability to regulate negative emotional states plays a defining role in the pathophysiology of PTSD. In order to improve our understanding of how emotion dysregulation manifests in this illness, functional neuroimaging research over the past 20 years provides great insight into underlying neuroanatomy of each component of emotion dysregulation in the context of PTSD. While prior reviews exist on the topic of neuroimaging findings in PTSD, the present review synthesizes that work through the lens of emotion and its regulation. Studies that employed tasks of emotional responding and symptom provocation, implicit regulation (e.g., emotional Stroop and interference), explicit regulation (e.g., cognitive reappraisal), and fear conditioning/extinction were reviewed. Findings demonstrate that emotion dysregulation in PTSD arises from complications within a large neurocircuitry involving the amygdala, insula, hippocampus, anterior cingulate cortex, and prefrontal cortex. Although an exaggerated response in the amygdala and insula to negative emotional triggers is pervasive, PTSD is also marked by deficient appraisal, resolution, and management of negative emotional states subserved by the anterior cingulate cortex and prefrontal cortex during regulation. These findings further support the importance of studying emotion-regulation deficits in tandem with exaggerated symptom provocation in order to better understand the constellation of symptoms present in those with PTSD.

Interactive effects of music and prefrontal cortex stimulation in modulating response inhibition

Influential hypotheses propose that alterations in emotional state influence decision processes and executive control of behavior. Both music and transcranial direct current stimulation (tDCS) of prefrontal cortex affect emotional state, however interactive effects of music and tDCS on executive functions remain unknown. Learning to inhibit inappropriate responses is an important aspect of executive control which is guided by assessing the decision outcomes such as errors. We found that high-tempo music, but not low-tempo music or low-level noise, significantly influenced learning and implementation of inhibitory control. In addition, a brief period of tDCS over prefrontal cortex specifically interacted with high-tempo music and altered its effects on executive functions. Measuring event-related autonomic and arousal response of participants indicated that exposure to task demands and practice led to a decline in arousal response to the decision outcome and high-tempo music enhanced such practice-related processes. However, tDCS specifically moderated the high-tempo music effect on the arousal response to errors and concomitantly restored learning and improvement in executive functions. Here, we show that tDCS and music interactively influence the learning and implementation of inhibitory control. Our findings indicate that alterations in the arousal-emotional response to the decision outcome might underlie these interactive effects.

Internet-Based Self-Help for Ménière's Disease: Details and Outcome of a Single-Group Open Trial

PURPOSE

In this article, we present the details and the pilot outcome of an Internet-based self-help program for Ménière's disease (MD).

METHOD

The Norton-Kaplan model is applied to construct a strategic, person-focused approach in the enablement process. The program assesses the disorder profile and diagnosis. In the therapeutic component of the program, the participant defines vision and time frame, inspects confounding factors, determines goals, establishes a strategy, and starts to work on the important problems caused by the disorder. The program works interactively, utilizes collaboration with significant others, and enhances positive thinking. Participants took part in an Internet-based self-help program. Data were collected interactively using open-ended and structured questionnaires on various disease-specific and general health aspects. The pilot outcome of 41 patients with MD was evaluated.

RESULTS

The analysis of the pilot data showed statistically significant improvement in their general health-related quality of life (p < .001). Also, the outcome of the Posttraumatic Growth Inventory (Cann et al., 2010) showed small to moderate change as a result of the intervention.

CONCLUSIONS

The Internet-based self-help program can be helpful in the rehabilitation of patients with MD to supplement medical therapy.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

Disease Profiling for Computerized Peer Support of Ménière's Disease

Background

Peer support is an emerging form of person-driven active health care. Chronic conditions such as Ménière’s disease (a disorder of the inner ear) need continuing rehabilitation and support that is beyond the scope of routine clinical medical practice. Hence, peer-support programs can be helpful in supplementing some of the rehabilitation aspects.

Objective

The aim of this study was to design a computerized data collection system for the peer support of Menière’s disease that is capable in profiling the subject for diagnosis and in assisting with problem solving.

Methods

The expert program comprises several data entries focusing on symptoms, activity limitations, participation restrictions, quality of life, attitude and personality trait, and an evaluation of disease-specific impact. Data was collected from 740 members of the Finnish Ménière’s Federation and utilized in the construction and evaluation of the program.

Results

The program verifies the diagnosis of a person by using an expert system, and the inference engine selects 50 cases with matched symptom severity by using a nearest neighbor algorithm. These cases are then used as a reference group to compare with the person’s attitude, sense of coherence, and anxiety. The program provides feedback for the person and uses this information to guide the person through the problem-solving process.

Conclusions

This computer-based peer-support program is the first example of an advanced computer-oriented approach using artificial intelligence, both in the profiling of the disease and in profiling the person’s complaints for hearing loss, tinnitus, and vertigo.

Targeting the neurophysiology of cognitive systems with transcranial alternating current stimulation

Cognitive impairment represents one of the most debilitating and most difficult symptom to treat of many psychiatric illnesses. Human neurophysiology studies have suggested that specific pathologies of cortical network activity correlate with cognitive impairment. However, we lack demonstration of causal relationships between specific network activity patterns and cognitive capabilities and treatment modalities that directly target impaired network dynamics of cognition. Transcranial alternating current stimulation (tACS), a novel non-invasive brain stimulation approach, may provide a crucial tool to tackle these challenges. Here, we propose that tACS can be used to elucidate the causal role of cortical synchronization in cognition and, eventually, to enhance pathologically weakened synchrony that may underlie cognitive deficits. To accelerate such development of tACS as a treatment for cognitive deficits, we discuss studies on tACS and cognition performed in healthy participants, according to the Research Domain Criteria of the National Institute of Mental Health.

Pain anticipation: an activation likelihood estimation meta-analysis of brain imaging studies

The anticipation of pain has been investigated in a variety of brain imaging studies. Importantly, today there is no clear overall picture of the areas that are involved in different studies and the exact role of these regions in pain expectation remains especially unexploited. To address this issue, we used activation likelihood estimation meta-analysis to analyze pain anticipation in several neuroimaging studies. A total of 19 functional magnetic resonance imaging were included in the analysis to search for the cortical areas involved in pain anticipation in human experimental models. During anticipation, activated foci were found in the dorsolateral prefrontal, midcingulate and anterior insula cortices, medial and inferior frontal gyri, inferior parietal lobule, middle and superior temporal gyrus, thalamus, and caudate. Deactivated foci were found in the anterior cingulate, superior frontal gyrus, parahippocampal gyrus and in the claustrum. The results of the meta-analytic connectivity analysis provide an overall view of the brain responses triggered by the anticipation of a noxious stimulus. Such a highly distributed perceptual set of self-regulation may prime brain regions to process information where emotion, action and perception as well as their related subcategories play a central role. Not only do these findings provide important information on the neural events when anticipating pain, but also they may give a perspective into nocebo responses, whereby negative expectations may lead to pain worsening.

Modulation of neural activity by reward in medial intraparietal cortex is sensitive to temporal sequence of reward

To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance.

Neural substrates of approach-avoidance conflict decision-making

Animal approach-avoidance conflict paradigms have been used extensively to operationalize anxiety, quantify the effects of anxiolytic agents, and probe the neural basis of fear and anxiety. Results from human neuroimaging studies support that a frontal-striatal-amygdala neural circuitry is important for approach-avoidance learning. However, the neural basis of decision-making is much less clear in this context. Thus, we combined a recently developed human approach-avoidance paradigm with functional magnetic resonance imaging (fMRI) to identify neural substrates underlying approach-avoidance conflict decision-making. Fifteen healthy adults completed the approach-avoidance conflict (AAC) paradigm during fMRI. Analyses of variance were used to compare conflict to nonconflict (avoid-threat and approach-reward) conditions and to compare level of reward points offered during the decision phase. Trial-by-trial amplitude modulation analyses were used to delineate brain areas underlying decision-making in the context of approach/avoidance behavior. Conflict trials as compared to the nonconflict trials elicited greater activation within bilateral anterior cingulate cortex, anterior insula, and caudate, as well as right dorsolateral prefrontal cortex (PFC). Right caudate and lateral PFC activation was modulated by level of reward offered. Individuals who showed greater caudate activation exhibited less approach behavior. On a trial-by-trial basis, greater right lateral PFC activation related to less approach behavior. Taken together, results suggest that the degree of activation within prefrontal-striatal-insula circuitry determines the degree of approach versus avoidance decision-making. Moreover, the degree of caudate and lateral PFC activation related to individual differences in approach-avoidance decision-making. Therefore, the approach-avoidance conflict paradigm is ideally suited to probe anxiety-related processing differences during approach-avoidance decision-making.

Brain networks for exploration decisions utilizing distinct modeled information types during contextual learning

Exploration permits acquisition of the most relevant information during learning. However, the specific information needed, the influences of this information on decision making, and the relevant neural mechanisms remain poorly understood. We modeled distinct information types available during contextual association learning and used model-based fMRI in conjunction with manipulation of exploratory decision making to identify neural activity associated with information-based decisions. We identified hippocampal-prefrontal contributions to advantageous decisions based on immediately available novel information, distinct from striatal contributions to advantageous decisions based on the sum total available (accumulated) information. Furthermore, network-level interactions among these regions during exploratory decision making were related to learning success. These findings link strategic exploration decisions during learning to quantifiable information and advance understanding of adaptive behavior by identifying the distinct and interactive nature of brain-network contributions to decisions based on distinct information types.

The neuroeconomics of alcohol demand: an initial investigation of the neural correlates of alcohol cost-benefit decision making in heavy drinking men

Neuroeconomics integrates concepts and methods from psychology, economics, and cognitive neuroscience to understand how the brain makes decisions. In economics, demand refers to the relationship between a commodity's consumption and its cost, and, in behavioral studies, high alcohol demand has been consistently associated with greater alcohol misuse. Relatively little is known about how the brain processes demand decision making, and the current study is an initial investigation of the neural correlates of alcohol demand among heavy drinkers. Using an event-related functional magnetic resonance imaging (fMRI) paradigm, participants (N=24) selected how much they would drink under varying levels of price. These choices determined access to alcohol during a subsequent bar laboratory self-administration period. During decisions to drink in general, greater activity was present in multiple distinct subunits of the prefrontal and parietal cortices. In contrast, during decisions to drink that were demonstrably affected by the cost of alcohol, significantly greater activation was evident in frontostriatal regions, suggesting an active interplay between cognitive deliberation and subjective reward value. These choices were also characterized by significant deactivation in default mode network regions, suggesting suppression resulting from greater cognitive load. Across choice types, the anterior insula was notably recruited in diverse roles, further implicating the importance of interoceptive processing in decision-making behavior. These findings reveal the neural signatures subserving alcohol cost-benefit decision making, providing a foundation for future clinical applications of this paradigm and extending this approach to understanding the neural correlates of demand for other addictive commodities.

Transcranial direct current stimulation of right dorsolateral prefrontal cortex does not affect model-based or model-free reinforcement learning in humans

There is broad consensus that the prefrontal cortex supports goal-directed, model-based decision-making. Consistent with this, we have recently shown that model-based control can be impaired through transcranial magnetic stimulation of right dorsolateral prefrontal cortex in humans. We hypothesized that an enhancement of model-based control might be achieved by anodal transcranial direct current stimulation of the same region. We tested 22 healthy adult human participants in a within-subject, double-blind design in which participants were given Active or Sham stimulation over two sessions. We show Active stimulation had no effect on model-based control or on model-free ('habitual') control compared to Sham stimulation. These null effects are substantiated by a power analysis, which suggests that our study had at least 60% power to detect a true effect, and by a Bayesian model comparison, which favors a model of the data that assumes stimulation had no effect over models that assume stimulation had an effect on behavioral control. Although we cannot entirely exclude more trivial explanations for our null effect, for example related to (faults in) our experimental setup, these data suggest that anodal transcranial direct current stimulation over right dorsolateral prefrontal cortex does not improve model-based control, despite existing evidence that transcranial magnetic stimulation can disrupt such control in the same brain region.

Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans

Human choice behavior often reflects a competition between inflexible computationally efficient control on the one hand and a slower more flexible system of control on the other. This distinction is well captured by model-free and model-based reinforcement learning algorithms. Here, studying human subjects, we show it is possible to shift the balance of control between these systems by disruption of right dorsolateral prefrontal cortex, such that participants manifest a dominance of the less optimal model-free control. In contrast, disruption of left dorsolateral prefrontal cortex impaired model-based performance only in those participants with low working memory capacity.

•

Disrupting right dorsolateral prefrontal cortex impairs flexible model-based choice

•

This drives behavior toward simpler, model-free control

•

Results provide causal evidence for neural underpinnings of flexible choice

Cortical signals for rewarded actions and strategic exploration

In stable environments, decision makers can exploit their previously learned strategies for optimal outcomes, while exploration might lead to better options in unstable environments. Here, to investigate the cortical contributions to exploratory behavior, we analyzed single-neuron activity recorded from four different cortical areas of monkeys performing a matching-pennies task and a visual search task, which encouraged and discouraged exploration, respectively. We found that neurons in multiple regions in the frontal and parietal cortex tended to encode signals related to previously rewarded actions more reliably than unrewarded actions. In addition, signals for rewarded choices in the supplementary eye field were attenuated during the visual search task and were correlated with the tendency to switch choices during the matching-pennies task. These results suggest that the supplementary eye field might play a unique role in encouraging animals to explore alternative decision-making strategies.

Agency modulates the lateral and medial prefrontal cortex responses in belief-based decision making

Many real-life decisions in complex and changing environments are guided by the decision maker's beliefs, such as her perceived control over decision outcomes (i.e., agency), leading to phenomena like the "illusion of control". However, the neural mechanisms underlying the "agency" effect on belief-based decisions are not well understood. Using functional imaging and a card guessing game, we revealed that the agency manipulation (i.e., either asking the subjects (SG) or the computer (CG) to guess the location of the winning card) not only affected the size of subjects' bets, but also their "world model" regarding the outcome dependency. Functional imaging results revealed that the decision-related activation in the lateral and medial prefrontal cortex (PFC) was significantly modulated by agency and previous outcome. Specifically, these PFC regions showed stronger activation when subjects made decisions after losses than after wins under the CG condition, but the pattern was reversed under the SG condition. Furthermore, subjects with high external attribution of negative events were more affected by agency at the behavioral and neural levels. These results suggest that the prefrontal decision-making system can be modulated by abstract beliefs, and are thus vulnerable to factors such as false agency and attribution.

Signal detection in animal psychoacoustics: analysis and simulation of sensory and decision-related influences

Signal detection theory (SDT) provides a framework for interpreting psychophysical experiments, separating the putative internal sensory representation and the decision process. SDT was used to analyse ferret behavioural responses in a (yes–no) tone-in-noise detection task. Instead of measuring the receiver-operating characteristic (ROC), we tested SDT by comparing responses collected using two common psychophysical data collection methods. These (Constant Stimuli, Limits) differ in the set of signal levels presented within and across behavioural sessions. The results support the use of SDT as a method of analysis: SDT sensory component was unchanged between the two methods, even though decisions depended on the stimuli presented within a behavioural session. Decision criterion varied trial-by-trial: a ‘yes’ response was more likely after a correct rejection trial than a hit trial. Simulation using an SDT model with several decision components reproduced the experimental observations accurately, leaving only ∼10% of the variance unaccounted for. The model also showed that trial-by-trial dependencies were unlikely to influence measured psychometric functions or thresholds. An additional model component suggested that inattention did not contribute substantially. Further analysis showed that ferrets were changing their decision criteria, almost optimally, to maximise the reward obtained in a session. The data suggest trial-by-trial reward-driven optimization of the decision process. Understanding the factors determining behavioural responses is important for correlating neural activity and behaviour. SDT provides a good account of animal psychoacoustics, and can be validated using standard psychophysical methods and computer simulations, without recourse to ROC measurements.

Dynamic afferent synapses to decision-making networks improve performance in tasks requiring stimulus associations and discriminations

Animals must often make opposing responses to similar complex stimuli. Multiple sensory inputs from such stimuli combine to produce stimulus-specific patterns of neural activity. It is the differences between these activity patterns, even when small, that provide the basis for any differences in behavioral response. In the present study, we investigate three tasks with differing degrees of overlap in the inputs, each with just two response possibilities. We simulate behavioral output via winner-takes-all activity in one of two pools of neurons forming a biologically based decision-making layer. The decision-making layer receives inputs either in a direct stimulus-dependent manner or via an intervening recurrent network of neurons that form the associative layer, whose activity helps distinguish the stimuli of each task. We show that synaptic facilitation of synapses to the decision-making layer improves performance in these tasks, robustly increasing accuracy and speed of responses across multiple configurations of network inputs. Conversely, we find that synaptic depression worsens performance. In a linearly nonseparable task with exclusive-or logic, the benefit of synaptic facilitation lies in its superlinear transmission: effective synaptic strength increases with presynaptic firing rate, which enhances the already present superlinearity of presynaptic firing rate as a function of stimulus-dependent input. In linearly separable single-stimulus discrimination tasks, we find that facilitating synapses are always beneficial because synaptic facilitation always enhances any differences between inputs. Thus we predict that for optimal decision-making accuracy and speed, synapses from sensory or associative areas to decision-making or premotor areas should be facilitating.

Lateral prefrontal cortex contributes to maladaptive decisions

Humans consistently make suboptimal decisions involving random events, yet the underlying neural mechanisms remain elusive. Using functional MRI and a matching pennies game that captured subjects' increasing tendency to predict the break of a streak as it continued [i.e., the "gambler's fallacy" (GF)], we found that a strong blood oxygen level-dependent response in the left lateral prefrontal cortex (LPFC) to the current outcome preceded the use of the GF strategy 10 s later. Furthermore, anodal transcranial direct current stimulation over the left LPFC, which enhances neuronal firing rates and cerebral excitability, increased the use of the GF strategy, and made the decisions more "sticky." These results reveal a causal role of the LPFC in implementing suboptimal decision strategy guided by false world models, especially when such strategy requires great resources for cognitive control.

Transcranial alternating current stimulation increases risk-taking behavior in the balloon analog risk task

The process of evaluating risks and benefits involves a complex neural network that includes the dorsolateral prefrontal cortex (DLPFC). It has been proposed that in conflict and reward situations, theta-band (4–8 Hz) oscillatory activity in the frontal cortex may reflect an electrophysiological mechanism for coordinating neural networks monitoring behavior, as well as facilitating task-specific adaptive changes. The goal of the present study was to investigate the hypothesis that theta-band oscillatory balance between right and left frontal and prefrontal regions, with a predominance role to the right hemisphere (RH), is crucial for regulatory control during decision-making under risk. In order to explore this hypothesis, we used transcranial alternating current stimulation, a novel technique that provides the opportunity to explore the functional role of neuronal oscillatory activities and to establish a causal link between specific oscillations and functional lateralization in risky decision-making situations. For this aim, healthy participants were randomly allocated to one of three stimulation groups (LH stimulation/RH stimulation/Sham stimulation), with active AC stimulation delivered in a frequency-dependent manner (at 6.5 Hz; 1 mA peak-to-peak). During the AC stimulation, participants performed the Balloon Analog Risk Task. This experiment revealed that participants receiving LH stimulation displayed riskier decision-making style compared to sham and RH stimulation groups. However, there was no difference in decision-making behaviors between sham and RH stimulation groups. The current study extends the notion that DLPFC activity is critical for adaptive decision-making in the context of risk-taking and emphasis the role of theta-band oscillatory activity during risky decision-making situations.

Decoding the formation of reward predictions across learning

The predicted reward of different behavioral options plays an important role in guiding decisions. Previous research has identified reward predictions in prefrontal and striatal brain regions. Moreover, it has been shown that the neural representation of a predicted reward is similar to the neural representation of the actual reward outcome. However, it has remained unknown how these representations emerge over the course of learning and how they relate to decision making. Here, we sought to investigate learning of predicted reward representations using functional magnetic resonance imaging and multivariate pattern classification. Using a pavlovian conditioning procedure, human subjects learned multiple novel cue-outcome associations in each scanning run. We demonstrate that across learning activity patterns in the orbitofrontal cortex, the dorsolateral prefrontal cortex (DLPFC), and the dorsal striatum, coding the value of predicted rewards become similar to the patterns coding the value of actual reward outcomes. Furthermore, we provide evidence that predicted reward representations in the striatum precede those in prefrontal regions and that representations in the DLPFC are linked to subsequent value-based choices. Our results show that different brain regions represent outcome predictions by eliciting the neural representation of the actual outcome. Furthermore, they suggest that reward predictions in the DLPFC are directly related to value-based choices.

Frontal Oscillatory Dynamics Predict Feedback Learning and Action Adjustment

Frontal oscillatory dynamics in the theta (4–8 Hz) and beta (20–30 Hz) frequency bands have been implicated in cognitive control processes. Here we investigated the changes in coordinated activity within and between frontal brain areas during feedback-based response learning. In a time estimation task, participants learned to press a button after specific, randomly selected time intervals (300–2000 msec) using the feedback after each button press (correct, too fast, too slow). Consistent with previous findings, theta-band activity over medial frontal scalp sites (presumably reflecting medial frontal cortex activity) was stronger after negative feedback, whereas beta-band activity was stronger after positive feedback. Theta-band power predicted learning only after negative feedback, and beta-band power predicted learning after positive and negative feedback. Furthermore, negative feedback increased theta-band intersite phase synchrony (a millisecond resolution measure of functional connectivity) among right lateral prefrontal, medial frontal, and sensorimotor sites. These results demonstrate the importance of frontal theta- and beta-band oscillations and intersite communication in the realization of reinforcement learning.

A neural correlate of predicted and actual reward-value information in monkey pedunculopontine tegmental and dorsal raphe nucleus during saccade tasks

Dopamine, acetylcholine, and serotonin, the main modulators of the central nervous system, have been proposed to play important roles in the execution of movement, control of several forms of attentional behavior, and reinforcement learning. While the response pattern of midbrain dopaminergic neurons and its specific role in reinforcement learning have been revealed, the role of the other neuromodulators remains rather elusive. Here, we review our recent studies using extracellular recording from neurons in the pedunculopontine tegmental nucleus, where many cholinergic neurons exist, and the dorsal raphe nucleus, where many serotonergic neurons exist, while monkeys performed eye movement tasks to obtain different reward values. The firing patterns of these neurons are often tonic throughout the task period, while dopaminergic neurons exhibited a phasic activity pattern to the task event. The different modulation patterns, together with the activity of dopaminergic neurons, reveal dynamic information processing between these different neuromodulator systems.

Frontal pole cortex: encoding ends at the end of the endbrain

Considerable neuroimaging research in humans indicates that the frontal pole cortex (FPC), also known as Brodmann area 10, contributes to many aspects of cognition. Despite these findings, however, its fundamental function and mechanism remain unclear. Recent neurophysiological results from the FPC of monkeys have implications about both. Neurons in the FPC seem to encode chosen goals at feedback time and nothing else. Goals, the places and objects that serve as targets for action, come in many forms and arise from many cognitive processes. The FPC's signal, although surprisingly simple for neurons at the apex of a prefrontal hierarchy, could promote learning about which kinds of goals and goal-generating processes produce particular costs and benefits, thereby improving future choices.

Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior

In attentional models of learning, associations between actions and subsequent rewards are stronger when outcomes are surprising, regardless of their valence. Despite the behavioral evidence that surprising outcomes drive learning, neural correlates of unsigned reward prediction errors remain elusive. Here we show that in a probabilistic choice task, trial-to-trial variations in preference track outcome surprisingness. Concordant with this behavioral pattern, responses of neurons in macaque (Macaca mulatta) dorsal anterior cingulate cortex (dACC) to both large and small rewards were enhanced when the outcome was surprising. Moreover, when, on some trials, probabilities were hidden, neuronal responses to rewards were reduced, consistent with the idea that the absence of clear expectations diminishes surprise. These patterns are inconsistent with the idea that dACC neurons track signed errors in reward prediction, as dopamine neurons do. Our results also indicate that dACC neurons do not signal conflict. In the context of other studies of dACC function, these results suggest a link between reward-related modulations in dACC activity and attention and motor control processes involved in behavioral adjustment. More speculatively, these data point to a harmonious integration between reward and learning accounts of ACC function on one hand, and attention and cognitive control accounts on the other.

Catecholamine influences on dorsolateral prefrontal cortical networks

The symptoms of attention-deficit/hyperactivity disorder (ADHD) involve impairments in prefrontal cortical top-down regulation of attention and behavior. All current pharmacological treatments for ADHD facilitate catecholamine transmission, and basic research suggests that these compounds have prominent actions in the prefrontal cortex (PFC). The dorsolateral PFC is especially sensitive to levels of norepinephrine and dopamine, whereby either too little or too much markedly impairs PFC function. Recent physiological studies have shown that norepinephrine strengthens PFC network connectivity and maintains persistent firing during a working memory task through stimulation of postsynaptic α(2A)-adrenoceptors on PFC neurons. Conversely, dopamine acts at D1 receptors to narrow spatial tuning, sculpting network inputs to decrease noise (i.e., stabilization of the representation). The stimulant medications and atomoxetine appear to enhance PFC function by indirectly increasing these catecholamine actions through blockade of norepinephrine and/or dopamine transporters. In contrast, guanfacine mimics the enhancing effects of norepinephrine at postsynaptic α(2A)-receptors in the PFC, strengthening network connectivity. Stronger PFC regulation of attention, behavior, and emotion likely contributes to the therapeutic effects of these medications for the treatment of ADHD.

Posterior cingulate cortex: adapting behavior to a changing world

When has the world changed enough to warrant a new approach? The answer depends on current needs, behavioral flexibility and prior knowledge about the environment. Formal approaches solve the problem by integrating the recent history of rewards, errors, uncertainty and context via Bayesian inference to detect changes in the world and alter behavioral policy. Neuronal activity in posterior cingulate cortex - a key node in the default network - is known to vary with learning, memory, reward and task engagement. We propose that these modulations reflect the underlying process of change detection and motivate subsequent shifts in behavior.

Dopamine in motivational control: rewarding, aversive, and alerting

Midbrain dopamine neurons are well known for their strong responses to rewards and their critical role in positive motivation. It has become increasingly clear, however, that dopamine neurons also transmit signals related to salient but nonrewarding experiences such as aversive and alerting events. Here we review recent advances in understanding the reward and nonreward functions of dopamine. Based on this data, we propose that dopamine neurons come in multiple types that are connected with distinct brain networks and have distinct roles in motivational control. Some dopamine neurons encode motivational value, supporting brain networks for seeking, evaluation, and value learning. Others encode motivational salience, supporting brain networks for orienting, cognition, and general motivation. Both types of dopamine neurons are augmented by an alerting signal involved in rapid detection of potentially important sensory cues. We hypothesize that these dopaminergic pathways for value, salience, and alerting cooperate to support adaptive behavior.