A Network for Computing Value Equilibrium in the Human Medial Prefrontal Cortex

Resting-state functional connectivity of goal-directed and habitual-learning systems: The efficacy of cognitive-behavioral therapy for obsessive-compulsive disorder

BACKGROUND

There is an imbalance between goal-directed and habitual-learning system in patients with obsessive-compulsive disorder (OCD). At present, the relationship between cognitive behavior therapy (CBT) as a first-line therapy and goal-directed and habitual-learning disorder is still unclear. We attempted to discuss the effect of CBT treatment in patients with OCD, using abnormalities in goal-directed and habitual-learning-related brain regions at baseline as predictive factors.

METHODS

A total of 71 subjects, including 35 OCD patients and 36 healthy controls, were recruited. The OCD patients underwent 8 weeks of CBT. These patients were divided into two groups based on treatment response (Nresponders = 18, Nnonresponders = 17). Further subgroup analysis was conducted based on disease duration (Nshort = 17, Nlong = 18) and age of onset (Nearly = 14, Nlate = 21). We collected resting-state ROI-ROI functional connectivity data and apply repeated-measures linear mixed-effects models to investigate the differences of different subgroups.

RESULTS

CBT led to symptom improvement in OCD patients, with varying degrees of effectiveness across subgroups. The orbitofrontal cortex (OFC) and insula, key regions for goal-directed behavior and habitual-learning, respectively, showed significant impacts on CBT efficacy in subgroups with different disease durations and ages of onset.

CONCLUSION

The findings suggest that the goal-directed system may influence the efficacy of CBT through goal selection, maintenance, and emotion regulation. Furthermore, we found that disease duration and age of onset may affect treatment outcomes by modulating functional connectivity between goal-directed and habitual-learning brain regions.

Goal commitment is supported by vmPFC through selective attention

When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.

The computational structure of consummatory anhedonia

Anhedonia is a reduction in enjoyment, motivation, or interest. It is common across mental health disorders and a harbinger of poor treatment outcomes. The enjoyment aspect, termed 'consummatory anhedonia', in particular poses fundamental questions about how the brain constructs rewards: what processes determine how intensely a reward is experienced? Here, we outline limitations of existing computational conceptualisations of consummatory anhedonia. We then suggest a richer reinforcement learning (RL) account of consummatory anhedonia with a reconceptualisation of subjective hedonic experience in terms of goal progress. This accounts qualitatively for the impact of stress, dysfunctional cognitions, and maladaptive beliefs on hedonic experience. The model also offers new views on the treatments for anhedonia.

Curiosity and the dynamics of optimal exploration

What drives our curiosity remains an elusive and hotly debated issue, with multiple hypotheses proposed but a cohesive account yet to be established. This review discusses traditional and emergent theories that frame curiosity as a desire to know and a drive to learn, respectively. We adopt a model-based approach that maps the temporal dynamics of various factors underlying curiosity-based exploration, such as uncertainty, information gain, and learning progress. In so doing, we identify the limitations of past theories and posit an integrated account that harnesses their strengths in describing curiosity as a tool for optimal environmental exploration. In our unified account, curiosity serves as a 'common currency' for exploration, which must be balanced with other drives such as safety and hunger to achieve efficient action.

Differential functional organization of amygdala-medial prefrontal cortex networks in macaque and human

Over the course of evolution, the amygdala (AMG) and medial frontal cortex (mPFC) network, involved in behavioral adaptation, underwent structural changes in the old-world monkey and human lineages. Yet, whether and how the functional organization of this network differs remains poorly understood. Using resting-state functional magnetic resonance imagery, we show that the functional connectivity (FC) between AMG nuclei and mPFC regions differs between humans and awake macaques. In humans, the AMG-mPFC FC displays U-shaped pattern along the corpus callosum: a positive FC with the ventromedial prefrontal (vmPFC) and anterior cingulate cortex (ACC), a negative FC with the anterior mid-cingulate cortex (MCC), and a positive FC with the posterior MCC. Conversely, in macaques, the negative FC shifted more ventrally at the junction between the vmPFC and the ACC. The functional organization divergence of AMG-mPFC network between humans and macaques might help understanding behavioral adaptation abilities differences in their respective socio-ecological niches.

Fractionation of neural reward processing into independent components by novel decoding principle

How to retrieve latent neurobehavioural processes from complex neurobiological signals is an important yet unresolved challenge. Here, we develop a novel approach, orthogonal-Decoding multi-Cognitive Processes (DeCoP), to reveal underlying latent neurobehavioural processing and show that its performance is superior to traditional non-orthogonal decoding in terms of both false inference and robustness. Processing value and salience information are two fundamental but mutually confounded pathways of reward reinforcement essential for decision making. During reward/punishment anticipation, we applied DeCoP to decode brain-wide responses into spatially overlapping, yet functionally independent, evaluation and readiness processes, which are modulated differentially by meso‑limbic vs nigro-striatal dopamine systems. Using DeCoP, we further demonstrated that most brain regions only encoded abstract information but not the exact input, except for dorsal anterior cingulate cortex and insula. Furthermore, we anticipate our novel analytical principle to be applied generally in decoding multiple latent neurobehavioral processes and thus advance both the design and hypothesis testing for cognitive tasks.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

Multiple routes of communication within the amygdala-mPFC network: A comparative approach in humans and macaques

The network formed by the amygdala (AMG) and the medial Prefrontal Cortex (mPFC), at the interface between our internal and external environment, has been shown to support some important aspects of behavioral adaptation. Whether and how the anatomo-functional organization of this network evolved across primates remains unclear. Here, we compared AMG nuclei morphological characteristics and their functional connectivity with the mPFC in humans and macaques to identify potential homologies and differences between these species. Based on selected studies, we highlight two subsystems within the AMG-mPFC circuits, likely involved in distinct temporal dynamics of integration during behavioral adaptation. We also show that whereas the mPFC displays a large expansion but a preserved intrinsic anatomo-functional organization, the AMG displays a volume reduction and morphological changes related to specific nuclei. We discuss potential commonalities and differences in the dialogue between AMG nuclei and mPFC in humans and macaques based on available data.

Intrinsic rewards explain context-sensitive valuation in reinforcement learning

When observing the outcome of a choice, people are sensitive to the choice's context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms-reflecting a different theoretical viewpoint-may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new "intrinsically enhanced" RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.

Frontopolar cortex represents complex features and decision value during choice between environments

Important decisions often involve choosing between complex environments that define future item encounters. Despite its importance for adaptive behavior and distinct computational challenges, decision-making research primarily focuses on item choice, ignoring environment choice altogether. Here we contrast previously studied item choice in ventromedial prefrontal cortex with lateral frontopolar cortex (FPl) linked to environment choice. Furthermore, we propose a mechanism for how FPl decomposes and represents complex environments during decision making. Specifically, we trained a choice-optimized, brain-naive convolutional neural network (CNN) and compared predicted CNN activation with actual FPl activity. We showed that the high-dimensional FPl activity decomposes environment features to represent the complexity of an environment to make such choice possible. Moreover, FPl functionally connects with posterior cingulate cortex for guiding environment choice. Further probing FPl's computation revealed a parallel processing mechanism in extracting multiple environment features.

Altered intrinsic functional network connectivity is associated with impulsivity and emotion dysregulation in drug-naïve young patients with borderline personality disorder

BACKGROUND

Despite impulse control and emotion regulation being altered in borderline personality disorder (BPD), the specific mechanism of these clinical features remains unclear. This study investigated the functional connectivity (FC) abnormalities within- and between- default mode network (DMN), salience network (SN), and central executive network (CEN) in BPD, and examined the association between aberrant FC and clinical features. We aimed to explore whether the abnormal large-scale networks underlie the pathophysiology of impulsivity and emotion dysregulation in BPD.

METHODS

Forty-one young, drug-naïve patients with BPD (24.98 ± 3.12 years, 20 males) and 42 healthy controls (HCs; 24.74 ± 1.29 years, 17 males) were included in resting-state functional magnetic resonance imaging analyses. Independent component analysis was performed to extract subnetworks of the DMN, CEN, and SN. Additionally, partial correlation was performed to explore the association between brain imaging variables and clinical features in BPD.

RESULTS

Compared with HCs, BPD showed significant decreased intra-network FC of right medial prefrontal cortex in the anterior DMN and of right angular gyrus in the right CEN. Intra-network FC of right angular gyrus in the anterior DMN was significantly negatively correlated with attention impulsivity in BPD. The patients also showed decreased inter-network FC between the posterior DMN and left CEN, which was significantly negatively correlated with emotion dysregulation.

CONCLUSION

These findings suggest that impaired intra-network FC may underlie the neurophysiological mechanism of impulsivity, and abnormal inter-network FC may elucidate the neurophysiological mechanism of emotion dysregulation in BPD.

Optogenetic stimulation in the medial prefrontal cortex modulates stimulus valence from rewarding and aversive to neutral states

Introduction

Understanding the modulations of the medial prefrontal cortex (mPFC) in the valence of the stimulus from rewarding and aversive status to neutral status is crucial for the development of novel treatments for drug addiction. This study addressed this issue and examined whether optogenetic ChR2 photostimulation in the cingulate, prelimbic, and infralimbic cortices of the mPFC regulated the valence of saccharin solution consumption from the rewarding property, the aversive property induced by morphine's conditioning, and the neutral states via saccharin extinction processes after morphine's conditioning.

Methods

All rats received virus infection, buried optical fiber, optical stimulation, water deprivation, and saccharin solution consumption phases. In Experiment 1, rats were given ChR2 virus infection into the cingulate cortex (Cg1), prelimbic cortex (PrL), and infralimbic cortex (IL) to influence the rewarding saccharin solution consumption under photostimulation. In Experiment 2, rats were given ChR2 or EYFP virus infection into the Cg1, PrL, and IL to alter the saccharin solution consumption in the morphine-induced aversively conditioned taste aversion (CTA) and the saccharin solution consumption in the neutral state following the extinction process under photostimulation. Later, the immunohistochemical staining with c-Fos protein was performed for the Cg1, IL, PrL, nucleus accumbens core, nucleus accumbens shell, central amygdala, basolateral amygdala, ventral tegmental area, and dentate gyrus.

Results

The results showed that optogenetic PrL stimulation decreased the rewarding valence of saccharin solution consumption and increased the morphine-induced, aversive valence of saccharin solution consumption. PrL stimulation decreased the neutral valence of saccharin solution consumption via the extinction process. Cg1 optogenetic stimulation increased the rewarding valence of saccharin solution consumption and the aversive valence of saccharin solution consumption induced by morphine in conditioning. Optogenetic IL stimulation increased the aversive valence of saccharin solution consumption induced by morphine via conditioning.

Conclusion

Altogether, optogenetic stimulation in the subareas of the mPFC modulated the reward, aversion, and neutral valences of the stimulus and altered neuronal activity in the mPFC, amygdala, nucleus accumbens, and hippocampus. Notably, the change of valence was temporary alternation during light-on related to the light-off periods. However, the findings may provide insights in the development of novel treatments for addictive symptoms.

The physiological control of eating: signals, neurons, and networks

During the past 30 yr, investigating the physiology of eating behaviors has generated a truly vast literature. This is fueled in part by a dramatic increase in obesity and its comorbidities that has coincided with an ever increasing sophistication of genetically based manipulations. These techniques have produced results with a remarkable degree of cell specificity, particularly at the cell signaling level, and have played a lead role in advancing the field. However, putting these findings into a brain-wide context that connects physiological signals and neurons to behavior and somatic physiology requires a thorough consideration of neuronal connections: a field that has also seen an extraordinary technological revolution. Our goal is to present a comprehensive and balanced assessment of how physiological signals associated with energy homeostasis interact at many brain levels to control eating behaviors. A major theme is that these signals engage sets of interacting neural networks throughout the brain that are defined by specific neural connections. We begin by discussing some fundamental concepts, including ones that still engender vigorous debate, that provide the necessary frameworks for understanding how the brain controls meal initiation and termination. These include key word definitions, ATP availability as the pivotal regulated variable in energy homeostasis, neuropeptide signaling, homeostatic and hedonic eating, and meal structure. Within this context, we discuss network models of how key regions in the endbrain (or telencephalon), hypothalamus, hindbrain, medulla, vagus nerve, and spinal cord work together with the gastrointestinal tract to enable the complex motor events that permit animals to eat in diverse situations.

A Domain-General Developmental "Do-GooD" Network Model of Prosocial Cognition in Adolescence: A Systematic Review

Adolescence is a period of substantial neural and social development, and prosocial decisions are beneficial to personal well-being, the well-being of others, and the functioning of society. Advances in network neuroscience call for a systematic synthesis and reappraisal of prosocial neural correlates during adolescent development. In this systematic review, we aim to outline the progress made in this field, identify the similarities between study results, and propose a model for prosocial cognition in adolescents to young adults. A total of 25 articles were included in this review. After reviewing and synthesizing the literature, we propose a DOmain-General Developmental "Do-GooD" network model of prosocial cognition that aligns with the reviewed literature, accounts for development, and combines elements of the value-based decision-making model with distinct value contributions from the default mode network, salience network, and control network. We offer predictions to test the "Do-GooD" model and propose new future directions for studying prosocial behavior and its development during adolescence, which in turn may lead to improving education and the development of better health interventions for adolescents.

A Computational View on the Nature of Reward and Value in Anhedonia

Anhedonia - a common feature of depression and other neuropsychiatric disorders - encompasses a reduction in the subjective experience and anticipation of rewarding events, and a reduction in the motivation to seek out such events. The presence of anhedonia often predicts or accompanies treatment resistance, and as such better interventions and treatments are important. Yet the mechanisms giving rise to anhedonia are not well understood. In this chapter, we briefly review existing computational conceptualisations of anhedonia. We argue that they are mostly descriptive and fail to provide an explanatory account of why anhedonia may occur. Working within the framework of reinforcement learning, we examine two potential computational mechanisms that could give rise to anhedonic phenomena. First, we show how anhedonia can arise in multi-dimensional drive-reduction settings through a trade-off between different rewards or needs. We then generalise this in terms of model-based value inference and identify a key role for associational belief structure. We close with a brief discussion of treatment implications of both of these conceptualisations. In summary, computational accounts of anhedonia have provided a useful descriptive framework. Recent advances in reinforcement learning suggest promising avenues by which the mechanisms underlying anhedonia may be teased apart, potentially motivating novel approaches to treatment.

Imbalance in Functional and Structural Connectivity Underlying Goal-Directed and Habitual Learning Systems in Obsessive-Compulsive Disorder

An imbalance between the goal-directed and habitual learning systems has been proposed to underlie compulsivity in obsessive-compulsive disorder (OCD). In addition, the overall balance between these systems may be influenced by stress hormones. We examined the multimodal networks underlying these dual learning systems. Both functional and structural measures indicated reduced connectivity within the goal-directed subnetwork (FC: P = 0.042; SC-FN: P = 0.014) and reduced connectivity between the goal-directed and habitual subnetworks (FC: P = 0.014; SC-FA: P = 0.052), but no differences within the habitual subnetwork in patients with OCD compared with controls. Path modeling indicated that anatomical connectivity in the goal-directed subnetwork influenced compulsive symptoms (R2 = 0.41), whereas functional connectivity within the habit subnetwork and between goal-directed and habitual subnetworks influenced obsessive symptoms (R2 = 0.63). In addition, the relationship between anatomical connectivity in the goal-directed subnetwork and compulsion was moderated by the stress hormone ACTH (adrenocorticotropic hormone), such that at low levels of ACTH greater connectivity resulted in lower compulsion, but at high levels of ACTH this relationship was reversed. These results provide new insights into the neural correlates of the imbalance between dual learning systems, and their relationship with symptom dimensions in patients with OCD. It may further support the reconceptualization of OCD as "compulsive-obsessive disorder," with a greater focus on the transdiagnostic dimension of compulsivity.

Formalizing planning and information search in naturalistic decision-making

Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.

Meta-analytic clustering dissociates brain activity and behavior profiles across reward processing paradigms

Reward learning is a ubiquitous cognitive mechanism guiding adaptive choices and behaviors, and when impaired, can lead to considerable mental health consequences. Reward-related functional neuroimaging studies have begun to implicate networks of brain regions essential for processing various peripheral influences (e.g., risk, subjective preference, delay, social context) involved in the multifaceted reward processing construct. To provide a more complete neurocognitive perspective on reward processing that synthesizes findings across the literature while also appreciating these peripheral influences, we used emerging meta-analytic techniques to elucidate brain regions, and in turn networks, consistently engaged in distinct aspects of reward processing. Using a data-driven, meta-analytic, k-means clustering approach, we dissociated seven meta-analytic groupings (MAGs) of neuroimaging results (i.e., brain activity maps) from 749 experimental contrasts across 176 reward processing studies involving 13,358 healthy participants. We then performed an exploratory functional decoding approach to gain insight into the putative functions associated with each MAG. We identified a seven-MAG clustering solution that represented dissociable patterns of convergent brain activity across reward processing tasks. Additionally, our functional decoding analyses revealed that each of these MAGs mapped onto discrete behavior profiles that suggested specialized roles in predicting value (MAG-1 & MAG-2) and processing a variety of emotional (MAG-3), external (MAG-4 & MAG-5), and internal (MAG-6 & MAG-7) influences across reward processing paradigms. These findings support and extend aspects of well-accepted reward learning theories and highlight large-scale brain network activity associated with distinct aspects of reward processing.

The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex

Despite continual debate for the past 30 years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. However, recent computational modeling work has provided insight into this question. Here we review computational models that illustrate three core principles of ACC function, related to hierarchy, world models, and cost. We also discuss four constraints on the neural implementation of these principles, related to modularity, binding, encoding, and learning and regulation. These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning (HMB-HRL), which instantiates a mechanism motivating the execution of high-level plans.

Chimpanzee histology and functional brain imaging show that the paracingulate sulcus is not human-specific

The paracingulate sulcus -PCGS- has been considered for a long time to be specific to the human brain. Its presence/absence has been discussed in relation to interindividual variability of personality traits and cognitive abilities. Recently, a putative PCGS has been observed in chimpanzee brains. To demonstrate that this newly discovered sulcus is the homologue of the PCGS in the human brain, we analyzed cytoarchitectonic and resting-state functional magnetic resonance imaging data in chimpanzee brains which did or did not display a PCGS. The results show that the organization of the mid-cingulate cortex of the chimpanzee brain is comparable to that of the human brain, both cytoarchitectonically and in terms of functional connectivity with the lateral frontal cortex. These results demonstrate that the PCGS is not human-specific but is a shared feature of the primate brain since at least the last common ancestor to humans and great apes ~6 mya.

The paracingulate sulcus (PCGS) is a brain structure long thought to be specific to humans, and variation in this structure has been linked to personality traits and cognitive abilities. In this study, Céline Amiez and Jérôme Sallet et al. analyze brain imaging data from humans and chimpanzees to demonstrate that the PCGS is in fact present in our closest relative and its functional connectivity in chimpanzees is comparable to that in humans.

Dynamic integration of forward planning and heuristic preferences during multiple goal pursuit

Selecting goals and successfully pursuing them in an uncertain and dynamic environment is an important aspect of human behaviour. In order to decide which goal to pursue at what point in time, one has to evaluate the consequences of one’s actions over future time steps by forward planning. However, when the goal is still temporally distant, detailed forward planning can be prohibitively costly. One way to select actions at minimal computational costs is to use heuristics. It is an open question how humans mix heuristics with forward planning to balance computational costs with goal reaching performance. To test a hypothesis about dynamic mixing of heuristics with forward planning, we used a novel stochastic sequential two-goal task. Comparing participants’ decisions with an optimal full planning agent, we found that at the early stages of goal-reaching sequences, in which both goals are temporally distant and planning complexity is high, on average 42% (SD = 19%) of participants’ choices deviated from the agent’s optimal choices. Only towards the end of the sequence, participant’s behaviour converged to near optimal performance. Subsequent model-based analyses showed that participants used heuristic preferences when the goal was temporally distant and switched to forward planning when the goal was close.

When we pursue our goals, there is often a moment when we recognize that we did not make the progress that we hoped for. What should we do now? Persevere to achieve the original goal, or switch to another goal? Two features of real-world goal pursuit make these decisions particularly complex. First, goals can lie far into an unpredictable future and second, there are many potential goals to pursue. When potential goals are temporally distant, human decision makers cannot use an exhaustive planning strategy, rendering simpler rules of thumb more appropriate. An important question is how humans adjust the rule of thumb approach once they get closer to the goal. We addressed this question using a novel sequential two-goal task and analysed the choice data using a computational model which arbitrates between a rule of thumb and accurate planning. We found that participants’ decision making progressively improved as the goal came closer and that this improvement was most likely caused by participants starting to plan ahead.

Threat Prediction from Schemas as a Source of Bias in Pain Perception

Our sensory impressions of pain are generally thought to represent the noxious properties of an agent but can be influenced by the predicted level of threat. Predictions can be sourced from higher-order cognitive processes, such as schemas, but the extent to which schemas can influence pain perception relative to bottom-up sensory inputs and the underlying neural underpinnings of such a phenomenon are unclear. Here, we investigate how threat predictions generated from learning a cognitive schema lead to inaccurate sensory impressions of the pain stimulus. Healthy male and female participants first detected a linear association between cue values and stimulus intensity and rated pain to reflect the linear schema when compared with uncued heat stimuli. The effect of bias on pain ratings was reduced when prediction errors (PEs) increased, but pain perception was only partially updated when measured against stepped increases in PEs. Cognitive, striatal, and sensory regions graded their responses to changes in predicted threat despite the PEs (p < 0.05, corrected). Individuals with more catastrophic thinking about pain and with low mindfulness were significantly more reliant on the schema than on the sensory evidence from the pain stimulus. These behavioral differences mapped to variability in responses of the striatum and ventromedial prefrontal cortex. Thus, this study demonstrates a significant role of higher-order schemas in pain perception and indicates that pain perception is biased more toward predictions and less toward nociceptive inputs in individuals who report less mindfulness and more fear of pain.SIGNIFICANCE STATEMENT This study demonstrates that threat predictions generated from cognitive schemas continue to influence pain perception despite increasing prediction errors arising in pain pathways. Individuals first formed a cognitive schema of linearity in the relationship between the cued threat value and the stimulus intensity. Subsequently, the linearity was reduced gradually, and participants partially updated their evaluations of pain in relation to the stepped increases in prediction errors. Individuals who continued to rate pain based more on the predicted threat than on changes in nociceptive inputs reported high pain catastrophizing and less mindful-awareness scores. These two affects mapped to activity in the ventral and dorsal striatum, respectively. These findings direct us to a significant role of top-down processes in pain perception.

Where Does Value Come From?

The computational framework of reinforcement learning (RL) has allowed us to both understand biological brains and build successful artificial agents. However, in this opinion, we highlight open challenges for RL as a model of animal behaviour in natural environments. We ask how the external reward function is designed for biological systems, and how we can account for the context sensitivity of valuation. We summarise both old and new theories proposing that animals track current and desired internal states and seek to minimise the distance to a goal across multiple value dimensions. We suggest that this framework readily accounts for canonical phenomena observed in the fields of psychology, behavioural ecology, and economics, and recent findings from brain-imaging studies of value-guided decision-making.

Looking into the Brain of Buridan's Ass

To survive, animals must maximize the minimum-take care of the least satisfied among their basic needs. In this issue of Neuron, a study by Juechems et al. (2019) illustrates that this core principle might shape the way medial prefrontal regions of the human brain drive and valuate sequential choices between different types of reward.