Necessary Contributions of Human Frontal Lobe Subregions to Reward Learning in a Dynamic, Multidimensional Environment

Prefrontal cortex state representations shape human credit assignment

People learn adaptively from feedback, but the rate of such learning differs drastically across individuals and contexts. Here, we examine whether this variability reflects differences in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative reward learning task, we link the specificity of credit assignment-how well people are able to appropriately attribute outcomes to their causes-to the precision of neural codes in the prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared vto nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in concert to match the neural codes from feedback to those at choice, and the strength of these common neural codes predicts credit assignment precision. Together this work provides a window into how neural representations drive adaptive learning.

Sinful pleasures and pious woes? Using fMRI to examine evaluative and hedonic emotion knowledge

Traditionally, lust and pride have been considered pleasurable, yet sinful in the West. Conversely, guilt is often considered aversive, yet valuable. These emotions illustrate how evaluations about specific emotions and beliefs about their hedonic properties may often diverge. Evaluations about specific emotions may shape important aspects of emotional life (e.g. in emotion regulation, emotion experience and acquisition of emotion concepts). Yet these evaluations are often understudied in affective neuroscience. Prior work in emotion regulation, affective experience, evaluation/attitudes and decision-making point to anterior prefrontal areas as candidates for supporting evaluative emotion knowledge. Thus, we examined the brain areas associated with evaluative and hedonic emotion knowledge, with a focus on the anterior prefrontal cortex. Participants (N = 25) made evaluative and hedonic ratings about emotion knowledge during functional magnetic resonance imaging (fMRI). We found that greater activity in the medial prefrontal cortex (mPFC), ventromedial PFC (vmPFC) and precuneus was associated with an evaluative (vs hedonic) focus on emotion knowledge. Our results suggest that the mPFC and vmPFC, in particular, may play a role in evaluating discrete emotions.

Prefrontal cortex interactions with the amygdala in primates

This review addresses functional interactions between the primate prefrontal cortex (PFC) and the amygdala, with emphasis on their contributions to behavior and cognition. The interplay between these two telencephalic structures contributes to adaptive behavior and to the evolutionary success of all primate species. In our species, dysfunction in this circuitry creates vulnerabilities to psychopathologies. Here, we describe amygdala-PFC contributions to behaviors that have direct relevance to Darwinian fitness: learned approach and avoidance, foraging, predator defense, and social signaling, which have in common the need for flexibility and sensitivity to specific and rapidly changing contexts. Examples include the prediction of positive outcomes, such as food availability, food desirability, and various social rewards, or of negative outcomes, such as threats of harm from predators or conspecifics. To promote fitness optimally, these stimulus-outcome associations need to be rapidly updated when an associative contingency changes or when the value of a predicted outcome changes. We review evidence from nonhuman primates implicating the PFC, the amygdala, and their functional interactions in these processes, with links to experimental work and clinical findings in humans where possible.

Distinct Causal Influences of Dorsolateral Prefrontal Cortex and Posterior Parietal Cortex in Multiple-Option Decision Making

Our knowledge about neural mechanisms underlying decision making is largely based on experiments that involved few options. However, it is more common in daily life to choose between many options, in which processing choice information selectively is particularly important. The current study examined whether the dorsolateral prefrontal cortex (dlPFC) and posterior parietal cortex (PPC) are of particular importance to multiple-option decision making. Sixty-eight participants received anodal high definition-transcranial direct current stimulation (HD-tDCS) to focally enhance dlPFC or PPC in a double-blind sham-controlled design. Participants then performed a multiple-option decision making task. We found longer fixations on poorer options were related to less optimal decisions. Interestingly, this negative impact was attenuated after applying anodal HD-tDCS over dlPFC, especially in choices with many options. This suggests that dlPFC has a causal role in filtering choice-irrelevant information. In contrast, these effects were absent after participants received anodal HD-tDCS over PPC. Instead, the choices made by these participants were more biased towards the best options presented on the side contralateral to the stimulation. This suggests PPC has a causal role in value-based spatial selection. To conclude, the dlPFC has a role in filtering undesirable options, whereas the PPC emphasizes the desirable contralateral options.

Supramodal executive control of attention: Evidence from unimodal and crossmodal dual conflict effects

Although we have demonstrated that the executive control of attention acts supramodally as shown by significant correlation between conflict effect measures in visual and auditory tasks, no direct evidence of the equivalence in the computational mechanisms governing the allocation of executive control resources within and across modalities has been found. Here, in two independent groups of 40 participants each, we examined the interaction effect of conflict processing in both unimodal (visual) and crossmodal (visual and auditory) dual-conflict paradigms (flanker conflict processing in Task 1 and then in the following Task 2) with a manipulation of the stimulus onset asynchrony (SOA). In both the unimodal and the crossmodal dual-conflict paradigms, the conflict processing of Task 1 significantly interfered with the processing of Task 2 when the SOA was short, as shown by an additive interference effect of Task 1 on Task 2 under the time constraints. These results suggest that there is a unified supramodal entity that supports conflict processing by implementing comparable mechanisms in unimodal and crossmodal scenarios.

Experimentally revealed stochastic preferences for multicomponent choice options

Realistic, everyday rewards contain multiple components. An apple has taste and size. However, we choose in single dimensions, simply preferring some apples to others. How can such single-dimensional preference relationships refer to multicomponent choice options? Here, we measured how stochastic choices revealed preferences for 2-component milkshakes. The preferences were intuitively graphed as indifference curves that represented the orderly integration of the 2 components as trade-off: parts of 1 component were given up for obtaining 1 additional unit of the other component without a change in preference. The well-ordered, nonoverlapping curves satisfied leave-one-out tests, followed predictions by machine learning decoders and correlated with single-dimensional Becker-DeGroot-Marschak (BDM) auction-like bids for the 2-component rewards. This accuracy suggests a decision process that integrates multiple reward components into single-dimensional estimates in a systematic fashion. In interspecies comparisons, human performance matched that of highly experienced laboratory monkeys, as measured by accuracy of the critical trade-off between bundle components. These data describe the nature of choices of multicomponent choice options and attest to the validity of the rigorous economic concepts and their convenient graphic schemes for explaining choices of human and nonhuman primates. The results encourage formal behavioral and neural investigations of normal, irrational, and pathological economic choices. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Under construction: ventral and lateral frontal lobe contributions to value-based decision-making and learning

Even apparently simple choices, like selecting a dessert in a pastry shop, involve options characterized by multiple motivationally relevant attributes. Neuroeconomic research suggests that the human brain may track the subjective value of such options, allowing disparate reward-predictive information to be compared in a common currency. However, the brain mechanisms involved in identifying value-predictive features and combining these to assess the value of each decision option remain unclear. Here, we review recent evidence from studies of multi-attribute decision-making in people with focal frontal lobe damage and in healthy people undergoing functional magnetic resonance imaging. This work suggests that ventromedial and lateral prefrontal cortex and orbitofrontal cortex are important for forming value judgments under conditions of complexity. We discuss studies supporting the involvement of these regions in selecting among and evaluating option attributes during value judgment and decision-making and when learning from reward feedback. These findings are consistent with roles for these regions in guiding value construction. They argue for a more nuanced understanding of how ventral and lateral prefrontal cortex contribute to discovering and recognizing value, processes that are required under the complex conditions typical of many everyday decisions.

Ventromedial prefrontal cortex compression during concept learning

Prefrontal cortex (PFC) is thought to support the ability to focus on goal-relevant information by filtering out irrelevant information, a process akin to dimensionality reduction. Here, we test this dimensionality reduction hypothesis by relating a data-driven approach to characterizing the complexity of neural representation with a theoretically-supported computational model of learning. We find evidence of goal-directed dimensionality reduction within human ventromedial PFC during learning. Importantly, by using computational predictions of each participant’s attentional strategies during learning, we find that that the degree of neural compression predicts an individual’s ability to selectively attend to concept-specific information. These findings suggest a domain-general mechanism of learning through compression in ventromedial PFC.

Efficient learning is akin to goal-directed dimensionality reduction, in which relevant information is highlighted and irrelevant input is ignored. Here, the authors show that ventromedial prefrontal cortex uniquely supports such learning by compressing neural codes to represent goal-specific information.

Ventromedial Frontal Lobe Damage Alters how Specific Attributes are Weighed in Subjective Valuation

The concept of subjective value is central to current neurobiological views of economic decision-making. Much of this work has focused on signals in the ventromedial frontal lobe (VMF) that correlate with the subjective value of a variety of stimuli (e.g., food, monetary gambles), and are thought to support decision-making. However, the neural processes involved in assessing and integrating value information from the attributes of such complex options remain to be defined. Here, we tested the necessary role of VMF in weighting attributes of naturalistic stimuli during value judgments. We asked how distinct attributes of visual artworks influenced the subjective value ratings of subjects with VMF damage, compared to healthy participants and a frontal lobe damaged control group. Subjects with VMF damage were less influenced by the energy (emotion, complexity) and color radiance (warmth, saturation) of the artwork, while they were similar to control groups in considering saliency, balance and concreteness. These dissociations argue that VMF is critical for allowing certain affective content to influence subjective value, while sparing the influence of perceptual or representational information. These distinctions are important for better defining the often-underspecified concept of subjective value and developing more detailed models of the brain mechanisms underlying decision behavior.

Beyond a rod through the skull: A systematic review of lesion studies of the human ventromedial frontal lobe

Neuropsychological studies from the past century have associated damage to the ventromedial frontal lobes (VMF) with impairments in a variety of domains, including memory, executive function, emotion, social cognition, and valuation. A central question in the literature is whether these seemingly distinct functions are subserved by different sub-regions within the VMF, or whether VMF supports a broader cognitive process that is crucial to these varied domains. In this comprehensive review of the neuropsychological literature from the last two decades, we present a qualitative synthesis of 184 papers that have examined the psychological impairments that result from VMF damage. We discuss these findings in the context of several theoretical frameworks and advocate for the view that VMF is critical for the formation and representation of schema and cognitive maps.

Specializations for reward-guided decision-making in the primate ventral prefrontal cortex

The estimated values of choices, and therefore decision-making based on those values, are influenced by both the chance that the chosen items or goods can be obtained (availability) and their current worth (desirability) as well as by the ability to link the estimated values to choices (a process sometimes called credit assignment). In primates, the prefrontal cortex (PFC) has been thought to contribute to each of these processes; however, causal relationships between particular subdivisions of the PFC and specific functions have been difficult to establish. Recent lesion-based research studies have defined the roles of two different parts of the primate PFC - the orbitofrontal cortex (OFC) and the ventral lateral frontal cortex (VLFC) - and their subdivisions in evaluating each of these factors and in mediating credit assignment during reward-based decision-making.

A Critical Role for Human Ventromedial Frontal Lobe in Value Comparison of Complex Objects Based on Attribute Configuration

In making decisions, we often choose from among options with multiple value-relevant attributes. Neuroeconomic models propose that the value associated with each attribute is integrated in a global value for each option. However, some evidence from patients with ventromedial frontal lobe (VMF) damage argues against a very general role for this region in value integration, suggesting instead that it contributes critically to a specific value inference or comparison process. Here, we tested value-based decision-making involving artificial multiattribute objects in humans with focal damage to the VMF (N = 12) compared with a healthy group matched for age and education (N = 24) and with a group with frontal lobe damage sparing the VMF (N = 12). In a "configural" condition, overall object value was predicted by the conjunction of two attributes, while in an "elemental" condition, object value could be assessed by combining the independent values of individual attributes. Patients with VMF damage were impaired in making choices when value was uniquely predicted by the configuration of attributes, but intact when choosing based on elemental attribute values. This is evidence that the VMF is critical for inferring the value of whole objects in a multiattribute choice. These findings have implications for models of value-based choice and add to emerging views of how this region may interact with medial temporal lobe systems involved in configural object processing and relational memory.SIGNIFICANCE STATEMENT Neuroeconomic models propose that the ventromedial frontal lobe (VMF) supports multiattribute decisions by integrating the values of attributes. However, researchers have been uncertain about the underlying mechanisms for this process. Patients with VMF damage made multiattribute choices under two conditions: in one, attribute values could be summed to guide choice; in the other, value was predicted by the conjunction of attributes. VMF damage impaired only the latter. This argues that the VMF is critical for inferring value from configural information to guide multiattribute object choice. This region may be key for judging the emergent "value of the forest," rather than for integrating the individual "value of each tree."

Understanding psychiatric disorder by capturing ecologically relevant features of learning and decision-making

Recent research in cognitive neuroscience has begun to uncover the processes underlying increasingly complex voluntary behaviours, including learning and decision-making. Partly this success has been possible by progressing from simple experimental tasks to paradigms that incorporate more ecological features. More specifically, the premise is that to understand cognitions and brain functions relevant for real life, we need to introduce some of the ecological challenges that we have evolved to solve. This often entails an increase in task complexity, which can be managed by using computational models to help parse complex behaviours into specific component mechanisms. Here we propose that using computational models with tasks that capture ecologically relevant learning and decision-making processes may provide a critical advantage for capturing the mechanisms underlying symptoms of disorders in psychiatry. As a result, it may help develop mechanistic approaches towards diagnosis and treatment. We begin this review by mapping out the basic concepts and models of learning and decision-making. We then move on to consider specific challenges that emerge in realistic environments and describe how they can be captured by tasks. These include changes of context, uncertainty, reflexive/emotional biases, cost-benefit decision-making, and balancing exploration and exploitation. Where appropriate we highlight future or current links to psychiatry. We particularly draw examples from research on clinical depression, a disorder that greatly compromises motivated behaviours in real-life, but where simpler paradigms have yielded mixed results. Finally, we highlight several paradigms that could be used to help provide new insights into the mechanisms of psychiatric disorders.

Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes

Advantageous foraging choices benefit from an estimation of two aspects of a resource's value: its current desirability and availability. Both orbitofrontal and ventrolateral prefrontal areas contribute to updating these valuations, but their precise roles remain unclear. To explore their specializations, we trained macaque monkeys on two tasks: one required updating representations of a predicted outcome's desirability, as adjusted by selective satiation, and the other required updating representations of an outcome's availability, as indexed by its probability. We evaluated performance on both tasks in three groups of monkeys: unoperated controls and those with selective, fiber-sparing lesions of either the OFC or VLPFC. Representations that depend on the VLPFC but not the OFC play a necessary role in choices based on outcome availability; in contrast, representations that depend on the OFC but not the VLPFC play a necessary role in choices based on outcome desirability.

Contrasting Effects of Medial and Lateral Orbitofrontal Cortex Lesions on Credit Assignment and Decision-Making in Humans

The orbitofrontal cortex is critical for goal-directed behavior. Recent work in macaques has suggested the lateral orbitofrontal cortex (lOFC) is relatively more concerned with assignment of credit for rewards to particular choices during value-guided learning, whereas the medial orbitofrontal cortex (often referred to as ventromedial prefrontal cortex in humans; vmPFC/mOFC) is involved in constraining the decision to the relevant options. We examined whether people with damage restricted to subregions of prefrontal cortex showed the patterns of impairment observed in prior investigations of the effects of lesions to homologous regions in macaques. Groups of patients with either lOFC (predominantly right hemisphere), mOFC/vmPFC, or dorsomedial prefrontal (DMF), and a comparison group of healthy age- and education-matched controls performed a probabilistic 3-choice decision-making task. We report anatomically specific patterns of impairment. We found that credit assignment, as indexed by the normal influence of contingent relationships between choice and reward, is reduced in lOFC patients compared with Controls and mOFC/vmPFC patients. Moreover, the effects of reward contingency on choice were similar for patients with lesions in DMF or mOFC/vmPFC, compared with Controls. By contrast, mOFC/vmPFC-lesioned patients made more stochastic choices than Controls when the decision was framed by valuable distracting alternatives, suggesting that value comparisons were no longer independent of irrelevant options. Once again, there was evidence of regional specialization: patients with lOFC lesions were unimpaired relative to Controls. As in macaques, human lOFC and mOFC/vmPFC are necessary for contingent learning and value-guided decision-making, respectively.SIGNIFICANCE STATEMENT The lateral and medial regions of the orbitofrontal cortex are cytoarchitectonically distinct and have different anatomical connections. Previous investigations in macaques have shown these anatomical differences are accompanied by functional specialization for learning and decision-making. Here, for the first time, we test the predictions made by macaque studies in an experiment with humans with frontal lobe lesions, asking whether behavioral impairments can be linked to lateral or medial orbitofrontal cortex. Using equivalent tasks and computational analyses, our findings broadly replicate the pattern reported after selective lesions in monkeys. Patients with lateral orbitofrontal damage had impaired credit assignment, whereas damage to medial orbitofrontal cortex meant that patients were more likely to be distracted by irrelevant options.