Neural correlates of strategic reasoning during competitive games

Altered dynamic functional connectivity in rectal cancer patients with and without chemotherapy: a resting-state fMRI study

Purpose: Understanding the mechanism of brain functional alterations in rectal cancer (RC) patients is of great significance to improve the prognosis and quality of life of patients. Additionally, the influence of chemotherapy on brain function in RC patients is still unclear. In this study, we aimed to investigate the alterations of brain functional network dynamics in RC patients and explore the effects of chemotherapy on temporal dynamics of dynamic functional connectivity (DFC).

Methods: The group independent component analysis (GICA) and sliding window method were applied to investigate abnormalities of DFC based on resting-state functional magnetic resonance imaging (rs-fMRI) of 18 RC patients without chemotherapy (RC_NC), 21 RC patients with chemotherapy (RC_C) and 33 healthy controls (HC). Then, the Spearman correlation between aberrant properties and clinical measures was calculated.

Results: Two discrete states were identified. Compared to HC, RC_NC exhibited increased mean dwell time (MDT) and fractional windows (FW) in state 2 and decreased transition numbers between the two states. Notably, three temporal properties in RC_C showed an intermediate trend in comparison with RC_NC and HC. Furthermore, RC_C also demonstrated abnormal intra- and inter-network connections, involving the visual (VIS), default mode (DM), and cognitive control (CC) networks, and most connections related to VIS were correlated with the severity of anxiety and depression.

Conclusions: Our study suggested that abnormal DFC patterns could be manifested in RC patients and chemotherapy would further correct abnormalities of network dynamics, which may provide new insights into the brain functional alterations in patients with RC from the time-varying connectivity perspective.

Therapeutic doses of ketamine acutely attenuate the aversive effect of losses during decision-making

The discovery of rapid-acting antidepressant, ketamine has opened a pathway to a new generation of treatments for depression, and inspired neuroscientific investigation based on a new perspective that non-adaptive changes in the intrinsic excitatory and inhibitory circuitry might underlie the pathophysiology of depression. Nevertheless, it still remains largely unknown how the hypothesized molecular and synaptic levels of changes in the circuitry might mediate behavioral and neuropsychological changes underlying depression, and how ketamine might restore adaptive behavior. Here, we used computational models to analyze behavioral changes induced by therapeutic doses of ketamine, while rhesus macaques were iteratively making decisions based on gains and losses of tokens. When administered intramuscularly or intranasally, ketamine reduced the aversiveness of undesirable outcomes such as losses of tokens without significantly affecting the evaluation of gains, behavioral perseveration, motivation, and other cognitive aspects of learning such as temporal credit assignment and time scales of choice and outcome memory. Ketamine's potentially antidepressant effect was separable from other side effects such as fixation errors, which unlike outcome evaluation, was readily countered with strong motivation to avoid errors. We discuss how the acute effect of ketamine to reduce the initial impact of negative events could potentially mediate longer-term antidepressant effects through mitigating the cumulative effect of those events produced by slowly decaying memory, and how the disruption-resistant affective memory might pose challenges in treating depression. Our study also invites future investigations on ketamine's antidepressant action over diverse mood states and with affective events exerting their impacts at diverse time scales.

Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others

Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.

ACC neural ensemble dynamics are structured by strategy prevalence

Medial frontal cortical areas are thought to play a critical role in the brain's ability to flexibly deploy strategies that are effective in complex settings, yet the underlying circuit computations remain unclear. Here, by examining neural ensemble activity in male rats that sample different strategies in a self-guided search for latent task structure, we observe robust tracking during strategy execution of a summary statistic for that strategy in recent behavioral history by the anterior cingulate cortex (ACC), especially by an area homologous to primate area 32D. Using the simplest summary statistic - strategy prevalence in the last 20 choices - we find that its encoding in the ACC during strategy execution is wide-scale, independent of reward delivery, and persists through a substantial ensemble reorganization that accompanies changes in global context. We further demonstrate that the tracking of reward by the ACC ensemble is also strategy-specific, but that reward prevalence is insufficient to explain the observed activity modulation during strategy execution. Our findings argue that ACC ensemble dynamics is structured by a summary statistic of recent behavioral choices, raising the possibility that ACC plays a role in estimating - through statistical learning - which actions promote the occurrence of events in the environment.

Tasks and their role in visual neuroscience

Vision is widely used as a model system to gain insights into how sensory inputs are processed and interpreted by the brain. Historically, careful quantification and control of visual stimuli have served as the backbone of visual neuroscience. There has been less emphasis, however, on how an observer's task influences the processing of sensory inputs. Motivated by diverse observations of task-dependent activity in the visual system, we propose a framework for thinking about tasks, their role in sensory processing, and how we might formally incorporate tasks into our models of vision.

Competitive and cooperative games for probing the neural basis of social decision-making in animals

In a social environment, it is essential for animals to consider the behavior of others when making decisions. To quantitatively assess such social decisions, games offer unique advantages. Games may have competitive and cooperative components, modeling situations with antagonistic and shared objectives between players. Games can be analyzed by mathematical frameworks, including game theory and reinforcement learning, such that an animal's choice behavior can be compared against the optimal strategy. However, so far games have been underappreciated in neuroscience research, particularly for rodent studies. In this review, we survey the varieties of competitive and cooperative games that have been tested, contrasting strategies employed by non-human primates and birds with rodents. We provide examples of how games can be used to uncover neural mechanisms and explore species-specific behavioral differences. We assess critically the limitations of current paradigms and propose improvements. Together, the synthesis of current literature highlights the advantages of using games to probe the neural basis of social decisions for neuroscience studies.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

Long-range inhibition synchronizes and updates prefrontal task activity

Changes in patterns of activity within the medial prefrontal cortex enable rodents, non-human primates and humans to update their behaviour to adapt to changes in the environment-for example, during cognitive tasks^1-5. Parvalbumin-expressing inhibitory neurons in the medial prefrontal cortex are important for learning new strategies during a rule-shift task^6-8, but the circuit interactions that switch prefrontal network dynamics from maintaining to updating task-related patterns of activity remain unknown. Here we describe a mechanism that links parvalbumin-expressing neurons, a new callosal inhibitory connection, and changes in task representations. Whereas nonspecifically inhibiting all callosal projections does not prevent mice from learning rule shifts or disrupt the evolution of activity patterns, selectively inhibiting only callosal projections of parvalbumin-expressing neurons impairs rule-shift learning, desynchronizes the gamma-frequency activity that is necessary for learning⁸ and suppresses the reorganization of prefrontal activity patterns that normally accompanies rule-shift learning. This dissociation reveals how callosal parvalbumin-expressing projections switch the operating mode of prefrontal circuits from maintenance to updating by transmitting gamma synchrony and gating the ability of other callosal inputs to maintain previously established neural representations. Thus, callosal projections originating from parvalbumin-expressing neurons represent a key circuit locus for understanding and correcting the deficits in behavioural flexibility and gamma synchrony that have been implicated in schizophrenia and related conditions^9,10.

Home alone: A population neuroscience investigation of brain morphology substrates

As a social species, ready exchange with peers is a pivotal asset - our "social capital". Yet, single-person households have come to pervade metropolitan cities worldwide, with unknown consequences in the long run. Here, we systematically explore the morphological manifestations associated with singular living in ∼40,000 UK Biobank participants. The uncovered population-level signature spotlights the highly associative default mode network, in addition to findings such as in the amygdala central, cortical and corticoamygdaloid nuclei groups, as well as the hippocampal fimbria and dentate gyrus. Both positive effects, equating to greater gray matter volume associated with living alone, and negative effects, which can be interpreted as greater gray matter associations with not living alone, were found across the cortex and subcortical structures Sex-stratified analyses revealed male-specific neural substrates, including somatomotor, saliency and visual systems, while female-specific neural substrates centered on the dorsomedial prefrontal cortex. In line with our demographic profiling results, the discovered neural pattern of living alone is potentially linked to alcohol and tobacco consumption, anxiety, sleep quality as well as daily TV watching. The persistent trend for solitary living will require new answers from public-health decision makers. SIGNIFICANCE STATEMENT: Living alone has profound consequences for mental and physical health. Despite this, there has been a rapid increase in single-person households worldwide, with the long-term consequences yet unknown. In the largest study of its kind, we investigate how the objective lack of everyday social interaction, through living alone, manifests in the brain. Our population neuroscience approach uncovered a gray matter signature that converged on the 'default network', alongside targeted subcortical, sex and demographic profiling analyses. The human urge for social relationships is highlighted by the evolving COVID-19 pandemic. Better understanding of how social isolation relates to the brain will influence health and social policy decision-making of pandemic planning, as well as social interventions in light of global shifts in houseful structures.

The Aversive Lens: Stress effects on the prefrontal-cingulate cortical pathways that regulate emotion

ARNSTEN, A.F.T., M.K.P. Joyce and A.C. Roberts. The Aversive Lens: Stress effects on the prefrontal-cingulate cortical pathways that regulate emotion. NEUROSCI BIOBEHAV REV XXX-XXX, 2022. The symptoms of major-depressive-disorder include psychic pain and anhedonia, i.e. seeing the world through an "aversive lens". The neurobiology underlying this shift in worldview is emerging. Here these data are reviewed, focusing on how activation of subgenual cingulate (BA25) induces an "aversive lens", and how higher prefrontal cortical (PFC) areas (BA46/10/32) provide top-down regulation of BA25 but are weakened by excessive dopamine and norepinephrine release during stress exposure, and dendritic spine loss with chronic stress exposure. These changes may generate an attractor state, which maintains the brain under the control of BA25, requiring medication or neuromodulatory treatments to return connectivity to a more flexible state. In line with this hypothesis, effective anti-depressant treatments reduce the activity of BA25 and restore top-down regulation by higher circuits, e.g. as seen with SSRI medications, ketamine, deep brain stimulation of BA25, or rTMS to strengthen dorsolateral PFC. This research has special relevance in an era of chronic stress caused by the COVID19 pandemic, political unrest and threat of climate change.

A neuronal prospect theory model in the brain reward circuitry

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

It is unclear how the activity of individual neurons conform to prospect theory. Here, the authors demonstrate that the activity of single neurons in various reward-related regions in the monkey brain can be described as encoding a multiplicative combination of utility and probability weighting, and that this subjective valuation process is achieved via a distributed coding scheme.

Neurocomputations of strategic behavior: From iterated to novel interactions

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge.

This article is categorized under:

Economics > Interactive Decision‐Making

Psychology > Reasoning and Decision Making

Neuroscience > Cognition

Medial and orbital frontal cortex in decision-making and flexible behavior

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Pupil Correlates of Decision Variables in Mice Playing a Competitive Mixed-Strategy Game

In a competitive game involving an animal and an opponent, the outcome is contingent on the choices of both players. To succeed, the animal must continually adapt to competitive pressure, or else risk being exploited and lose out on rewards. In this study, we demonstrate that head-fixed male mice can be trained to play the iterative competitive game "matching pennies" against a virtual computer opponent. We find that the animals' performance is well described by a hybrid computational model that includes Q-learning and choice kernels. Comparing between matching pennies and a non-competitive two-armed bandit task, we show that the tasks encourage animals to operate at different regimes of reinforcement learning. To understand the involvement of neuromodulatory mechanisms, we measure fluctuations in pupil size and use multiple linear regression to relate the trial-by-trial transient pupil responses to decision-related variables. The analysis reveals that pupil responses are modulated by observable variables, including choice and outcome, as well as latent variables for value updating, but not action selection. Collectively, these results establish a paradigm for studying competitive decision-making in head-fixed mice and provide insights into the role of arousal-linked neuromodulation in the decision process.

Comparative connectomics of the primate social brain

Social interaction is thought to provide a selection pressure for human intelligence, yet little is known about its neurobiological basis and evolution throughout the primate lineage. Recent advances in neuroimaging have enabled whole brain investigation of brain structure, function, and connectivity in humans and non-human primates (NHPs), leading to a nascent field of comparative connectomics. However, linking social behavior to brain organization across the primates remains challenging. Here, we review the current understanding of the macroscale neural mechanisms of social behaviors from the viewpoint of system neuroscience. We first demonstrate an association between the number of cortical neurons and the size of social groups across primates, suggesting a link between neural information-processing capacity and social capabilities. Moreover, by capitalizing on recent advances in species-harmonized functional MRI, we demonstrate that portions of the mirror neuron system and default-mode networks, which are thought to be important for representation of the other's actions and sense of self, respectively, exhibit similarities in functional organization in macaque monkeys and humans, suggesting possible homologies. With respect to these two networks, we describe recent developments in the neurobiology of social perception, joint attention, personality and social complexity. Together, the Human Connectome Project (HCP)-style comparative neuroimaging, hyperscanning, behavioral, and other multi-modal investigations are expected to yield important insights into the evolutionary foundations of human social behavior.

Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning, and Attention-augmented Reinforcement Learning

Flexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we aim to disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six monkeys is consistently best predicted with a model combining (i) fast working memory and (ii) slower reinforcement learning from differently weighted positive and negative prediction errors as well as (iii) selective suppression of nonchosen feature values and (iv) a meta-learning mechanism that enhances exploration rates based on a memory trace of recent errors. The optimal model parameter settings suggest that these mechanisms cooperate differently at low and high attentional loads. Whereas working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and suggest how they may cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.

Dissecting functional contributions of the social brain to strategic behavior

Social interactions routinely lead to neural activity in a "social brain network" comprising, among other regions, the temporoparietal junction (TPJ) and the dorsomedial prefrontal cortex (dmPFC). But what is the function of these areas? Are they specialized for behavior in social contexts or do they implement computations required for dealing with any reactive process, even non-living entities? Here, we use fMRI and a game paradigm separating the need for these two aspects of cognition. We find that most social-brain areas respond to both social and non-social reactivity rather than just to human opponents. However, the TPJ shows a dissociation from the dmPFC: its activity and connectivity primarily reflect context-dependent outcome processing and reactivity detection, while dmPFC engagement is linked to implementation of a behavioral strategy. Our results characterize an overarching computational property of the social brain but also suggest specialized roles for subregions of this network.

If I do A, B will happen: Dissecting circuits detecting causal relations between actions and outcomes in marmoset prefrontal cortex

In this issue of Neuron, Duan et al. (2021) use pharmacological manipulation to reveal opposing influences of anterior cingulate and orbitofrontal cortex of marmosets on decisions that are based on action-outcome associations.

Computational modelling of social cognition and behaviour-a reinforcement learning primer

Social neuroscience aims to describe the neural systems that underpin social cognition and behaviour. Over the past decade, researchers have begun to combine computational models with neuroimaging to link social computations to the brain. Inspired by approaches from reinforcement learning theory, which describes how decisions are driven by the unexpectedness of outcomes, accounts of the neural basis of prosocial learning, observational learning, mentalizing and impression formation have been developed. Here we provide an introduction for researchers who wish to use these models in their studies. We consider both theoretical and practical issues related to their implementation, with a focus on specific examples from the field.

Human dorsomedial prefrontal cortex delineates the self and other against the tendency to form interdependent social representations

In this issue of Neuron, using non-invasive brain stimulation, Wittmann et al. (2021) highlight a causal role of the dorsomedial prefrontal cortex in keeping separate estimations for the self and others, protecting against a default human tendency to form interdependent social representations.

Causal manipulation of self-other mergence in the dorsomedial prefrontal cortex

To navigate social environments, people must simultaneously hold representations about their own and others’ abilities. During self-other mergence, people estimate others’ abilities not only on the basis of the others’ past performance, but the estimates are also influenced by their own performance. For example, if we perform well, we overestimate the abilities of those with whom we are co-operating and underestimate competitors. Self-other mergence is associated with specific activity patterns in the dorsomedial prefrontal cortex (dmPFC). Using a combination of non-invasive brain stimulation, functional magnetic resonance imaging, and computational modeling, we show that dmPFC neurostimulation silences these neural signatures of self-other mergence in relation to estimation of others’ abilities. In consequence, self-other mergence behavior increases, and our assessments of our own performance are projected increasingly onto other people. This suggests an inherent tendency to form interdependent social representations and a causal role of the dmPFC in separating self and other representations.

•

During self-other mergence (SOM), people confuse one’s own with another’s performance

•

Brain stimulation over dorsomedial prefrontal cortex (dmPFC) alters neural SOM

•

Brain stimulation over dmPFC simultaneously alters behavioral SOM

•

This suggests a causal role of dmPFC in separating self and other representations

From the field to the lab and back: neuroethology of primate social behavior

Social mammals with more numerous and stronger social relationships live longer, healthier lives. Despite the established importance of social relationships, our understanding of the neurobiological mechanisms by which they are pursued, formed, and maintained in primates remains largely confined to highly controlled laboratory settings which do not allow natural, dynamic social interactions to unfold. In this review, we argue that the neurobiological study of primate social behavior would benefit from adopting a neuroethological approach, that is, a perspective grounded in natural, species-typical behavior, with careful selection of animal models according to the scientific question at hand. We highlight macaques and marmosets as key animal models for human social behavior and summarize recent findings in the social domain for both species. We then review pioneering studies of dynamic social behaviors in small animals, which can inspire studies in larger primates where the technological landscape is now ripe for an ethological overhaul.

The anterior cingulate cortex directs exploration of alternative strategies

The ability to adjust one's behavioral strategy in complex environments is at the core of cognition. Doing so efficiently requires monitoring the reliability of the ongoing strategy and, when appropriate, switching away from it to evaluate alternatives. Studies in humans and non-human primates have uncovered signals in the anterior cingulate cortex (ACC) that reflect the pressure to switch away from the ongoing strategy, whereas other ACC signals relate to the pursuit of alternatives. However, whether these signals underlie computations that actually underpin strategy switching or merely reflect tracking of related variables remains unclear. Here we provide causal evidence that the rodent ACC actively arbitrates between persisting with the ongoing behavioral strategy and temporarily switching away to re-evaluate alternatives. Furthermore, by individually perturbing distinct output pathways, we establish that the two associated computations-determining whether to switch strategy and committing to the pursuit of a specific alternative-are segregated in the ACC microcircuitry.

Reading between the lines: Listener's vmPFC simulates speaker cooperative choices in communication games

Humans have a remarkable ability to understand what is and is not being said by conversational partners. It has been hypothesized that listeners decode the intended meaning of a communicative signal by assuming speakers speak cooperatively, rationally simulating the speaker's choice process and inverting it to recover the speaker's most probable meaning. We investigated whether and how rational simulations of speakers are represented in the listener's brain, by combining referential communication games with functional neuroimaging. We show that listeners' ventromedial prefrontal cortex encodes the probabilistic inference of what a cooperative speaker should say given a communicative goal and context, even when such inferences are irrelevant for reference resolution. The listener's striatum encodes the amount of update on intended meaning, consistent with inverting a simulated mental model. These findings suggest a neural generative mechanism, subserved by the frontal-striatal circuits, that underlies our ability to understand communicative and, more generally, social actions.

Learning, exploitation and bias in games

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.

Social processing by the primate medial frontal cortex

The primate medial frontal cortex is comprised of several brain regions that are consistently implicated in regulating complex social behaviors. The medial frontal cortex is also critically involved in many non-social behaviors, such as those involved in reward, affective, and decision-making processes, broadly implicating the fundamental role of the medial frontal cortex in internally guided cognition. An essential question therefore is what unique contributions, if any, does the medial frontal cortex make to social behaviors? In this chapter, we outline several neural algorithms necessary for mediating adaptive social interactions and discuss selected evidence from behavioral neurophysiology experiments supporting the role of the medial frontal cortex in implementing these algorithms. By doing so, we primarily focus on research in nonhuman primates and examine several key attributes of the medial frontal cortex. Specifically, we review neuronal substrates in the medial frontal cortex uniquely suitable for enabling social monitoring, observational and vicarious learning, as well as predicting the behaviors of social partners. Moreover, by utilizing the three levels of organization in information processing systems proposed by Marr (1982) and recently adapted by Lockwood, Apps, and Chang (2020) for social information processing, we survey selected social functions of the medial frontal cortex through the lens of socially relevant algorithms and implementations. Overall, this chapter provides a broad overview of the behavioral neurophysiology literature endorsing the importance of socially relevant neural algorithms implemented by the primate medial frontal cortex for regulating social interactions.

Impact of internal and external factors on prosocial choices in rhesus macaques

While traditional economic models assume that agents are self-interested, humans and most non-human primates are social species. Therefore, many of decisions they make require the integration of information about other social agents. This study asks to what extent information about social status and the social context in which decisions are taken impact on reward-guided decisions in rhesus macaques. We tested 12 monkeys of varying dominance status in several experimental versions of a two-choice task in which reward could be delivered to self only, only another monkey, both the self and another monkey, or neither. Results showed dominant animals were more prone to make prosocial choices than subordinates, but only when the decision was between a reward for self only and a reward for both self and other. If the choice was between a reward for self only and a reward for other only, no animal expressed altruistic behaviour. Finally, prosocial choices were true social decisions as they were strikingly reduced when the social partner was replaced by a non-social object. These results showed that as in humans, rhesus macaques' social decisions are adaptive and modulated by social status and the cost associated with being prosocial. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Activation and disruption of a neural mechanism for novel choice in monkeys

Neural mechanisms that mediate the ability to make value-guided decisions have received substantial attention in humans and animals^1-6. Experiments in animals typically involve long training periods. By contrast, choices in the real world often need to be made between new options spontaneously. It is therefore possible that the neural mechanisms targeted in animal studies differ from those required for new decisions, which are typical of human imaging studies. Here we show that the primate medial frontal cortex (MFC)⁷ is involved in making new inferential choices when the options have not been previously experienced. Macaques spontaneously inferred the values of new options via similarities with the component parts of previously encountered options. Functional magnetic resonance imaging (fMRI) suggested that this ability was mediated by the MFC, which is rarely investigated in monkeys³; MFC activity reflected different processes of comparison for unfamiliar and familiar options. Multidimensional representations of options in the MFC used a coding scheme resembling that of grid cells, which is well known in spatial navigation^8,9, to integrate dimensions in this non-physical space¹⁰ during novel decision-making. By contrast, the orbitofrontal cortex held specific object-based value representations^1,11. In addition, minimally invasive ultrasonic disruption¹² of MFC, but not adjacent tissue, altered the estimation of novel choice values.

The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection

Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions.

Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex

Everyday life often requires arbitrating between pursuing an ongoing action plan by possibly adjusting it versus exploring a new action plan instead. Resolving this so-called exploitation-exploration dilemma involves the medial prefrontal cortex (mPFC). Using human intracranial electrophysiological recordings, we discovered that neural activity in the ventral mPFC infers and tracks the reliability of the ongoing plan to proactively encode upcoming action outcomes as either learning signals or potential triggers to explore new plans. By contrast, the dorsal mPFC exhibits neural responses to action outcomes, which results in either improving or abandoning the ongoing plan. Thus, the mPFC resolves the exploitation-exploration dilemma through a two-stage, predictive coding process: a proactive ventromedial stage that constructs the functional signification of upcoming action outcomes and a reactive dorsomedial stage that guides behavior in response to action outcomes.

Self-other overlap and interpersonal neural synchronization serially mediate the effect of behavioral synchronization on prosociality

Behavioral synchronization has been found to facilitate social bonding and prosociality but the neural mechanisms underlying such effects are not well understood. In the current study, 60 dyads were hyperscanned using functional near-infrared spectroscopy while they performed either a synchronous key-pressing task or a control task. After the task, they were asked to perform the dictator game to assess their prosocial behavior. We also measured three potential mediating variables: self–other overlap, perceived similarity and interpersonal neural synchronization. Results showed that dyads in the synchronization group were higher in behavioral synchronization, interpersonal neural synchronization (INS) at the right dorsolateral prefrontal cortex, self–other overlap, perceived similarity and prosociality than those in the control group. INS was significantly associated with prosocial behaviors and self–other overlap. After testing four meditation models, we found that self–other overlap and INS played a serial mediation role in the effect of behavioral synchronization on prosociality. These results contribute to our understanding of the neural and cognitive mechanisms underlying the effect of behavioral synchronization on prosocial behavior.

Stability or Plasticity? - A Hierarchical Allostatic Regulation Model of Medial Prefrontal Cortex Function for Social Valuation

The medial prefrontal cortex (mPFC) has long been recognized as the key component of the neurocircuitry involved in various social as well as non-social behaviors, however, little is known regarding the organizing principle of distinctive subregions in the mPFC that integrates a wide range of mPFC functions. The present study proposes a hierarchical model of mPFC functionality, where three functionally dissociable subregions, namely, the ventromedial prefrontal cortex (vmPFC), rostromedial prefrontal cortex (rmPFC), and dorsomedial prefrontal cortex (dmPFC), are differentially involved in computing values of decision-making. According to this model, the mPFC subregions interact with each other in such a way that more dorsal regions utilize additional external sensory information from environment to predict and prevent conflicts occurring in more ventral regions tuned to internal bodily signals, thereby exerting the hierarchically organized allostatic regulatory control over homeostatic reflexes. This model also emphasizes the role of the thalamic reticular nucleus (TRN) in arbitrating the transitions between different thalamo-cortical loops, detecting conflicts between competing options for decision-making, and in shifting flexibly between decision modes. The hierarchical architecture of the mPFC working in conjunction with the TRN may play a key role in adjusting the internal (bodily) needs to suit the constraints of external (environmental) variables better, thus effectively addressing the stability-plasticity dilemma.

Neural computations underlying strategic social decision-making in groups

When making decisions in groups, the outcome of one's decision often depends on the decisions of others, and there is a tradeoff between short-term incentives for an individual and long-term incentives for the groups. Yet, little is known about the neurocomputational mechanisms at play when weighing different utilities during repeated social interactions. Here, using model-based fMRI and Public-good-games, we find that the ventromedial prefrontal cortex encodes immediate expected rewards as individual utility while the lateral frontopolar cortex encodes group utility (i.e., pending rewards of alternative strategies beneficial for the group). When it is required to change one's strategy, these brain regions exhibited changes in functional interactions with brain regions engaged in switching strategies. Moreover, the anterior cingulate cortex and the temporoparietal junction updated beliefs about the decision of others during interactions. Together, our findings provide a neurocomputational account of how the brain dynamically computes effective strategies to make adaptive collective decisions.

A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans

During competitive interactions, such as predator-prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non-strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed-strategy games, how brain activity differs in the context of strategic interactions versus non-strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed-strategy interactions from non-strategic choices, and we used task-based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed-strategy game, "matching pennies," against a dynamic computer opponent designed to exploit predictability in players' response patterns. Results were contrasted with a non-strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed-strategy game was associated with activation of a distributed cortico-striatal network compared to the non-strategic task. We propose that choosing in mixed-strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.

Subcortical Substrates of Explore-Exploit Decisions in Primates

The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

Bayesian Computation through Cortical Latent Dynamics

Statistical regularities in the environment create prior beliefs that we rely on to optimize our behavior when sensory information is uncertain. Bayesian theory formalizes how prior beliefs can be leveraged and has had a major impact on models of perception, sensorimotor function, and cognition. However, it is not known how recurrent interactions among neurons mediate Bayesian integration. By using a time-interval reproduction task in monkeys, we found that prior statistics warp neural representations in the frontal cortex, allowing the mapping of sensory inputs to motor outputs to incorporate prior statistics in accordance with Bayesian inference. Analysis of recurrent neural network models performing the task revealed that this warping was enabled by a low-dimensional curved manifold and allowed us to further probe the potential causal underpinnings of this computational strategy. These results uncover a simple and general principle whereby prior beliefs exert their influence on behavior by sculpting cortical latent dynamics.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

Understanding psychiatric disorder by capturing ecologically relevant features of learning and decision-making

Recent research in cognitive neuroscience has begun to uncover the processes underlying increasingly complex voluntary behaviours, including learning and decision-making. Partly this success has been possible by progressing from simple experimental tasks to paradigms that incorporate more ecological features. More specifically, the premise is that to understand cognitions and brain functions relevant for real life, we need to introduce some of the ecological challenges that we have evolved to solve. This often entails an increase in task complexity, which can be managed by using computational models to help parse complex behaviours into specific component mechanisms. Here we propose that using computational models with tasks that capture ecologically relevant learning and decision-making processes may provide a critical advantage for capturing the mechanisms underlying symptoms of disorders in psychiatry. As a result, it may help develop mechanistic approaches towards diagnosis and treatment. We begin this review by mapping out the basic concepts and models of learning and decision-making. We then move on to consider specific challenges that emerge in realistic environments and describe how they can be captured by tasks. These include changes of context, uncertainty, reflexive/emotional biases, cost-benefit decision-making, and balancing exploration and exploitation. Where appropriate we highlight future or current links to psychiatry. We particularly draw examples from research on clinical depression, a disorder that greatly compromises motivated behaviours in real-life, but where simpler paradigms have yielded mixed results. Finally, we highlight several paradigms that could be used to help provide new insights into the mechanisms of psychiatric disorders.

The application of computational models to social neuroscience: promises and pitfalls

Interactions with conspecifics are key to any social species. In order to navigate this social world, it is crucial for individuals to learn from and about others. From learning new skills by observing parents perform them to making complex collective decisions, understanding the mechanisms underlying social cognitive processes has been of considerable interest to psychologists and neuroscientists. Here, we review studies that have used computational modelling techniques, combined with neuroimaging, to shed light on how people learn and make decisions in social contexts. As opposed to standard social neuroscience methods, the computational approach allows one to directly examine where in the brain particular computations, as estimated by models of behavior, are implemented. Findings suggest that people use several strategies to learn from others: vicarious reward learning, where one learns from observing the reward outcomes of another agent; action imitation, which relies on encoding a prediction error between the expected and actual actions of the other agent; and social inference, where one learns by inferring the goals and intentions of others. These computations are implemented in distinct neural networks, which may be recruited adaptively depending on task demands, the environment and other social factors.

Ventral anterior cingulate cortex and social decision-making

Studies in the field of social neuroscience have recently made use of computational models of decision-making to provide new insights into how we learn about the self and others during social interactions. Importantly, these studies have increasingly drawn attention to brain areas outside of classical cortical "social brain" regions that may be critical for social processing. In particular, two portions of the ventral anterior cingulate cortex (vACC), subgenual anterior cingulate cortex and perigenual anterior cingulate cortex, have been linked to social and self learning signals, respectively. Here we discuss the emerging parallels between these studies. Uncovering the function of vACC during social interactions could provide important new avenues to understand social decision-making in health and disease.

On the Flexibility of Basic Risk Attitudes in Monkeys

Monkeys and other animals appear to share with humans two risk attitudes predicted by prospect theory: an inverse-S-shaped probability-weighting (PW) function and a steeper utility curve for losses than for gains. These findings suggest that such preferences are stable traits with common neural substrates. We hypothesized instead that animals tailor their preferences to subtle changes in task contexts, making risk attitudes flexible. Previous studies used a limited number of outcomes, trial types, and contexts. To gain a broader perspective, we examined two large datasets of male macaques' risky choices: one from a task with real (juice) gains and another from a token task with gains and losses. In contrast to previous findings, monkeys were risk seeking for both gains and losses (i.e., lacked a reflection effect) and showed steeper gain than loss curves (loss seeking). Utility curves for gains were substantially different in the two tasks. Monkeys showed nearly linear PWs in one task and S-shaped ones in the other; neither task produced a consistent inverse-S-shaped curve. To account for these observations, we developed and tested various computational models of the processes involved in the construction of reward value. We found that adaptive differential weighting of prospective gamble outcomes could partially account for the observed differences in the utility functions across the two experiments and thus provide a plausible mechanism underlying flexible risk attitudes. Together, our results support the idea that risky choices are constructed flexibly at the time of elicitation and place important constraints on neural models of economic choice.SIGNIFICANCE STATEMENT We respond in reliable ways to risk, but are our risk preferences stable traits or ephemeral states? Using various computational models, we examined two large datasets of macaque risky choices in two different tasks. We observed several deviations from "classic" risk preferences seen in humans and monkeys: no reflection effect, loss seeking as opposed to loss aversion, and linear and S-shaped, as opposed to inverse-S-shaped, probability distortion. These results challenge the idea that our risk attitudes are evolved traits shared with the last common ancestor of macaques and humans, suggesting instead that behavioral flexibility is the hallmark of risky choice in primates. We show how this flexibility can emerge partly as a result of interactions between attentional and reward systems.

Neural Mechanisms of Social Cognition in Primates

Activity in a network of areas spanning the superior temporal sulcus, dorsomedial frontal cortex, and anterior cingulate cortex is concerned with how nonhuman primates negotiate the social worlds in which they live. Central aspects of these circuits are retained in humans. Activity in these areas codes for primates' interactions with one another, their attempts to find out about one another, and their attempts to prevent others from finding out too much about themselves. Moreover, important features of the social world, such as dominance status, cooperation, and competition, modulate activity in these areas. We consider the degree to which activity in these regions is simply encoding an individual's own actions and choices or whether this activity is especially and specifically concerned with social cognition. Recent advances in comparative anatomy and computational modeling may help us to gain deeper insights into the nature and boundaries of primate social cognition.

Social Observation Increases Functional Segregation between MPFC Subregions Predicting Prosocial Consumer Decisions

Although it is now well documented that observation by others can be a powerful elicitor of prosocial behaviour, the underlying neural mechanism is yet to be explored. In the present fMRI study, we replicated the previously reported observer effect in ethical consumption, in that participants were more likely to purchase social products that are sold to support people in need than non-social products when being observed by others. fMRI data revealed that the anterior cingulate cortex (ACC) and the dorsomedial prefrontal cortex (dmPFC) encoded subject-specific value parameters of purchase decisions for social and non-social products, respectively, under social observation. The ACC showed strong functional coupling with the amygdala and the anterior insula when participants in the observation condition were making purchases of social versus non-social products. Finally, ventromedial prefrontal cortex (vmPFC) activity predicted faster reaction time and increased prosocial behavior during decisions to purchase social versus non-social products, regardless of social observation. The present findings suggest that subregions of the mPFC, namely the dmPFC, ACC, and vmPFC, are hierarchically organized to encode different levels of decision values from the value of context-sensitive reputation to that of internalized prosociality.

Gender differences in neural correlates of stress-induced anxiety

Although gender differences have been identified as a crucial factor for understanding stress-related anxiety and associated clinical disorders, the neural mechanisms underlying these differences remain unclear. To explore gender differences in the neural correlates of stress-induced anxiety, the current study used functional magnetic resonance imaging to examine brain responses in 96 healthy men and women with commensurable levels of trait anxiety as they engaged in a personalized guided imagery paradigm to provoke stress and neutral-relaxing experiences. During the task, a significant gender main effect emerged, with men displaying greater responses in the caudate, cingulate gyrus, midbrain, thalamus, and cerebellum. In contrast, women showed greater responses in the posterior insula, temporal gyrus, and occipital lobe. Additionally, a significant anxiety ratings × gender interaction from whole-brain regression analyses was observed in the dorsomedial prefrontal cortex, left inferior parietal lobe, left temporal gyrus, occipital gyrus, and cerebellum (P < 0.05, whole-brain family-wise error corrected), with positive associations between activity in these regions and stress-induced anxiety in women, but negative associations in men, indicating that men and women differentially use neural resources when experiencing stress-induced anxiety. The findings suggest that in response to stress, there is a greater use of the medial prefrontal-parietal cortices in experiencing subjective anxiety in women, while decreased use of this circuit was associated with increased subjective anxiety states in men. The current study has implications for understanding gender-specific differences in stress-induced anxiety and vulnerability to stress-related clinical disorders, and for developing more effective treatment strategies tailored to each gender. © 2016 Wiley Periodicals, Inc.

Adaptive coordination of working-memory and reinforcement learning in non-human primates performing a trial-and-error problem solving task

Accumulating evidence suggest that human behavior in trial-and-error learning tasks based on decisions between discrete actions may involve a combination of reinforcement learning (RL) and working-memory (WM). While the understanding of brain activity at stake in this type of tasks often involve the comparison with non-human primate neurophysiological results, it is not clear whether monkeys use similar combined RL and WM processes to solve these tasks. Here we analyzed the behavior of five monkeys with computational models combining RL and WM. Our model-based analysis approach enables to not only fit trial-by-trial choices but also transient slowdowns in reaction times, indicative of WM use. We found that the behavior of the five monkeys was better explained in terms of a combination of RL and WM despite inter-individual differences. The same coordination dynamics we used in a previous study in humans best explained the behavior of some monkeys while the behavior of others showed the opposite pattern, revealing a possible different dynamics of WM process. We further analyzed different variants of the tested models to open a discussion on how the long pretraining in these tasks may have favored particular coordination dynamics between RL and WM. This points towards either inter-species differences or protocol differences which could be further tested in humans.

Neural activity in macaque medial frontal cortex represents others' choices

Predicting the behavior of others is a fundamental skill in primate social life. We tested the role of medial frontal cortex in the prediction of other agents’ behavior in two male macaques, using a monkey-human interactive task in which their actor-observer roles were intermixed. In every trial, the observer monitored the actor’s choice to reject it for a different one when he became the actor on the subsequent trial. In the delay period preceding the action, we identified neurons modulated by the agent’s identity, as well as a group of neurons encoding the agent’s future choice, some of which were neurons that showed differential patterns of activity between agents. The ability of these neurons to flexibly move from ‘self-oriented’ to ‘other-oriented’ representations could correspond to the “other side of the coin” of the simulative mirroring activity. Neurons that changed coding scheme, together with neurons exclusively involved in the prediction of the other agent’s choice, show a neural substrate for predicting or anticipating others’ choices beyond simulation.

Self-Other Mergence in the Frontal Cortex during Cooperation and Competition

To survive, humans must estimate their own ability and the abilities of others. We found that, although people estimated their abilities on the basis of their own performance in a rational manner, their estimates of themselves were partly merged with the performance of others. Reciprocally, their ability estimates for others also reflected their own, as well as the others', performance. Self-other mergence operated in a context-dependent manner: interacting with high or low performers, respectively, enhanced and diminished own ability estimates in cooperative contexts, but the opposite occurred in competitive contexts. Self-other mergence not only influenced subjective evaluations, it also affected how people subsequently objectively adjusted their performance. Perigenual anterior cingulate cortex tracked one's own performance. Dorsomedial frontal area 9 tracked others' performances, but also integrated contextual and self-related information. Self-other mergence increased with the strength of self and other representations in area 9, suggesting it carries interdependent representations of self and other.

Contrasting Effects of Medial and Lateral Orbitofrontal Cortex Lesions on Credit Assignment and Decision-Making in Humans

The orbitofrontal cortex is critical for goal-directed behavior. Recent work in macaques has suggested the lateral orbitofrontal cortex (lOFC) is relatively more concerned with assignment of credit for rewards to particular choices during value-guided learning, whereas the medial orbitofrontal cortex (often referred to as ventromedial prefrontal cortex in humans; vmPFC/mOFC) is involved in constraining the decision to the relevant options. We examined whether people with damage restricted to subregions of prefrontal cortex showed the patterns of impairment observed in prior investigations of the effects of lesions to homologous regions in macaques. Groups of patients with either lOFC (predominantly right hemisphere), mOFC/vmPFC, or dorsomedial prefrontal (DMF), and a comparison group of healthy age- and education-matched controls performed a probabilistic 3-choice decision-making task. We report anatomically specific patterns of impairment. We found that credit assignment, as indexed by the normal influence of contingent relationships between choice and reward, is reduced in lOFC patients compared with Controls and mOFC/vmPFC patients. Moreover, the effects of reward contingency on choice were similar for patients with lesions in DMF or mOFC/vmPFC, compared with Controls. By contrast, mOFC/vmPFC-lesioned patients made more stochastic choices than Controls when the decision was framed by valuable distracting alternatives, suggesting that value comparisons were no longer independent of irrelevant options. Once again, there was evidence of regional specialization: patients with lOFC lesions were unimpaired relative to Controls. As in macaques, human lOFC and mOFC/vmPFC are necessary for contingent learning and value-guided decision-making, respectively.SIGNIFICANCE STATEMENT The lateral and medial regions of the orbitofrontal cortex are cytoarchitectonically distinct and have different anatomical connections. Previous investigations in macaques have shown these anatomical differences are accompanied by functional specialization for learning and decision-making. Here, for the first time, we test the predictions made by macaque studies in an experiment with humans with frontal lobe lesions, asking whether behavioral impairments can be linked to lateral or medial orbitofrontal cortex. Using equivalent tasks and computational analyses, our findings broadly replicate the pattern reported after selective lesions in monkeys. Patients with lateral orbitofrontal damage had impaired credit assignment, whereas damage to medial orbitofrontal cortex meant that patients were more likely to be distracted by irrelevant options.

Social learning through prediction error in the brain

Learning about the world is critical to survival and success. In social animals, learning about others is a necessary component of navigating the social world, ultimately contributing to increasing evolutionary fitness. How humans and nonhuman animals represent the internal states and experiences of others has long been a subject of intense interest in the developmental psychology tradition, and, more recently, in studies of learning and decision making involving self and other. In this review, we explore how psychology conceptualizes the process of representing others, and how neuroscience has uncovered correlates of reinforcement learning signals to explore the neural mechanisms underlying social learning from the perspective of representing reward-related information about self and other. In particular, we discuss self-referenced and other-referenced types of reward prediction errors across multiple brain structures that effectively allow reinforcement learning algorithms to mediate social learning. Prediction-based computational principles in the brain may be strikingly conserved between self-referenced and other-referenced information.

Computational Properties of the Hippocampus Increase the Efficiency of Goal-Directed Foraging through Hierarchical Reinforcement Learning

The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide “goal-directed” behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals' ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, “forward sweeps” through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks.