Insights from the application of computational neuroimaging to social neuroscience

Humans can infer social preferences from decision speed alone

Humans are known to be capable of inferring hidden preferences and beliefs of their conspecifics when observing their decisions. While observational learning based on choices has been explored extensively, the question of how response times (RT) impact our learning of others' social preferences has received little attention. Yet, while observing choices alone can inform us about the direction of preference, they reveal little about the strength of this preference. In contrast, RT provides a continuous measure of strength of preference with faster responses indicating stronger preferences and slower responses signaling hesitation or uncertainty. Here, we outline a preregistered orthogonal design to investigate the involvement of both choices and RT in learning and inferring other's social preferences. Participants observed other people's behavior in a social preferences task (Dictator Game), seeing either their choices, RT, both, or no information. By coupling behavioral analyses with computational modeling, we show that RT is predictive of social preferences and that observers were able to infer those preferences even when receiving only RT information. Based on these findings, we propose a novel observational reinforcement learning model that closely matches participants' inferences in all relevant conditions. In contrast to previous literature suggesting that, from a Bayesian perspective, people should be able to learn equally well from choices and RT, we show that observers' behavior substantially deviates from this prediction. Our study elucidates a hitherto unknown sophistication in human observational learning but also identifies important limitations to this ability.

Observational reinforcement learning in children and young adults

Observational learning is essential for the acquisition of new behavior in educational practices and daily life and serves as an important mechanism for human cognitive and social-emotional development. However, we know little about its underlying neurocomputational mechanisms from a developmental perspective. In this study we used model-based fMRI to investigate differences in observational learning and individual learning between children and younger adults. Prediction errors (PE), the difference between experienced and predicted outcomes, related positively to striatal and ventral medial prefrontal cortex activation during individual learning and showed no age-related differences. PE-related activation during observational learning was more pronounced when outcomes were worse than predicted. Particularly, negative PE-coding in the dorsal medial prefrontal cortex was stronger in adults compared to children and was associated with improved observational learning in children and adults. The current findings pave the way to better understand observational learning challenges across development and educational settings.

A neuro-computational social learning framework to facilitate transdiagnostic classification and treatment across psychiatric disorders

Social deficits are among the core and most striking psychiatric symptoms, present in most psychiatric disorders. Here, we introduce a novel social learning framework, which consists of neuro-computational models that combine reinforcement learning with various types of social knowledge structures. We outline how this social learning framework can help specify and quantify social psychopathology across disorders and provide an overview of the brain regions that may be involved in this type of social learning. We highlight how this framework can specify commonalities and differences in the social psychopathology of individuals with autism spectrum disorder (ASD), personality disorders (PD), and major depressive disorder (MDD) and improve treatments on an individual basis. We conjecture that individuals with psychiatric disorders rely on rigid social knowledge representations when learning about others, albeit the nature of their rigidity and the behavioral consequences can greatly differ. While non-clinical cohorts tend to efficiently adapt social knowledge representations to relevant environmental constraints, psychiatric cohorts may rigidly stick to their preconceived notions or overly coarse knowledge representations during learning.

Mind the gender gap: The social neuroscience of belonging

Gender gaps persist in the 21st century, in many aspects of society and in many types of organisation. There are earnings gaps in almost all domains, reports of glass ceilings and the “missing middle” in business, finance, law and politics, and dramatic under-representation of women in many branches of science, even in the most “gender equal” countries. This is despite decades of effort to address them, including targeted legislation and many Diversity and Inclusion initiatives. Early essentialist, competence-based explanations for the existence of gender gaps have been largely discredited at the research level, although their persistence in the public consciousness and at the level of education and training can still negatively bias both individual self-belief and organisational processes. Contemporary essentialist explanations are now emerging, with claims that such gaps are the manifestations of the presence or absence of endogenous, brain-based characteristics underpinning career progression or career preferences. The focus remains on the individual as the source of gender imbalances. Less attention has been paid to the contextual aspects of organisations where gender gaps are evident, to inclusion (or the lack of it), or the availability of unbiased reward and progression pathways. Advances in 21st century social cognitive neuroscience are revealing the importance of external organisational processes as powerful brain-changing forces, with their potentially negative impact on self-belief and a sense of belonging. Key research is demonstrating the cortical and behavioural consequences of negative social experiences, with the activation of core inhibitory pathways associated with low self-esteem, lack of engagement, and eventual withdrawal. This paper will argue that reference to such research will provide better explanations for the persistence of gender gaps, and offer evidence-based insights into addressing gender gap issues. Importantly, this is not a rejection of an endogenous, brain-based explanation for gender gaps but the elaboration of a better-informed 21st century model, flagging up the need to take factors such as cultural stereotyping and organisational bias into account in any drive toward true gender equity, or genuinely levelled playing fields.

Neurocomputational mechanism of real-time distributed learning on social networks

Social networks shape our decisions by constraining what information we learn and from whom. Yet, the mechanisms by which network structures affect individual learning and decision-making remain unclear. Here, by combining a real-time distributed learning task with functional magnetic resonance imaging, computational modeling and social network analysis, we studied how humans learn from observing others' decisions on seven-node networks with varying topological structures. We show that learning on social networks can be approximated by a well-established error-driven process for observational learning, supported by an action prediction error encoded in the lateral prefrontal cortex. Importantly, learning is flexibly weighted toward well-connected neighbors, according to activity in the dorsal anterior cingulate cortex, but only insofar as social observations contain secondhand, potentially intertwining, information. These data suggest a neurocomputational mechanism of network-based filtering on the sources of information, which may give rise to biased learning and the spread of misinformation in an interconnected society.

Neurocomputational mechanisms of young children's observational learning of delayed gratification

The ability to delay gratification is crucial for a successful and healthy life. An effective way for young children to learn this ability is to observe the action of adult models. However, the underlying neurocomputational mechanism remains unknown. Here, we tested the hypotheses that children employed either the simple imitation strategy or the goal-inference strategy when learning from adult models in a high-uncertainty context. Results of computational modeling indicated that children used the goal-inference strategy regardless of whether the adult model was their mother or a stranger. At the neural level, results showed that successful learning of delayed gratification was associated with enhanced interpersonal neural synchronization (INS) between children and the adult models in the dorsal lateral prefrontal cortex but was not associated with children's own single-brain activity. Moreover, the discounting of future reward's value obtained from computational modeling of the goal-inference strategy was positively correlated with the strength of INS. These findings from our exploratory study suggest that, even for 3-year-olds, the goal-inference strategy is used to learn delayed gratification from adult models, and the learning strategy is associated with neural interaction between the brains of children and adult models.

Incorporating social knowledge structures into computational models

To navigate social interactions successfully, humans need to continuously learn about the personality traits of other people (e.g., how helpful or aggressive is the other person?). However, formal models that capture the complexities of social learning processes are currently lacking. In this study, we specify and test potential strategies that humans can employ for learning about others. Standard Rescorla-Wagner (RW) learning models only capture parts of the learning process because they neglect inherent knowledge structures and omit previously acquired knowledge. We therefore formalize two social knowledge structures and implement them in hybrid RW models to test their usefulness across multiple social learning tasks. We name these concepts granularity (knowledge structures about personality traits that can be utilized at different levels of detail during learning) and reference points (previous knowledge formalized into representations of average people within a social group). In five behavioural experiments, results from model comparisons and statistical analyses indicate that participants efficiently combine the concepts of granularity and reference points-with the specific combinations in models depending on the people and traits that participants learned about. Overall, our experiments demonstrate that variants of RW algorithms, which incorporate social knowledge structures, describe crucial aspects of the dynamics at play when people interact with each other.

Neurocomputations of strategic behavior: From iterated to novel interactions

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge.

This article is categorized under:

Economics > Interactive Decision‐Making

Psychology > Reasoning and Decision Making

Neuroscience > Cognition

Neural Processing of Facial Attractiveness and Romantic Love: An Overview and Suggestions for Future Empirical Studies

Romantic love is universally observed in human communities, and the manner in which a person chooses a long-term romantic partner has been a central question in studies on close relationships. Numerous empirical psychological studies have demonstrated that facial attractiveness greatly impacts initial romantic attraction. This close link was further investigated by neuroimaging studies showing that both viewing attractive faces and having romantic thoughts recruit the reward system. However, it remains unclear how our brains integrate perceived facial attractiveness into initial romantic attraction. In addition, it remains unclear how our brains shape a persistent attraction to a particular person through interactions; this persistent attraction is hypothesized to contribute to a long-term relationship. After reviewing related studies, I introduce methodologies that could help address these questions.

Characterization of the Core Determinants of Social Influence From a Computational and Cognitive Perspective

Most human decisions are made among social others, and in what social context the choices are made is known to influence individuals' decisions. Social influence has been noted as an important factor that may nudge individuals to take more risks (e.g., initiation of substance use), but ironically also help individuals to take safer actions (e.g., successful abstinence). Such bi-directional impacts of social influence hint at the complexity of social information processing. Here, we first review the recent computational approaches that shed light on neural and behavioral mechanisms underlying social influence following basic computations involved in decision-making: valuation, action selection, and learning. We next review the studies on social influence from various fields including neuroeconomics, developmental psychology, social psychology, and cognitive neuroscience, and highlight three dimensions of determinants-who are the recipients, how the social contexts are presented, and to what domains and processes of decisions the influence is applied-that modulate the extent to which individuals are influenced by others. Throughout the review, we also introduce the brain regions that were suggested as neural instantiations of social influence from a large body of functional neuroimaging studies. Finally, we outline the remaining questions to be addressed in the translational application of computational and cognitive theories of social influence to psychopathology and health.

Seven computations of the social brain

The social environment presents the human brain with the most complex information processing demands. The computations that the brain must perform occur in parallel, combine social and nonsocial cues, produce verbal and nonverbal signals and involve multiple cognitive systems, including memory, attention, emotion and learning. This occurs dynamically and at timescales ranging from milliseconds to years. Here, we propose that during social interactions, seven core operations interact to underwrite coherent social functioning; these operations accumulate evidence efficiently-from multiple modalities-when inferring what to do next. We deconstruct the social brain and outline the key components entailed for successful human-social interaction. These include (i) social perception; (ii) social inferences, such as mentalizing; (iii) social learning; (iv) social signaling through verbal and nonverbal cues; (v) social drives (e.g. how to increase one's status); (vi) determining the social identity of agents, including oneself and (vii) minimizing uncertainty within the current social context by integrating sensory signals and inferences. We argue that while it is important to examine these distinct aspects of social inference, to understand the true nature of the human social brain, we must also explain how the brain integrates information from the social world.

Dorsolateral and dorsomedial prefrontal cortex track distinct properties of dynamic social behavior

Understanding how humans make competitive decisions in complex environments is a key goal of decision neuroscience. Typical experimental paradigms constrain behavioral complexity (e.g. choices in discrete-play games), and thus, the underlying neural mechanisms of dynamic social interactions remain incompletely understood. Here, we collected fMRI data while humans played a competitive real-time video game against both human and computer opponents, and then, we used Bayesian non-parametric methods to link behavior to neural mechanisms. Two key cognitive processes characterized behavior in our task: (i) the coupling of one’s actions to another’s actions (i.e. opponent sensitivity) and (ii) the advantageous timing of a given strategic action. We found that the dorsolateral prefrontal cortex displayed selective activation when the subject’s actions were highly sensitive to the opponent’s actions, whereas activation in the dorsomedial prefrontal cortex increased proportionally to the advantageous timing of actions to defeat one’s opponent. Moreover, the temporoparietal junction tracked both of these behavioral quantities as well as opponent social identity, indicating a more general role in monitoring other social agents. These results suggest that brain regions that are frequently implicated in social cognition and value-based decision-making also contribute to the strategic tracking of the value of social actions in dynamic, multi-agent contexts.

Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

Decision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

Watch and Learn: The Cognitive Neuroscience of Learning from Others' Actions

The mirror neuron system has dominated understanding of observational learning from a cognitive neuroscience perspective. Our review highlights the value of observational learning frameworks that integrate a more diverse and distributed set of cognitive and brain systems, including those implicated in sensorimotor transformations, as well as in more general processes such as executive control, reward, and social cognition. We argue that understanding how observational learning occurs in the real world will require neuroscientific frameworks that consider how visuomotor processes interface with more general aspects of cognition, as well as how learning context and action complexity shape mechanisms supporting learning from watching others.

Mapping social reward and punishment processing in the human brain: A voxel-based meta-analysis of neuroimaging findings using the social incentive delay task

Social rewards or punishments motivate human learning and behaviour, and alterations in the brain circuits involved in the processing of these stimuli have been linked with several neuropsychiatric disorders. However, questions still remain about the exact neural substrates implicated in social reward and punishment processing. Here, we conducted four Anisotropic Effect Size Signed Differential Mapping voxel-based meta-analyses of fMRI studies investigating the neural correlates of the anticipation and receipt of social rewards and punishments using the Social Incentive Delay task. We found that the anticipation of both social rewards and social punishment avoidance recruits a wide network of areas including the basal ganglia, the midbrain, the dorsal anterior cingulate cortex, the supplementary motor area, the anterior insula, the occipital gyrus and other frontal, temporal, parietal and cerebellar regions not captured in previous coordinate-based meta-analysis. We identified decreases in the BOLD signal during the anticipation of both social reward and punishment avoidance in regions of the default-mode network that were missed in individual studies likely due to a lack of power. Receipt of social rewards engaged a robust network of brain regions including the ventromedial frontal and orbitofrontal cortices, the anterior cingulate cortex, the amygdala, the hippocampus, the occipital cortex and the brainstem, but not the basal ganglia. Receipt of social punishments increased the BOLD signal in the orbitofrontal cortex, superior and inferior frontal gyri, lateral occipital cortex and the insula. In contrast to the receipt of social rewards, we also observed a decrease in the BOLD signal in the basal ganglia in response to the receipt of social punishments. Our results provide a better understanding of the brain circuitry involved in the processing of social rewards and punishment. Furthermore, they can inform hypotheses regarding brain areas where disruption in activity may be associated with dysfunctional social incentive processing during disease.

A Neuro-computational Account of Arbitration between Choice Imitation and Goal Emulation during Human Observational Learning

When individuals learn from observing the behavior of others, they deploy at least two distinct strategies. Choice imitation involves repeating other agents' previous actions, whereas emulation proceeds from inferring their goals and intentions. Despite the prevalence of observational learning in humans and other social animals, a fundamental question remains unaddressed: how does the brain decide which strategy to use in a given situation? In two fMRI studies (the second a pre-registered replication of the first), we identify a neuro-computational mechanism underlying arbitration between choice imitation and goal emulation. Computational modeling, combined with a behavioral task that dissociated the two strategies, revealed that control over behavior was adaptively and dynamically weighted toward the most reliable strategy. Emulation reliability, the model's arbitration signal, was represented in the ventrolateral prefrontal cortex, temporoparietal junction, and rostral cingulate cortex. Our replicated findings illuminate the computations by which the brain decides to imitate or emulate others.

Breaking human social decision making into multiple components and then putting them together again

Most of our waking time as human beings is spent interacting with other individuals. In order to make good decisions in this social milieu, it is often necessary to make inferences about the internal states, traits and intentions of others. Recently, some progress has been made toward uncovering the neural computations underlying human social decision-making by combining functional magnetic resonance neuroimaging (fMRI) with computational modeling of behavior. Modeling of behavioral data allows us to identify the key computations necessary for social decision-making and to determine how these computations are integrated. Furthermore, by correlating these variables against neuroimaging data, it has become possible to elucidate where in the brain various computations are implemented. Here we review the current state of knowledge in the domain of social computational neuroscience. Findings to date have emphasized that social decisions are driven by multiple computations conducted in parallel, and implemented in distinct brain regions. We suggest that further progress is going to depend on identifying how and where such variables get integrated in order to yield a coherent behavioral output.

Patients with basal ganglia damage show preserved learning in an economic game

Both basal ganglia (BG) and orbitofrontal cortex (OFC) have been widely implicated in social and non-social decision-making. However, unlike OFC damage, BG pathology is not typically associated with disturbances in social functioning. Here we studied the behavior of patients with focal lesions to either BG or OFC in a multi-strategy competitive game known to engage these regions. We find that whereas OFC patients are significantly impaired, BG patients show intact learning in the economic game. By contrast, when information about the strategic context is absent, both cohorts are significantly impaired. Computational modeling further shows a preserved ability in BG patients to learn by anticipating and responding to the behavior of others using the strategic context. These results suggest that apparently divergent findings on BG contribution to social decision-making may instead reflect a model where higher-order learning processes are dissociable from trial-and-error learning, and can be preserved despite BG damage.

Neuroimaging evidence implicates basal ganglia (BG) in social decision-making, yet causal evidence remains lacking. Here, the authors show that learning in strategic (but not non-strategic) games is spared in patients with BG damage, suggesting social decision making is not fully reliant on the BG.

Spreading inequality: neural computations underlying paying-it-forward reciprocity

People tend to pay the generosity they receive from a person forward to someone else even if they have no chance to reciprocate directly. This phenomenon, known as paying-it-forward (PIF) reciprocity, crucially contributes to the maintenance of a cooperative human society by passing kindness among strangers and has been widely studied in evolutionary biology. To further examine its neural implementation and underlying computations, we used functional magnetic resonance imaging together with computational modeling. In a modified PIF paradigm, participants first received a monetary split (i.e. greedy, equal or generous) from either a human partner or a computer. They then chose between two options involving additional amounts of money to be allocated between themselves and an uninvolved person. Behaviorally, people forward the previously received greed/generosity towards a third person. The social impact of previous treatments is integrated into computational signals in the ventromedial prefrontal cortex and the right temporoparietal junction during subsequent decision making. Our findings provide insights to understand the proximal origin of PIF reciprocity.

The application of computational models to social neuroscience: promises and pitfalls

Interactions with conspecifics are key to any social species. In order to navigate this social world, it is crucial for individuals to learn from and about others. From learning new skills by observing parents perform them to making complex collective decisions, understanding the mechanisms underlying social cognitive processes has been of considerable interest to psychologists and neuroscientists. Here, we review studies that have used computational modelling techniques, combined with neuroimaging, to shed light on how people learn and make decisions in social contexts. As opposed to standard social neuroscience methods, the computational approach allows one to directly examine where in the brain particular computations, as estimated by models of behavior, are implemented. Findings suggest that people use several strategies to learn from others: vicarious reward learning, where one learns from observing the reward outcomes of another agent; action imitation, which relies on encoding a prediction error between the expected and actual actions of the other agent; and social inference, where one learns by inferring the goals and intentions of others. These computations are implemented in distinct neural networks, which may be recruited adaptively depending on task demands, the environment and other social factors.

Social Hierarchy Representation in the Primate Amygdala Reflects the Emotional Ambiguity of Our Social Interactions

Group living can help individuals defend against predators and acquire nutrition. However, conflicts between group members can arise (food sharing, mating, etc), requiring individuals to know the social status of each member to promote survival. In our recent paper, we sought to understand how the brain represents the social status of monkeys living in the same colony. Primates learn the social status of their peers through experience, including observation and direct interactions, just like they learn the rewarding or aversive nature of stimuli that predict different types of reinforcement. Group members may thereby be viewed as differing in value. We found in the amygdala, a brain area specialized for emotion, a neural representation of social hierarchy embedded in the same neuronal ensemble engaged in the assignment of motivational significance to previously neutral stimuli. Interestingly, we found 2 subpopulations of amygdala neurons encoding the social status of individuals in an opposite manner. In response to a stimulus, one population encodes similarly appetitive nonsocial images and dominant monkeys as well as aversive nonsocial stimuli and submissive monkeys. The other population encodes the opposite pattern later in time. This mechanism could reflect the emotional ambiguity we face in social situations as each interaction is potentially positive (eg, food access, protection, promotion) or negative (eg, aggression, bullying).

The Role of Intelligence in Social Learning

Studies in cultural evolution have uncovered many types of social learning strategies that are adaptive in certain environments. The efficiency of these strategies also depends on the individual characteristics of both the observer and the demonstrator. We investigate the relationship between intelligence and the ways social and individual information is utilised to make decisions in an uncertain environment. We measure fluid intelligence and study experimentally how individuals learn from observing the choices of a demonstrator in a 2-armed bandit problem with changing probabilities of a reward. Participants observe a demonstrator with high or low fluid intelligence. In some treatments they are aware of the intelligence score of the demonstrator and in others they are not. Low fluid intelligence individuals imitate the demonstrator more when her fluid intelligence is known than when it is not. Conversely, individuals with high fluid intelligence adjust their use of social information, as the observed behaviour changes, independently of the knowledge of the intelligence of the demonstrator. We provide evidence that intelligence determines how social and individual information is integrated in order to make choices in a changing uncertain environment.

Neural computations underlying inverse reinforcement learning in the human brain

In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.

The involvement of model-based but not model-free learning signals during observational reward learning in the absence of choice

A major open question is whether computational strategies thought to be used during experiential learning, specifically model-based and model-free reinforcement learning, also support observational learning. Furthermore, the question of how observational learning occurs when observers must learn about the value of options from observing outcomes in the absence of choice has not been addressed. In the present study we used a multi-armed bandit task that encouraged human participants to employ both experiential and observational learning while they underwent functional magnetic resonance imaging (fMRI). We found evidence for the presence of model-based learning signals during both observational and experiential learning in the intraparietal sulcus. However, unlike during experiential learning, model-free learning signals in the ventral striatum were not detectable during this form of observational learning. These results provide insight into the flexibility of the model-based learning system, implicating this system in learning during observation as well as from direct experience, and further suggest that the model-free reinforcement learning system may be less flexible with regard to its involvement in observational learning.

Goal-directed, habitual and Pavlovian prosocial behavior

Although prosocial behaviors have been widely studied across disciplines, the mechanisms underlying them are not fully understood. Evidence from psychology, biology and economics suggests that prosocial behaviors can be driven by a variety of seemingly opposing factors: altruism or egoism, intuition or deliberation, inborn instincts or learned dispositions, and utility derived from actions or their outcomes. Here we propose a framework inspired by research on reinforcement learning and decision making that links these processes and explains characteristics of prosocial behaviors in different contexts. More specifically, we suggest that prosocial behaviors inherit features of up to three decision-making systems employed to choose between self- and other- regarding acts: a goal-directed system that selects actions based on their predicted consequences, a habitual system that selects actions based on their reinforcement history, and a Pavlovian system that emits reflexive responses based on evolutionarily prescribed priors. This framework, initially described in the field of cognitive neuroscience and machine learning, provides insight into the potential neural circuits and computations shaping prosocial behaviors. Furthermore, it identifies specific conditions in which each of these three systems should dominate and promote other- or self- regarding behavior.

Neural mechanisms underlying human consensus decision-making

Consensus building in a group is a hallmark of animal societies, yet little is known about its underlying computational and neural mechanisms. Here, we applied a computational framework to behavioral and fMRI data from human participants performing a consensus decision-making task with up to five other participants. We found that participants reached consensus decisions through integrating their own preferences with information about the majority group members' prior choices, as well as inferences about how much each option was stuck to by the other people. These distinct decision variables were separately encoded in distinct brain areas-the ventromedial prefrontal cortex, posterior superior temporal sulcus/temporoparietal junction, and intraparietal sulcus-and were integrated in the dorsal anterior cingulate cortex. Our findings provide support for a theoretical account in which collective decisions are made through integrating multiple types of inference about oneself, others, and environments, processed in distinct brain modules.

Deconstructing and reconstructing theory of mind

Usage of the term 'theory of mind' (ToM) has exploded across fields ranging from developmental psychology to social neuroscience and psychiatry research. However, its meaning is often vague and inconsistent, its biological bases are a subject of debate, and the methods used to study it are highly heterogeneous. Most crucially, its original definition does not permit easy downward translation to more basic processes such as those studied by behavioral neuroscience, leaving the interpretation of neuroimaging results opaque. We argue for a reformulation of ToM through a systematic two-stage approach, beginning with a deconstruction of the construct into a comprehensive set of basic component processes, followed by a complementary reconstruction from which a scientifically tractable concept of ToM can be recovered.

A universal role of the ventral striatum in reward-based learning: evidence from human studies

Reinforcement learning enables organisms to adjust their behavior in order to maximize rewards. Electrophysiological recordings of dopaminergic midbrain neurons have shown that they code the difference between actual and predicted rewards, i.e., the reward prediction error, in many species. This error signal is conveyed to both the striatum and cortical areas and is thought to play a central role in learning to optimize behavior. However, in human daily life rewards are diverse and often only indirect feedback is available. Here we explore the range of rewards that are processed by the dopaminergic system in human participants, and examine whether it is also involved in learning in the absence of explicit rewards. While results from electrophysiological recordings in humans are sparse, evidence linking dopaminergic activity to the metabolic signal recorded from the midbrain and striatum with functional magnetic resonance imaging (fMRI) is available. Results from fMRI studies suggest that the human ventral striatum (VS) receives valuation information for a diverse set of rewarding stimuli. These range from simple primary reinforcers such as juice rewards over abstract social rewards to internally generated signals on perceived correctness, suggesting that the VS is involved in learning from trial-and-error irrespective of the specific nature of provided rewards. In addition, we summarize evidence that the VS can also be implicated when learning from observing others, and in tasks that go beyond simple stimulus-action-outcome learning, indicating that the reward system is also recruited in more complex learning tasks.

Multiplexing signals in reinforcement learning with internal models and dopamine

A fundamental challenge for computational and cognitive neuroscience is to understand how reward-based learning and decision-making are made and how accrued knowledge and internal models of the environment are incorporated. Remarkable progress has been made in the field, guided by the midbrain dopamine reward prediction error hypothesis and the underlying reinforcement learning framework, which does not involve internal models ('model-free'). Recent studies, however, have begun not only to address more complex decision-making processes that are integrated with model-free decision-making, but also to include internal models about environmental reward structures and the minds of other agents, including model-based reinforcement learning and using generalized prediction errors. Even dopamine, a classic model-free signal, may work as multiplexed signals using model-based information and contribute to representational learning of reward structure.

Toward a neural basis for social behavior

Nearly 25 years ago, the shared interests of psychologists and biologists in understanding the neural basis of social behavior led to the inception of social neuroscience. In the past decade, this field has exploded, in large part due to the infusion of studies that use fMRI. At the same time, tensions have arisen about how to prioritize a diverse range of questions and about the authority of neurobiological data in answering them. The field is now poised to tackle some of the most interesting and important questions about human and animal behavior but at the same time faces uncertainty about how to achieve focus in its research and cohesion among the scientists who tackle it. The next 25 years offer the opportunity to alleviate some of these growing pains, as well as the challenge of answering large questions that encompass the nature and bounds of diverse social interactions (in humans, including interactions through the internet); how to characterize, and treat, social dysfunction in psychiatric illness; and how to compare social cognition in humans with that in other animals.

Theory of mind: a neural prediction problem

Predictive coding posits that neural systems make forward-looking predictions about incoming information. Neural signals contain information not about the currently perceived stimulus, but about the difference between the observed and the predicted stimulus. We propose to extend the predictive coding framework from high-level sensory processing to the more abstract domain of theory of mind; that is, to inferences about others' goals, thoughts, and personalities. We review evidence that, across brain regions, neural responses to depictions of human behavior, from biological motion to trait descriptions, exhibit a key signature of predictive coding: reduced activity to predictable stimuli. We discuss how future experiments could distinguish predictive coding from alternative explanations of this response profile. This framework may provide an important new window on the neural computations underlying theory of mind.