Foraging optimally in social neuroscience: computations and methodological considerations

How adolescents learn to build social bonds: A developmental computational account of social explore-exploit decision-making

Building social bonds is a critical task of adolescence that affords opportunities for learning, identity formation, and social support. Failing to develop close relationships in adolescence hinders adult interpersonal functioning and contributes to problems such as loneliness and depression. During adolescence, increased reward sensitivity and greater social flexibility both contribute to healthy social development, yet we lack a clear theory of how these processes interact to support social functioning. Here, we propose synthesizing these two literatures using a computational reinforcement learning framework that recasts how adolescents pursue and learn from social rewards as a social explore-exploit problem. To become socially skilled, adolescents must balance both their efforts to form individual bonds within specific groups and manage memberships across multiple groups to maximize access to social resources. We draw on insights from sociological studies on social capital in collective networks and neurocognitive research on foraging and cooperation to describe the social explore-exploit dilemma faced by adolescents navigating a modern world with increasing access to diverse resources and group memberships. Our account provides important new directions for examining the dynamics of adolescent behavior in social groups and understanding how social value computations can support positive relationships into adulthood.

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic answer is that we calculate the value of each option and then compare the values to choose the most rewarding, modulo some exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold, at which point we start exploring other options. In order to determine which view better describes human decision-making, we developed a novel, foraging-inspired sequential decision-making model and used it to ask whether humans compare to threshold ("Forage") or compare alternatives ("Reinforcement-Learn" [RL]). We found that the foraging model was a better fit for participant behavior, better predicted the participants' tendency to repeat choices, and predicted the existence of held-out participants with a pattern of choice that was almost impossible under RL. Together, these results suggest that humans use foraging computations, rather than RL, even in classic reinforcement learning tasks.

Social environment-based opportunity costs dictate when people leave social interactions

There is an ever-increasing understanding of the cognitive mechanisms underlying how we process others' behaviours during social interactions. However, little is known about how people decide when to leave an interaction. Are these decisions shaped by alternatives in the environment - the opportunity-costs of connecting to other people? Here, participants chose when to leave partners who treated them with varying degrees of fairness, and connect to others, in social environments with different opportunity-costs. Across four studies we find people leave partners more quickly when opportunity-costs are high, both the average fairness of people in the environment and the effort required to connect to another partner. People's leaving times were accounted for by a fairness-adapted evidence accumulation model, and modulated by depression and loneliness scores. These findings demonstrate the computational processes underlying decisions to leave, and highlight atypical social time allocations as a marker of poor mental health.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety

In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.

Neural implementation of computational mechanisms underlying the continuous trade-off between cooperation and competition

Social interactions evolve continuously. Sometimes we cooperate, sometimes we compete, while at other times we strategically position ourselves somewhere in between to account for the ever-changing social contexts around us. Research on social interactions often focuses on a binary dichotomy between competition and cooperation, ignoring people's evolving shifts along a continuum. Here, we develop an economic game - the Space Dilemma - where two players change their degree of cooperativeness over time in cooperative and competitive contexts. Using computational modelling we show how social contexts bias choices and characterise how inferences about others' intentions modulate cooperativeness. Consistent with the modelling predictions, brain regions previously linked to social cognition, including the temporo-parietal junction, dorso-medial prefrontal cortex and the anterior cingulate gyrus, encode social prediction errors and context-dependent signals, correlating with shifts along a cooperation-competition continuum. These results provide a comprehensive account of the computational and neural mechanisms underlying the continuous trade-off between cooperation and competition.

Synthetic Spatial Foraging With Active Inference in a Geocaching Task

Humans are highly proficient in learning about the environments in which they operate. They form flexible spatial representations of their surroundings that can be leveraged with ease during spatial foraging and navigation. To capture these abilities, we present a deep Active Inference model of goal-directed behavior, and the accompanying belief updating. Active Inference rests upon optimizing Bayesian beliefs to maximize model evidence or marginal likelihood. Bayesian beliefs are probability distributions over the causes of observable outcomes. These causes include an agent's actions, which enables one to treat planning as inference. We use simulations of a geocaching task to elucidate the belief updating-that underwrites spatial foraging-and the associated behavioral and neurophysiological responses. In a geocaching task, the aim is to find hidden objects in the environment using spatial coordinates. Here, synthetic agents learn about the environment via inference and learning (e.g., learning about the likelihoods of outcomes given latent states) to reach a target location, and then forage locally to discover the hidden object that offers clues for the next location.

Delay discounting and under-valuing of recent information predict poorer adherence to social distancing measures during the COVID-19 pandemic

The COVID-19 pandemic has brought about unprecedented global changes in individual and collective behaviour. To reduce the spread of the virus, public health bodies have promoted social distancing measures while attempting to mitigate their mental health consequences. The current study aimed to identify cognitive predictors of social distancing adherence and mental health symptoms, using computational models derived from delay discounting (the preference for smaller, immediate rewards over larger, delayed rewards) and patch foraging (the ability to trade-off between exploiting a known resource and exploring an unknown one). In a representative sample of the UK population (N = 442), we find that steeper delay discounting predicted poorer adherence to social distancing measures and greater sensitivity to reward magnitude during delay discounting predicted higher levels of anxiety symptoms. Furthermore, under-valuing recently sampled information during foraging independently predicted greater violation of lockdown guidance. Our results suggest that those who show greater discounting of delayed rewards struggle to maintain social distancing. Further, those who adapt faster to new information are better equipped to change their behaviour in response to public health measures. These findings can inform interventions that seek to increase compliance with social distancing measures whilst minimising negative repercussions for mental health.

Uncertainty drives deviations in normative foraging decision strategies

Nearly all animals forage to acquire energy for survival through efficient search and resource harvesting. Patch exploitation is a canonical foraging behaviour, but there is a need for more tractable and understandable mathematical models describing how foragers deal with uncertainty. To provide such a treatment, we develop a normative theory of patch foraging decisions, proposing mechanisms by which foraging behaviours emerge in the face of uncertainty. Our model foragers statistically and sequentially infer patch resource yields using Bayesian updating based on their resource encounter history. A decision to leave a patch is triggered when the certainty of the patch type or the estimated yield of the patch falls below a threshold. The time scale over which uncertainty in resource availability persists strongly impacts behavioural variables like patch residence times and decision rules determining patch departures. When patch depletion is slow, as in habitat selection, departures are characterized by a reduction of uncertainty, suggesting that the forager resides in a low-yielding patch. Uncertainty leads patch-exploiting foragers to overharvest (underharvest) patches with initially low (high) resource yields in comparison with predictions of the marginal value theorem. These results extend optimal foraging theory and motivate a variety of behavioural experiments investigating patch foraging behaviour.

Computational methods in social neuroscience: recent advances, new tools and future directions

Recent years have seen a surge of exciting developments in the computational tools available to social neuroscientists. This paper highlights and synthesizes recent advances that have been enabled by the application of such tools, as well as methodological innovations likely to be of interest and utility to social neuroscientists, but that have been concentrated in other sub-fields. Papers in this special issue are emphasized—many of which contain instructive materials (e.g. tutorials and code) for researchers new to the highlighted methods. These include approaches for modeling social decisions, characterizing multivariate neural response patterns at varying spatial scales, using decoded neurofeedback to draw causal links between specific neural response patterns and psychological and behavioral phenomena, examining time-varying patterns of connectivity between brain regions, and characterizing the social networks in which social thought and behavior unfold in everyday life. By combining computational methods for characterizing participants’ rich social environments—at the levels of stimuli, paradigms and the webs of social relationships that surround people—with those for capturing the psychological processes that undergird social behavior and the wealth of information contained in neuroimaging datasets, social neuroscientists can gain new insights into how people create, understand and navigate their complex social worlds.

Pandemic buying: Testing a psychological model of over-purchasing and panic buying using data from the United Kingdom and the Republic of Ireland during the early phase of the COVID-19 pandemic

The over-purchasing and hoarding of necessities is a common response to crises, especially in developed economies where there is normally an expectation of plentiful supply. This behaviour was observed internationally during the early stages of the Covid-19 pandemic. In the absence of actual scarcity, this behaviour can be described as 'panic buying' and can lead to temporary shortages. However, there have been few psychological studies of this phenomenon. Here we propose a psychological model of over-purchasing informed by animal foraging theory and make predictions about variables that predict over-purchasing by either exacerbating or mitigating the anticipation of future scarcity. These variables include additional scarcity cues (e.g. loss of income), distress (e.g. depression), psychological factors that draw attention to these cues (e.g. neuroticism) or to reassuring messages (eg. analytical reasoning) or which facilitate over-purchasing (e.g. income). We tested our model in parallel nationally representative internet surveys of the adult general population conducted in the United Kingdom (UK: N = 2025) and the Republic of Ireland (RoI: N = 1041) 52 and 31 days after the first confirmed cases of COVID-19 were detected in the UK and RoI, respectively. About three quarters of participants reported minimal over-purchasing. There was more over-purchasing in RoI vs UK and in urban vs rural areas. When over-purchasing occurred, in both countries it was observed across a wide range of product categories and was accounted for by a single latent factor. It was positively predicted by household income, the presence of children at home, psychological distress (depression, death anxiety), threat sensitivity (right wing authoritarianism) and mistrust of others (paranoia). Analytic reasoning ability had an inhibitory effect. Predictor variables accounted for 36% and 34% of the variance in over-purchasing in the UK and RoI respectively. With some caveats, the data supported our model and points to strategies to mitigate over-purchasing in future crises.

Is There a 'Social' Brain? Implementations and Algorithms

A fundamental question in psychology and neuroscience is the extent to which cognitive and neural processes are specialised for social behaviour, or are shared with other 'non-social' cognitive, perceptual, and motor faculties. Here we apply the influential framework of Marr (1982) across research in humans, monkeys, and rodents to propose that information processing can be understood as 'social' or 'non-social' at different levels. We argue that processes can be socially specialised at the implementational and/or the algorithmic level, and that changing the goal of social behaviour can also change social specificity. This framework could provide important new insights into the nature of social behaviour across species, facilitate greater integration, and inspire novel theoretical and empirical approaches.

Dopamine Modulates Dynamic Decision-Making during Foraging

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by traveling elsewhere (background reward rate), to determine the opportunity cost of staying versus leaving. We hypothesized that dopamine specifically modulates the influence of background, but not foreground, reward information when making a dynamic comparison of these variables for optimal behavior. On a novel foraging task based on an ecological account of animal behavior (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background reward rates. In line with theoretical accounts, people's decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signaling the opportunity cost of rewards, not value per se. Using this ecologically derived framework, we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared.SIGNIFICANCE STATEMENT Many decisions, across economic, political, and social spheres, involve choices to "leave". Such decisions depend on a continuous comparison of a current location's value, with that of other locations you could move on to. However, how the brain makes such decisions is poorly understood. Here, we developed a computerized task, based around theories of how animals make decisions to move on when foraging for food. Healthy human participants had to decide when to leave collecting financial rewards in a location, and travel to collect rewards elsewhere. Using a pharmacological manipulation, we show that the activity of dopamine in the brain modulates decisions to move on, with people valuing other locations differently depending on their dopaminergic state.