Latent goal models for dynamic strategic interaction

Neuronal mechanisms of dynamic strategic competition

Competitive social interactions, as in chess or poker, often involve multiple moves and countermoves deployed tactically within a broader strategic plan. Such maneuvers are supported by mentalizing or theory-of-mind-reasoning about the beliefs, plans, and goals of an opponent. The neuronal mechanisms underlying strategic competition remain largely unknown. To address this gap, we studied humans and monkeys playing a virtual soccer game featuring continuous competitive interactions. Humans and monkeys deployed similar tactics within broadly identical strategies, which featured unpredictable trajectories and precise timing for kickers, and responsiveness to opponents for goalies. We used Gaussian Process (GP) classification to decompose continuous gameplay into a series of discrete decisions predicated on the evolving states of self and opponent. We extracted relevant model parameters as regressors for neuronal activity in macaque mid-superior temporal sulcus (mSTS), the putative homolog of human temporo-parietal junction (TPJ), an area selectively engaged during strategic social interactions. We discovered two spatially-segregated populations of mSTS neurons that signaled actions of self and opponent, sensitivities to state changes, and previous and current trial outcomes. Inactivating mSTS reduced kicker unpredictability and impaired goalie responsiveness. These findings demonstrate mSTS neurons multiplex information about the current states of self and opponent as well as history of previous interactions to support ongoing strategic competition, consistent with hemodynamic activity found in human TPJ.

Dorsolateral and dorsomedial prefrontal cortex track distinct properties of dynamic social behavior

Understanding how humans make competitive decisions in complex environments is a key goal of decision neuroscience. Typical experimental paradigms constrain behavioral complexity (e.g. choices in discrete-play games), and thus, the underlying neural mechanisms of dynamic social interactions remain incompletely understood. Here, we collected fMRI data while humans played a competitive real-time video game against both human and computer opponents, and then, we used Bayesian non-parametric methods to link behavior to neural mechanisms. Two key cognitive processes characterized behavior in our task: (i) the coupling of one’s actions to another’s actions (i.e. opponent sensitivity) and (ii) the advantageous timing of a given strategic action. We found that the dorsolateral prefrontal cortex displayed selective activation when the subject’s actions were highly sensitive to the opponent’s actions, whereas activation in the dorsomedial prefrontal cortex increased proportionally to the advantageous timing of actions to defeat one’s opponent. Moreover, the temporoparietal junction tracked both of these behavioral quantities as well as opponent social identity, indicating a more general role in monitoring other social agents. These results suggest that brain regions that are frequently implicated in social cognition and value-based decision-making also contribute to the strategic tracking of the value of social actions in dynamic, multi-agent contexts.

Continuous decisions

Humans and other animals evolved to make decisions that extend over time with continuous and ever-changing options. Nonetheless, the academic study of decision-making is mostly limited to the simple case of choice between two options. Here, we advocate that the study of choice should expand to include continuous decisions. Continuous decisions, by our definition, involve a continuum of possible responses and take place over an extended period of time during which the response is continuously subject to modification. In most continuous decisions, the range of options can fluctuate and is affected by recent responses, making consideration of reciprocal feedback between choices and the environment essential. The study of continuous decisions raises new questions, such as how abstract processes of valuation and comparison are co-implemented with action planning and execution, how we simulate the large number of possible futures our choices lead to, and how our brains employ hierarchical structure to make choices more efficiently. While microeconomic theory has proven invaluable for discrete decisions, we propose that engineering control theory may serve as a better foundation for continuous ones. And while the concept of value has proven foundational for discrete decisions, goal states and policies may prove more useful for continuous ones. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

The neural basis of predictive pursuit

It remains unclear whether and, if so, how nonhuman animals make on-the-fly predictions during pursuit. Here we used a novel laboratory pursuit task that incentivizes the prediction of future prey positions. We trained three macaques to perform a joystick-controlled pursuit task in which prey follow intelligent escape algorithms. Subjects aimed toward the likely future positions of the prey, which indicated that they generate internal predictions and use these to guide behavior. We then developed a generative model that explains real-time pursuit trajectories and showed that our subjects use prey position, velocity and acceleration to make predictions. We identified neurons in the dorsal anterior cingulate cortex whose responses track these three variables. These neurons multiplexed prediction-related variables with a distinct and explicit representation of the future position of the prey. Our results provide a clear demonstration that the brain can explicitly represent future predictions and highlight the critical role of anterior cingulate cortex for future-oriented cognition.