Probability Distortion Depends on Choice Sequence in Rhesus Monkeys

The role of dorsal premotor cortex in joint action inhibition

Behavioral interpersonal coordination requires smooth negotiation of actions in time and space (joint action-JA). Inhibitory control may play a role in fine-tuning appropriate coordinative responses. To date, little research has been conducted on motor inhibition during JA and on the modulatory influence that premotor areas might exert on inhibitory control. Here, we used an interactive task in which subjects were required to reach and open a bottle using one hand. The bottle was held and stabilized by a co-actor (JA) or by a mechanical holder (vice clamp, no-JA). We recorded two TMS-based indices of inhibition (short-interval intracortical inhibition-sICI; cortical silent period-cSP) during the reaching phase of the task. These reflect fast intracortical (GABAa-mediated) and slow corticospinal (GABAb-mediated) inhibition. Offline continuous theta burst stimulation (cTBS) was used to interfere with dorsal premotor cortex (PMd), ventral premotor cortex (PMv), and control site (vertex) before the execution of the task. Our results confirm a dissociation between fast and slow inhibition during JA coordination and provide evidence that premotor areas drive only slow inhibitory mechanisms, which in turn may reflect behavioral co-adaptation between trials. Exploratory analyses further suggest that PMd, more than PMv, is the key source of modulatory drive sculpting movements, according to the socio-interactive context.

Balancing risk-return decisions by manipulating the mesofrontal circuits in primates

Decision-making is always coupled with some level of risk, with more pathological forms of risk-taking decisions manifesting as gambling disorders. In macaque monkeys trained in a high risk-high return (HH) versus low risk-low return (LL) choice task, we found that the reversible pharmacological inactivation of ventral Brodmann area 6 (area 6V) impaired the risk dependency of decision-making. Selective optogenetic activation of the mesofrontal pathway from the ventral tegmental area (VTA) to the ventral aspect of 6V resulted in stronger preference for HH, whereas activation of the pathway from the VTA to the dorsal aspect of 6V led to LL preference. Finally, computational decoding captured the modulations of behavioral preference. Our results suggest that VTA inputs to area 6V determine the decision balance between HH and LL.

Worth the Work? Monkeys Discount Rewards by a Subjective Adapting Effort Cost

All life must solve how to allocate limited energy resources to maximize benefits from scarce opportunities. Economic theory posits decision makers optimize choice by maximizing the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here, we test how well the subtractive model of effort disutility explains the behavior of two male nonhuman primates (Macaca mulatta) in a binary choice task in which reward quantity and physical effort to obtain were varied. Applying random utility modeling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared with parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility depends on previous experience of effort: in analogy to work from behavioral labor economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choices can be explained by the subtractive cost model of effort. Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step.SIGNIFICANCE STATEMENT All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here, we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys' experience of effort in previous trials.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

Systematic comparison of risky choices in humans and monkeys

The past decades have seen tremendous progress in fundamental studies on economic choice in humans. However, elucidation of the underlying neuronal processes requires invasive neurophysiological studies that are met with difficulties in humans. Monkeys as evolutionary closest relatives offer a solution. The animals display sophisticated and well-controllable behavior that allows to implement key constructs of proven economic choice theories. However, the similarity of economic choice between the two species has never been systematically investigated. We investigated compliance with the independence axiom (IA) of expected utility theory as one of the most demanding choice tests and compared IA violations between humans and monkeys. Using generalized linear modeling and cumulative prospect theory (CPT), we found that humans and monkeys made comparable risky choices, although their subjective values (utilities) differed. These results suggest similar fundamental choice mechanism across these primate species and encourage to study their underlying neurophysiological mechanisms.

A neuronal prospect theory model in the brain reward circuitry

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

It is unclear how the activity of individual neurons conform to prospect theory. Here, the authors demonstrate that the activity of single neurons in various reward-related regions in the monkey brain can be described as encoding a multiplicative combination of utility and probability weighting, and that this subjective valuation process is achieved via a distributed coding scheme.

How does decision-making change during challenging times?

Prospect Theory, proposed and developed by Kahneman and Tversky, demonstrated that people do not make rational decisions based on expected utility, but are instead biased by specific cognitive tendencies leading to neglect, under- or over- consider information, depending on the context of presentation. In this vein, the present paper focuses on whether and how individual decision-making attitudes are prone to change in the presence of globally challenging events. We ran three partial replications of the Kahneman and Tversky (1979) paper, focusing on a set of eight prospects, after a terror attack (Paris, November 2015, 134 subjects) and during the Covid-19 pandemic, both during the first lockdown in Italy (Spring 2020, 176 subjects) and after the first reopening (140 subjects). The results confirm patterns of choice characterizing uncertain times, as shown by previous literature. In particular, we note significant increase of risk aversion, both in the gain and in the loss domains, that consistently emerged in the three replications. Given the nature of our sample, and the heterogeneity between the three periods investigated, we suggest that the phenomenon we present can be explained stress-related effects on decision making rather than by other economic effects, such as the income effect.

Risky choice: Probability weighting explains independence axiom violations in monkeys

Expected Utility Theory (EUT) provides axioms for maximizing utility in risky choice. The Independence Axiom (IA) is its most demanding axiom: preferences between two options should not change when altering both options equally by mixing them with a common gamble. We tested common consequence (CC) and common ratio (CR) violations of the IA over several months in thousands of stochastic choices using a large variety of binary option sets. Three monkeys showed consistently few outright Preference Reversals (8%) but substantial graded Preference Changes (46%) between the initial preferred gamble and the corresponding altered gamble. Linear Discriminant Analysis (LDA) indicated that gamble probabilities predicted most Preference Changes in CC (72%) and CR (88%) tests. The Akaike Information Criterion indicated that probability weighting within Cumulative Prospect Theory (CPT) explained choices better than models using Expected Value (EV) or EUT. Fitting by utility and probability weighting functions of CPT resulted in nonlinear and non-parallel indifference curves (IC) in the Marschak-Machina triangle and suggested IA non-compliance of models using EV or EUT. Indeed, CPT models predicted Preference Changes better than EV and EUT models. Indifference points in out-of-sample tests were closer to CPT-estimated ICs than EV and EUT ICs. Finally, while the few outright Preference Reversals may reflect the long experience of our monkeys, their more graded Preference Changes corresponded to those reported for humans. In benefitting from the wide testing possibilities in monkeys, our stringent axiomatic tests contribute critical information about risky decision-making and serves as basis for investigating neuronal decision mechanisms.

Supplementary information

The online version contains supplementary material available at 10.1007/s11166-022-09388-7.

Comparing utility functions between risky and riskless choice in rhesus monkeys

Decisions can be risky or riskless, depending on the outcomes of the choice. Expected utility theory describes risky choices as a utility maximization process: we choose the option with the highest subjective value (utility), which we compute considering both the option's value and its associated risk. According to the random utility maximization framework, riskless choices could also be based on a utility measure. Neuronal mechanisms of utility-based choice may thus be common to both risky and riskless choices. This assumption would require the existence of a utility function that accounts for both risky and riskless decisions. Here, we investigated whether the choice behavior of two macaque monkeys in risky and riskless decisions could be described by a common underlying utility function. We found that the utility functions elicited in the two choice scenarios were different from each other, even after taking into account the contribution of subjective probability weighting. Our results suggest that distinct utility representations exist for risky and riskless choices, which could reflect distinct neuronal representations of the utility quantities, or distinct brain mechanisms for risky and riskless choices. The different utility functions should be taken into account in neuronal investigations of utility-based choice.

Primate anterior insular cortex represents economic decision variables proposed by prospect theory

In humans, risk attitude is highly context-dependent, varying with wealth levels or for different potential outcomes, such as gains or losses. These behavioral effects have been modelled using prospect theory, with the key assumption that humans represent the value of each available option asymmetrically as a gain or loss relative to a reference point. It remains unknown how these computations are implemented at the neuronal level. Here we show that macaques, like humans, change their risk attitude across wealth levels and gain/loss contexts using a token gambling task. Neurons in the anterior insular cortex (AIC) encode the ‘reference point’ (i.e., the current wealth level of the monkey) and reflect ‘loss aversion’ (i.e., option value signals are more sensitive to change in the loss than in the gain context) as postulated by prospect theory. In addition, changes in the activity of a subgroup of AIC neurons correlate with the inter-trial fluctuations in choice and risk attitude. Taken together, we show that the primate AIC in risky decision-making may be involved in monitoring contextual information used to guide the animal’s willingness to accept risk.

Prospect theory predicts irrational effects in human decision-making, but relies on ad-hoc assumptions. Here, authors provide a neural basis for this by showing that anterior insular cortex encodes key economic variables proposed by prospect theory.

Adaptation of utility functions to reward distribution in rhesus monkeys

This study investigated how the experience of different reward distributions would shape the utility functions that can be inferred from economic choice. Despite the generally accepted notion that utility functions are not insensitive to external references, the exact way in which such changes take place remains largely unknown. Here we benefitted from the capacity to engage in thorough and prolonged empirical tests of economic choice by one of our evolutionary cousins, the rhesus macaque. We analyzed data from thousands of binary choices and found that the animals' preferences changed depending on the statistics of rewards experienced in the past (up to weeks) and that these changes could reflect monkeys' adapting their expectations of reward. The utility functions we elicited from their choices stretched and shifted over several months of sequential changes in the mean and range of rewards that the macaques experienced. However, this adaptation was usually incomplete, suggesting that - even after months - past experiences held weight when monkeys' assigned value to future rewards. Rather than having stable and fixed preferences assumed by normative economic models, our results demonstrate that rhesus macaques flexibly shape their preferences around the past and present statistics of their environment. That is, rather than relying on a singular reference-point, reference-dependent preferences are likely to capture a monkey's range of expectations.

Nonhuman Primates Satisfy Utility Maximization in Compliance with the Continuity Axiom of Expected Utility Theory

Expected Utility Theory (EUT), the first axiomatic theory of risky choice, describes choices as a utility maximization process: decision makers assign a subjective value (utility) to each choice option and choose the one with the highest utility. The continuity axiom, central to Expected Utility Theory and its modifications, is a necessary and sufficient condition for the definition of numerical utilities. The axiom requires decision makers to be indifferent between a gamble and a specific probabilistic combination of a more preferred and a less preferred gamble. While previous studies demonstrated that monkeys choose according to combinations of objective reward magnitude and probability, a concept-driven experimental approach for assessing the axiomatically defined conditions for maximizing utility by animals is missing. We experimentally tested the continuity axiom for a broad class of gamble types in 4 male rhesus macaque monkeys, showing that their choice behavior complied with the existence of a numerical utility measure as defined by the economic theory. We used the numerical quantity specified in the continuity axiom to characterize subjective preferences in a magnitude-probability space. This mapping highlighted a trade-off relation between reward magnitudes and probabilities, compatible with the existence of a utility function underlying subjective value computation. These results support the existence of a numerical utility function able to describe choices, allowing for the investigation of the neuronal substrates responsible for coding such rigorously defined quantity.SIGNIFICANCE STATEMENT A common assumption of several economic choice theories is that decisions result from the comparison of subjectively assigned values (utilities). This study demonstrated the compliance of monkey behavior with the continuity axiom of Expected Utility Theory, implying a subjective magnitude-probability trade-off relation, which supports the existence of numerical utility directly linked to the theoretical economic framework. We determined a numerical utility measure able to describe choices, which can serve as a correlate for the neuronal activity in the quest for brain structures and mechanisms guiding decisions.

The description-experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

The experimental investigation of decision-making in humans relies on two distinct types of paradigms, involving either description- or experience-based choices. In description-based paradigms, decision variables (i.e. payoffs and probabilities) are explicitly communicated by means of symbols. In experience-based paradigms decision variables are learnt from trial-by-trial feedback. In the decision-making literature, 'description-experience gap' refers to the fact that different biases are observed in the two experimental paradigms. Remarkably, well-documented biases of description-based choices, such as under-weighting of rare events and loss aversion, do not apply to experience-based decisions. Here, we argue that the description-experience gap represents a major challenge, not only to current decision theories, but also to the neuroeconomics research framework, which relies heavily on the translation of neurophysiological findings between human and non-human primate research. In fact, most non-human primate neurophysiological research relies on behavioural designs that share features of both description- and experience-based choices. As a consequence, it is unclear whether the neural mechanisms built from non-human primate electrophysiology should be linked to description-based or experience-based decision-making processes. The picture is further complicated by additional methodological gaps between human and non-human primate neuroscience research. After analysing these methodological challenges, we conclude proposing new lines of research to address them. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Mechanisms for mutual support in motor interactions

What is the key to successful interaction? Is it sufficient to represent a common goal, or does the way our partner achieves that goal count as well? How do we react when our partner misbehaves? We used a turn-taking music-like task requiring participants to play sequences of notes together with a partner, and we investigated how people adapt to a partner's error that violates their expectations. Errors consisted of either playing a wrong note of a sequence that the agents were playing together (thus preventing the achievement of the joint goal) or playing the expected note with an unexpected action. In both cases, we found post-error slowing and inaccuracy suggesting the participants' implicit tendency to correct the partner's error and produce the action that the partner should have done. We argue that these "joint" monitoring processes depend on the motor predictions made within a (dyadic) motor plan and may represent a basic mechanism for mutual support in motor interactions.

The adaptive value of probability distortion and risk-seeking in macaques' decision-making

In humans, the attitude toward risk is not neutral and is dissimilar between bets involving gains and bets involving losses. The existence and prevalence of these decision features in non-human primates are unclear. In addition, only a few studies have tried to simulate the evolution of agents based on their attitude toward risk. Therefore, we still ignore to what extent Prospect theory's claims are evolutionarily rooted. To shed light on this issue, we collected data from nine macaques that performed bets involving gains or losses. We confirmed that their overall behaviour is coherent with Prospect theory's claims. In parallel, we used a genetic algorithm to simulate the evolution of a population of agents across several generations. We showed that the algorithm selects progressively agents that exhibit risk-seeking, and has an inverted S-shape distorted perception of probability. We compared these two results and found that monkeys' attitude toward risk is only congruent with the simulation when they are facing losses. This result is consistent with the idea that gambling in the loss domain is analogous to deciding in a context of life-threatening challenges where a certain level of risk-seeking behaviour and probability distortion may be adaptive. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Investigating the origin and consequences of endogenous default options in repeated economic choices

Classical value-based decision theories state that economic choices are solely based on the value of available options. Experimental evidence suggests, however, that individuals’ choices are biased towards default options, prompted by the framing of decisions. Although the effects of default options created by exogenous framing–such as how choice options are displayed–are well-documented, little is known about the potential effects and properties of endogenous framing, that is, originating from an individual's internal state. In this study, we investigated the existence and properties of endogenous default options in a task involving choices between risky lotteries. By manipulating and examining the effects of three experimental features–time pressure, time spent on task and relative choice proportion towards a specific option–, we reveal and dissociate two features of endogenous default options which bias individuals’ choices: a natural tendency to prefer certain types of options (natural default), and the tendency to implicitly learn a default option from past choices (learned default). Additional analyses suggest that while the natural default may bias the standard choice process towards an option category, the learned default effects may be attributable to a second independent choice process. Overall, these investigations provide a first experimental evidence of how individuals build and apply diverse endogenous default options in economic decision-making and how this biases their choices.

Flexible combination of reward information across primates

A fundamental but rarely contested assumption in economics and neuroeconomics is that decision-makers compute subjective values of risky options by multiplying functions of reward probability and magnitude. In contrast, an additive strategy for valuation allows flexible combination of reward information required in uncertain or changing environments. We hypothesized that the level of uncertainty in the reward environment should determine the strategy used for valuation and choice. To test this hypothesis, we examined choice between risky options in humans and monkeys across three tasks with different levels of uncertainty. We found that whereas humans and monkeys adopted a multiplicative strategy under risk when probabilities are known, both species spontaneously adopted an additive strategy under uncertainty when probabilities must be learned. Additionally, the level of volatility influenced relative weighting of certain and uncertain reward information and this was reflected in the encoding of reward magnitude by neurons in the dorsolateral prefrontal cortex.

Trait and State-Dependent Risk Attitude of Monkeys Measured in a Single-Option Response Task

Humans and animals show diverse preferences for risks (“trait-like” risk attitude) and shift their preference depending on the state or current needs (“state-dependent” risk attitude). For a better understanding of the neural mechanisms underlying risk-sensitive decisions, useful animal models have been required. Here we examined the risk attitude of three male monkeys in a single-option response task, in which an instrumental lever-release was required to obtain a chance of reward. In each trial, reward condition, either deterministic (100% of 1, 2, 3, and 4 drops of juice) or probabilistic (25, 50, 75, and 100% of 4-drop juice) was randomly selected and assigned by a unique visual cue, allowing the monkeys to evaluate the forthcoming reward. The subjective value of the reward was inferred from their performance. Model-based analysis incorporating known economic models revealed non-linear probability distortion in monkeys; unlike previous studies, they showed a simple convex or concave probability distortion curve. The direction of risk preference was consistent between early and late phases of the testing period, suggesting that our observation reflected the trait-like risk attitude of monkeys, at least under the current experimental setting. Regardless of the baseline risk preference, all monkeys showed an enhancement of risk preference in a session according to the satiation level (i.e., state-dependent risk attitude). Our results suggest that, without choice or cognitive demand, monkeys show naturalistic risk attitude – diverse and flexible like humans. Our novel approach may provide a useful animal model of risk-sensitive decisions, facilitating the investigation of the neural mechanisms of decision-making under risk.