High monetary reward rates and caloric rewards decrease temporal persistence

Lesions to different regions of frontal cortex have dissociable effects on voluntary persistence

Deciding how long to keep waiting for uncertain future rewards is a complex problem. Previous research has shown that choosing to stop waiting results from an evaluative process that weighs the subjective value of the awaited reward against the opportunity cost of waiting. In functional neuroimaging data, activity in ventromedial prefrontal cortex (vmPFC) tracks the dynamics of this evaluation, while activation in the dorsomedial prefrontal cortex (dmPFC) and anterior insula (AI) ramps up before a decision to quit is made. Here, we provide causal evidence of the necessity of these brain regions for successful performance in a willingness-to-wait task. 28 participants with frontal lobe lesions were tested on their ability to adaptively calibrate how long they waited for monetary rewards. We grouped the participants based on the location of their lesions, which were primarily in ventromedial, dorsomedial, or lateral parts of their prefrontal cortex (vmPFC, dmPFC, and lPFC, respectively), or in the anterior insula. We compared the performance of each subset of lesion participants to behavior in a control group without lesions (n=18). Finally, we fit a newly developed computational model to the data to glean a more mechanistic understanding of how lesions affect the cognitive processes underlying choice. We found that participants with lesions to the vmPFC waited less overall, while participants with lesions to the dmPFC and anterior insula were specifically impaired at calibrating their level of persistence to the environment. These behavioral effects were accounted for by systematic differences in parameter estimates from a computational model of task performance: while the vmPFC group showed reduced initial willingness to wait, lesions to the dmPFC/anterior insula were associated with slower learning from negative feedback. These findings corroborate the notion that failures of persistence can be driven by sophisticated cost-benefit analyses rather than lapses in self-control. They also support the functional specialization of different parts of the prefrontal cortex in service of voluntary persistence.

Pupillometric evidence for a temporal expectations-based account of persistence under temporal uncertainty

People often quit waiting for delayed rewards when the exact timing of those rewards is uncertain. This behavior often has been attributed to self-control failure. Another possibility is that quitting is the result of a rational decision-making process in the face of uncertainty, based on the decision-maker's expectations about the possible arrival times of the awaited reward. There are forms of temporal expectations (e.g., heavy-tailed) under which the expected time remaining until a reward arrives actually increases as time elapses. In those cases, the rational strategy is to quit waiting when the expected reward is no longer worth the expected time remaining. To arbitrate between the "limited self-control" and "temporal expectations" accounts of persistence, we measured pupil diameter during a persistence task, as a physiological marker of surprise (phasic responses) and effort (pre-decision diameter). Phasic pupil responses were elevated in response to reward receipt. Critically, the extent to which pupils dilated following rewards depended on the delay: people showed larger pupillary surprise responses the more delayed the reward was. This result suggests that people expect the reward less the longer they wait for it-a form of temporal expectations under which limiting persistence is rational. Moreover, predecision pupil diameter before quit events was not associated with how long the participant had been waiting, but rather, depended on how atypical the quit decision was compared with the participant's usual behavior. These data provide physiological evidence for a temporal expectations account of persistence under temporal uncertainty.

Temporal discounting when outcomes are experienced in the moment: Validation of a novel paradigm and comparison with a classic hypothetical intertemporal choice task

When faced with intertemporal choices, people typically devalue rewards available in the future compared to rewards more immediately available, a phenomenon known as temporal discounting. Decisions involving intertemporal choices arise daily, with critical impact on health and financial wellbeing. Although many such decisions are "experiential" in that they involve delays and rewards that are experienced in real-time and can inform subsequent choices, most studies have focused on intertemporal choices with hypothetical outcomes (or outcomes delivered after all decisions are made). The present study focused on experiential intertemporal choices. First, a novel intertemporal choice task was developed and validated, using delays experienced in real time and artistic photographs as consumable perceptual rewards. Second, performance on the experiential task was compared to performance on a classic intertemporal choice task with hypothetical outcomes. Involvement of distinct processes across tasks was probed by examining differential relations to state and trait anxiety. A two-parameter logistic function framework was proposed to fit indifference point data. This approach accounts for individual variability not only in the delay at which an individual switches from choosing the delayed to more immediate option, but also in the slope of that switch. Fit results indicated that the experiential task elicited temporal discounting, with effective trade-off between delay and perceptual reward. Comparison with the hypothetical intertemporal choice task suggested distinct mechanisms: first, temporal discounting across the two tasks was not correlated; and second, state and trait anxiety both were associated with choice behavior in the experiential task, albeit in distinct ways, whereas neither was significantly associated with choice behavior in the hypothetical task. The engagement of different processes in the experiential compared to hypothetical task may align with neural evidence for the recruitment of the hippocampus in animal but not in classic human intertemporal choice studies.

Uncertainty and computational complexity

Modern theories of decision-making typically model uncertainty about decision options using the tools of probability theory. This is exemplified by the Savage framework, the most popular framework in decision-making research. There, decision-makers are assumed to choose from among available decision options as if they maximized subjective expected utility, which is given by the utilities of outcomes in different states weighted with subjective beliefs about the occurrence of those states. Beliefs are captured by probabilities and new information is incorporated using Bayes' Law. The primary concern of the Savage framework is to ensure that decision-makers' choices are rational. Here, we use concepts from computational complexity theory to expose two major weaknesses of the framework. Firstly, we argue that in most situations, subjective utility maximization is computationally intractable, which means that the Savage axioms are implausible. We discuss empirical evidence supporting this claim. Secondly, we argue that there exist many decision situations in which the nature of uncertainty is such that (random) sampling in combination with Bayes' Law is an ineffective strategy to reduce uncertainty. We discuss several implications of these weaknesses from both an empirical and a normative perspective. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

Resource Signaling via Blood Glucose in Embodied Decision Making

Food, money, and time are exchangeable resources essential for survival and reproduction. Individuals live within finite budgets of these resources and make tradeoffs between money and time when making intertemporal choices between an immediate smaller reward and a delayed lager reward. In this paper, I examine signaling functions of blood glucose in regulating behaviors related to resource regulations beyond caloric metabolisms. These behavioral regulations include choices between energy expenditure and energy conservation, monetary intertemporal choices, and self-control in overcoming temptations. I begin by comparing potential embodied signals for resource forecasting and proactive decision making in terms of their pros and cons as a signal for regulating both metabolism and behavioral decision making and self-control. Based on this analysis, circulating glucose emerges as not only the designated fuel for brain metabolism but also a privileged resource forecasting signal for regulating immediate, short-term, and long-term behavioral adaptations to the resource budget of the decision maker. In the context of an on-going debate between the limited resource model and the motivation accounts of behavioral effects of blood glucose, I propose a dual functions (caloric provision and resource forecasting) and dual signaling (glucose taste and ingestion) hypothesis of circulating glucose in resource management, and provide behavioral and neurophysiological evidence of the separate effects of glucose taste to motivate effort for resource acquisition and glucose ingestion to promote resource conservation and future orientation. Accumulating evidence indicates that the body is able to detect fake signals of non-caloric sweeteners and react to such "caloric crisis" with an enhanced preference for immediate rewards over future rewards, revealing the wisdom of the body.

The effects of acute stress on the calibration of persistence

People frequently fail to wait for delayed rewards after choosing them. These preference reversals are sometimes thought to reflect self-control failure. Other times, however, continuing to wait for a delayed reward may be counterproductive (e.g., when reward timing uncertainty is high). Research has demonstrated that people can calibrate how long to wait for rewards in a given environment. Thus, the role of self-control might be to integrate information about the environment to flexibly adapt behavior, not merely to promote waiting. Here we tested effects of acute stress, which has been shown to tax control processes, on persistence, and the calibration of persistence, in young adult human participants. Half the participants (n = 60) performed a task in which persistence was optimal, and the other half (n = 60) performed a task in which it was optimal to quit waiting for reward soon after each trial began. Each participant completed the task either after cold pressor stress or no stress. Stress did not influence persistence or optimal calibration of persistence. Nevertheless, an exploratory analysis revealed an "inverted-U" relationship between cortisol increase and performance in the stress groups, suggesting that choosing the adaptive waiting policy may be facilitated with some stress and impaired with severe stress.

High monetary reward rates and caloric rewards decrease temporal persistence

Temporal persistence refers to an individual's capacity to wait for future rewards, while forgoing possible alternatives. This requires a trade-off between the potential value of delayed rewards and opportunity costs, and is relevant to many real-world decisions, such as dieting. Theoretical models have previously suggested that high monetary reward rates, or positive energy balance, may result in decreased temporal persistence. In our study, 50 fasted participants engaged in a temporal persistence task, incentivised with monetary rewards. In alternating blocks of this task, rewards were delivered at delays drawn randomly from distributions with either a lower or higher maximum reward rate. During some blocks participants received either a caloric drink or water. We used survival analysis to estimate participants' probability of quitting conditional on the delay distribution and the consumed liquid. Participants had a higher probability of quitting in blocks with the higher reward rate. Furthermore, participants who consumed the caloric drink had a higher probability of quitting than those who consumed water. Our results support the predictions from the theoretical models, and importantly, suggest that both higher monetary reward rates and physiologically relevant rewards can decrease temporal persistence, which is a crucial determinant for survival in many species.