Multiple timescales of temporal context in risky choice: Behavioral identification and relationships to physiological arousal

Context-dependence is fundamental to risky monetary decision-making. A growing body of evidence suggests that temporal context, or recent events, alters risk-taking at a minimum of three timescales: immediate (e.g. trial-by-trial), neighborhood (e.g. a group of consecutive trials), and global (e.g. task-level). To examine context effects, we created a novel monetary choice set with intentional temporal structure in which option values shifted between multiple levels of value magnitude ("contexts") several times over the course of the task. This structure allowed us to examine whether effects of each timescale were simultaneously present in risky choice behavior and the potential mechanistic role of arousal, an established correlate of risk-taking, in context-dependency. We found that risk-taking was sensitive to immediate, neighborhood, and global timescales: risk-taking decreased following large (vs. small) outcome amounts, increased following large positive (but not negative) shifts in context, and increased when cumulative earnings exceeded expectations. We quantified arousal with skin conductance responses, which were related to the global timescale, increasing with cumulative earnings, suggesting that physiological arousal captures a task-level assessment of performance. Our results both replicate and extend prior research by demonstrating that risky decision-making is consistently dynamic at multiple timescales and that the role of arousal in risk-taking extends to some, but not all timescales of context-dependence.

Prolonged Physiological Stress Is Associated With a Lower Rate of Exploratory Learning That Is Compounded by Depression

BACKGROUND

Stress is a major risk factor for depression, and both are associated with important changes in decision-making patterns. However, decades of research have only weakly connected physiological measurements of stress to the subjective experience of depression. Here, we examined the relationship between prolonged physiological stress, mood, and explore-exploit decision making in a population navigating a dynamic environment under stress: health care workers during the COVID-19 pandemic.

METHODS

We measured hair cortisol levels in health care workers who completed symptom surveys and performed an explore-exploit restless-bandit decision-making task; 32 participants were included in the final analysis. Hidden Markov and reinforcement learning models assessed task behavior.

RESULTS

Participants with higher hair cortisol exhibited less exploration (r = -0.36, p = .046). Higher cortisol levels predicted less learning during exploration (β = -0.42, false discovery rate [FDR]-corrected p [pFDR] = .022). Importantly, mood did not independently correlate with cortisol concentration, but rather explained additional variance (β = 0.46, pFDR = .022) and strengthened the relationship between higher cortisol and lower levels of exploratory learning (β = -0.47, pFDR = .022) in a joint model. These results were corroborated by a reinforcement learning model, which revealed less learning with higher hair cortisol and low mood (β = -0.67, pFDR = .002).

CONCLUSIONS

These results imply that prolonged physiological stress may limit learning from new information and lead to cognitive rigidity, potentially contributing to burnout. Decision-making measures link subjective mood states to measured physiological stress, suggesting that they should be incorporated into future biomarker studies of mood and stress conditions.

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BACKGROUND

Reinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.

METHODS

Twenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.

RESULTS

Individuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.

CONCLUSIONS

Despite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients' negative symptom, especially amotivation.

Effect of uncertainty-dependent residential mobility on loss aversion

The current research aimed to examine the relationship between residential mobility and loss aversion, a ubiquitous decision-making bias conceptualized in prospect theory. Combining correlational, experimental, and electroencephalography approaches, we tested the hypothesis that residential mobility may increase loss aversion. The results revealed a positive correlation between residential mobility and loss aversion (Study 1). These effects were moderated by individuals' residential mobility/stability mindsets. Behaviorally, we observed a positive association between residential mobility history and loss aversion among individuals with a high (vs. low) subjective uncertainty after manipulating their residential mobility mindset by priming them with a mobile or stable lifestyle (Study 2). On the neural level, we found that the mindset of residential mobility (vs. stability) enhanced the differential feedback-related negativity (FRN) modulations between the large-win and large-loss conditions among Chinese participants with a high subjective uncertainty (Study 3). In conclusion, residential mobility elevated loss aversion by enhancing feelings of subjective uncertainty. Our findings provide novel insights into how changes in the socioecological environment shape individuals' decision-making bias.

Pairing facts with imagined consequences improves pandemic-related risk perception

The COVID-19 pandemic reached staggering new peaks during a global resurgence more than a year after the crisis began. Although public health guidelines initially helped to slow the spread of disease, widespread pandemic fatigue and prolonged harm to financial stability and mental well-being contributed to this resurgence. In the late stage of the pandemic, it became clear that new interventions were needed to support long-term behavior change. Here, we examined subjective perceived risk about COVID-19 and the relationship between perceived risk and engagement in risky behaviors. In study 1 (n = 303), we found that subjective perceived risk was likely inaccurate but predicted compliance with public health guidelines. In study 2 (n = 735), we developed a multifaceted intervention designed to realign perceived risk with actual risk. Participants completed an episodic simulation task; we expected that imagining a COVID-related scenario would increase the salience of risk information and enhance behavior change. Immediately following the episodic simulation, participants completed a risk estimation task with individualized feedback about local viral prevalence. We found that information prediction error, a measure of surprise, drove beneficial change in perceived risk and willingness to engage in risky activities. Imagining a COVID-related scenario beforehand enhanced the effect of prediction error on learning. Importantly, our intervention produced lasting effects that persisted after a 1- to 3-wk delay. Overall, we describe a fast and feasible online intervention that effectively changed beliefs and intentions about risky behaviors.

The effect of testosterone on economic risk-taking: A multi-study, multi-method investigation

Testosterone has been suggested to influence individuals' economic decision making, yet the effects of testosterone on economic behavior are not well-understood and existing research is equivocal. In response, in three studies, we examined the extent to which testosterone affected or was associated with several different facets of economic decision making. Study 1 was a double-blind, placebo-controlled, within-subjects study examining loss aversion and risk-taking (N = 26), whereas Study 2 was a larger double-blind, placebo-controlled, between-subjects study examining loss aversion and risk-taking behavior (N = 117). As a methodological compliment, Study 3 was a larger correlational design (N = 213) with a highly accurate measure of endogenous testosterone examining a wider range of economic behaviors and trait-like preferences. Broadly, the results of all three studies suggest no consistent relationship between testosterone and financial behavior or preferences. Although there were significant effects in specific cases, these findings did not replicate in other studies or would not remain significant when controlling for family-wise error rate. We consider potential contextual moderators that may determine under what circumstances testosterone affects economic decision making.

Quantifying the immediate computational effects of preceding outcomes on subsequent risky choices

Forty years ago, prospect theory introduced the notion that risky options are evaluated relative to their recent context, causing a significant shift in the study of risky monetary decision-making in psychology, economics, and neuroscience. Despite the central role of past experiences, it remains unclear whether, how, and how much past experiences quantitatively influence risky monetary choices moment-to-moment in a nominally learning-free setting. We analyzed a large dataset of risky monetary choices with trial-by-trial feedback to quantify how past experiences, or recent events, influence risky choice behavior and the underlying processes. We found larger recent outcomes both negatively influence subsequent risk-taking and positively influence the weight put on potential losses. Using a hierarchical Bayesian framework to fit a modified version of prospect theory, we demonstrated that the same risks will be evaluated differently given different past experiences. The computations underlying risky decision-making are fundamentally dynamic, even if the environment is not.

Loss Aversion and Risk Aversion in Non-Clinical Negative Symptoms and Hypomania

In the field of behavioral decision-making, "loss aversion" is a behavioral phenomenon in which individuals show a higher sensitivity to potential losses than to gains. Conversely, "risk averse" individuals have an enhanced sensitivity/aversion to options with uncertain consequences. Here we examine whether hypomania or negative symptoms predict the degree of these choice biases. We chose to study these two symptom dimensions because they present a common theme across many syndromes with compromised decision-making. In our exploratory study, we employed a non-clinical sample to dissociate the hypomanic from negative symptom dimension regarding choice behavior. We randomly selected a sample of 45 subjects from a student population (18-37 years) without self-reported psychiatric diagnoses (n = 835). We stratified them based on percentiles into a low hypomania/low negative symptoms (n = 15), a hypomania (n = 15), and a negative symptoms group (n = 15) using the hypomanic personality scale (HPS-30) and community assessment of psychic experiences (CAPE). Participants completed a loss aversion task consisting of forced binary choices between a monetary gamble and a riskless choice without gain or loss. We found a reduced loss aversion in participants with higher negative symptoms. In addition, risk aversion was reduced in participants with higher hypomania and negative symptoms compared to low hypomania/negative symptoms. This study adds to the understanding of underlying psychological mechanisms of loss and risk aversion. Given the partially opposing nature of hypomania and negative symptoms, further work is needed to examine whether they affect loss and risk aversion via dissociable mechanisms.

Asymmetric valuation of gains and losses in effort-based decision making

Our decisions are often swayed by a desire to avoid losses over a desire to acquire gains. While loss aversion has been confirmed for decisions about money or commodities, it is unclear how individuals generally value gains relative to losses in effort-based decisions. For example, do individuals avoid greater work more than they seek out less work? We examined this question in the context of physical effort, using an arm-reaching task in which decreased effort was framed as a gain and increased effort was framed as a loss. Subjects performed reaching movements against different levels of resistance that increased or decreased the effort demands of the reaches. They then chose to accept or reject various lotteries, each with a possibility of performing less effortful reaches and a possibility of performing more effortful reaches, compared to the certain outcome of performing reaches against a fixed reference level of effort. Subjects avoided higher effort conditions more than they sought lower effort conditions, demonstrating asymmetric valuation of gains and losses. Using prospect theory, we explored various model formulations to determine subject-specific valuation of effort in these mixed gambles. A nonlinear model of effort valuation demonstrating increasing sensitivity to absolute effort best described the effort lottery choices. In contrast to the loss-aversion observed in financial decisions, there was no evidence of loss aversion in effort-based decisions. Rather, we observed moderate relief-seeking behavior. This model confirms that gains and losses are valued asymmetrically. This is due to the combined effects of increasing sensitivity to absolute effort and moderate relief-seeking, leading to a net effect of greater avoidance of higher effort. Asymmetric valuation was magnified on a later day of testing. In contrast, subjects were loss-averse in a comparable financial task. We suggest that consideration of nonlinear effort valuation can inform future studies of sensorimotor control and exercise motivation.

Insensitivity to Losses: A Core Feature in Patients With Anorexia Nervosa?

BACKGROUND

Patients with anorexia nervosa (AN) demonstrate aberrations in choice behavior, including impairments in laboratory measures of decision making. Although a wealth of studies suggest that these aberrations arise from alterations in value processing, it remains unclear by which core component of value processing this is mediated.

METHODS

We fit trial-by-trial data of patients with AN (n = 60 first cohort, n = 216 second cohort) and healthy control participants (n = 55) performing the Iowa Gambling Task to a computational model based on prospect utility theory. We determined, per participant, the best-fit model parameters and compared these between the groups.

RESULTS

Analyses revealed a decreased estimate of model parameter λ in patients with AN, indicative of an attenuation of loss-aversive behavior in the Iowa Gambling Task. In comparison, measures of reward sensitivity, value-based learning, and exploration versus exploitation were unaltered in patients with AN. A measurement in a second independent cohort replicated the finding that loss aversion, typically observed in healthy individuals, is reduced in patients with AN.

CONCLUSIONS

We show that patients with AN, in contrast to healthy control participants, demonstrate reduced loss-aversive behavior. This finding provides important fundamental insights into the decision-making capacity of patients with AN, suggesting alterations in the mechanisms involved in value processing related to negative feedback.

The Psychological and Neural Basis of Loss Aversion

Loss aversion is a central element of prospect theory, the dominant theory of decision making under uncertainty for the past four decades, and refers to the overweighting of potential losses relative to equivalent gains, a critical determinant of risky decision making. Recent advances in affective and decision neuroscience have shed new light on the psychological and neurobiological mechanisms underlying loss aversion. Here, integrating disparate literatures from the level of neurotransmitters to subjective reports of emotion, we propose a novel neural and computational framework that links norepinephrine to loss aversion and identifies a distinct role for dopamine in risk taking for rewards. We also propose that loss aversion specifically relates to anticipated emotions and aspects of the immediate experience of realized gains and losses but not their long-term emotional consequences, highlighting an underappreciated temporal structure. Finally, we discuss challenges to loss aversion and the relevance of loss aversion to understanding psychiatric disorders. Refining models of loss aversion will have broad consequences for the science of decision making and for how we understand individual variation in economic preferences and psychological well-being across both healthy and psychiatric populations.

Context-dependent sensitivity to losses: Range and skew manipulations

The assumption that losses loom larger than gains is widely used to explain many behavioral phenomena in judgment and decision-making. It is also generally accepted that loss aversion is a stable, traitlike individual difference characterizing people’s sensitivity to gains and losses. This interpretation was recently challenged by Walasek and Stewart (2015), who showed that by manipulating the range of the gains and losses used in the accept−reject task it is possible to find loss aversion, loss neutrality, and a reversal of loss aversion. Here, we reexamined the claim that these context effects arise as a result of people being sensitive to the rank position of a given gain among other gains and the rank position of a loss among other losses. We used skewed distributions of outcomes to manipulate the rank position of gains and losses while keeping the range of possible outcomes constant. We found a small but robust effect of skew on the propensity to accept mixed gambles. We compared the sizes of skew and range effects and found that they are of similar magnitude but that the range effects are smaller than those reported by Walasek and Stewart. We were able to attenuate loss aversion, but we were not able to replicate Walasek and Stewart’s reversal of loss aversion. We conclude that rank effects are, at least in part, responsible for the loss aversion seen in the accept−reject task.

No Evidence for a Relationship Between Hair Testosterone Concentrations and 2D:4D Ratio or Risk Taking

Using a recently developed alternative assay procedure to measure hormone levels from hair samples, we examined the relationships between testosterone, cortisol, 2D:4D ratio, overconfidence and risk taking. A total of 162 (53 male) participants provided a 3 cm sample of hair, a scanned image of their right and left hands from which we determined 2D:4D ratios, and completed measures of overconfidence and behavioral risk taking. While our sample size for males was less than ideal, our results revealed no evidence for a relationship between hair testosterone concentrations, 2D:4D ratios and risk taking. No relationships with overconfidence emerged. Partially consistent with the Dual Hormone Hypothesis, we did find evidence for the interacting effect of testosterone and cortisol on risk taking but only in men. Hair testosterone concentrations were positively related to risk taking when levels of hair cortisol concentrations were low, in men. Our results lend support to the suggestion that endogenous testosterone and 2D:4D ratio are unrelated and might then exert diverging activating vs. organizing effects on behavior. Comparing our results to those reported in the existing literature we speculate that behavioral correlates of testosterone such as direct effects on risk taking may be more sensitive to state-based fluctuations than baseline levels of testosterone.

Combined Effects of Glucocorticoid and Noradrenergic Activity on Loss Aversion

Loss aversion is a well-known behavioral regularity in financial decision making, describing humans' tendency to overweigh losses compared to gains of the same amount. Recent research indicates that stress and associated hormonal changes affect loss aversion, yet the underlying neuroendocrine mechanisms are still poorly understood. Here, we investigated the causal influence of two major stress neuromodulators, cortisol and noradrenaline, on loss aversion during financial decision making. In a double-blind, placebo-controlled between-subject design, we orally administered either the α2-adrenergic antagonist yohimbine (increasing noradrenergic stimulation), hydrocortisone, both substances, or a placebo to healthy young men. We tested the treatments' influence on a financial decision-making task measuring loss aversion and risk attitude. We found that both drugs combined, relative to either drug by itself, reduced loss aversion in the absence of an effect on risk attitude or choice consistency. Our data suggest that concurrent glucocorticoid and noradrenergic activity prompts an alignment of reward- with loss-sensitivity, and thus diminishes loss aversion. Our results have implications for the understanding of the susceptibility to biases in decision making.

Increased Loss Aversion in Unmedicated Patients with Obsessive-Compulsive Disorder

INTRODUCTION

Obsessive-compulsive disorder (OCD) patients show abnormalities in decision-making and, clinically, appear to show heightened sensitivity to potential negative outcomes. Despite the importance of these cognitive processes in OCD, few studies have examined the disorder within an economic decision-making framework. Here, we investigated loss aversion, a key construct in the prospect theory that describes the tendency for individuals to be more sensitive to potential losses than gains when making decisions.

METHODS

Across two study sites, groups of unmedicated OCD patients (n = 14), medicated OCD patients (n = 29), and healthy controls (n = 34) accepted or rejected a series of 50/50 gambles containing varying loss/gain values. Loss aversion was calculated as the ratio of the likelihood of rejecting a gamble with increasing potential losses to the likelihood of accepting a gamble with increasing potential gains. Decision times to accept or reject were also examined and correlated with loss aversion.

RESULTS

Unmedicated OCD patients exhibited significantly more loss aversion compared to medicated OCD or controls, an effect that was replicated across both sites and remained significant even after controlling for OCD symptom severity, trait anxiety, and sex. Post hoc analyses further indicated that unmedicated patients' increased likelihood to reject a gamble as its loss value increased could not be explained solely by greater risk aversion among patients. Unmedicated patients were also slower to accept than reject gambles, effects that were not found in the other two groups. Loss aversion was correlated with decision times in unmedicated patients but not in the other two groups.

DISCUSSION

These data identify abnormalities of decision-making in a subgroup of OCD patients not taking psychotropic medication. The findings help elucidate the cognitive mechanisms of the disorder and suggest that future treatments could aim to target abnormalities of loss/gain processing during decision-making in this population.

Effects of mineralocorticoid-receptor stimulation on risk taking behavior in young healthy men and women

Risk taking is influenced by stress, with riskier decisions after exposure to an acute stressor and consecutively elevated cortisol levels. In the brain, cortisol acts on two receptors with different functional profiles: the glucocorticoid receptor (GR) and the mineralocorticoid receptor (MR). In the current study we investigated the effect of MR stimulation on risk taking behavior in 80 young healthy participants (40 women, mean age=23.9). We administered 0.4mg fludrocortisone, a MR agonist, in a between-subjects, placebo controlled design. Subsequently, participants conducted an established risk taking paradigm, the Balloon-Analogue-Risk-Task (BART). We also used two questionnaires to assess risk taking and decision behavior as trait measures. We found a treatment effect with riskier decisions in the fludrocortisone group. Furthermore, we found a sex effect with more risk taking in men. There was no statistically significant interaction between both factors. Our results indicate that acute MR stimulation leads to riskier decisions in women and men. Our findings argue for an important role of the MR in decision making processes.

The impact of stress on tournament entry

Individual willingness to enter competitive environments predicts career choices and labor market outcomes. Meanwhile, many people experience competitive contexts as stressful. We use two laboratory experiments to investigate whether factors related to stress can help explain individual differences in tournament entry. Experiment 1 studies whether stress responses (measured as salivary cortisol) to taking part in a mandatory tournament predict individual willingness to participate in a voluntary tournament. We find that competing increases stress levels. This cortisol response does not predict tournament entry for men but is positively and significantly correlated with choosing to enter the tournament for women. In Experiment 2, we exogenously induce physiological stress using the cold-pressor task. We find a positive causal effect of stress on tournament entry for women but no effect for men. Finally, we show that although the effect of stress on tournament entry differs between the genders, stress reactions cannot explain the well-documented gender difference in willingness to compete.

Acute stress does not affect risky monetary decision-making

The ubiquitous and intense nature of stress responses necessitate that we understand how they affect decision-making. Despite a number of studies examining risky decision-making under stress, it is as yet unclear whether and in what way stress alters the underlying processes that shape our choices. This is in part because previous studies have not separated and quantified dissociable valuation and decision-making processes that can affect choices of risky options, including risk attitudes, loss aversion, and choice consistency, among others. Here, in a large, fully-crossed two-day within-subjects design, we examined how acute stress alters risky decision-making. On each day, 120 participants completed either the cold pressor test or a control manipulation with equal probability, followed by a risky decision-making task. Stress responses were assessed with salivary cortisol. We fit an econometric model to choices that dissociated risk attitudes, loss aversion, and choice consistency using hierarchical Bayesian techniques to both pool data and allow heterogeneity in decision-making. Acute stress was found to have no effect on risk attitudes, loss aversion, or choice consistency, though participants did become more loss averse and more consistent on the second day relative to the first. In the context of an inconsistent previous literature on risk and acute stress, our findings provide strong and specific evidence that acute stress does not affect risk attitudes, loss aversion, or consistency in risky monetary decision-making.

Stress potentiates decision biases: A stress induced deliberation-to-intuition (SIDI) model

Humans often make decisions in stressful situations, for example when the stakes are high and the potential consequences severe, or when the clock is ticking and the task demand is overwhelming. In response, a whole train of biological responses to stress has evolved to allow organisms to make a fight-or-flight response. When under stress, fast and effortless heuristics may dominate over slow and demanding deliberation in making decisions under uncertainty. Here, I review evidence from behavioral studies and neuroimaging research on decision making under stress and propose that stress elicits a switch from an analytic reasoning system to intuitive processes, and predict that this switch is associated with diminished activity in the prefrontal executive control regions and exaggerated activity in subcortical reactive emotion brain areas. Previous studies have shown that when stressed, individuals tend to make more habitual responses than goal-directed choices, be less likely to adjust their initial judgment, and rely more on gut feelings in social situations. It is possible that stress influences the arbitration between the emotion responses in subcortical regions and deliberative processes in the prefrontal cortex, so that final decisions are based on unexamined innate responses. Future research may further test this 'stress induced deliberation-to-intuition' (SIDI) model and examine its underlying neural mechanisms.