Why we should use animals to study economic decision making - a perspective

A Double Hit of Social and Economic Stress in Mice Precipitates Changes in Decision-Making Strategies

BACKGROUND

Economic stress can serve as a second hit for people who have already accumulated a history of adverse life experiences. How one recovers from a setback is a core feature of resilience but is seldom captured in animal studies.

METHODS

We challenged mice in a novel 2-hit stress model by first exposing them to chronic social defeat stress and then testing adaptations to increasing reward scarcity on a neuroeconomic task. Mice were tested across months on the Restaurant Row task, during which they foraged daily for their primary source of food while on a limited time budget in a closed-economy system. An abrupt transition into a reward-scarce environment elicits an economic challenge, precipitating a drop in food intake and body weight to which mice must respond to survive.

RESULTS

We found that mice with a history of social stress mounted a robust behavioral response to this economic challenge that was achieved through a complex redistribution of time allocation among competing opportunities. Interestingly, we found that mice with a history of social defeat displayed changes in the development of decision-making policies during the recovery process that are important not only for ensuring food security necessary for survival but also prioritizing subjective value and that these changes emerged only for certain types of choices.

CONCLUSIONS

These findings indicate that an individual's capacity to recover from economic challenges depends on that person's prior history of stress and can affect multiple decision-making aspects of subjective well-being, thus highlighting a motivational balance that may be altered in stress-related disorders such as depression.

Ketamine reverses stress-induced hypersensitivity to sunk costs

How mood interacts with information processing in the brain is thought to mediate the maladaptive behaviors observed in depressed individuals. However, the neural mechanisms underlying impairments in emotion-cognition interactions are poorly understood. This includes influencing the balance between how past-sensitive vs. future-looking one is during decision-making. Recent insights from the field of neuroeconomics offer novel approaches to study changes in such valuation processes in a manner that is biologically tractable and readily translatable across species. We recently discovered that rodents are sensitive to "sunk costs" - a feature of higher cognition previously thought to be unique to humans. The sunk costs bias describes the phenomenon in which an individual overvalues and escalates commitment to continuing an ongoing endeavor, even if suboptimal, as a function of irrecoverable past (sunk) losses - information that, according to classic economic theory, should be ignored. In the present study, mice were exposed to chronic social defeat stress paradigm, a well-established animal model used for the study of depression. Mice were then tested on our longitudinal neuroeconomic foraging task, Restaurant Row. We found mice exposed to this severe stressor displayed an increased sensitivity to sunk costs, without altering overall willingness to wait. Mice were then randomly assigned to receive a single intraperitoneal injection of either saline or ketamine (20 mg/kg). We discovered that stress-induced hypersensitivity to sunk costs was renormalized following a single dose of ketamine. Interestingly, in non-defeated mice, ketamine treatment completely abolished sunk cost sensitivity, causing mice to no longer value irrecoverable losses during re-evaluation decisions who instead based choices solely on the future investment required to obtain a goal. These findings suggest that the antidepressant effects of ketamine may be mediated in part through changes in the processing of past-sensitive information during on-going decision-making, reducing its weight as a potential source of cognitive dissonance that could modulate behavior and instead promoting more future-thinking behavior.

Diabetes alters neuroeconomically dissociable forms of mental accounting

Those with diabetes mellitus are at high-risk of developing psychiatric disorders, yet the link between hyperglycemia and alterations in motivated behavior has not been explored in detail. We characterized value-based decision-making behavior of a streptozocin-induced diabetic mouse model on a naturalistic neuroeconomic foraging paradigm called Restaurant Row. Mice made self-paced choices while on a limited time-budget accepting or rejecting reward offers as a function of cost (delays cued by tone-pitch) and subjective value (flavors), tested daily in a closed-economy system across months. We found streptozocin-treated mice disproportionately undervalued less-preferred flavors and inverted their meal-consumption patterns shifted toward a more costly strategy that overprioritized high-value rewards. We discovered these foraging behaviors were driven by impairments in multiple decision-making systems, including the ability to deliberate when engaged in conflict and cache the value of the passage of time in the form of sunk costs. Surprisingly, diabetes-induced changes in behavior depended not only on the type of choice being made but also the salience of reward-scarcity in the environment. These findings suggest complex relationships between glycemic regulation and dissociable valuation algorithms underlying unique cognitive heuristics and sensitivity to opportunity costs can disrupt fundamentally distinct computational processes and could give rise to psychiatric vulnerabilities.

Arc regulates a second-guessing cognitive bias during naturalistic foraging through effects on discrete behavior modules

Foraging in animals relies on innate decision-making heuristics that can result in suboptimal cognitive biases in some contexts. The mechanisms underlying these biases are not well understood, but likely involve strong genetic effects. To explore this, we studied fasted mice using a naturalistic foraging paradigm and discovered an innate cognitive bias called "second-guessing." This involves repeatedly investigating an empty former food patch instead of consuming available food, which hinders the mice from maximizing feeding benefits. The synaptic plasticity gene Arc is revealed to play a role in this bias, as Arc-deficient mice did not exhibit second-guessing and consumed more food. In addition, unsupervised machine learning decompositions of foraging identified specific behavior sequences, or "modules", that are affected by Arc. These findings highlight the genetic basis of cognitive biases in decision making, show links between behavior modules and cognitive bias, and provide insight into the ethological roles of Arc in naturalistic foraging.

Noncanonical genomic imprinting in the monoamine system determines naturalistic foraging and brain-adrenal axis functions

Noncanonical genomic imprinting can cause biased expression of one parental allele in a tissue; however, the functional relevance of such biases is unclear. To investigate ethological roles for noncanonical imprinting in dopa decarboxylase (Ddc) and tyrosine hydroxylase (Th), we use machine learning to decompose naturalistic foraging in maternal and paternal allele mutant heterozygous mice. We uncover distinct roles for the maternal versus paternal alleles on foraging, where maternal alleles affect sons while daughters are under paternal allelic control. Each parental allele controls specific action sequences reflecting decisions in naive or familiar contexts. The maternal Ddc allele is preferentially expressed in subsets of hypothalamic GABAergic neurons, while the paternal allele predominates in subsets of adrenal cells. Each Ddc allele affects distinct molecular and endocrine components of the brain-adrenal axis. Thus, monoaminergic noncanonical imprinting has ethological roles in foraging and endocrine functions and operates by affecting discrete subsets of cells.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Dopamine receptors regulate preference between high-effort and high-risk rewards

RATIONALE

Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.

OBJECTIVES

We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.

METHODS

In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.

RESULTS

Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.

CONCLUSIONS

The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.

Non-human primate token use shows possibilities but also limitations for establishing a form of currency

Non-human primates evaluate choices based on quantitative information and subjective valuation of options. Non-human primates can learn to value tokens as placeholders for primary rewards (such as food). With those tokens established as a potential form of 'currency', it is then possible to examine how they respond to opportunities to earn and use tokens in ways such as accumulating tokens or exchanging tokens with each other or with human experimenters to gain primary rewards. Sometimes, individuals make efficient and beneficial choices to obtain tokens and then exchange them at the right moments to gain optimal reward. Sometimes, they even accumulate such rewards through extended delay of gratification, or through other exchange-based interactions. Thus, non-human primates are capable of associating value to arbitrary tokens that may function as currency-like stimuli, but there also are strong limitations on how non-human primates can integrate such tokens into choice situations or use such tokens to fully 'symbolize' economic decision-making. These limitations are important to acknowledge when considering the evolutionary emergence of currency use in our species. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Are capuchin monkeys (Sapajus spp.) sensitive to lost opportunities? The role of opportunity costs in intertemporal choice

Principles of economics predict that the costs associated with obtaining rewards can influence choice. When individuals face choices between a smaller, immediate option and a larger, later option, they often experience opportunity costs associated with waiting for delayed rewards because they must forego the opportunity to make other choices. We evaluated how reducing opportunity costs affects delay tolerance in capuchin monkeys. After choosing the larger option, in the High cost condition, subjects had to wait for the delay to expire, whereas in the Low cost different and Low cost same conditions, they could perform a new choice during the delay. To control for the effect of intake rate on choices, the Low cost same condition had the same intake rate ratio as the High cost condition. We found that capuchins attended both to intake rates and to opportunity costs. They chose the larger option more often in the Low cost different and Low cost same conditions than in the High cost condition, and more often in the Low cost different condition than in the Low cost same condition. Understanding how non-human primates represent and use costs in making decisions not only helps to develop theoretical frameworks to explain their choices but also addresses similarities with and differences from human decision-making. These outcomes provide insights into the origins of human economic behaviour. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

How animals do business

The field of comparative behavioural economics investigates decisions about the acquisition and exchange of goods and services. It does so in both humans and other species on the assumption that the cognition and emotions involved have a shared evolutionary background. This preface roughly defines the field and reviews a few selected early studies and concepts to offer a taste of what economic behaviour means in relation to species other than our own. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

Are the roots of human economic systems shared with non-human primates?

We review and analyze evidence for an evolutionary rooting of human economic behaviors and organization in non-human primates. Rather than focusing on the direct application of economic models that a priori account for animal decision behavior, we adopt an inductive definition of economic behavior in terms of the contribution of individual cognitive capacities to the provision of resources within an exchange structure. We spell out to what extent non-human primates' individual and strategic decision behaviors are shared with humans. We focus on the ability to trade, through barter or token-mediated exchanges, as a landmark of an economic system among members of the same species. It is an open question why only humans have reached a high level of economic sophistication. While primates have many of the necessary cognitive abilities (symbolic and computational) in isolation, one plausible issue we identify is the limits in exerting cognitive control to combine several sources of information. The difference between human and non-human primates' economies might well then be in degree rather than kind.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Anabolic-androgenic steroids alter decision making in a balanced rodent model of the Iowa gambling task

Anabolic-androgenic steroid (AAS) abuse is implicated in maladaptive decision making such as increased risk taking and problem gambling. Endogenous testosterone correlates with economic risk taking in both the stock market (Coates & Herbert, 2008) and in the laboratory, as measured by the Iowa Gambling Task (Stanton, Liening, & Schultheiss, 2011). Additionally, AAS use has been associated with problem gambling behavior in adolescents (Proimos, DuRant, Pierce, & Goodman, 1998). Thus, AAS may impair economic decision making. However, studies of human AAS users cannot control for preexisting risky behavior or normalize androgen levels. Accordingly, the present study investigated AAS effects on decision making in rats using a novel, balanced rodent model of the IGT. Adolescent male Long-Evans rats were treated chronically with high-dose testosterone (7.5 mg/kg) or vehicle (13% cyclodextrin in water) sc, and trained to work for sugar pellets in an operant chamber equipped with 4 levers, each associated with a different schedule of reward magnitude (number of pellets), probability, and punishment (time-out) duration. By RM-ANOVA, there was a main effect of lever (F3,78 = 25.33, p < .05), such that all rats preferred lever L4 offering a large reward (4 pellets), but with low probability (45%) and a long (35 sec) time-out. There was also a significant interaction of testosterone × lever (F3,78 = 2.78, p < .05), with testosterone increasing preference for L4 and decreasing preference for the other levers, relative to vehicle-treated controls. These data extend our previous findings of altered decision making in AAS-treated rats, and suggest that AAS may alter economic decision making in human users. (PsycINFO Database Record

Sensitivity to "sunk costs" in mice, rats, and humans

Sunk costs are irrecoverable investments that should not influence decisions, because decisions should be made on the basis of expected future consequences. Both human and nonhuman animals can show sensitivity to sunk costs, but reports from across species are inconsistent. In a temporal context, a sensitivity to sunk costs arises when an individual resists ending an activity, even if it seems unproductive, because of the time already invested. In two parallel foraging tasks that we designed, we found that mice, rats, and humans show similar sensitivities to sunk costs in their decision-making. Unexpectedly, sensitivity to time invested accrued only after an initial decision had been made. These findings suggest that sensitivity to temporal sunk costs lies in a vulnerability distinct from deliberation processes and that this distinction is present across species.

Economic Decision-Making in Parrots

Economic decision-making involves weighing up differently beneficial alternatives to maximise payoff. This sometimes requires the ability to forego one's desire for immediate satisfaction. This ability is considered cognitively challenging because it not only requires inhibiting impulses, but also evaluating expected outcomes in order to decide whether waiting is worthwhile. We tested four parrot species in a token exchange task. The subjects were first trained to exchange three types of tokens for a food item of low, medium, and high value and successfully learned to exchange these in an order according to their value. Subsequently, they were confronted with a choice between a food item and a token that could be exchanged for higher-quality food. In additional control conditions however, choosing a token led to an equal or lower payoff. Individuals of all species were capable of deciding economically, yet only large macaws outperformed the other species in one of the crucial controls. For some individuals, particularly African grey parrots, the token apparently had an intrinsic value, which prevented them from choosing economically in some control conditions and which should be considered as potentially confounding by future token exchange studies.

Beyond simple tests of value: measuring addiction as a heterogeneous disease of computation-specific valuation processes

Addiction is considered to be a neurobiological disorder of learning and memory because addiction is capable of producing lasting changes in the brain. Recovering addicts chronically struggle with making poor decisions that ultimately lead to relapse, suggesting a view of addiction also as a neurobiological disorder of decision-making information processing. How the brain makes decisions depends on how decision-making processes access information stored as memories in the brain. Advancements in circuit-dissection tools and recent theories in neuroeconomics suggest that neurally dissociable valuation processes access distinct memories differently, and thus are uniquely susceptible as the brain changes during addiction. If addiction is to be considered a neurobiological disorder of memory, and thus decision-making, the heterogeneity with which information is both stored and processed must be taken into account in addiction studies. Addiction etiology can vary widely from person to person. We propose that addiction is not a single disease, nor simply a disorder of learning and memory, but rather a collection of symptoms of heterogeneous neurobiological diseases of distinct circuit-computation-specific decision-making processes.

Mice learn to avoid regret

Regret can be defined as the subjective experience of recognizing that one has made a mistake and that a better alternative could have been selected. The experience of regret is thought to carry negative utility. This typically takes two distinct forms: augmenting immediate postregret valuations to make up for losses, and augmenting long-term changes in decision-making strategies to avoid future instances of regret altogether. While the short-term changes in valuation have been studied in human psychology, economics, neuroscience, and even recently in nonhuman-primate and rodent neurophysiology, the latter long-term process has received far less attention, with no reports of regret avoidance in nonhuman decision-making paradigms. We trained 31 mice in a novel variant of the Restaurant Row economic decision-making task, in which mice make decisions of whether to spend time from a limited budget to achieve food rewards of varying costs (delays). Importantly, we tested mice longitudinally for 70 consecutive days, during which the task provided their only source of food. Thus, decision strategies were interdependent across both trials and days. We separated principal commitment decisions from secondary reevaluation decisions across space and time and found evidence for regret-like behaviors following change-of-mind decisions that corrected prior economically disadvantageous choices. Immediately following change-of-mind events, subsequent decisions appeared to make up for lost effort by altering willingness to wait, decision speed, and pellet consumption speed, consistent with past reports of regret in rodents. As mice were exposed to an increasingly reward-scarce environment, we found they adapted and refined distinct economic decision-making strategies over the course of weeks to maximize reinforcement rate. However, we also found that even without changes in reinforcement rate, mice transitioned from an early strategy rooted in foraging to a strategy rooted in deliberation and planning that prevented future regret-inducing change-of-mind episodes from occurring. These data suggest that mice are learning to avoid future regret, independent of and separate from reinforcement rate maximization.

Regret describes a unique postdecision phenomenon in which losses are realized as a fault of one’s own actions. Regret is often hypothesized to have an inherent negative utility, and humans will often incur costs so as to avoid the risk of future regret. However, current models of nonhuman decision-making are based on reward maximization hypotheses. We recently found that rats express regret behaviorally and neurophysiologically on neuroeconomic foraging tasks; however, it remains unknown whether nonhuman animals will change strategies so as to avoid regret, even in the absence of changes in the achieved rate of reinforcement. Here, we provide the first evidence that mice change strategies to avoid future regret, independent of and separate from reinforcement rate maximization. Our data suggest mice accomplish this by shifting from a foraging decision-making strategy that produces change-of-mind decisions after investment mistakes to one rooted in deliberation that learns to plan ahead.

Altering gain of the infralimbic-to-accumbens shell circuit alters economically dissociable decision-making algorithms

The nucleus accumbens shell (NAcSh) is involved in reward valuation. Excitatory projections from infralimbic cortex (IL) to NAcSh undergo synaptic remodeling in rodent models of addiction and enable the extinction of disadvantageous behaviors. However, how the strength of synaptic transmission of the IL-NAcSh circuit affects decision-making information processing and reward valuation remains unknown, particularly because these processes can conflict within a given trial and particularly given recent data suggesting that decisions arise from separable information-processing algorithms. The approach of many neuromodulation studies is to disrupt information flow during on-going behaviors; however, this limits the interpretation of endogenous encoding of computational processes. Furthermore, many studies are limited by the use of simple behavioral tests of value which are unable to dissociate neurally distinct decision-making algorithms. We optogenetically altered the strength of synaptic transmission between glutamatergic IL-NAcSh projections in mice trained on a neuroeconomic task capable of separating multiple valuation processes. We found that induction of long-term depression in these synapses produced lasting changes in foraging processes without disrupting deliberative processes. Mice displayed inflated reevaluations to stay when deciding whether to abandon continued reward-seeking investments but displayed no changes during initial commitment decisions. We also developed an ensemble-level measure of circuit-specific plasticity that revealed individual differences in foraging valuation tendencies. Our results demonstrate that alterations in projection-specific synaptic strength between the IL and the NAcSh are capable of augmenting self-control economic valuations within a particular decision-making modality and suggest that the valuation mechanisms for these multiple decision-making modalities arise from different circuits.

Principles of Economic Rationality in Mice

Humans and non-human animals frequently violate principles of economic rationality, such as transitivity, independence of irrelevant alternatives, and regularity. The conditions that lead to these violations are not completely understood. Here we report a study on mice tested in automated home-cage setups using rewards of drinking water. Rewards differed in one of two dimensions, volume or probability. Our results suggest that mouse choice conforms to the principles of economic rationality for options that differ along a single reward dimension. A psychometric analysis of mouse choices further revealed that mice responded more strongly to differences in probability than to differences in volume, despite equivalence in return rates. This study also demonstrates the synergistic effect between the principles of economic rationality and psychophysics in making quantitative predictions about choices of healthy laboratory mice. This opens up new possibilities for the analyses of multi-dimensional choice and the use of mice with cognitive impairments that may violate economic rationality.

Valuation of opportunity costs by rats working for rewarding electrical brain stimulation

Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an “opportunity cost:” the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones. In an operant chamber, rewarding electrical brain stimulation was delivered when the cumulative time a lever had been depressed reached a criterion duration. The value of the activities forgone during this duration is the opportunity cost of the electrical reward. We determined which of four functions best describes how objective opportunity costs, expressed as the required duration of lever depression, are translated into their subjective equivalents. The simplest account is the identity function, which equates subjective and objective opportunity costs. A variant of this function called the “sigmoidal-slope function,” converges on the identity function at longer durations but deviates from it at shorter durations. The sigmoidal-slope function has the form of a hockey stick. The flat “blade” denotes a range over which opportunity costs are subjectively equivalent; these durations are too short to allow substitution of more beneficial activities. The blade extends into an upward-curving portion over which costs become discriminable and finally into the straight “handle,” over which objective and subjective costs match. The two remaining functions are based on hyperbolic and exponential temporal discounting, respectively. The results are best described by the sigmoidal-slope function. That this is so suggests that different principles of intertemporal choice are involved in the evaluation of time spent working for a reward or waiting for its delivery. The subjective opportunity-cost function plays a key role in the evaluation and selection of goals. An accurate description of its form and parameters is essential to successful modeling and prediction of instrumental performance and reward-related decision making.

A Social Reinforcement Learning Hypothesis of Mutual Reward Preferences in Rats

Although the use of neuroimaging techniques has revealed much about the neural correlates of social decision making (SDM) in humans, it remains poorly understood how social stimuli are represented, and how social decisions are implemented at the neural level in humans and in other species. To address this issue, the establishment of novel animal paradigms allowing a broad spectrum of neurobiological causal manipulations and neurophysiological recordings provides an exciting tool to investigate the neural implementation of social valuation in the brain. Here, we discuss the potential of a rodent model, Rattus norvegicus, for the understanding of SDM and its neural underpinnings. Particularly, we consider recent data collected in a rodent prosocial choice task within a social reinforcement framework and discuss factors that could drive SDM in rodents.

Reward and decision processes in the brains of humans and nonhuman primates

Choice behavior requires weighing multiple decision variables, such as utility, uncertainty, delay, or effort, that combine to define a subjective value for each considered option or course of action. This capacity is based on prior learning about potential rewards (and punishments) that result from prior actions. When made in a social context, decisions can involve strategic thinking about the intentions of others and about the impact of others' behavior on one's own outcome. Valuation is also influenced by different emotions that serve to adaptively regulate our choices in order to, for example, stay away from excessively risky gambles, prevent future regrets, or avoid personal rejection or conflicts. Drawing on economic theory and on advances in the study of neuronal mechanisms, we review relevant recent experiments in nonhuman primates and clinical observations made in neurologically impaired patients suffering from impaired decision-making capacities.

Temporal dynamics of choice behavior in rats and humans: an examination of pre- and post-choice latencies

Identifying similarities and differences in choice behavior across species is informative about how basic mechanisms give rise to more complex processes. In the present study, we compared pre- and post-choice latencies between rats and humans under two paradigms. In Experiment 1, we used a cued choice paradigm where subjects were presented with a cue that directed them as to which of two options to respond for rewards. In Experiment 2, subjects were free to choose between two options in order to procure rewards. In both Experiments rewards were delivered with distinct probabilities. The trial structure used in these experiments allowed the choice process to be decomposed into pre- and post-choice processes. Overall, post-choice latencies reflected the difference in reward probability between the two options, where latencies for the option with higher probability of reward were longer than those for the option with lower probability of reward. An interesting difference between rats and humans was observed: the choice behavior for humans, but not rats, was sensitive to the free-choice aspect of the tasks, such that in free-choice trials post-choice latencies no longer reflected the difference in reward probabilities between the two options.

Budget Constraints Affect Male Rats' Choices between Differently Priced Commodities

Demand theory can be applied to analyse how a human or animal consumer changes her selection of commodities within a certain budget in response to changes in price of those commodities. This change in consumption assessed over a range of prices is defined as demand elasticity. Previously, income-compensated and income-uncompensated price changes have been investigated using human and animal consumers, as demand theory predicts different elasticities for both conditions. However, in these studies, demand elasticity was only evaluated over the entirety of choices made from a budget. As compensating budgets changes the number of attainable commodities relative to uncompensated conditions, and thus the number of choices, it remained unclear whether budget compensation has a trivial effect on demand elasticity by simply sampling from a different total number of choices or has a direct effect on consumers’ sequential choice structure. If the budget context independently changes choices between commodities over and above price effects, this should become apparent when demand elasticity is assessed over choice sets of any reasonable size that are matched in choice opportunities between budget conditions. To gain more detailed insight in the sequential choice dynamics underlying differences in demand elasticity between budget conditions, we trained N=8 rat consumers to spend a daily budget by making a number of nosepokes to obtain two liquid commodities under different price regimes, in sessions with and without budget compensation. We confirmed that demand elasticity for both commodities differed between compensated and uncompensated budget conditions, also when the number of choices considered was matched, and showed that these elasticity differences emerge early in the sessions. These differences in demand elasticity were driven by a higher choice rate and an increased reselection bias for the preferred commodity in compensated compared to uncompensated budget conditions, suggesting a budget context effect on relative valuation.

Relative gains, losses, and reference points in probabilistic choice in rats

Theoretical reference points have been proposed to differentiate probabilistic gains from probabilistic losses in humans, but such a phenomenon in non-human animals has yet to be thoroughly elucidated. Three experiments evaluated the effect of reward magnitude on probabilistic choice in rats, seeking to determine reference point use by examining the effect of previous outcome magnitude(s) on subsequent choice behavior. Rats were trained to choose between an outcome that always delivered reward (low-uncertainty choice) and one that probabilistically delivered reward (high-uncertainty). The probability of high-uncertainty outcome receipt and the magnitudes of low-uncertainty and high-uncertainty outcomes were manipulated within and between experiments. Both the low- and high-uncertainty outcomes involved variable reward magnitudes, so that either a smaller or larger magnitude was probabilistically delivered, as well as reward omission following high-uncertainty choices. In Experiments 1 and 2, the between groups factor was the magnitude of the high-uncertainty-smaller (H-S) and high-uncertainty-larger (H-L) outcome, respectively. The H-S magnitude manipulation differentiated the groups, while the H-L magnitude manipulation did not. Experiment 3 showed that manipulating the probability of differential losses as well as the expected value of the low-uncertainty choice produced systematic effects on choice behavior. The results suggest that the reference point for probabilistic gains and losses was the expected value of the low-uncertainty choice. Current theories of probabilistic choice behavior have difficulty accounting for the present results, so an integrated theoretical framework is proposed. Overall, the present results have implications for understanding individual differences and corresponding underlying mechanisms of probabilistic choice behavior.

Rats prefer mutual rewards in a prosocial choice task

Pro-sociality, i.e., the preference for outcomes that produce benefits for other individuals, is ubiquitous in humans. Recently, cross-species comparisons of social behavior have offered important new insights into the evolution of pro-sociality. Here, we present a rodent analog of the Pro-social Choice Task that controls strategic components, de-confounds other-regarding choice motives from the animals' natural tendencies to maximize own food access and directly tests the effect of social context on choice allocation. We trained pairs of rats—an actor and a partner rat—in a double T-maze task where actors decided between two alternatives only differing in the reward delivered to the partner. The “own reward” choice yielded a reward only accessible to the actor whereas the “both reward” choice produced an additional reward for a partner (partner condition) or an inanimate toy (toy Condition), located in an adjacent compartment. We found that actors chose “both reward” at levels above chance and more often in the partner than in the toy condition. Moreover, we show that this choice pattern adapts to the current social context and that the observed behavior is stable over time.

Mechanisms of Individual Differences in Impulsive and Risky Choice in Rats

Individual differences in impulsive and risky choice are key risk factors for a variety of maladaptive behaviors such as drug abuse, gambling, and obesity. In our rat model, ordered individual differences are stable across choice parameters, months of testing, and span a broad spectrum, suggesting that rats, like humans, exhibit trait-level impulsive and risky choice behaviors. In addition, impulsive and risky choices are highly correlated, suggesting a degree of correlation between these two traits. An examination of the underlying cognitive mechanisms has suggested an important role for timing processes in impulsive choice. In addition, in an examination of genetic factors in impulsive choice, the Lewis rat strain emerged as a possible animal model for studying disordered impulsive choice, with this strain demonstrating deficient delay processing. Early rearing environment also affected impulsive behaviors, with rearing in an enriched environment promoting adaptable and more self-controlled choices. The combined results with impulsive choice suggest an important role for timing and reward sensitivity in moderating impulsive behaviors. Relative reward valuation also affects risky choice, with manipulation of objective reward value (relative to an alternative reference point) resulting in loss chasing behaviors that predicted overall risky choice behaviors. The combined results are discussed in relation to domain-specific versus domain-general subjective reward valuation processes and the potential neural substrates of impulsive and risky choice.

The impact of stress in decision making in the context of uncertainty

For a number of decades, different fields of knowledge, including psychology, economics, and neurosciences, have focused their research efforts on a better understanding of the decision-making process. Making decisions based on the probability of future events is routine in everyday life; it occurs whenever individuals select an option from several alternatives, each one associated with a specific value. Sometimes subjects decide knowing the precise outcomes of each option, but commonly they have to decide without knowing the consequences (because either ambiguity or risk is involved). Stress has a broad impact on animal behaviors, affects brain regions involved in decision-making processes, and, when maladaptive, is a trigger for neuropsychiatric disorders. This Mini-Review provides a comprehensive overview on how stress impacts decision-making processes, particularly under uncertain conditions. Understanding this can prove to be useful for intervention related to impairments to decision-making processes that present in several stress-triggered neuropsychiatric disorders.

The effect of probability discounting on reward seeking: a three-dimensional perspective

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Mechanisms of impulsive choice: I. Individual differences in interval timing and reward processing

Impulsive choice behavior incorporates the psychological mechanisms involved in the processing of the anticipated magnitude and delay until reward. The goal of the present experiment was to determine whether individual differences in such processes related to individual differences in impulsive choice behavior. Two groups of rats (Delay Group and Magnitude Group) were initially exposed to an impulsive choice task with choices between smaller-sooner (SS) and larger-later (LL) rewards. The Delay Group was subsequently exposed to a temporal discrimination task followed by a progressive interval task, whereas the Magnitude Group was exposed to a reward magnitude sensitivity task followed by a progressive ratio task. Intertask correlations revealed that the rats in the Delay Group that made more self-controlled (LL) choices also displayed lower standard deviations in the temporal bisection task and greater delay tolerance in the progressive interval task. Impulsive choice behavior in the Magnitude Group did not display any substantial correlations with the reward magnitude sensitivity and progressive ratio tasks. The results indicate the importance of core timing processes in impulsive choice behavior, and encourage further research examining the effects of changes in core timing processes on impulsive choice.

Optimal short-sighted rules

The aim of this paper is to assess the relevance of methodological transfers from behavioral ecology to experimental economics with respect to the elicitation of intertemporal preferences. More precisely our discussion will stem from the analysis of Stephens and Anderson’s (2001) seminal article. In their study with blue jays they document that foraging behavior typically implements short-sighted choice rules which are beneficial in the long run. Such long-term profitability of short-sighted behavior cannot be evidenced when using a self-control paradigm (one which contrasts in a binary way sooner smaller and later larger payoffs) but becomes apparent when ecological patch-paradigms (replicating economic situations in which the main trade-off consists in staying on a food patch or leaving for another patch) are implemented. We transfer this methodology in view of contrasting foraging strategies and self-control in human intertemporal choices.

Mechanistic classification of neural circuit dysfunctions: insights from neuroeconomics research in animals

Many psychiatric conditions present complex behavioral symptoms, and the type and magnitude of underlying neural dysfunction may vary drastically. This review introduces a classification scheme for psychiatric symptoms, describing them in terms of the state of a dysfunctional neural circuit. We provide examples of two kinds of functional deficits: variance-shifted functionality, in which a damaged circuit continues to function albeit suboptimally, and state-shifted functionality, resulting in an absent or qualitatively different functional state. We discuss, from the perspective of neuroeconomics and related areas of behavioral investigation, three broad classes of commonly occurring symptoms in psychopathology based on selected studies of decision making in animals: temporal discounting, social preferences, and decision making under environmental volatility. We conclude that the proposed mechanistic categorization scheme offers promise for understanding neural circuit dysfunctions underlying psychopathology.

Scarce means with alternative uses: robbins' definition of economics and its extension to the behavioral and neurobiological study of animal decision making

Almost 80 years ago, Lionel Robbins proposed a highly influential definition of the subject matter of economics: the allocation of scarce means that have alternative ends. Robbins confined his definition to human behavior, and he strove to separate economics from the natural sciences in general and from psychology in particular. Nonetheless, I extend his definition to the behavior of non-human animals, rooting my account in psychological processes and their neural underpinnings. Some historical developments are reviewed that render such a view more plausible today than would have been the case in Robbins’ time. To illustrate a neuroeconomic perspective on decision making in non-human animals, I discuss research on the rewarding effect of electrical brain stimulation. Central to this discussion is an empirically based, functional/computational model of how the subjective intensity of the electrical reward is computed and combined with subjective costs so as to determine the allocation of time to the pursuit of reward. Some successes achieved by applying the model are discussed, along with limitations, and evidence is presented regarding the roles played by several different neural populations in processes posited by the model. I present a rationale for marshaling convergent experimental methods to ground psychological and computational processes in the activity of identified neural populations, and I discuss the strengths, weaknesses, and complementarity of the individual approaches. I then sketch some recent developments that hold great promise for advancing our understanding of structure–function relationships in neuroscience in general and in the neuroeconomic study of decision making in particular.