Optimal coding and neuronal adaptation in economic decisions

Efficient value synthesis in the orbitofrontal cortex explains how loss aversion adapts to the ranges of gain and loss prospects

Is irrational behavior the incidental outcome of biological constraints imposed on neural information processing? In this work, we consider the paradigmatic case of gamble decisions, where gamble values integrate prospective gains and losses. Under the assumption that neurons have a limited firing response range, we show that mitigating the ensuing information loss within artificial neural networks that synthetize value involves a specific form of self-organized plasticity. We demonstrate that the ensuing efficient value synthesis mechanism induces value range adaptation. We also reveal how the ranges of prospective gains and/or losses eventually determine both the behavioral sensitivity to gains and losses and the information content of the network. We test these predictions on two fMRI datasets from the OpenNeuro.org initiative that probe gamble decision-making but differ in terms of the range of gain prospects. First, we show that peoples' loss aversion eventually adapts to the range of gain prospects they are exposed to. Second, we show that the strength with which the orbitofrontal cortex (in particular: Brodmann area 11) encodes gains and expected value also depends upon the range of gain prospects. Third, we show that, when fitted to participant's gambling choices, self-organizing artificial neural networks generalize across gain range contexts and predict the geometry of information content within the orbitofrontal cortex. Our results demonstrate how self-organizing plasticity aiming at mitigating information loss induced by neurons' limited response range may result in value range adaptation, eventually yielding irrational behavior.

The representation of decision variables in orbitofrontal cortex is longitudinally stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the choice input and the choice output, suggesting that the cell groups found in the OFC constitute the building blocks of a decision circuit. Here, we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we record from the OFC of mice engaged in a juice-choice task. Imaging of individual cells continues for up to 40 weeks. For each cell and each session pair, we compare activity profiles using cosine similarity, and we assess whether the neuron encodes the same variable in both sessions. We find a high degree of stability and a modest representational drift. Quantitative estimates indicate that this drift would not randomize the circuit within the animal's lifetime.

Irrational choices via a curvilinear representational geometry for value

We make decisions by comparing values, but it is not yet clear how value is represented in the brain. Many models assume, if only implicitly, that the representational geometry of value is linear. However, in part due to a historical focus on noisy single neurons, rather than neuronal populations, this hypothesis has not been rigorously tested. Here, we examine the representational geometry of value in the ventromedial prefrontal cortex (vmPFC), a part of the brain linked to economic decision-making, in two male rhesus macaques. We find that values are encoded along a curved manifold in vmPFC. This curvilinear geometry predicts a specific pattern of irrational decision-making: that decision-makers will make worse choices when an irrelevant, decoy option is worse in value, compared to when it is better. We observe this type of irrational choices in behavior. Together, these results not only suggest that the representational geometry of value is nonlinear, but that this nonlinearity could impose bounds on rational decision-making.

Recent Opioid Use Impedes Range Adaptation in Reinforcement Learning in Human Addiction

BACKGROUND

Drugs like opioids are potent reinforcers thought to co-opt value-based decisions by overshadowing other rewarding outcomes, but how this happens at a neurocomputational level remains elusive. Range adaptation is a canonical process of fine-tuning representations of value based on reward context. Here, we tested whether recent opioid exposure impacts range adaptation in opioid use disorder, potentially explaining why shifting decision making away from drug taking during this vulnerable period is so difficult.

METHODS

Participants who had recently (<90 days) used opioids (n = 34) or who had abstained from opioid use for ≥ 90 days (n = 20) and comparison control participants (n = 44) completed a reinforcement learning task designed to induce robust contextual modulation of value. Two models were used to assess the latent process that participants engaged while making their decisions: 1) a Range model that dynamically tracks context and 2) a standard Absolute model that assumes stationary, objective encoding of value.

RESULTS

Control participants and ≥90-days-abstinent participants with opioid use disorder exhibited choice patterns consistent with range-adapted valuation. In contrast, participants with recent opioid use were more prone to learn and encode value on an absolute scale. Computational modeling confirmed the behavior of most control participants and ≥90-days-abstinent participants with opioid use disorder (75%), but a minority in the recent use group (38%), was better fit by the Range model than the Absolute model. Furthermore, the degree to which participants relied on range adaptation correlated with duration of continuous abstinence and subjective craving/withdrawal.

CONCLUSIONS

Reduced context adaptation to available rewards could explain difficulty deciding about smaller (typically nondrug) rewards in the aftermath of drug exposure.

Rationality, preferences, and emotions with biological constraints: it all starts from our senses

Is the role of our sensory systems to represent the physical world as accurately as possible? If so, are our preferences and emotions, often deemed irrational, decoupled from these 'ground-truth' sensory experiences? We show why the answer to both questions is 'no'. Brain function is metabolically costly, and the brain loses some fraction of the information that it encodes and transmits. Therefore, if brains maximize objective functions that increase the fitness of their species, they should adapt to the objective-maximizing rules of the environment at the earliest stages of sensory processing. Consequently, observed 'irrationalities', preferences, and emotions stem from the necessity for our early sensory systems to adapt and process information while considering the metabolic costs and internal states of the organism.

The Representation of Decision Variables in Orbitofrontal Cortex is Longitudinally Stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the input and the output of the choice process, suggesting that the cell groups found in OFC constitute the building blocks of a decision circuit. Here we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we recorded from mice choosing between different juice flavors. Recordings of individual cells continued for up to 20 weeks. For each cell and each pair of sessions, we compared the activity profiles using cosine similarity, and we assessed whether the cell encoded the same variable in both sessions. These analyses revealed a high degree of stability and a modest representational drift. A quantitative estimate indicated this drift would not randomize the circuit within the animal's lifetime.

A neural mechanism for conserved value computations integrating information and rewards

Behavioral and economic theory dictate that we decide between options based on their values. However, humans and animals eagerly seek information about uncertain future rewards, even when this does not provide any objective value. This implies that decisions are made by endowing information with subjective value and integrating it with the value of extrinsic rewards, but the mechanism is unknown. Here, we show that human and monkey value judgements obey strikingly conserved computational principles during multi-attribute decisions trading off information and extrinsic reward. We then identify a neural substrate in a highly conserved ancient structure, the lateral habenula (LHb). LHb neurons signal subjective value, integrating information's value with extrinsic rewards, and the LHb predicts and causally influences ongoing decisions. Neurons in key input areas to the LHb largely signal components of these computations, not integrated value signals. Thus, our data uncover neural mechanisms of conserved computations underlying decisions to seek information about the future.

Working memory and attention in choice

We study the role of attention and working memory in choices where options are presented sequentially rather than simultaneously. We build a model where a costly attention effort is chosen, which can vary over time. Evidence is accumulated proportionally to this effort and the utility of the reward. Crucially, the evidence accumulated decays over time. Optimal attention allocation maximizes expected utility from final choice; the optimal solution takes the decay into account, so attention is preferentially devoted to later times; but convexity of the flow attention cost prevents it from being concentrated near the end. We test this model with a choice experiment where participants observe sequentially two options. In our data the option presented first is, everything else being equal, significantly less likely to be chosen. This recency effect has a natural explanation with appropriate parameter values in our model of leaky evidence accumulation, where the decline is stronger for the option observed first. Analysis of choice, response time and brain imaging data provide support for the model. Working memory plays an essential role. The recency bias is stronger for participants with weaker performance in working memory tasks. Also activity in parietal areas, coding the stored value in working, declines over time as predicted.

GREM1, LRPPRC and SLC39A4 as potential biomarkers of intervertebral disc degeneration: a bioinformatics analysis based on multiple microarray and single-cell sequencing data

BACKGROUND

Low back pain (LBP) has drawn much widespread attention and is a major global health concern. In this field, intervertebral disc degeneration (IVDD) is frequently the focus of classic studies. However, the mechanistic foundation of IVDD is unclear and has led to conflicting outcomes.

METHODS

Gene expression profiles (GSE34095, GSE147383) of IVDD patients alongside control groups were analyzed to identify differentially expressed genes (DEGs) in the GEO database. GSE23130 and GSE70362 were applied to validate the yielded key genes from DEGs by means of a best subset selection regression. Four machine-learning models were established to assess their predictive ability. Single-sample gene set enrichment analysis (ssGSEA) was used to profile the correlation between overall immune infiltration levels with Thompson grades and key genes. The upstream targeting miRNAs of key genes (GSE63492) were also analyzed. A single-cell transcriptome sequencing data (GSE160756) was used to define several cell clusters of nucleus pulposus (NP), annulus fibrosus (AF), and cartilaginous endplate (CEP) of human intervertebral discs and the distribution of key genes in different cell clusters was yielded.

RESULTS

By developing appropriate p-values and logFC values, a total of 6 DEGs was obtained. 3 key genes (LRPPRC, GREM1, and SLC39A4) were validated by an externally validated predictive modeling method. The ssGSEA results indicated that key genes were correlated with the infiltration abundance of multiple immune cells, such as dendritic cells and macrophages. Accordingly, these 4 key miRNAs (miR-103a-3p, miR-484, miR-665, miR-107) were identified as upstream regulators targeting key genes using the miRNet database and external GEO datasets. Finally, the spatial distribution of key genes in AF, CEP, and NP was plotted. Pseudo-time series and GSEA analysis indicated that the expression level of GREM1 and the differentiation trajectory of NP chondrocytes are generally consistent. GREM1 may mainly exacerbate the degeneration of NP cells in IVDD.

CONCLUSIONS

Our study gives a novel perspective for identifying reliable and effective gene therapy targets in IVDD.

Intrinsic rewards explain context-sensitive valuation in reinforcement learning

When observing the outcome of a choice, people are sensitive to the choice's context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms-reflecting a different theoretical viewpoint-may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new "intrinsically enhanced" RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.

Sensory perception relies on fitness-maximizing codes

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.

Human value learning and representation reflect rational adaptation to task demands

Humans and other animals routinely make choices between goods of different values. Choices are often made within identifiable contexts, such that an efficient learner may represent values relative to their local context. However, if goods occur across multiple contexts, a relative value code can lead to irrational choices. In this case, an absolute context-independent value is preferable to a relative code. Here we test the hypothesis that value representation is not fixed but rationally adapted to context expectations. In two experiments, we manipulated participants' expectations about whether item values learned within local contexts would need to be subsequently compared across contexts. Despite identical learning experiences, the group whose expectations included choices across local contexts went on to learn more absolute-like representation than the group whose expectations covered only fixed local contexts. Human value representation is thus neither relative nor absolute but efficiently and rationally tuned to task demands.

Efficiently irrational: deciphering the riddle of human choice

For the past half-century, cognitive and social scientists have struggled with the irrationalities of human choice behavior; people consistently make choices that are logically inconsistent. Is human choice behavior evolutionarily adaptive or is it an inefficient patchwork of competing mechanisms? In this review, I present an interdisciplinary synthesis arguing for a novel interpretation: choice is efficiently irrational. Connecting findings across disciplines suggests that observed choice behavior reflects a precise optimization of the trade-off between the costs of increasing the precision of the choice mechanism and the declining benefits that come as precision increases. Under these constraints, a rationally imprecise strategy emerges that works toward optimal efficiency rather than toward optimal rationality. This approach rationalizes many of the puzzling inconsistencies of human choice behavior, explaining why these inconsistencies arise as an optimizing solution in biological choosers.

Neuronal origins of reduced accuracy and biases in economic choices under sequential offers

Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here, we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in the orbitofrontal cortex revealed that these phenomena emerged at different computational stages. Lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey.

The aversive value of pain in human decision-making

BACKGROUND

In order to decide between avoiding pain or pursuing competing rewards, pain must be assigned an abstract value that can be traded against that of competing goods. To assess the relationship between subjectively perceived pain and its value, we conducted an experiment where participants had to accept or decline different intensities of painful electric shocks in exchange of monetary rewards.

METHODS

Participants (n = 90) were divided into three groups that were exposed to different distributions of monetary rewards. Monetary offers ranged linearly from $0 to $5 or $10 in groups 1 and 2, respectively, and exponentially from $0 to $5 in group 3. Pain offers ranged from pain detection to pain tolerance thresholds. The value of pain was assessed by identifying the indifference points corresponding to a 50% chance of accepting a certain level of pain for a given monetary reward.

RESULTS

The value of pain increased quadratically as a function of the anticipated pain intensity and was found to be relative to the mean and standard deviation of monetary offers. Moreover, decision times increased as a function of the intensity of accepted painful stimulations. Finally, inter-individual differences in psychological traits related to harm avoidance and persistence influenced the value of pain.

CONCLUSIONS

This is the first demonstration that the value of pain follows a curvilinear function and is relative to the mean and standard deviation of competing monetary rewards. These new observations significantly contribute to our understanding of how pain is assigned value when making decisions between avoiding pain and obtaining rewards.

SIGNIFICANCE

This work provides a description of the pain value function indicating how much people are willing to pay to avoid different intensities of pain. We found that the function was curvilinear, suggesting that the same unit of subjective pain has more value in the high vs. low pain range. Moreover, the pain value was influenced by the experimental manipulation of the rewards distribution and of the inter-individual differences in harm avoidance and persistence. Altogether, the present study provides a detailed account of how subjectively experienced pain is assigned value.

The Neural Instantiation of an Abstract Cognitive Map for Economic Choice

Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value, may project to common downstream targets. However, it is unclear whether reward history, which may subserve lOFC's well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al., 2019). Here, we characterize functional subpopulations of lOFC neurons during behavior, including their encoding of task variables. We found five distinct clusters of lOFC neurons, either based on clustering of their trial-averaged peristimulus time histograms (PSTHs), or a feature space defined by their average conditional firing rates aligned to different task variables. We observed weak encoding of reward attributes, but stronger encoding of reward history, the animal's left or right choice, and reward receipt across all clusters. Only one cluster, however, encoded the animal's reward history at the time shortly preceding the choice, suggesting a possible role in integrating previous and current trial outcomes at the time of choice. This cluster also exhibits qualitatively similar responses to identified corticostriatal projection neurons in a recent study (Hirokawa et al., 2019), and suggests a possible role for subpopulations of lOFC neurons in mediating trial-by-trial learning.

Range-Adaptive Value Representation in Different Stages of Schizophrenia: A Proof of Concept Study

Amotivation is related to value representation. A comprehensive account of amotivation requires a mechanistic understanding of how the brain exploits external information to represent value. To achieve maximal value discriminability, brain valuation system will dynamically adapt its coding sensitivity to the range of values available in any given condition, so-called range adaptive coding. We administered an experimental task to 30 patients with chronic schizophrenia (C-SCZ), 30 first-episode schizophrenia (FE-SCZ), 34 individuals with high social anhedonia (HSoA), and their paired controls to assess range adaptation ability. C-SCZ patients exhibited over-adaptation and their performances were negatively correlated with avolition symptoms and positive symptoms and positively correlated with blunted-affect symptoms and self-reported consummatory interpersonal pleasure scores, though the results were non-significant. FE-SCZ patients exhibited reduced adaptation, which was significantly and negatively correlated with avolition symptoms and positively correlated with the overall proportion of choosing to exert more effort. Although HSoA participants exhibited comparable range adaptation to controls, their performances were significantly and negatively correlated with the proportion of choosing to exert more effort under the lowest value condition. Our results suggest that different stages of schizophrenia spectrum showed distinct range adaptation patterns. Range adaptation impairments may index a possible underlying mechanism for amotivation symptoms in FE-SCZ and more complicated and pervasive effects on clinical symptoms in C-SCZ.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex

The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.

Adaptation of utility functions to reward distribution in rhesus monkeys

This study investigated how the experience of different reward distributions would shape the utility functions that can be inferred from economic choice. Despite the generally accepted notion that utility functions are not insensitive to external references, the exact way in which such changes take place remains largely unknown. Here we benefitted from the capacity to engage in thorough and prolonged empirical tests of economic choice by one of our evolutionary cousins, the rhesus macaque. We analyzed data from thousands of binary choices and found that the animals' preferences changed depending on the statistics of rewards experienced in the past (up to weeks) and that these changes could reflect monkeys' adapting their expectations of reward. The utility functions we elicited from their choices stretched and shifted over several months of sequential changes in the mean and range of rewards that the macaques experienced. However, this adaptation was usually incomplete, suggesting that - even after months - past experiences held weight when monkeys' assigned value to future rewards. Rather than having stable and fixed preferences assumed by normative economic models, our results demonstrate that rhesus macaques flexibly shape their preferences around the past and present statistics of their environment. That is, rather than relying on a singular reference-point, reference-dependent preferences are likely to capture a monkey's range of expectations.

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning

Evidence suggests that economic values are rescaled as a function of the range of the available options. Although locally adaptive, range adaptation has been shown to lead to suboptimal choices, particularly notable in reinforcement learning (RL) situations when options are extrapolated from their original context to a new one. Range adaptation can be seen as the result of an adaptive coding process aiming at increasing the signal-to-noise ratio. However, this hypothesis leads to a counterintuitive prediction: Decreasing task difficulty should increase range adaptation and, consequently, extrapolation errors. Here, we tested the paradoxical relation between range adaptation and performance in a large sample of participants performing variants of an RL task, where we manipulated task difficulty. Results confirmed that range adaptation induces systematic extrapolation errors and is stronger when decreasing task difficulty. Last, we propose a range-adapting model and show that it is able to parsimoniously capture all the behavioral results.

Values encoded in orbitofrontal cortex are causally related to economic choices

In the eighteenth century, Daniel Bernoulli, Adam Smith and Jeremy Bentham proposed that economic choices rely on the computation and comparison of subjective values1. This hypothesis continues to inform modern economic theory2 and research in behavioural economics3, but behavioural measures are ultimately not sufficient to verify the proposal4. Consistent with the hypothesis, when agents make choices, neurons in the orbitofrontal cortex (OFC) encode the subjective value of offered and chosen goods5. Value-encoding cells integrate multiple dimensions6-9, variability in the activity of each cell group correlates with variability in choices10,11 and the population dynamics suggests the formation of a decision12. However, it is unclear whether these neural processes are causally related to choices. More generally, the evidence linking economic choices to value signals in the brain13-15 remains correlational16. Here we show that neuronal activity in the OFC is causal to economic choices. We conducted two experiments using electrical stimulation in rhesus monkeys (Macaca mulatta). Low-current stimulation increased the subjective value of individual offers and thus predictably biased choices. Conversely, high-current stimulation disrupted both the computation and the comparison of subjective values, and thus increased choice variability. These results demonstrate a causal chain linking subjective values encoded in OFC to valuation and choice.

Efficient sampling and noisy decisions

Human decisions are based on finite information, which makes them inherently imprecise. But what determines the degree of such imprecision? Here, we develop an efficient coding framework for higher-level cognitive processes in which information is represented by a finite number of discrete samples. We characterize the sampling process that maximizes perceptual accuracy or fitness under the often-adopted assumption that full adaptation to an environmental distribution is possible, and show how the optimal process differs when detailed information about the current contextual distribution is costly. We tested this theory on a numerosity discrimination task, and found that humans efficiently adapt to contextual distributions, but in the way predicted by the model in which people must economize on environmental information. Thus, understanding decision behavior requires that we account for biological restrictions on information coding, challenging the often-adopted assumption of precise prior knowledge in higher-level decision systems.

The bounded rationality of probability distortion

In decision making under risk (DMR) participants' choices are based on probability values systematically different from those that are objectively correct. Similar systematic distortions are found in tasks involving relative frequency judgments (JRF). These distortions limit performance in a wide variety of tasks and an evident question is, Why do we systematically fail in our use of probability and relative frequency information? We propose a bounded log-odds model (BLO) of probability and relative frequency distortion based on three assumptions: 1) log-odds: probability and relative frequency are mapped to an internal log-odds scale, 2) boundedness: the range of representations of probability and relative frequency are bounded and the bounds change dynamically with task, and 3) variance compensation: the mapping compensates in part for uncertainty in probability and relative frequency values. We compared human performance in both DMR and JRF tasks to the predictions of the BLO model as well as 11 alternative models, each missing one or more of the underlying BLO assumptions (factorial model comparison). The BLO model and its assumptions proved to be superior to any of the alternatives. In a separate analysis, we found that BLO accounts for individual participants' data better than any previous model in the DMR literature. We also found that, subject to the boundedness limitation, participants' choice of distortion approximately maximized the mutual information between objective task-relevant values and internal values, a form of bounded rationality.

Value-based attention but not divisive normalization influences decisions with multiple alternatives

Violations of economic rationality principles in choices between three or more options are critical for understanding the neural and cognitive mechanisms of decision-making. A recent study reported that the relative choice accuracy between two options decreases as the value of a third (distractor) option increases and attributed this effect to divisive normalization of neural value representations. In two preregistered experiments, a direct replication and an eye-tracking experiment, we assessed the replicability of this effect and tested an alternative account that assumes value-based attention to mediate the distractor effect. Surprisingly, we could not replicate the distractor effect in our experiments. However, we found a dynamic influence of distractor value on fixations to distractors as predicted by the value-based attention theory. Computationally, we show that extending an established sequential sampling decision-making model by a value-based attention mechanism offers a comprehensive account of the interplay between value, attention, response times and decisions.

The Effect of Counterfactual Information on Outcome Value Coding in Medial Prefrontal and Cingulate Cortex: From an Absolute to a Relative Neural Code

Adaptive coding of stimuli is well documented in perception, where it supports efficient encoding over a broad range of possible percepts. Recently, a similar neural mechanism has been reported also in value-based decision, where it allows optimal encoding of vast ranges of values in PFC: neuronal response to value depends on the choice context (relative coding), rather than being invariant across contexts (absolute coding). Additionally, value learning is sensitive to the amount of feedback information: providing complete feedback (both obtained and forgone outcomes) instead of partial feedback (only obtained outcome) improves learning. However, it is unclear whether relative coding occurs in all PFC regions and how it is affected by feedback information. We systematically investigated univariate and multivariate feedback encoding in various mPFC regions and compared three modes of neural coding: absolute, partially-adaptive and fully-adaptive.Twenty-eight human participants (both sexes) performed a learning task while undergoing fMRI scanning. On each trial, they chose between two symbols associated with a certain outcome. Then, the decision outcome was revealed. Notably, in one-half of the trials participants received partial feedback, whereas in the other half they got complete feedback. We used univariate and multivariate analysis to explore value encoding in different feedback conditions.We found that both obtained and forgone outcomes were encoded in mPFC, but with opposite sign in its ventral and dorsal subdivisions. Moreover, we showed that increasing feedback information induced a switch from absolute to relative coding. Our results suggest that complete feedback information enhances context-dependent outcome encoding.SIGNIFICANCE STATEMENT This study offers a systematic investigation of the effect of the amount of feedback information (partial vs complete) on univariate and multivariate outcome value encoding, within multiple regions in mPFC and cingulate cortex that are critical for value-based decisions and behavioral adaptation. Moreover, we provide the first comparison of three possible models of neural coding (i.e., absolute, partially-adaptive, and fully-adaptive coding) of value signal in these regions, by using commensurable measures of prediction accuracy. Taken together, our results help build a more comprehensive picture of how the human brain encodes and processes outcome value. In particular, our results suggest that simultaneous presentation of obtained and foregone outcomes promotes relative value representation.

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected sequential effects, including increased willingness to take risks following risky wins, and spatial 'win-stay/lose-shift' biases. Recordings from lateral orbitofrontal cortex (lOFC) revealed encoding of reward history and receipt, and optogenetic inhibition of lOFC eliminated rats' increased preference for risk following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and the lOFC's role in adaptive behavior promotes learning of more abstract biases (here, biases for the risky option), but not spatial ones.

Combinations of low-level and high-level neural processes account for distinct patterns of context-dependent choice

Context effects have been explained by either low-level neural adjustments or high-level cognitive processes but not their combination. It is currently unclear how these processes interact to shape individuals’ responses to context. Here, we used a large cohort of human subjects in experiments involving choice between two or three gambles in order to study the dependence of context effects on neural adaptation and individuals’ risk attitudes. Our experiments did not provide any evidence that neural adaptation on long timescales (~100 trials) contributes to context effects. Using post-hoc analyses we identified two groups of subjects with distinct patterns of responses to decoys, both of which depended on individuals’ risk aversion. Subjects in the first group exhibited strong, consistent decoy effects and became more risk averse due to decoy presentation. In contrast, subjects in the second group did not show consistent decoy effects and became more risk seeking. The degree of change in risk aversion due to decoy presentation was positively correlated with the original degrees of risk aversion. To explain these results and reveal underlying neural mechanisms, we developed new models incorporating both low- and high-level processes and used these models to fit individuals’ choice behavior. We found that observed distinct patterns of decoy effects can be explained by a combination of adjustments in neural representations and competitive weighting of reward attributes, both of which depend on risk aversion but in opposite directions. Altogether, our results demonstrate how a combination of low- and high-level processes shapes choice behavior in more naturalistic settings, modulates overall risk preference, and explains distinct behavioral phenotypes.

A large body of experimental work has illustrated that the introduction of a new, and often irrelevant, option can influence preference among the existing options, a phenomenon referred to as context or decoy effects. For example, introducing a new option that is worse than one of the two existing options in all its attributes but better than the alternative option in some attributes (and thus should not ever be selected) can increase the preference for the former option. Context effects have been explained by high-level cognitive processes—such as comparisons and competitions between attributes—or low-level adjustments of neural representations. However, it is unclear how these processes interact to shape individuals’ responses to context. Here, we show that both high-level cognitive processes and low-level neural adjustments shift risk preference during choice between multiple risky options but in opposite directions. Moreover, we demonstrate that combinations of these processes can account for distinct patterns of context effects in human subjects.

Reference effects on decision-making elicited by previous rewards

Substantial evidence has highlighted reference effects occurring during decision-making, whereby subjective value is not calculated in absolute terms but relative to the distribution of rewards characterizing a context. Among these, within-choice effects are exerted by options simultaneously available during choice. These should be distinguished from between-choice effects, which depend on the distribution of options presented in the past. Influential theories on between-choice effects include Decision-by-Sampling, Expectation-as-Reference and Divisive Normalization. Surprisingly, previous literature has focused on each theory individually disregarding the others. Thus, similarities and differences among theories remain to be systematically examined. Here we fill this gap by offering an overview of the state-of-the-art of research about between-choice reference effects. Our comparison of alternative theories shows that, at present, none of them is able to account for the full range of empirical data. To address this, we propose a model inspired by previous perspectives and based on a logistic framework, hence called logistic model of subjective value. Predictions of the model are analysed in detail about reference effects and risky decision-making. We conclude that our proposal offers a compelling framework for interpreting the multifaceted manifestations of between-choice reference effects.

Partial Adaptation to the Value Range in the Macaque Orbitofrontal Cortex

Values available for choice in different behavioral contexts can vary immensely. To compensate for this variability, neuronal circuits underlying economic decisions undergo adaptation. In orbitofrontal cortex (OFC), neurons encode the subjective value of offered and chosen goods in a quasilinear way. Previous experiments found that the gain of the encoding is lower when the value range is wider. However, the parameters OFC neurons adapted to remained unclear. Furthermore, previous studies did not examine additive changes in neuronal responses. Computational considerations indicate that these factors can directly impact choice behavior. Here we investigated how OFC neurons adapt to changes in the value range. We recorded from two male rhesus monkeys during a juice choice task. Each session was divided into two blocks of trials. In each block, juices were offered within a set range of values, and ranges changed between blocks. Across blocks, neuronal responses adapted to both the maximum and the minimum value, but only partially. As a result, the minimum neural activity was elevated in some value ranges relative to others. Through simulation of a linear decision model, we showed that increasing the minimum response increases choice variability, lowering the expected payoff. This effect is modulated by the balance between cells with positive and negative encoding. The presence of these two populations induces a non-monotonic relationship between the value range and choice efficacy, such that the expected payoff is highest for decisions in an intermediate value range.SIGNIFICANCE STATEMENT Economic decisions are thought to rely on the orbitofrontal cortex (OFC). The values available for choice vary enormously in different contexts. Previous work showed that neurons in OFC encode values in a linear way, and that the gain of encoding is inversely related to the range of available values. However, the specific parameters driving adaptation remained unclear. Here we show that OFC neurons adapt to both the maximum and minimum value in the current context. However, adaptation is partial, leading to contextual changes in the response offset. Interestingly, increasing the activity offset negatively affects choices in a simulated network. Partial adaptation may allow the circuit to maintain information about context value at the cost of slightly reduced payoff.

Efficient coding of subjective value

Preference-based decisions are essential for survival, for instance when deciding what we should (not) eat. Despite their importance, preference-based decisions are surprisingly variable and can appear irrational in ways that have defied mechanistic explanations. Here we propose that subjective valuation results from an inference process that accounts for the structure of values in the environment and that maximizes information in value representations in line with demands imposed by limited coding resources. A model of this inference process explains the variability in both subjective value reports and preference-based choices, and predicts a new preference illusion that we validate with empirical data. Interestingly, the same model explains the level of confidence associated with these reports. Our results imply that preference-based decisions reflect information-maximizing transmission and statistically optimal decoding of subjective values by a limited-capacity system. These findings provide a unified account of how humans perceive and valuate the environment to optimally guide behavior.

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

A review of inflammatory mechanism in airway diseases

BACKGROUND

Inflammation in the lung is the body's natural response to injury. It acts to remove harmful stimuli such as pathogens, irritants, and damaged cells and initiate the healing process. Acute and chronic pulmonary inflammation are seen in different respiratory diseases such as; acute respiratory distress syndrome, chronic obstructive pulmonary disease (COPD), asthma, and cystic fibrosis (CF).

FINDINGS

In this review, we found that inflammatory response in COPD is determined by the activation of epithelial cells and macrophages in the respiratory tract. Epithelial cells and macrophages discharge transforming growth factor-β (TGF-β), which trigger fibroblast proliferation and tissue remodeling. Asthma leads to airway hyper-responsiveness, obstruction, mucus hyper-production, and airway-wall remodeling. Cytokines, allergens, chemokines, and infectious agents are the main stimuli that activate signaling pathways in epithelial cells in asthma. Mutation of the CF transmembrane conductance regulator (CFTR) gene results in CF. Mutations in CFTR influence the lung epithelial innate immune function that leads to exaggerated and ineffective airway inflammation that fails to abolish pulmonary pathogens. We present mechanistic computational models (based on ordinary differential equations, partial differential equations and agent-based models) that have been applied in studying the complex physiological and pathological mechanisms of chronic inflammation in different airway diseases.

CONCLUSION

The scope of the present review is to explore the inflammatory mechanism in airway diseases and highlight the influence of aging on airways' inflammation mechanism. The main goal of this review is to encourage research collaborations between experimentalist and modelers to promote our understanding of the physiological and pathological mechanisms that control inflammation in different airway diseases.

Context Effects in the Judgment of Visual Relative-Frequency: Trial-by-Trial Adaptation and Non-linear Sequential Effect

Humans' judgment of relative-frequency, similar to their use of probability in decision-making, is often distorted as an inverted-S-shape curve-small relative-frequency overestimated and large relative-frequency underestimated. Here we investigated how the judgment of relative-frequency, despite its natural reference points (0 and 1) and stereotyped distortion, may adapt to the environmental statistics. The task was to report the relative-frequency of black (or white) dots in a visual array of black and white dots. We found that participants' judgment was distorted in the typical inverted-S-shape, but the distortion curve was influenced by both the central tendency and spread of the distribution of objective relative-frequencies: the lower the central tendency, the higher the overall judgment (contrast effect); the higher the spread, the more curved the inverted-S-shape (curvature effect). These context effects are in the spirit of efficient coding but opposite to what would be predicted by Bayesian inference. We further modeled the context effects on the level of individual trials, through which we found not only a trial-by-trial adaptation, but also the non-linear sequential effects that were recently reported mainly in circularly distributed visual stimuli.

Multiple timescales of normalized value coding underlie adaptive choice behavior

Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.

Free choice shapes normalized value signals in medial orbitofrontal cortex

Normalization is a common cortical computation widely observed in sensory perception, but its importance in perception of reward value and decision making remains largely unknown. We examined (1) whether normalized value signals occur in the orbitofrontal cortex (OFC) and (2) whether changes in behavioral task context influence the normalized representation of value. We record medial OFC (mOFC) single neuron activity in awake-behaving monkeys during a reward-guided lottery task. mOFC neurons signal the relative values of options via a divisive normalization function when animals freely choose between alternatives. The normalization model, however, performed poorly in a variant of the task where only one of the two possible choice options yields a reward and the other was certain not to yield a reward (so called: “forced choice”). The existence of such context-specific value normalization may suggest that the mOFC contributes valuation signals critical for economic decision making when meaningful alternative options are available.

Neurons in prefrontal areas including the medial orbitofrontal cortex (mOFC) represent the relative reward value of choices. Here the authors report that mOFC neurons implement divisive normalization to encode the relative values of lottery options only when the decision involves free choice.

Orbitofrontal Cortex: A Neural Circuit for Economic Decisions

Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.