A neuronal theory of sequential economic choice

Semi-orthogonal subspaces for value mediate a binding and generalization trade-off

When choosing between options, we must associate their values with the actions needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. Here, in macaques performing a choice task, we show that neural populations in five reward-sensitive regions encode the values of offers presented on the left and right in distinct subspaces. This encoding is sufficient to bind offer values to their locations while preserving abstract value information. After offer presentation, all areas encode the value of the first and second offers in orthogonal subspaces; this orthogonalization also affords binding. Our binding-by-subspace hypothesis makes two new predictions confirmed by the data. First, behavioral errors should correlate with spatial, but not temporal, neural misbinding. Second, behavioral errors should increase when offers have low or high values, compared to medium values, even when controlling for value difference. Together, these results support the idea that the brain uses semi-orthogonal subspaces to bind features.

Rational inattention in neural coding for economic choice

Mental operations like computing the value of an option are computationally expensive. Even before we evaluate options, we must decide how much attentional effort to invest in the evaluation process. More precise evaluation will improve choice accuracy, and thus reward yield, but the gain may not justify the cost. Rational Inattention theories provide an accounting of the internal economics of attentionally effortful economic decisions. To understand this process, we examined choices and neural activity in several brain regions in six macaques making risky choices. We extended the rational inattention framework to incorporate the foraging theoretic understanding of local environmental richness or reward rate, which we predict will promote attentional effort. Consistent with this idea, we found local reward rate positively predicted choice accuracy. Supporting the hypothesis that this effect reflects variations in attentional effort, richer contexts were associated with increased baseline and evoked pupil size. Neural populations likewise showed systematic baseline coding of reward rate context. During increased reward rate contexts, ventral striatum and orbitofrontal cortex showed both an increase in value decodability and a rotation in the population geometries for value. This confluence of these results suggests a mechanism of attentional effort that operates by controlling gain through using partially distinct population codes for value. Additionally, increased reward rate accelerated value code dynamics, which have been linked to improved signal-to-noise. These results extend the theory of rational inattention to static and stationary contexts and align theories of rational inattention with specific costly, neural processes.

Gaze-centered gating, reactivation, and reevaluation of economic value in orbitofrontal cortex

During economic choice, options are often considered in alternation, until commitment. Nonetheless, neuroeconomics typically ignores the dynamic aspects of deliberation. We trained two male macaques to perform a value-based decision-making task in which two risky offers were presented in sequence at the opposite sides of the visual field, each followed by a delay epoch where offers were invisible. Surprisingly, during the two delays, subjects tend to look at empty locations where the offers had previously appeared, with longer fixations increasing the probability of choosing the associated offer. Spiking activity in orbitofrontal cortex reflects the value of the gazed offer, or of the offer associated with the gazed empty spatial location, even if it is not the most recent. This reactivation reflects a reevaluation process, as fluctuations in neural spiking correlate with upcoming choice. Our results suggest that look-at-nothing gazing triggers the reactivation of a previously seen offer for further evaluation.

Semi-orthogonal subspaces for value mediate a tradeoff between binding and generalization

When choosing between options, we must associate their values with the action needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. To test this hypothesis, we examined neuronal responses in five reward-sensitive regions in macaques performing a risky choice task with sequential offers. Surprisingly, in all areas, the neural population encoded the values of offers presented on the left and right in distinct subspaces. We show that the encoding we observe is sufficient to bind the values of the offers to their respective positions in space while preserving abstract value information, which may be important for rapid learning and generalization to novel contexts. Moreover, after both offers have been presented, all areas encode the value of the first and second offers in orthogonal subspaces. In this case as well, the orthogonalization provides binding. Our binding-by-subspace hypothesis makes two novel predictions borne out by the data. First, behavioral errors should correlate with putative spatial (but not temporal) misbinding in the neural representation. Second, the specific representational geometry that we observe across animals also indicates that behavioral errors should increase when offers have low or high values, compared to when they have medium values, even when controlling for value difference. Together, these results support the idea that the brain makes use of semi-orthogonal subspaces to bind features together.

The orbitofrontal cortex: A goal-directed cognitive map framework for social and non-social behaviors

The orbitofrontal cortex (OFC) is regarded as one of the core brain areas in a variety of value-based behaviors. Over the past two decades, tremendous knowledge about the OFC function was gained from studying the behaviors of single subjects. As a result, our previous understanding of the OFC's function of encoding decision variables, such as the value and identity of choices, has evolved to the idea that the OFC encodes a more complex representation of the task space as a cognitive map. Accumulating evidence also indicates that the OFC importantly contributes to behaviors in social contexts, especially those involved in cooperative interactions. However, it remains elusive how exactly OFC neurons contribute to social functions and how non-social and social behaviors are related to one another in the computations performed by OFC neurons. In this review, we aim to provide an integrated view of the OFC function across both social and non-social behavioral contexts. We propose that seemingly complex functions of the OFC may be explained by its role in providing a goal-directed cognitive map to guide a wide array of adaptive reward-based behaviors in both social and non-social domains.

Hierarchical organization of rhesus macaque behavior

Primatologists, psychologists and neuroscientists have long hypothesized that primate behavior is highly structured. However, delineating that structure has been impossible due to the difficulties of precision behavioral tracking. Here we analyzed a dataset consisting of continuous measures of the 3D position of two male rhesus macaques (Macaca mulatta) performing three different tasks in a large unrestrained environment over several hours. Using an unsupervised embedding approach on the tracked joints, we identified commonly repeated pose patterns, which we call postures. We found that macaques' behavior is characterized by 49 distinct postures, lasting an average of 0.6 seconds. We found evidence that behavior is hierarchically organized, in that transitions between poses tend to occur within larger modules, which correspond to identifiable actions; these actions are further organized hierarchically. Our behavioral decomposition allows us to identify universal (cross-individual and cross-task) and unique (specific to each individual and task) principles of behavior. These results demonstrate the hierarchical nature of primate behavior, provide a method for the automated ethogramming of primate behavior, and provide important constraints on neural models of pose generation.

Subspace orthogonalization as a mechanism for binding values to space

When choosing between options, we must solve an important binding problem. The values of the options must be associated with information about the action needed to select them. We hypothesize that the brain solves this binding problem through use of distinct population subspaces. To test this hypothesis, we examined the responses of single neurons in five reward-sensitive regions in rhesus macaques performing a risky choice task. In all areas, neurons encoded the value of the offers presented on both the left and the right side of the display in semi-orthogonal subspaces, which served to bind the values of the two offers to their positions in space. Supporting the idea that this orthogonalization is functionally meaningful, we observed a session-to-session covariation between choice behavior and the orthogonalization of the two value subspaces: trials with less orthogonalized subspaces were associated with greater likelihood of choosing the less valued option. Further inspection revealed that these semi-orthogonal subspaces arose from a combination of linear and nonlinear mixed selectivity in the neural population. We show this combination of selectivity balances reliable binding with an ability to generalize value across different spatial locations. These results support the hypothesis that semi-orthogonal subspaces support reliable binding, which is essential to flexible behavior in the face of multiple options.

Reward salience but not spatial attention dominates the value representation in the orbitofrontal cortex

The orbitofrontal cortex (OFC) encodes value and plays a key role in value-based decision-making. However, the attentional modulation of the OFC's value encoding is poorly understood. We trained two monkeys to detect a luminance change at a cued location between a pair of visual stimuli, which were over-trained pictures associated with different amounts of juice reward and, thus, different reward salience. Both the monkeys' behavior and the dorsolateral prefrontal cortex neuronal activities indicated that the monkeys actively directed their spatial attention toward the cued stimulus during the task. However, the OFC's neuronal responses were dominated by the stimulus with higher reward salience and encoded its value. The value of the less salient stimulus was only weakly represented regardless of spatial attention. The results demonstrate that reward and spatial attention are distinctly represented in the prefrontal cortex and the OFC maintains a stable representation of reward salience minimally affected by attention.

Orbitofrontal cortex contributes to the comparison of values underlying economic choices

Economic choices between goods entail the computation and comparison of subjective values. Previous studies examined neuronal activity in the orbitofrontal cortex (OFC) of monkeys choosing between different types of juices. Three groups of neurons were identified: offer value cells encoding the value of individual offers, chosen juice cells encoding the identity of the chosen juice, and chosen value cells encoding the value of the chosen offer. The encoded variables capture both the input (offer value) and the output (chosen juice, chosen value) of the decision process, suggesting that values are compared within OFC. Recent work demonstrates that choices are causally linked to the activity of offer value cells. Conversely, the hypothesis that OFC contributes to value comparison has not been confirmed. Here we show that weak electrical stimulation of OFC specifically disrupts value comparison without altering offer values. This result implies that neuronal populations in OFC participate in value comparison.

The Promise of Behavioral Tracking Systems for Advancing Primate Animal Welfare

Recent years have witnessed major advances in the ability of computerized systems to track the positions of animals as they move through large and unconstrained environments. These systems have so far been a great boon in the fields of primatology, psychology, neuroscience, and biomedicine. Here, we discuss the promise of these technologies for animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare within the context of the three Rs of animal research (reduction, refinement, and replacement), and applying our understanding of animal behavior to increase the "natural" behaviors in captive and wild populations facing human impact challenges. We note that these benefits are often incidental to the designed purpose of these tracking systems, a reflection of the fact that animal welfare is not inimical to research progress, but instead, that the aligned interests between basic research and welfare hold great promise for improvements to animal well-being.

A structural and functional subdivision in central orbitofrontal cortex

Economic choice requires many cognitive subprocesses, including stimulus detection, valuation, motor output, and outcome monitoring; many of these subprocesses are associated with the central orbitofrontal cortex (cOFC). Prior work has largely assumed that the cOFC is a single region with a single function. Here, we challenge that unified view with convergent anatomical and physiological results from rhesus macaques. Anatomically, we show that the cOFC can be subdivided according to its much stronger (medial) or weaker (lateral) bidirectional anatomical connectivity with the posterior cingulate cortex (PCC). We call these subregions cOFCm and cOFCl, respectively. These two subregions have notable functional differences. Specifically, cOFCm shows enhanced functional connectivity with PCC, as indicated by both spike-field coherence and mutual information. The cOFCm-PCC circuit, but not the cOFCl-PCC circuit, shows signatures of relaying choice signals from a non-spatial comparison framework to a spatially framed organization and shows a putative bidirectional mutually excitatory pattern.

Deep imagination is a close to optimal policy for planning in large decision trees under limited resources

Many decisions involve choosing an uncertain course of action in deep and wide decision trees, as when we plan to visit an exotic country for vacation. In these cases, exhaustive search for the best sequence of actions is not tractable due to the large number of possibilities and limited time or computational resources available to make the decision. Therefore, planning agents need to balance breadth-considering many actions in the first few tree levels-and depth-considering many levels but few actions in each of them-to allocate optimally their finite search capacity. We provide efficient analytical solutions and numerical analysis to the problem of allocating finite sampling capacity in one shot to infinitely large decision trees, both in the time discounted and undiscounted cases. We find that in general the optimal policy is to allocate few samples per level so that deep levels can be reached, thus favoring depth over breadth search. In contrast, in poor environments and at low capacity, it is best to broadly sample branches at the cost of not sampling deeply, although this policy is marginally better than deep allocations. Our results can provide a theoretical foundation for why human reasoning is pervaded by imagination-based processes.

Optimal Allocation of Finite Sampling Capacity in Accumulator Models of Multialternative Decision Making

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here, we study the problem of how to optimally allocate limited sampling time to multiple options, modeled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sublinearly, closely following a power law as a function of capacity for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multialternative decisions are endowed with breadth–depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

Neuroscience needs evolution

The nervous system is a product of evolution. That is, it was constructed through a long series of modifications, within the strong constraints of heredity, and continuously subjected to intense selection pressures. As a result, the organization and functions of the brain are shaped by its history. We believe that this fact, underappreciated in contemporary systems neuroscience, offers an invaluable aid for helping us resolve the brain's mysteries. Indeed, we think that the consideration of evolutionary history ought to take its place alongside other intellectual tools used to understand the brain, such as behavioural experiments, studies of anatomical structure and functional characterization based on recordings of neural activity. In this introduction, we argue for the importance of evolution by highlighting specific examples of ways that evolutionary theory can enhance neuroscience. The rest of the theme issue elaborates this point, emphasizing the conservative nature of neural evolution, the important consequences of specific transitions that occurred in our history, and the ways in which considerations of evolution can shed light on issues ranging from specific mechanisms to fundamental principles of brain organization.

This article is part of the theme issue ‘Systems neuroscience through the lens of evolutionary theory’.

The whole prefrontal cortex is premotor cortex

We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Economic Choices under Simultaneous or Sequential Offers Rely on the Same Neural Circuit

A series of studies in which monkeys chose between two juices offered in variable amounts identified in the orbitofrontal cortex (OFC) different groups of neurons encoding the value of individual options (offer value), the binary choice outcome (chosen juice), and the chosen value. These variables capture both the input and the output of the choice process, suggesting that the cell groups identified in OFC constitute the building blocks of a decision circuit. Several lines of evidence support this hypothesis. However, in previous experiments offers were presented simultaneously, raising the question of whether current notions generalize to when goods are presented or are examined in sequence. Recently, Ballesta and Padoa-Schioppa (2019) examined OFC activity under sequential offers. An analysis of neuronal responses across time windows revealed that a small number of cell groups encoded specific sequences of variables. These sequences appeared analogous to the variables identified under simultaneous offers, but the correspondence remained tentative. Thus, in the present study, we examined the relation between cell groups found under sequential versus simultaneous offers. We recorded from the OFC while monkeys chose between different juices. Trials with simultaneous and sequential offers were randomly interleaved in each session. We classified cells in each choice modality, and we examined the relation between the two classifications. We found a strong correspondence; in other words, the cell groups measured under simultaneous offers and under sequential offers were one and the same. This result indicates that economic choices under simultaneous or sequential offers rely on the same neural circuit.SIGNIFICANCE STATEMENT Research in the past 20 years has shed light on the neuronal underpinnings of economic choices. A large number of results indicates that decisions between goods are formed in a neural circuit within the orbitofrontal cortex. In most previous studies, subjects chose between two goods offered simultaneously. Yet, in daily situations, goods available for choice are often presented or examined in sequence. Here we recorded neuronal activity in the primate orbitofrontal cortex alternating trials under simultaneous and under sequential offers. Our analyses demonstrate that the same neural circuit supports choices in the two modalities. Hence, current notions on the neuronal mechanisms underlying economic decisions generalize to choices under sequential offers.

The population doctrine in cognitive neuroscience

A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.

The road towards understanding embodied decisions

Most current decision-making research focuses on classical economic scenarios, where choice offers are prespecified and where action dynamics play no role in the decision. However, our brains evolved to deal with different choice situations: "embodied decisions". As examples of embodied decisions, consider a lion that has to decide which gazelle to chase in the savannah or a person who has to select the next stone to jump on when crossing a river. Embodied decision settings raise novel questions, such as how people select from time-varying choice options and how they track the most relevant choice attributes; but they have long remained challenging to study empirically. Here, we summarize recent progress in the study of embodied decisions in sports analytics and experimental psychology. Furthermore, we introduce a formal methodology to identify the relevant dimensions of embodied choices (present and future affordances) and to map them into the attributes of classical economic decisions (probabilities and utilities), hence aligning them. Studying embodied decisions will greatly expand our understanding of what decision-making is.

Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex

Choice-relevant brain regions in prefrontal cortex may progressively transform information about options into choices. Here, we examine responses of neurons in four regions of the medial prefrontal cortex as macaques performed two-option risky choices. All four regions encode economic variables in similar proportions and show similar putative signatures of key choice-related computations. We provide evidence to support a gradient of function that proceeds from areas 14 to 25 to 32 to 24. Specifically, we show that decodability of twelve distinct task variables increases along that path, consistent with the idea that regions that are higher in the anatomical hierarchy make choice-relevant variables more separable. We also show progressively longer intrinsic timescales in the same series. Together these results highlight the importance of the medial wall in choice, endorse a specific gradient-based organization, and argue against a modular functional neuroanatomy of choice.

Partial integration of the components of value in anterior cingulate cortex

Evaluation often involves integrating multiple determinants of value, such as the different possible outcomes in risky choice. A brain region can be placed either before or after a presumed evaluation stage by measuring how responses of its neurons depend on multiple determinants of value. A brain region could also, in principle, show partial integration, which would indicate that it occupies a middle position between (preevaluative) nonintegration and (postevaluative) full integration. Existing mathematical techniques cannot distinguish full from partial integration and therefore risk misidentifying regional function. Here we use a new Bayesian regression-based approach to analyze responses of neurons in dorsal anterior cingulate cortex (dACC) to risky offers. We find that dACC neurons only partially integrate across outcome dimensions, indicating that dACC cannot be assigned to either a pre- or postevaluative position. Neurons in dACC also show putative signatures of value comparison, thereby demonstrating that comparison does not require complete evaluation before proceeding. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Multicentric tracking of multiple agents by anterior cingulate cortex during pursuit and evasion

Successful pursuit and evasion require rapid and precise coordination of navigation with adaptive motor control. We hypothesize that the dorsal anterior cingulate cortex (dACC), which communicates bidirectionally with both the hippocampal complex and premotor/motor areas, would serve a mapping role in this process. We recorded responses of dACC ensembles in two macaques performing a joystick-controlled continuous pursuit/evasion task. We find that dACC carries two sets of signals, (1) world-centric variables that together form a representation of the position and velocity of all relevant agents (self, prey, and predator) in the virtual world, and (2) avatar-centric variables, i.e. self-prey distance and angle. Both sets of variables are multiplexed within an overlapping set of neurons. Our results suggest that dACC may contribute to pursuit and evasion by computing and continuously updating a multicentric representation of the unfolding task state, and support the hypothesis that it plays a high-level abstract role in the control of behavior.

Prefrontal cortex represents heuristics that shape choice bias and its integration into future behavior

Goal-directed behavior requires integrating sensory information with prior knowledge about the environment. Behavioral biases that arise from these priors could increase positive outcomes when the priors match the true structure of the environment, but mismatches also happen frequently and could cause unfavorable outcomes. Biases that reduce gains and fail to vanish with training indicate fundamental suboptimalities arising from ingrained heuristics of the brain. Here, we report systematic, gain-reducing choice biases in highly trained monkeys performing a motion direction discrimination task where only the current stimulus is behaviorally relevant. The monkey's bias fluctuated at two distinct time scales: slow, spanning tens to hundreds of trials, and fast, arising from choices and outcomes of the most recent trials. Our findings enabled single trial prediction of biases, which influenced the choice especially on trials with weak stimuli. The pre-stimulus activity of neuronal ensembles in the monkey prearcuate gyrus represented these biases as an offset along the decision axis in the state space. This offset persisted throughout the stimulus viewing period, when sensory information was integrated, leading to a biased choice. The pre-stimulus representation of history-dependent bias was functionally indistinguishable from the neural representation of upcoming choice before stimulus onset, validating our model of single-trial biases and suggesting that pre-stimulus representation of choice could be fully defined by biases inferred from behavioral history. Our results indicate that the prearcuate gyrus reflects intrinsic heuristics that compute bias signals, as well as the mechanisms that integrate them into the oculomotor decision-making process.

Continuous decisions

Humans and other animals evolved to make decisions that extend over time with continuous and ever-changing options. Nonetheless, the academic study of decision-making is mostly limited to the simple case of choice between two options. Here, we advocate that the study of choice should expand to include continuous decisions. Continuous decisions, by our definition, involve a continuum of possible responses and take place over an extended period of time during which the response is continuously subject to modification. In most continuous decisions, the range of options can fluctuate and is affected by recent responses, making consideration of reciprocal feedback between choices and the environment essential. The study of continuous decisions raises new questions, such as how abstract processes of valuation and comparison are co-implemented with action planning and execution, how we simulate the large number of possible futures our choices lead to, and how our brains employ hierarchical structure to make choices more efficiently. While microeconomic theory has proven invaluable for discrete decisions, we propose that engineering control theory may serve as a better foundation for continuous ones. And while the concept of value has proven foundational for discrete decisions, goal states and policies may prove more useful for continuous ones. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Dynamics of a Mutual Inhibition Circuit between Pyramidal Neurons Compared to Human Perceptual Competition

Neural competition plays an essential role in active selection processes of noisy and ambiguous input signals, and it is assumed to underlie emergent properties of brain functioning, such as perceptual organization and decision-making. Despite ample theoretical research on neural competition, experimental tools to allow neurophysiological investigation of competing neurons have not been available. We developed a "hybrid" system where real-life neurons and a computer-simulated neural circuit interacted. It enabled us to construct a mutual inhibition circuit between two real-life pyramidal neurons. We then asked what dynamics this minimal unit of neural competition exhibits and compared them with the known behavioral-level dynamics of neural competition. We found that the pair of neurons shows bistability when activated simultaneously by current injections. The addition of modeled synaptic noise and changes in the activation strength showed that the dynamics of the circuit are strikingly similar to the known properties of bistable visual perception: The distribution of dominance durations showed a right-skewed shape, and the changes of the activation strengths caused changes in dominance, dominance durations, and reversal rates as stated in the well-known empirical laws of bistable perception known as Levelt's propositions.SIGNIFICANCE STATEMENT Visual perception emerges as the result of neural systems actively organizing visual signals that involves selection processes of competing neurons. While the neural competition, realized by a "mutual inhibition" circuit has been examined in many theoretical studies, its properties have not been investigated in real neurons. We have developed a "hybrid" system where two real-life pyramidal neurons in a mouse brain slice interact through a computer-simulated mutual inhibition circuit. We found that simultaneous activation of the neurons leads to bistable activity. We investigated the effect of noise and the effect of changes in the activation strength on the dynamics. We observed that the pair of neurons exhibit dynamics strikingly similar to the known properties of bistable visual perception.

Heuristics and optimal solutions to the breadth-depth dilemma

In multialternative risky choice, we are often faced with the opportunity to allocate our limited information-gathering capacity between several options before receiving feedback. In such cases, we face a natural trade-off between breadth-spreading our capacity across many options-and depth-gaining more information about a smaller number of options. Despite its broad relevance to daily life, including in many naturalistic foraging situations, the optimal strategy in the breadth-depth trade-off has not been delineated. Here, we formalize the breadth-depth dilemma through a finite-sample capacity model. We find that, if capacity is small (∼10 samples), it is optimal to draw one sample per alternative, favoring breadth. However, for larger capacities, a sharp transition is observed, and it becomes best to deeply sample a very small fraction of alternatives, which roughly decreases with the square root of capacity. Thus, ignoring most options, even when capacity is large enough to shallowly sample all of them, is a signature of optimal behavior. Our results also provide a rich casuistic for metareasoning in multialternative decisions with bounded capacity using close-to-optimal heuristics.

Choosing fast and simply: Construction of preferences by starlings through parallel option valuation

The integration of normative and descriptive analyses of decision processes in humans struggles with the fact that measuring preferences by different procedures yields different rankings and that humans appear irrationally impulsive (namely, show maladaptive preference for immediacy). Failure of procedure invariance has led to the widespread hypothesis that preferences are constructed "on the spot" by cognitive evaluations performed at choice time, implying that choices should take extra time in order to perform the necessary comparisons. We examine this issue in experiments with starlings (Sturnus vulgaris) and show that integrating normative and descriptive arguments is possible and may help reinterpreting human decision results. Our main findings are that (1) ranking alternatives through direct rating (response time) accurately predicts preference in choice, overcoming failures of procedure invariance; (2) preference is not constructed at choice time nor does it involve extra time (we show that the opposite is true); and (3) starlings' choices are not irrationally impulsive but are instead directly interpretable in terms of profitability ranking. Like all nonhuman research, our protocols examine decisions by experience rather than by description, and hence support the conjecture that irrationalities that prevail in research with humans may not be observed in decisions by experience protocols.

Representation of probabilistic outcomes during risky decision-making

Goal-directed behaviour requires prospectively retrieving and evaluating multiple possible action outcomes. While a plethora of studies suggested sequential retrieval for deterministic choice outcomes, it remains unclear whether this is also the case when integrating multiple probabilistic outcomes of the same action. We address this question by capitalising on magnetoencephalography (MEG) in humans who made choices in a risky foraging task. We train classifiers to distinguish MEG field patterns during presentation of two probabilistic outcomes (reward, loss), and then apply these to decode such patterns during deliberation. First, decoded outcome representations have a temporal structure, suggesting alternating retrieval of the outcomes. Moreover, the probability that one or the other outcome is being represented depends on loss magnitude, but not on loss probability, and it predicts the chosen action. In summary, we demonstrate decodable outcome representations during probabilistic decision-making, which are sequentially structured, depend on task features, and predict subsequent action.

The Transition from Evaluation to Selection Involves Neural Subspace Reorganization in Core Reward Regions

Economic choice proceeds from evaluation, in which we contemplate options, to selection, in which we weigh options and choose one. These stages must be differentiated so that decision makers do not proceed to selection before evaluation is complete. We examined responses of neurons in two core reward regions, orbitofrontal (OFC) and ventromedial prefrontal cortex (vmPFC), during two-option choice with asynchronous offer presentation. Our data suggest that neurons selective during the first (presumed evaluation) and second (presumed comparison and selection) offer epochs come from a single pool. Stage transition is accompanied by a shift toward orthogonality in the low-dimensional population response manifold. Nonetheless, the relative position of each option in driving responses in the population subspace is preserved. The orthogonalization we observe supports the hypothesis that the transition from evaluation to selection leads to reorganization of response subspace and suggests a mechanism by which value-related signals are prevented from prematurely driving choice.

Activity in orbitofrontal neuronal ensembles reflects inhibitory control

Stopping, or inhibition, is a form of self-control that is a core element of flexible and adaptive behavior. Its neural origins remain unclear. Some views hold that inhibition decisions reflect the aggregation of widespread and diverse pieces of information, including information arising in ostensible core reward regions (i.e., outside the canonical executive system). We recorded activity of single neurons in the orbitofrontal cortex (OFC) of macaques, a region associated with economic decisions, and whose role in inhibition is debated. Subjects performed a classic inhibition task known as the stop signal task. Ensemble decoding analyses reveal a clear firing rate pattern that distinguishes successful from failed inhibition and that begins after the stop signal and before the stop signal reaction time (SSRT). We also found a different and orthogonal ensemble pattern that distinguishes successful from failed stopping before the beginning of the trial. These signals were distinct from, and orthogonal to, value encoding, which was also observed in these neurons. The timing of the early and late signals was, respectively, consistent with the idea that neuronal activity in OFC encodes inhibition both proactively and reactively.

Economic Decisions through Circuit Inhibition

Economic choices between goods are thought to rely on the orbitofrontal cortex (OFC), but the decision mechanisms remain poorly understood. To shed light on this fundamental issue, we recorded from the OFC of monkeys choosing between two juices offered sequentially. An analysis of firing rates across time windows revealed the presence of different groups of neurons similar to those previously identified under simultaneous offers. This observation suggested that economic decisions in the two modalities are formed in the same neural circuit. We then examined several hypotheses on the decision mechanisms. OFC neurons encoded good identities and values in a juice-based representation (labeled lines). Contrary to previous assessments, our data argued against the idea that decisions rely on mutual inhibition at the level of offer values. In fact, we showed that previous arguments for mutual inhibition were confounded by differences in value ranges. Instead, decisions seemed to involve mechanisms of circuit inhibition, whereby each offer value indirectly inhibited neurons encoding the opposite choice outcome. Our results reconcile a variety of previous findings and provide a general account for the neuronal underpinnings of economic choices.

Why has evolution not selected for perfect self-control?

Self-control refers to the ability to deliberately reject tempting options and instead select ones that produce greater long-term benefits. Although some apparent failures of self-control are, on closer inspection, reward maximizing, at least some self-control failures are clearly disadvantageous and non-strategic. The existence of poor self-control presents an important evolutionary puzzle because there is no obvious reason why good self-control should be more costly than poor self-control. After all, a rock is infinitely patient. I propose that self-control failures result from cases in which well-learned (and thus routinized) decision-making strategies yield suboptimal choices. These mappings persist in the decision-makers' repertoire because they result from learning processes that are adaptive in the broader context, either on the timescale of learning or of evolution. Self-control, then, is a form of cognitive control and the subjective feeling of effort likely reflects the true costs of cognitive control. Poor self-control, in this view, is ultimately a result of bounded optimality. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications.

Opposing pupil responses to offered and anticipated reward values

Previous studies have shown that the pupils dilate more in anticipation of larger rewards. This finding raises the possibility of a more general association between reward amount and pupil size. We tested this idea by characterizing macaque pupil responses to offered rewards during evaluation and comparison in a binary choice task. To control attention, we made use of a design in which offers occurred in sequence. By looking at pupil responses after choice but before reward, we confirmed the previously observed positive association between pupil size and anticipated reward values. Surprisingly, however, we find that pupil size is negatively correlated with the value of offered gambles before choice, during both evaluation and comparison stages of the task. These results demonstrate a functional distinction between offered and anticipated rewards and present evidence against a narrow version of the simulation hypothesis; the idea that we represent offers by reactivating states associated with anticipating them. They also suggest that pupil size is correlated with relative, not absolute, values of offers, suggestive of an accept-reject model of comparison.

Economic Choice as an Untangling of Options into Actions

We propose that economic choice can be understood as a gradual transformation from a domain of options to one of the actions. We draw an analogy with the idea of untangling information in the form vision system and propose that form vision and economic choice may be two aspects of a larger process that sculpts actions based on sensory inputs. From this viewpoint, choice results from the accumulated effect of repetitions of simple computations. These may consist primarily of relative valuations (evaluations relative to the value of rejection, perhaps in a manner akin to divisive normalization) applied to individual offers. With regard to economic choice, cortical brain regions differ primarily in their position and in what information they prioritize, and do not-with a few exceptions-have categorically distinct roles. Each region's specific contribution is determined largely by its inputs; thus, understanding connectivity is crucial for understanding choice. This view suggests that there is no single site of choice, that there is no meaningful distinction between pre- and post-decisionality, and that there is no explicit representation of value in the brain.