Brain mechanisms for perceptual and reward-related decision-making

The memory systems of the human brain and generative artificial intelligence

Generative Artificial Intelligence foundation models (for example Generative Pre-trained Transformer - GPT - models) can generate the next token given a sequence of tokens. How can this 'generative AI' be compared with the 'real' intelligence of the human brain, when for example a human generates a whole memory in response to an incomplete retrieval cue, and then generates further prospective thoughts? Here these two types of generative intelligence, artificial in machines and real in the human brain are compared, and it is shown how when whole memories are generated by hippocampal recall in response to an incomplete retrieval cue, what the human brain computes, and how it computes it, are very different from generative AI. Key differences are the use of local associative learning rules in the hippocampal memory system, and of non-local backpropagation of error learning in AI. Indeed, it is argued that the whole operation of the human brain is performed computationally very differently to what is implemented in generative AI. Moreover, it is emphasized that the primate including human hippocampal system includes computations about spatial view and where objects and people are in scenes, whereas in rodents the emphasis is on place cells and path integration by movements between places. This comparison with generative memory and processing in the human brain has interesting implications for the further development of generative AI and for neuroscience research.

Taste the emotions: pilot for a novel, sensors-based approach to emotional analysis during coffee tasting

BACKGROUND

Coffee is a natural drink with important properties for the human body and mind, capable of delivering energy and strong emotions, thus being appreciated since ancient times. The qualitative and quantitative assessment of the coffee properties is normally performed by trained panelists, though relying on standardized questionnaires, with possible biases arising. In this study, for the first time in the scientific literature, we applied a technology-based approach, based on the use of wearable sensors, to study the implicit emotional responses of a small cohort of experienced coffee judges, thus taking this chance to assess the feasibility of this approach in such a scenario. The merging of different technologies for capturing biomedical signals, including electrocardiogram, galvanic skin response, and electroencephalogram, was therefore adopted to retrieve results in terms of the relationships between implicit (i.e. psychophysiological) and explicit (i.e. derived from questionnaires) measurements.

RESULTS

Significant correlations were obtained between biomedical signals and data from the questionnaires within all the sensory domains (olfaction, vision, taste) investigated, particularly concerning autonomic-related features.

CONCLUSIONS

The results obtained confirmed the viability of this new approach in the psychophysical and emotional assessment in coffee tasting judges, paving the way for a new perspective into the universe of coffee quality assessment panels, eventually transferable to broader scale investigations, somewhat dealing with consumer satisfaction and neuromarketing at large. © 2023 Society of Chemical Industry.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here we provide evidence that the energy landscape around attractor basins in population neural activity in the prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays to reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here, we provide evidence that the energy landscape around attractor basins in population neural activity in prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays-to-reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

A neural mechanism for terminating decisions

The brain makes decisions by accumulating evidence until there is enough to stop and choose. Neural mechanisms of evidence accumulation are established in association cortex, but the site and mechanism of termination are unknown. Here, we show that the superior colliculus (SC) plays a causal role in terminating decisions, and we provide evidence for a mechanism by which this occurs. We recorded simultaneously from neurons in the lateral intraparietal area (LIP) and SC while monkeys made perceptual decisions. Despite similar trial-averaged activity, we found distinct single-trial dynamics in the two areas: LIP displayed drift-diffusion dynamics and SC displayed bursting dynamics. We hypothesized that the bursts manifest a threshold mechanism applied to signals represented in LIP to terminate the decision. Consistent with this hypothesis, SC inactivation produced behavioral effects diagnostic of an impaired threshold sensor and prolonged the buildup of activity in LIP. The results reveal the transformation from deliberation to commitment.

Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala

The orbitofrontal cortex and amygdala are involved in emotion and in motivation, but the relationship between these functions performed by these brain structures is not clear. To address this, a unified theory of emotion and motivation is described in which motivational states are states in which instrumental goal-directed actions are performed to obtain rewards or avoid punishers, and emotional states are states that are elicited when the reward or punisher is or is not received. This greatly simplifies our understanding of emotion and motivation, for the same set of genes and associated brain systems can define the primary or unlearned rewards and punishers such as sweet taste or pain. Recent evidence on the connectivity of human brain systems involved in emotion and motivation indicates that the orbitofrontal cortex is involved in reward value and experienced emotion with outputs to cortical regions including those involved in language, and is a key brain region involved in depression and the associated changes in motivation. The amygdala has weak effective connectivity back to the cortex in humans, and is implicated in brainstem-mediated responses to stimuli such as freezing and autonomic activity, rather than in declarative emotion. The anterior cingulate cortex is involved in learning actions to obtain rewards, and with the orbitofrontal cortex and ventromedial prefrontal cortex in providing the goals for navigation and in reward-related effects on memory consolidation mediated partly via the cholinergic system.

Structure and function in artificial, zebrafish and human neural networks

Network science provides a set of tools for the characterization of the structure and functional behavior of complex systems. Yet a major problem is to quantify how the structural domain is related to the dynamical one. In other words, how the diversity of dynamical states of a system can be predicted from the static network structure? Or the reverse problem: starting from a set of signals derived from experimental recordings, how can one discover the network connections or the causal relations behind the observed dynamics? Despite the advances achieved over the last two decades, many challenges remain concerning the study of the structure-dynamics interplay of complex systems. In neuroscience, progress is typically constrained by the low spatio-temporal resolution of experiments and by the lack of a universal inferring framework for empirical systems. To address these issues, applications of network science and artificial intelligence to neural data have been rapidly growing. In this article, we review important recent applications of methods from those fields to the study of the interplay between structure and functional dynamics of human and zebrafish brain. We cover the selection of topological features for the characterization of brain networks, inference of functional connections, dynamical modeling, and close with applications to both the human and zebrafish brain. This review is intended to neuroscientists who want to become acquainted with techniques from network science, as well as to researchers from the latter field who are interested in exploring novel application scenarios in neuroscience.

Neural Mechanisms That Make Perceptual Decisions Flexible

Neural mechanisms of perceptual decision making have been extensively studied in experimental settings that mimic stable environments with repeating stimuli, fixed rules, and payoffs. In contrast, we live in an ever-changing environment and have varying goals and behavioral demands. To accommodate variability, our brain flexibly adjusts decision-making processes depending on context. Here, we review a growing body of research that explores the neural mechanisms underlying this flexibility. We highlight diverse forms of context dependency in decision making implemented through a variety of neural computations. Context-dependent neural activity is observed in a distributed network of brain structures, including posterior parietal, sensory, motor, and subcortical regions, as well as the prefrontal areas classically implicated in cognitive control. We propose that investigating the distributed network underlying flexible decisions is key to advancing our understanding and discuss a path forward for experimental and theoretical investigations.

The orbitofrontal cortex, food reward, body weight and obesity

In primates including humans, the orbitofrontal cortex is the key brain region representing the reward value and subjective pleasantness of the sight, smell, taste and texture of food. At stages of processing before this, in the insular taste cortex and inferior temporal visual cortex, the identity of the food is represented, but not its affective value. In rodents, the whole organisation of reward systems appears to be different, with reward value reflected earlier in processing systems. In primates and humans, the amygdala is overshadowed by the great development of the orbitofrontal cortex. Social and cognitive factors exert a top-down influence on the orbitofrontal cortex, to modulate the reward value of food that is represented in the orbitofrontal cortex. Recent evidence shows that even in the resting state, with no food present as a stimulus, the liking for food, and probably as a consequence of that body mass index, is correlated with the functional connectivity of the orbitofrontal cortex and ventromedial prefrontal cortex. This suggests that individual differences in these orbitofrontal cortex reward systems contribute to individual differences in food pleasantness and obesity. Implications of how these reward systems in the brain operate for understanding, preventing and treating obesity are described.

New insight into brain disease therapy: nanomedicines-crossing blood-brain barrier and extracellular space for drug delivery

INTRODUCTION

Brain diseases including brain tumor, Alzheimer's disease, Parkinson's disease, etc. are difficult to treat. The blood-brain barrier (BBB) is a major obstacle for drug delivery into the brain. Although nano-package and receptor-mediated delivery of nanomedicine markedly increases BBB penetration, it yet did not extensively improve clinical cure rate. Recently, brain extracellular space (ECS) and interstitial fluid (ISF) drainage in ECS have been found to determine whether a drug dissolved in ISF can reach its target cells. Notably, an increase in tortuosity of ECS associated with slower ISF drainage induced by the accumulated harmful substances, such as: amyloid-beta (Aβ), α-synuclein, and metabolic wastes, causes drug delivery failure.

AREAS COVERED

The methods of nano-package and receptor-mediated drug delivery and the penetration efficacy of nanomedicines across BBB and ECS are assessed.

EXPERT OPINION

Invasive delivering drug via ECS and noninvasive near-infrared photo-sensitive nanomedicines may provide a promising benefit to patients with brain disease.

Multistability, perceptual value, and internal foraging

Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.

E-Senses, Panel Tests and Wearable Sensors: A Teamwork for Food Quality Assessment and Prediction of Consumer’s Choices

At present, food quality is of utmost importance, not only to comply with commercial regulations, but also to meet the expectations of consumers; this aspect includes sensory features capable of triggering emotions through the citizen’s perception. To date, key parameters for food quality assessment have been sought through analytical methods alone or in combination with a panel test, but the evaluation of panelists’ reactions via psychophysiological markers is now becoming increasingly popular. As such, the present review investigates recent applications of traditional and novel methods to the specific field. These include electronic senses (e-nose, e-tongue, and e-eye), sensory analysis, and wearables for emotion recognition. Given the advantages and limitations highlighted throughout the review for each approach (both traditional and innovative ones), it was possible to conclude that a synergy between traditional and innovative approaches could be the best way to optimally manage the trade-off between the accuracy of the information and feasibility of the investigation. This evidence could help in better planning future investigations in the field of food sciences, providing more reliable, objective, and unbiased results, but it also has important implications in the field of neuromarketing related to edible compounds.

Donut-like organization of inhibition underlies categorical neural responses in the midbrain

Categorical neural responses underlie various forms of selection and decision-making. Such binary-like responses promote robust signaling of the winner in the presence of input ambiguity and neural noise. Here, we show that a ‘donut-like’ inhibitory mechanism in which each competing option suppresses all options except itself, is highly effective at generating categorical neural responses. It surpasses motifs of feedback inhibition, recurrent excitation, and divisive normalization invoked frequently in decision-making models. We demonstrate experimentally not only that this mechanism operates in the midbrain spatial selection network in barn owls, but also that it is necessary for categorical signaling by it. The functional pattern of neural inhibition in the midbrain forms an exquisitely structured ‘multi-holed’ donut consistent with this network’s combinatorial inhibitory function for stimulus selection. Additionally, modeling reveals a generalizable neural implementation of the donut-like motif for categorical selection. Self-sparing inhibition may, therefore, be a powerful circuit module central to categorization.

Decision making is facilitated by categorical neuronal responses, which robustly signal a winner despite input noise. In this study, the authors demonstrate that a donut-like inhibition motif effectively generates such categorical responses.

The Mental Maxwell Relations: A Thermodynamic Allegory for Higher Brain Functions

The theoretical framework of classical thermodynamics unifies vastly diverse natural phenomena and captures once-elusive effects in concrete terms. Neuroscience confronts equally varied, equally ineffable phenomena in the mental realm, but has yet to unite or to apprehend them rigorously, perhaps due to an insufficient theoretical framework. The terms for mental phenomena, the mental variables, typically used in neuroscience are overly numerous and imprecise. Unlike in thermodynamics or other branches of physics, in neuroscience, there are no core mental variables from which all others formally derive and it is unclear which variables are distinct and which overlap. This may be due to the nature of mental variables themselves. Unlike the variables of physics, perhaps they cannot be interpreted as composites of a small number of axioms. However, it is well worth exploring if they can, as that would allow more parsimonious theories of higher brain function. Here we offer a theoretical exercise in the spirit of the National Institutes of Health Research Domain Criteria (NIH RDoC) Initiative and the Cognitive Atlas Project, which aim to remedy this state of affairs. Imitating classical thermodynamics, we construct a formal framework for mental variables, an extended analogy - an allegory - between mental and thermodynamic quantities. Starting with mental correlates of the physical indefinables length, time, mass or force, and charge, we pursue the allegory up to mental versions of the thermodynamic Maxwell Relations. The Maxwell Relations interrelate the thermodynamic quantities volume, pressure, temperature, and entropy and were chosen since they are easy to derive, yet capable of generating nontrivial, nonobvious predictions. Our "Mental Maxwell Relations" interlink the mental variables consciousness, salience, arousal, and distraction and make nontrivial, nonobvious statements about mental phenomena. The mental system thus constructed is internally consistent, in harmony with introspection, and respects the RDoC criteria of employing only psychologically valid constructs with some evidence of a brain basis. We briefly apply these concepts to the problem of decision-making and sketch how some of them might be tested empirically.

Quantifying the Landscape of Decision Making From Spiking Neural Networks

The decision making function is governed by the complex coupled neural circuit in the brain. The underlying energy landscape provides a global picture for the dynamics of the neural decision making system and has been described extensively in the literature, but often as illustrations. In this work, we explicitly quantified the landscape for perceptual decision making based on biophysically-realistic cortical network with spiking neurons to mimic a two-alternative visual motion discrimination task. Under certain parameter regions, the underlying landscape displays bistable or tristable attractor states, which quantify the transition dynamics between different decision states. We identified two intermediate states: the spontaneous state which increases the plasticity and robustness of changes of minds and the "double-up" state which facilitates the state transitions. The irreversibility of the bistable and tristable switches due to the probabilistic curl flux demonstrates the inherent non-equilibrium characteristics of the neural decision system. The results of global stability of decision-making quantified by barrier height inferred from landscape topography and mean first passage time are in line with experimental observations. These results advance our understanding of the stochastic and dynamical transition mechanism of decision-making function, and the landscape and kinetic path approach can be applied to other cognitive function related problems (such as working memory) in brain networks.

Neuronal Circuits for Social Decision-Making and Their Clinical Implications

Social living facilitates individual access to rewards, cognitive resources, and objects that would not be otherwise accessible. There are, however, some drawbacks to social living, particularly when competing for scarce resources. Furthermore, variability in our ability to make social decisions can be associated with neuropsychiatric disorders. The neuronal mechanisms underlying social decision-making are beginning to be understood. The momentum to study this phenomenon has been partially carried over by the study of economic decision-making. Yet, because of the similarities between these different types of decision-making, it is unclear what is a social decision. Here, we propose a definition of social decision-making as choices taken in a context where one or more conspecifics are involved in the decision or the consequences of it. Social decisions can be conceptualized as complex economic decisions since they are based on the subjective preferences between different goods. During social decisions, individuals choose based on their internal value estimate of the different alternatives. These are complex decisions given that conspecifics beliefs or actions could modify the subject's internal valuations at every choice. Here, we first review recent developments in our collective understanding of the neuronal mechanisms and circuits of social decision-making in primates. We then review literature characterizing populations with neuropsychiatric disorders showing deficits in social decision-making and the underlying neuronal circuitries associated with these deficits.

The neuroscience of emotional disorders

Emotions can be defined as states elicited by rewards or punishments, and indeed the neurology of emotional disorders can be understood in terms of this foundation. The orbitofrontal cortex in humans and other primates is a critical area in emotion processing, determining the value of stimuli and whether they are rewarding or nonrewarding. The cortical processing that occurs before the orbitofrontal cortex primarily involves defining the identity of stimuli, i.e., "what" is present and not reward value. There is evidence that this holds true for taste, visual, somatosensory, and olfactory stimuli. The human medial orbitofrontal cortex is important in processing many different types of reward, and the lateral orbitofrontal cortex in processing nonreward and punishment. Humans with damage to the orbitofrontal cortex have an impaired ability to identify facial and voice expressions of emotions, and impaired subjective experience of emotion. They can have an altered personality and be impulsive because they are impaired at processing failures to receive expected rewards and at processing punishments. In humans, the role of the amygdala in the processing of emotions is reduced because of the great evolutionary development of the orbitofrontal cortex: amygdala damage has much less effect on emotion than does orbitofrontal cortex damage. The orbitofrontal cortex projects reward value information to the anterior cingulate cortex, which is involved in learning those actions required to obtain rewards and avoid punishments. The cingulate cortex thus provides an output route for emotional behavior. In depression, the medial orbitofrontal cortex has decreased connectivity and sensitivity to reward, and the lateral orbitofrontal cortex has increased connectivity and sensitivity to nonreward. The orbitofrontal cortex has major projections to the anterior cingulate cortex, including its subcommissural region, and the anterior cingulate cortex is also implicated in depression.

The connections of neocortical pyramidal cells can implement the learning of new categories, attractor memory, and top-down recall and attention

Neocortical pyramidal cells have three key classes of excitatory input: forward inputs from the previous cortical area (or thalamus); recurrent collateral synapses from nearby pyramidal cells; and backprojection inputs from the following cortical area. The neocortex performs three major types of computation: (1) unsupervised learning of new categories, by allocating neurons to respond to combinations of inputs from the preceding cortical stage, which can be performed using competitive learning; (2) short-term memory, which can be performed by an attractor network using the recurrent collaterals; and (3) recall of what has been learned by top–down backprojections from the following cortical area. There is only one type of excitatory neuron involved, pyramidal cells, with these three types of input. It is proposed, and tested by simulations of a neuronal network model, that pyramidal cells can implement all three types of learning simultaneously, and can subsequently usefully categorise the forward inputs; keep them active in short-term memory; and later recall the representations using the backprojection input. This provides a new approach to understanding how one type of excitatory neuron in the neocortex can implement these three major types of computation, and provides a conceptual advance in understanding how the cerebral neocortex may work.

Secondary rewards acquire enhanced incentive motivation via increasing anticipatory activity of the lateral orbitofrontal cortex

The motivation to strive for and consume primary rewards such as palatable food is bound by devaluation mechanisms, yet secondary rewards such as money may not be bound by these regulatory mechanisms. The present study therefore aimed at determining diverging devaluation trajectories for primary (chocolate milk) and secondary (money) reinforcers on the behavioral and neural level. Devaluation procedures with repeated exposure to reward combined with a choice (Experiment 1) and an incentive delay (Experiment 2) paradigm consistently revealed decreasing hedonic value for the primary reward as reflected by decreasing hedonic evaluation and choice preference with repeated receipt, while hedonic value and preferences for the secondary reward increased. Concomitantly acquired functional near-infrared spectroscopy (fNIRS) data during the incentive delay paradigm revealed that increasing value of the secondary reward was accompanied by increasing anticipatory activation in the lateral orbitofrontal cortex, while during the consummatory phase the secondary reinforcer associated with higher medial orbitofrontal activity irrespective of devaluation stage. Overall, the findings suggest that-in contrast to primary reinforcers-secondary reinforcers, i.e. money, can acquire progressively enhanced incentive motivation with repeated receipt, suggesting a mechanism which could promote escalating striving to obtain secondary rewards.

Representational geometry of perceptual decisions in the monkey parietal cortex

Lateral intraparietal (LIP) neurons represent formation of perceptual decisions involving eye movements. In circuit models for these decisions, neural ensembles that encode actions compete to form decisions. Consequently, representation and readout of the decision variables (DVs) are implemented similarly for decisions with identical competing actions, irrespective of input and task context differences. Further, DVs are encoded as partially potentiated action plans through balance of activity of action-selective ensembles. Here, we test those core principles. We show that in a novel face-discrimination task, LIP firing rates decrease with supporting evidence, contrary to conventional motion-discrimination tasks. These opposite response patterns arise from similar mechanisms in which decisions form along curved population-response manifolds misaligned with action representations. These manifolds rotate in state space based on context, indicating distinct optimal readouts for different tasks. We show similar manifolds in lateral and medial prefrontal cortices, suggesting similar representational geometry across decision-making circuits.

Attractor cortical neurodynamics, schizophrenia, and depression

The local recurrent collateral connections between cortical neurons provide a basis for attractor neural networks for memory, attention, decision-making, and thereby for many aspects of human behavior. In schizophrenia, a reduction of the firing rates of cortical neurons, caused for example by reduced NMDA receptor function or reduced spines on neurons, can lead to instability of the high firing rate attractor states that normally implement short-term memory and attention in the prefrontal cortex, contributing to the cognitive symptoms. Reduced NMDA receptor function in the orbitofrontal cortex by reducing firing rates may produce negative symptoms, by reducing reward, motivation, and emotion. Reduced functional connectivity between some brain regions increases the temporal variability of the functional connectivity, contributing to the reduced stability and more loosely associative thoughts. Further, the forward projections have decreased functional connectivity relative to the back projections in schizophrenia, and this may reduce the effects of external bottom-up inputs from the world relative to internal top-down thought processes. Reduced cortical inhibition, caused by a reduction of GABA neurotransmission, can lead to instability of the spontaneous firing states of cortical networks, leading to a noise-induced jump to a high firing rate attractor state even in the absence of external inputs, contributing to the positive symptoms of schizophrenia. In depression, the lateral orbitofrontal cortex non-reward attractor network system is over-connected and has increased sensitivity to non-reward, providing a new approach to understanding depression. This is complemented by under-sensitivity and under-connectedness of the medial orbitofrontal cortex reward system in depression.

Propofol rather than Isoflurane Accelerates the Interstitial Fluid Drainage in the Deep Rat Brain

Objective: Different anesthetics have distinct effects on the interstitial fluid (ISF) drainage in the extracellular space (ECS) of the superficial rat brain, while their effects on ISF drainage in the ECS of the deep rat brain still remain unknown. Herein, we attempt to investigate and compare the effects of propofol and isoflurane on ECS structure and ISF drainage in the caudate-putamen (CPu) and thalamus (Tha) of the deep rat brain. Methods: Adult Sprague-Dawley rats were anesthetized with propofol or isoflurane, respectively. Twenty-four anesthetized rats were randomly divided into the propofol-CPu, isoflurane-CPu, propofol-Tha, and isoflurane-Tha groups. Tracer-based magnetic resonance imaging (MRI) and fluorescent-labeled tracer assay were utilized to quantify ISF drainage in the deep brain. Results: The half-life of ISF in the propofol-CPu and propofol-Tha groups was shorter than that in the isoflurane-CPu and isoflurane-Tha groups, respectively. The ECS volume fraction in the propofol-CPu and propofol-Tha groups was much higher than that in the isoflurane-CPu and isoflurane-Tha groups, respectively. However, the ECS tortuosity in the propofol-CPu and propofol-Tha groups was much smaller than that in isoflurane-CPu and isoflurane-Tha groups, respectively. Conclusions: Our results demonstrate that propofol rather than isoflurane accelerates the ISF drainage in the deep rat brain, which provides novel insights into the selective control of ISF drainage and guides selection of anesthetic agents in different clinical settings, and unravels the mechanism of how general anesthetics function.

The orbitofrontal cortex: reward, emotion and depression

The orbitofrontal cortex in primates including humans is the key brain area in emotion, and in the representation of reward value and in non-reward, that is not obtaining an expected reward. Cortical processing before the orbitofrontal cortex is about the identity of stimuli, i.e. 'what' is present, and not about reward value. There is evidence that this holds for taste, visual, somatosensory and olfactory stimuli. The human medial orbitofrontal cortex represents many different types of reward, and the lateral orbitofrontal cortex represents non-reward and punishment. Not obtaining an expected reward can lead to sadness, and feeling depressed. The concept is advanced that an important brain region in depression is the orbitofrontal cortex, with depression related to over-responsiveness and over-connectedness of the non-reward-related lateral orbitofrontal cortex, and to under-responsiveness and under-connectivity of the reward-related medial orbitofrontal cortex. Evidence from large-scale voxel-level studies and supported by an activation study is described that provides support for this hypothesis. Increased functional connectivity of the lateral orbitofrontal cortex with brain areas that include the precuneus, posterior cingulate cortex and angular gyrus is found in patients with depression and is reduced towards the levels in controls when treated with medication. Decreased functional connectivity of the medial orbitofrontal cortex with medial temporal lobe areas involved in memory is found in patients with depression. Some treatments for depression may act by reducing activity or connectivity of the lateral orbitofrontal cortex. New treatments that increase the activity or connectivity of the medial orbitofrontal cortex may be useful for depression. These concepts, and that of increased activity in non-reward attractor networks, have potential for advancing our understanding and treatment of depression. The focus is on the orbitofrontal cortex in primates including humans, because of differences of operation of the orbitofrontal cortex, and indeed of reward systems, in rodents. Finally, the hypothesis is developed that the orbitofrontal cortex has a special role in emotion and decision-making in part because as a cortical area it can implement attractor networks useful in maintaining reward and emotional states online, and in decision-making.

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Perceptual decisions entail the accumulation of evidence until a decision criterion is reached. The amount of noise in this process is inversely related to the behavioral performance of the decision-maker. Hence, reducing the amount of perceived noise could improve performance in perceptual decisions. In this study, we investigated whether providing monetary reward for correct responses in a perceptual decision-making task would enhance performance based on prior research linking noise reduction to the administration of reward. To this end, thirty-one healthy young adults carried out an incentivized dot tracking task (iDT) during recording of functional magnetic resonance imaging (fMRI). Behavioral responses were fitted to a Bayesian version of the drift-diffusion model that, among other parameters, also includes an estimate of sensory noise. Fifty percent of the trials were incentivized to compare rewarded with unrewarded trials regarding behavior, brain responses and estimates of model parameters. In order to establish a link between the noise parameter and fMRI activity, we correlated percent signal change (PSC) values from nucleus accumbens and caudate nucleus with noise levels in rewarded and unrewarded trials respectively. Although reward did not affect behavioral performance and model parameters, the fMRI analyses showed notable differences in nucleus accumbens, caudate nucleus and rostral anterior cingulate cortex in rewarded relative to unrewarded trials. Furthermore, higher PSC within nucleus accumbens was significantly associated with lower sensory noise levels, which was specific to rewarded trials. This work is consistent with previous findings on reward modulation of brain responses and marks a first step towards elucidating the effects of reward-induced noise suppression during perceptual decision-making.

The Generation of Time in the Hippocampal Memory System

We propose that ramping time cells in the lateral entorhinal cortex can be produced by synaptic adaptation and demonstrate this in an integrate-and-fire attractor network model. We propose that competitive networks in the hippocampal system can convert these entorhinal ramping cells into hippocampal time cells and demonstrate this in a competitive network. We propose that this conversion is necessary to provide orthogonal hippocampal time representations to encode the temporal sequence of events in hippocampal episodic memory, and we support that with analytic arguments. We demonstrate that this processing can produce hippocampal neuronal ensembles that not only show replay of the sequence later on, but can also do this in reverse order in reverse replay. This research addresses a major issue in neuroscience: the mechanisms by which time is encoded in the brain and how the time representations are then useful in the hippocampal memory of events and their order.

Mechanisms of competitive selection: A canonical neural circuit framework

Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.

Differentiating between integration and non-integration strategies in perceptual decision making

Many tasks used to study decision-making encourage subjects to integrate evidence over time. Such tasks are useful to understand how the brain operates on multiple samples of information over prolonged timescales, but only if subjects actually integrate evidence to form their decisions. We explored the behavioral observations that corroborate evidence-integration in a number of task-designs. Several commonly accepted signs of integration were also predicted by non-integration strategies. Furthermore, an integration model could fit data generated by non-integration models. We identified the features of non-integration models that allowed them to mimic integration and used these insights to design a motion discrimination task that disentangled the models. In human subjects performing the task, we falsified a non-integration strategy in each and confirmed prolonged integration in all but one subject. The findings illustrate the difficulty of identifying a decision-maker’s strategy and support solutions to achieve this goal.

Combination of Simultaneous Artificial Sensory Percepts to Identify Prosthetic Hand Postures: A Case Study

Multiple sources of sensory information are combined to develop hand posture percepts in the intact system, but the combination of multiple artificial somatosensory percepts by human prosthesis users has not been studied. Here, we report on a case study in which a person with transradial amputation identified prosthetic hand postures using artificial somatosensory feedback. He successfully combined five artificial somatosensory percepts to achieve above-chance performance of 95.0% and 75.7% in identifying four and seven postures, respectively. We studied how artificial somatosensation and the extant hand representation are combined in the decision-making process by providing two mappings between the prosthetic sensor and the location of the sensory percept: (1) congruent, and (2) incongruent. The participant's ability to combine and engage with the sensory feedback significantly differed between the two conditions. The participant was only able to successfully generalize prior knowledge to novel postures in the congruent mapping. Further, he learned postures more accurately and quickly in the congruent mapping. Finally, he developed an understanding of the relationships between postures in the congruent mapping instead of simply memorizing each individual posture. These experimental results are corroborated by a Bayesian decision-making model which tracked the participant's learning.

Macroscopic gradients of synaptic excitation and inhibition in the neocortex

With advances in connectomics, transcriptome and neurophysiological technologies, the neuroscience of brain-wide neural circuits is poised to take off. A major challenge is to understand how a vast diversity of functions is subserved by parcellated areas of mammalian neocortex composed of repetitions of a canonical local circuit. Areas of the cerebral cortex differ from each other not only in their input-output patterns but also in their biological properties. Recent experimental and theoretical work has revealed that such variations are not random heterogeneities; rather, synaptic excitation and inhibition display systematic macroscopic gradients across the entire cortex, and they are abnormal in mental illness. Quantitative differences along these gradients can lead to qualitatively novel behaviours in non-linear neural dynamical systems, by virtue of a phenomenon mathematically described as bifurcation. The combination of macroscopic gradients and bifurcations, in tandem with biological evolution, development and plasticity, provides a generative mechanism for functional diversity among cortical areas, as a general principle of large-scale cortical organization.

The Interstitial System of the Brain in Health and Disease

The brain interstitial fluid (ISF) and the cerebrospinal fluid (CSF) cushion and support the brain cells. The ISF occupies the brain interstitial system (ISS), whereas the CSF fills the brain ventricles and the subarachnoid space. The brain ISS is an asymmetrical, tortuous, and exceptionally confined space between neural cells and the brain microvasculature. Recently, with a newly developed in vivo measuring technique, a series of discoveries have been made in the brain ISS and the drainage of ISF. The goal of this review is to confer recent advances in our understanding of the brain ISS, including its structure, function, and the various processes mediating or disrupting ISF drainage in physiological and pathological conditions. The brain ISF in the deep brain regions has recently been demonstrated to drain in a compartmentalized ISS instead of a highly connected system, together with the drainage of ISF into the cerebrospinal fluid (CSF) at the surface of the cerebral cortex and the transportation from CSF into cervical lymph nodes. Besides, accumulation of tau in the brain ISS in conditions such as Alzheimer’s disease and its link to the sleep-wake cycle and sleep deprivation, clearance of ISF in a deep sleep via increased CSF flow, novel approaches to remove beta-amyloid from the brain ISS, and obstruction to the ISF drainage in neurological conditions are deliberated. Moreover, the role of ISS in the passage of extracellular vesicles (EVs) released from neural cells and the rapid targeting of therapeutic EVs into neural cells in the entire brain following an intranasal administration, and the promise and limitations of ISS based drug delivery approaches are discussed

The cingulate cortex and limbic systems for emotion, action, and memory

Evidence is provided for a new conceptualization of the connectivity and functions of the cingulate cortex in emotion, action, and memory. The anterior cingulate cortex receives information from the orbitofrontal cortex about reward and non-reward outcomes. The posterior cingulate cortex receives spatial and action-related information from parietal cortical areas. It is argued that these inputs allow the cingulate cortex to perform action-outcome learning, with outputs from the midcingulate motor area to premotor areas. In addition, because the anterior cingulate cortex connects rewards to actions, it is involved in emotion; and because the posterior cingulate cortex has outputs to the hippocampal system, it is involved in memory. These apparently multiple different functions of the cingulate cortex are related to the place of this proisocortical limbic region in brain connectivity.

Analysis of Biased Competition and Cooperation for Attention in the Cerebral Cortex

A new approach to understanding the interaction between cortical areas is provided by a mathematical analysis of biased competition, which describes many interactions between cortical areas, including those involved in top-down attention. The analysis helps to elucidate the principles of operation of such cortical systems, and in particular the parameter values within which biased competition operates. The analytic results are supported by simulations that illustrate the operation of the system with parameters selected from the analysis. The findings provide a detailed mathematical analysis of the operation of these neural systems with nodes connected by feedforward (bottom-up) and feedback (top-down) connections. The analysis provides the critical value of the top-down attentional bias that enables biased competition to operate for a range of input values to the network, and derives this as a function of all the parameters in the model. The critical value of the top-down bias depends linearly on the value of the other inputs, but the coefficients in the function reveal non-linear relations between the remaining parameters. The results provide reasons why the backprojections should not be very much weaker than the forward connections between two cortical areas. The major advantage of the analytical approach is that it discloses relations between all the parameters of the model.

Primate prefrontal neurons signal economic risk derived from the statistics of recent reward experience

Risk derives from the variation of rewards and governs economic decisions, yet how the brain calculates risk from the frequency of experienced events, rather than from explicit risk-descriptive cues, remains unclear. Here, we investigated whether neurons in dorsolateral prefrontal cortex process risk derived from reward experience. Monkeys performed in a probabilistic choice task in which the statistical variance of experienced rewards evolved continually. During these choices, prefrontal neurons signaled the reward-variance associated with specific objects ('object risk') or actions ('action risk'). Crucially, risk was not derived from explicit, risk-descriptive cues but calculated internally from the variance of recently experienced rewards. Support-vector-machine decoding demonstrated accurate neuronal risk discrimination. Within trials, neuronal signals transitioned from experienced reward to risk (risk updating) and from risk to upcoming choice (choice computation). Thus, prefrontal neurons encode the statistical variance of recently experienced rewards, complying with formal decision variables of object risk and action risk.

Foraging as an evidence accumulation process

The patch-leaving problem is a canonical foraging task, in which a forager must decide to leave a current resource in search for another. Theoretical work has derived optimal strategies for when to leave a patch, and experiments have tested for conditions where animals do or do not follow an optimal strategy. Nevertheless, models of patch-leaving decisions do not consider the imperfect and noisy sampling process through which an animal gathers information, and how this process is constrained by neurobiological mechanisms. In this theoretical study, we formulate an evidence accumulation model of patch-leaving decisions where the animal averages over noisy measurements to estimate the state of the current patch and the overall environment. We solve the model for conditions where foraging decisions are optimal and equivalent to the marginal value theorem, and perform simulations to analyze deviations from optimal when these conditions are not met. By adjusting the drift rate and decision threshold, the model can represent different “strategies”, for example an incremental, decremental, or counting strategy. These strategies yield identical decisions in the limiting case but differ in how patch residence times adapt when the foraging environment is uncertain. To describe sub-optimal decisions, we introduce an energy-dependent marginal utility function that predicts longer than optimal patch residence times when food is plentiful. Our model provides a quantitative connection between ecological models of foraging behavior and evidence accumulation models of decision making. Moreover, it provides a theoretical framework for potential experiments which seek to identify neural circuits underlying patch-leaving decisions.

Foraging is a ubiquitous animal behavior, performed by organisms as different as worms, birds, rats, and humans. Although the behavior has been extensively studied, it is not known how the brain processes information obtained during foraging activity to make subsequent foraging decisions. We form an evidence accumulation model of foraging decisions that describes the process through which an animal gathers information and uses it to make foraging decisions. By building on studies of the neural decision mechanisms within systems neuroscience, this model connects the foraging decision process with ecological models of patch-leaving decisions, such as the marginal value theorem. The model suggests the existence of different foraging strategies, which optimize for different environmental conditions and their potential implementation by neural decision making circuits. The model also shows how state-dependence, such as satiation level, can affect evidence accumulation to lead to sub-optimal foraging decisions. Our model provides a framework for future experimental studies which seek to elucidate how neural decision making mechanisms have been shaped by evolutionary forces in an animal’s surrounding environment.

Primate Amygdala Neurons Simulate Decision Processes of Social Partners

By observing their social partners, primates learn about reward values of objects. Here, we show that monkeys' amygdala neurons derive object values from observation and use these values to simulate a partner monkey's decision process. While monkeys alternated making reward-based choices, amygdala neurons encoded object-specific values learned from observation. Dynamic activities converted these values to representations of the recorded monkey's own choices. Surprisingly, the same activity patterns unfolded spontaneously before partner's choices in separate neurons, as if these neurons simulated the partner's decision-making. These "simulation neurons" encoded signatures of mutual-inhibitory decision computation, including value comparisons and value-to-choice conversions, resulting in accurate predictions of partner's choices. Population decoding identified differential contributions of amygdala subnuclei. Biophysical modeling of amygdala circuits showed that simulation neurons emerge naturally from convergence between object-value neurons and self-other neurons. By simulating decision computations during observation, these neurons could allow primates to reconstruct their social partners' mental states.

Relation between centro-parietal positivity and diffusion model parameters in both perceptual and memory-based decision making

Several studies have suggested that the centro-parietal positivity (CPP), an EEG potential occurring approximately 500 ms post-stimulus, reflects the accumulation of evidence for making a decision. Yet, most previous studies of the CPP focused exclusively on perceptual decisions with very simple stimuli. In this study, we examined how the dynamics of the CPP depended on the type of decision being made, and whether its slope was related to parameters of an accumulator model of decision making. We show initial evidence that memory- and perceptual decisions about carefully-controlled face stimuli exhibit similar dynamics, but offset by a time difference in decision onset. Importantly, the individual-trial slopes of the CPP are related to the accumulator model's drift parameter. These findings help to further understand the role of the CPP across different kinds of decisions.

Taste and smell processing in the brain

Taste pathways in humans and other primates project from the nucleus of the solitary tract directly to the taste thalamus, and then to the taste insula. The taste cortex in the anterior insula provides separate and combined representations of the taste, temperature, and texture of food in the mouth independently of hunger and thus of reward value and pleasantness. One synapse on, in the orbitofrontal cortex, these sensory inputs are for some neurons combined by associative learning with olfactory inputs received from the pyriform cortex, and visual inputs from the temporal lobe, and these neurons encode food reward value in that they only respond to food when hungry, and in that activations correlate linearly with subjective pleasantness. Cognitive factors, including word-level descriptions, and selective attention to affective value, modulate the representation of the reward value of taste, olfactory and flavor stimuli in the orbitofrontal cortex and a region to which it projects, the anterior cingulate cortex. These food reward representations are important in the control of appetite, and the liking of food. Individual differences in these reward representations may contribute to obesity, and there are age-related differences in these reward representations.

Why is the environment important for decision making? Local reservoir model for choice-based learning

Decision making based on behavioral and neural observations of living systems has been extensively studied in brain science, psychology, neuroeconomics, and other disciplines. Decision-making mechanisms have also been experimentally implemented in physical processes, such as single photons and chaotic lasers. The findings of these experiments suggest that there is a certain common basis in describing decision making, regardless of its physical realizations. In this study, we propose a local reservoir model to account for choice-based learning (CBL). CBL describes decision consistency as a phenomenon where making a certain decision increases the possibility of making that same decision again later. This phenomenon has been intensively investigated in neuroscience, psychology, and other related fields. Our proposed model is inspired by the viewpoint that a decision is affected by its local environment, which is referred to as a local reservoir. If the size of the local reservoir is large enough, consecutive decision making will not be affected by previous decisions, thus showing lower degrees of decision consistency in CBL. In contrast, if the size of the local reservoir decreases, a biased distribution occurs within it, which leads to higher degrees of decision consistency in CBL. In this study, an analytical approach for characterizing local reservoirs is presented, as well as several numerical demonstrations. Furthermore, a physical architecture for CBL based on single photons is discussed, and the effects of local reservoirs are numerically demonstrated. Decision consistency in human decision-making tasks and in recruiting empirical data is evaluated based on the local reservoir. This foundation based on a local reservoir offers further insights into the understanding and design of decision making.

Medial Frontal Circuit Dynamics Represents Probabilistic Choices for Unfamiliar Sensory Experience

Neurons in medial frontal cortex (MFC) receive sensory signals that are crucial for decision-making behavior. While decision-making is easy for familiar sensory signals, it becomes more elaborative when sensory signals are less familiar to animals. It remains unclear how the population of neurons enables the coordinate transformation of such a sensory input into ambiguous choice responses. Furthermore, whether and how cortical oscillations temporally coordinate neuronal firing during this transformation has not been extensively studied. Here, we recorded neuronal population responses to familiar or unfamiliar auditory cues in rat MFC and computed their probabilistic evolution. Population responses to familiar sounds organize into neuronal trajectories containing multiplexed sensory, motor, and choice information. Unfamiliar sounds, in contrast, evoke trajectories that travel under the guidance of familiar paths and eventually diverge to unique decision states. Local field potentials exhibited beta- (15-20 Hz) and gamma-band (50-60 Hz) oscillations to which neuronal firing showed modest phase locking. Interestingly, gamma oscillation, but not beta oscillation, increased its power abruptly at some timepoint by which neural trajectories for different choices were near maximally separated. Our results emphasize the importance of the evolution of neural trajectories in rapid probabilistic decisions that utilize unfamiliar sensory information.

It Pays to Prepare: Human Motor Preparation Depends on the Relative Value of Potential Response Options

Alternative motor responses can be prepared in parallel. Here, we used electroencephalography (EEG) to test whether the parallel preparation of alternative response options is modulated by their relative value. Participants performed a choice response task with three potential actions: isometric contraction of the left, the right, or both wrists. An imperative stimulus (IS) appeared after a warning cue, such that the initiation time of a required action was predictable, but the specific action was not. To encourage advanced preparation, the target was presented 200 ms prior to the IS, and only correct responses initiated within ±100 ms of the IS were rewarded. At baseline, all targets were equally rewarded and probable. Then, responses with one hand were made more valuable, either by increasing the probability that the left or right target would be required (Exp. 1; n = 31) or by increasing the reward magnitude of one target (Exp. 2, n = 36). We measured reaction times, movement vigor, and an EEG correlate of action preparation (value-based lateralized readiness potential) prior to target presentation. Participants responded earlier to more frequent and more highly rewarded targets, and movements to highly rewarded targets were more vigorous. The EEG was more negative over the hemisphere contralateral to the more repeated/rewarded hand, implying an increased neural preparation of more valuable actions. Thus, changing the value of alternative response options can lead to greater preparation of actions associated with more valuable outcomes. This preparation asymmetry likely contributes to behavioral biases that are typically observed toward repeated or rewarded targets.

Computations in the deep vs superficial layers of the cerebral cortex

A fundamental question is how the cerebral neocortex operates functionally, computationally. The cerebral neocortex with its superficial and deep layers and highly developed recurrent collateral systems that provide a basis for memory-related processing might perform somewhat different computations in the superficial and deep layers. Here we take into account the quantitative connectivity within and between laminae. Using integrate-and-fire neuronal network simulations that incorporate this connectivity, we first show that attractor networks implemented in the deep layers that are activated by the superficial layers could be partly independent in that the deep layers might have a different time course, which might because of adaptation be more transient and useful for outputs from the neocortex. In contrast the superficial layers could implement more prolonged firing, useful for slow learning and for short-term memory. Second, we show that a different type of computation could in principle be performed in the superficial and deep layers, by showing that the superficial layers could operate as a discrete attractor network useful for categorisation and feeding information forward up a cortical hierarchy, whereas the deep layers could operate as a continuous attractor network useful for providing a spatially and temporally smooth output to output systems in the brain. A key advance is that we draw attention to the functions of the recurrent collateral connections between cortical pyramidal cells, often omitted in canonical models of the neocortex, and address principles of operation of the neocortex by which the superficial and deep layers might be specialized for different types of attractor-related memory functions implemented by the recurrent collaterals.

The Effects of Aging on Orientation Discrimination

Visual perception relies on low-level encoding of local orientation. Recent studies show an age-dependent impairment in orientation discrimination of stimuli embedded in external noise, suggesting that encoding of orientation is inefficient in older adults. In the present study we ask whether aging also reduces decoding, i.e., selecting the neural representations of target orientation while discarding those conflicting with it. We compared younger and older participants capability (mean age 24 and 68 years respectively) in discriminating whether the orientation of a Gabor target was left or right from the vertical. We measured (d′), an index of discrimination sensitivity, for orientation offset ranging from 1° to 12°. In the isolated target condition, d′ was reduced by aging and, in the older group, did not increase with orientation offset, thus resulting in a larger group difference at large than small orientation offsets from the vertical. Moreover, oriented elements in the background impaired more discrimination in the older group. However, distractors reduced more d′ when target-background orientation offset was large than when target and flanker had similar orientation, indicating that the effect of the background was not local, i.e., due to target inhibition by similarly oriented flankers. Altogether, these results indicate that aging reduces the efficiency in discarding the response to orientations differing from the target. Our results suggest that neural decision-making mechanisms, involving not only signal enhancement but also non-signal inhibition, become inefficient with age. This suggestion is consistent with the neurophysiological evidence of inefficient visual cortical inhibition in aging.

Response repetition biases in human perceptual decisions are explained by activity decay in competitive attractor models

Animals and humans have a tendency to repeat recent choices, a phenomenon known as choice hysteresis. The mechanism for this choice bias remains unclear. Using an established, biophysically informed model of a competitive attractor network for decision making, we found that decaying tail activity from the previous trial caused choice hysteresis, especially during difficult trials, and accurately predicted human perceptual choices. In the model, choice variability could be directionally altered through amplification or dampening of post-trial activity decay through simulated depolarizing or hyperpolarizing network stimulation. An analogous intervention using transcranial direct current stimulation (tDCS) over left dorsolateral prefrontal cortex (dlPFC) yielded a close match between model predictions and experimental results: net soma depolarizing currents increased choice hysteresis, while hyperpolarizing currents suppressed it. Residual activity in competitive attractor networks within dlPFC may thus give rise to biases in perceptual choices, which can be directionally controlled through non-invasive brain stimulation.