The hierarchical construction of value

Efficient value synthesis in the orbitofrontal cortex explains how loss aversion adapts to the ranges of gain and loss prospects

Is irrational behavior the incidental outcome of biological constraints imposed on neural information processing? In this work, we consider the paradigmatic case of gamble decisions, where gamble values integrate prospective gains and losses. Under the assumption that neurons have a limited firing response range, we show that mitigating the ensuing information loss within artificial neural networks that synthetize value involves a specific form of self-organized plasticity. We demonstrate that the ensuing efficient value synthesis mechanism induces value range adaptation. We also reveal how the ranges of prospective gains and/or losses eventually determine both the behavioral sensitivity to gains and losses and the information content of the network. We test these predictions on two fMRI datasets from the OpenNeuro.org initiative that probe gamble decision-making but differ in terms of the range of gain prospects. First, we show that peoples' loss aversion eventually adapts to the range of gain prospects they are exposed to. Second, we show that the strength with which the orbitofrontal cortex (in particular: Brodmann area 11) encodes gains and expected value also depends upon the range of gain prospects. Third, we show that, when fitted to participant's gambling choices, self-organizing artificial neural networks generalize across gain range contexts and predict the geometry of information content within the orbitofrontal cortex. Our results demonstrate how self-organizing plasticity aiming at mitigating information loss induced by neurons' limited response range may result in value range adaptation, eventually yielding irrational behavior.

Distinct neural bases of visual art- and music-induced aesthetic experiences

Aesthetic experiences are characterized by a conscious, emotionally and hedonically rewarding perceptions of a stimulus's aesthetic qualities and are thought to arise from a unique combination of cognitive and affective processes. To pinpoint neural correlates of aesthetic experiences, in the present study, we performed a series of meta-analyses based on the existing functional Magnetic Resonance Imaging (fMRI) studies of art appreciation in visual art (34 experiments, 692 participants) and music (34 experiments, 718 participants). The Activation Likelihood Estimation (ALE) analyses showed that the frontal pole (FP), ventromedial prefrontal cortex (vmPFC), and inferior frontal gyrus (IFG) were commonly activated in visual-art-induced aesthetic experiences, whilst bilateral superior temporal gyrus (STG) and striatal areas were commonly activated in music appreciation. Additionally, task-independent Resting-state Functional Connectivity (RSFC), task-dependent Meta-analytical Connectivity Modelling (MACM) analyses, as well as Activation Network Modeling (ANM) further showed that visual art and music engaged quite distinct brain networks. Our findings support the domain-specific view of aesthetic appreciation and challenge the notion that there is a general "common neural currency" for aesthetic experiences across domains.

Modality-specific and modality-general representations of subjective value in frontal cortex

Neuroeconomics theories propose that the value associated with diverse rewards or reward-predicting stimuli is encoded along a common reference scale, irrespective of their sensory properties. However, in a dynamic environment with changing stimulus-reward pairings, the brain must also represent the sensory features of rewarding stimuli. The mechanism by which the brain balances these needs-deriving a common reference scale for valuation while maintaining sensitivity to sensory contexts-remains unclear. To investigate this, we conducted an fMRI study with human participants engaged in a dynamic foraging task, which required integrating the reward history of auditory or visual choice options and updating the subjective value for each sensory modality. Univariate fMRI analysis revealed modality-specific value representations in the orbitofrontal cortex (OFC) and modality-general value representations in the ventromedial prefrontal cortex (vmPFC), confirmed by an exploratory multivariate pattern classification approach. Crucially, modality-specific value representations were absent when the task involved instruction-based rather than value-based choices. Effective connectivity analysis showed that modality-specific value representations emerged from selective bidirectional interactions across the auditory and visual sensory cortices, the corresponding OFC clusters, and the vmPFC. These results illustrate how the brain enables a valuation process that is sensitive to the sensory context of rewarding stimuli.

Striatal GABA levels correlate with risk sensitivity in monetary loss

Background

Decision-making under risk is a common challenge. It is known that risk-taking behavior varies between contexts of reward and punishment, yet the mechanisms underlying this asymmetry in risk sensitivity remain unclear.

Methods

This study used a monetary task to investigate neurochemical mechanisms and brain dynamics underpinning risk sensitivity. Twenty-eight participants engaged in a task requiring selection of visual stimuli to maximize monetary gains and minimize monetary losses. We modeled participant trial-and-error processes using reinforcement learning.

Results

Participants with higher subjective utility parameters showed risk preference in the gain domain (r = -0.59) and risk avoidance in the loss domain (r = -0.77). Magnetic resonance spectroscopy (MRS) revealed that risk avoidance in the loss domain was associated with γ-aminobutyric acid (GABA) levels in the ventral striatum (r = -0.42), but not in the insula (r = -0.15). Using functional magnetic resonance imaging (fMRI), we tested whether risk-sensitive brain dynamics contribute to participant risky choices. Energy landscape analyses demonstrated that higher switching rates between brain states, including the striatum and insula, were correlated with risk avoidance in the loss domain (r = -0.59), a relationship not observed in the gain domain (r = -0.02).

Conclusions

These findings from MRS and fMRI suggest that distinct mechanisms are involved in gain/loss decision making, mediated by subcortical neurometabolite levels and brain dynamic transitions.

The online metacognitive control of decisions

Difficult decisions typically involve mental effort, which scales with the deployment of cognitive (e.g., mnesic, attentional) resources engaged in processing decision-relevant information. But how does the brain regulate mental effort? A possibility is that the brain optimizes a resource allocation problem, whereby the amount of invested resources balances its expected cost (i.e. effort) and benefit. Our working assumption is that subjective decision confidence serves as the benefit term of the resource allocation problem, hence the "metacognitive" nature of decision control. Here, we present a computational model for the online metacognitive control of decisions or oMCD. Formally, oMCD is a Markov Decision Process that optimally solves the ensuing resource allocation problem under agnostic assumptions about the inner workings of the underlying decision system. We demonstrate how this makes oMCD a quasi-optimal control policy for a broad class of decision processes, including -but not limited to- progressive attribute integration. We disclose oMCD's main properties (in terms of choice, confidence and response time), and show that they reproduce most established empirical results in the field of value-based decision making. Finally, we discuss the possible connections between oMCD and most prominent neurocognitive theories about decision control and mental effort regulation.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

A clearing in the objectivity of aesthetics?

As subjective experiences go, beauty matters. Although aesthetics has long been a topic of study, research in this area has not resulted in a level of interest and progress commensurate with its import. Here, we briefly discuss two recent advances, one computational and one neuroscientific, and their pertinence to aesthetic processing. First, we hypothesize that deep neural networks provide the capacity to model representations essential to aesthetic experiences. Second, we highlight the principal gradient as an axis of information processing that is potentially key to examining where and how aesthetic processing takes place in the brain. In concert with established neuroimaging tools, we suggest that these advances may cultivate a new frontier in the understanding of our aesthetic experiences.

Uncertainty in learning and decision-making: Introduction to the special issue

Uncertainty is a fundamental aspect of the environment. This special issue presents interdisciplinary research on decision-making and learning under uncertainty. Thirty-one research and review papers report the findings of the behavioral, neural, and computational bases of coping with uncertainty, as well as changes of these mechanisms in development, aging, and psychopathology. Taken together, this special issue presents extant research, identifies gaps in our knowledge, and offers paths for future directions.

Semantic knowledge of social interactions is mediated by the hedonic evaluation system in the brain

Attaching semantic meaning to sensory information received from both inside and outside our bodies is a fundamental function of the human brain. The theory of Controlled Semantic Cognition (CSC) proposes that the formation of semantic knowledge relies on connections between spatially distributed modality-specific spoke-nodes, and a modality-general hub in the anterior temporal lobes (ATLs). This theory can also be applied to social semantic knowledge, though certain domain-specific spoke-nodes may make a disproportionate contribution to the understanding of social concepts. The ATLs have strong connections with spoke-node structures such as the subgenual ACC (sgACC) and the orbitofrontal cortex (OFC) that play an important role in predicting the hedonic value of stimuli. We hypothesized that in addition to the ATL semantic hub, a social semantic task would also require input from hedonic evaluation structures. We used voxel based morphometry (VBM) to examine structural brain-behavior relationships in 152 patients with neurodegeneration (Alzheimer's disease [N = 12], corticobasal syndrome (N = 18], progressive supranuclear palsy [N = 13], behavioral variant frontotemporal dementia [N = 56], and primary progressive aphasia (PPA) [N = 53]) using the Social Interaction Vocabulary Task (SIVT). This task measures the ability to correctly match a social term (e.g. "gossiping") with a visual depiction of that social interaction. As predicted, VBM showed that worse SIVT scores corresponded with volume loss in bilateral ATL semantic hub regions, but also in the sgACC, OFC, caudate and putamen (pFWE <0.05). These results support the CSC model of a hub-and-spoke organization of social semantic knowledge with the ATL as a domain-general semantic hub, and ventromedial and striatal structures as domain specific spoke-nodes. Importantly, these results suggest that correct comprehension of social semantic concepts requires emotional 'tagging' of a concept by the evaluation system, and that the social deficits observed in some neurodegenerative disease syndromes may be caused by the break-down of this mechanism.

Neural mechanisms underlying the hierarchical construction of perceived aesthetic value

Little is known about how the brain computes the perceived aesthetic value of complex stimuli such as visual art. Here, we used computational methods in combination with functional neuroimaging to provide evidence that the aesthetic value of a visual stimulus is computed in a hierarchical manner via a weighted integration over both low and high level stimulus features contained in early and late visual cortex, extending into parietal and lateral prefrontal cortices. Feature representations in parietal and lateral prefrontal cortex may in turn be utilized to produce an overall aesthetic value in the medial prefrontal cortex. Such brain-wide computations are not only consistent with a feature-based mechanism for value construction, but also resemble computations performed by a deep convolutional neural network. Our findings thus shed light on the existence of a general neurocomputational mechanism for rapidly and flexibly producing value judgements across an array of complex novel stimuli and situations.

Counterfactual curiosity: motivated thinking about what might have been

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate 'counterfactual curiosity': they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally-the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Neural Representations of Food-Related Attributes in the Human Orbitofrontal Cortex during Choice Deliberation in Anorexia Nervosa

Decisions about what to eat recruit the orbitofrontal cortex (OFC) and involve the evaluation of food-related attributes such as taste and health. These attributes are used differently by healthy individuals and patients with disordered eating behavior, but it is unclear whether these attributes are decodable from activity in the OFC in both groups and whether neural representations of these attributes are differentially related to decisions about food. We used fMRI combined with behavioral tasks to investigate the representation of taste and health attributes in the human OFC and the role of these representations in food choices in healthy women and women with anorexia nervosa (AN). We found that subjective ratings of tastiness and healthiness could be decoded from patterns of activity in the OFC in both groups. However, health-related patterns of activity in the OFC were more related to the magnitude of choice preferences among patients with AN than healthy individuals. These findings suggest that maladaptive decision-making in AN is associated with more consideration of health information represented by the OFC during deliberation about what to eat.SIGNIFICANCE STATEMENT An open question about the OFC is whether it supports the evaluation of food-related attributes during deliberation about what to eat. We found that healthiness and tastiness information was decodable from patterns of neural activity in the OFC in both patients with AN and healthy controls. Critically, neural representations of health were more strongly related to choices in patients with AN, suggesting that maladaptive overconsideration of healthiness during deliberation about what to eat is related to activity in the OFC. More broadly, these results show that activity in the human OFC is associated with the evaluation of relevant attributes during value-based decision-making. These findings may also guide future research into the development of treatments for AN.

Reinforcement-learning in fronto-striatal circuits

We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.