Neurons in human pre-supplementary motor area encode key computations for value-based choice

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

Free Will as Defined by the Constrained Disorder Principle: a Restricted, Mandatory, Personalized, Regulated Process for Decision-Making

The concept of free will has challenged physicists, biologists, philosophers, and other professionals for decades. The constrained disorder principle (CDP) is a fundamental law that defines systems according to their inherent variability. It provides mechanisms for adapting to dynamic environments. This work examines the CDP's perspective of free will concerning various free will theories. Per the CDP, systems lack intentions, and the "freedom" to select and act is built into their design. The "freedom" is embedded within the response range determined by the boundaries of the systems' variability. This built-in and self-generating mechanism enables systems to cope with perturbations. According to the CDP, neither dualism nor an unknown metaphysical apparatus dictates choices. Brain variability facilitates cognitive adaptation to complex, unpredictable situations across various environments. Human behaviors and decisions reflect an underlying physical variability in the brain and other organs for dealing with unpredictable noises. Choices are not predetermined but reflect the ongoing adaptation processes to dynamic prssu½res. Malfunctions and disease states are characterized by inappropriate variability, reflecting an inability to respond adequately to perturbations. Incorporating CDP-based interventions can overcome malfunctions and disease states and improve decision processes. CDP-based second-generation artificial intelligence platforms improve interventions and are being evaluated to augment personal development, wellness, and health.

The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Encoding of Predictive Associations in Human Prefrontal and Medial Temporal Neurons During Pavlovian Appetitive Conditioning

Pavlovian conditioning is thought to involve the formation of learned associations between stimuli and values, and between stimuli and specific features of outcomes. Here, we leveraged human single neuron recordings in ventromedial prefrontal, dorsomedial frontal, hippocampus, and amygdala while patients of both sexes performed an appetitive Pavlovian conditioning task probing both stimulus-value and stimulus-stimulus associations. Ventromedial prefrontal cortex encoded predictive value along with the amygdala, and also encoded predictions about the identity of stimuli that would subsequently be presented, suggesting a role for neurons in this region in encoding predictive information beyond value. Unsigned error signals were found in dorsomedial frontal areas and hippocampus, potentially supporting learning of non-value related outcome features. Our findings implicate distinct human prefrontal and medial temporal neuronal populations in mediating predictive associations which could partially support model-based mechanisms during Pavlovian conditioning.

Temporally organized representations of reward and risk in the human brain

The value and uncertainty associated with choice alternatives constitute critical features relevant for decisions. However, the manner in which reward and risk representations are temporally organized in the brain remains elusive. Here we leverage the spatiotemporal precision of intracranial electroencephalography, along with a simple card game designed to elicit the unfolding computation of a set of reward and risk variables, to uncover this temporal organization. Reward outcome representations across wide-spread regions follow a sequential order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We further highlight the role of the anterior insula in generalizing between reward prediction error and risk prediction error codes. Together our results emphasize the importance of neural dynamics for understanding value-based decisions under uncertainty.

The Neural Correlates of Individual Differences in Reinforcement Learning during Pain Avoidance and Reward Seeking

Organisms learn to gain reward and avoid punishment through action-outcome associations. Reinforcement learning (RL) offers a critical framework to understand individual differences in this associative learning by assessing learning rate, action bias, pavlovian factor (i.e., the extent to which action values are influenced by stimulus values), and subjective impact of outcomes (i.e., motivation to seek reward and avoid punishment). Nevertheless, how these individual-level metrics are represented in the brain remains unclear. The current study leveraged fMRI in healthy humans and a probabilistic learning go/no-go task to characterize the neural correlates involved in learning to seek reward and avoid pain. Behaviorally, participants showed a higher learning rate during pain avoidance relative to reward seeking. Additionally, the subjective impact of outcomes was greater for reward trials and associated with lower response randomness. Our imaging findings showed that individual differences in learning rate and performance accuracy during avoidance learning were positively associated with activities of the dorsal anterior cingulate cortex, midcingulate cortex, and postcentral gyrus. In contrast, the pavlovian factor was represented in the precentral gyrus and superior frontal gyrus (SFG) during pain avoidance and reward seeking, respectively. Individual variation of the subjective impact of outcomes was positively predicted by activation of the left posterior cingulate cortex. Finally, action bias was represented by the supplementary motor area (SMA) and pre-SMA whereas the SFG played a role in restraining this action tendency. Together, these findings highlight for the first time the neural substrates of individual differences in the computational processes during RL.

Dorsomedial frontal cortex damage impairs error-based, but not reinforcement-based motor learning in humans

We adapt our movements to new and changing environments through multiple processes. Sensory error-based learning counteracts environmental perturbations that affect the sensory consequences of movements. Sensory errors also cause the upregulation of reflexes and muscle co-contraction. Reinforcement-based learning enhances the selection of movements that produce rewarding outcomes. Although some findings have identified dissociable neural substrates of sensory error- and reinforcement-based learning, correlative methods have implicated dorsomedial frontal cortex in both. Here, we tested the causal contributions of dorsomedial frontal to adaptive motor control, studying people with chronic damage to this region. Seven human participants with focal brain lesions affecting the dorsomedial frontal and 20 controls performed a battery of arm movement tasks. Three experiments tested: (i) the upregulation of visuomotor reflexes and muscle co-contraction in response to unpredictable mechanical perturbations, (ii) sensory error-based learning in which participants learned to compensate predictively for mechanical force-field perturbations, and (iii) reinforcement-based motor learning based on binary feedback in the absence of sensory error feedback. Participants with dorsomedial frontal damage were impaired in the early stages of force field adaptation, but performed similarly to controls in all other measures. These results provide evidence for a specific and selective causal role for the dorsomedial frontal in sensory error-based learning.

A cognitive map for value-guided choice in ventromedial prefrontal cortex

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Beta and theta oscillations track effort and previous reward in human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we investigate this with chronic intracranial recordings from prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12-20 Hz) oscillations tracking subjective effort on a single trial basis and PFC theta (4-7 Hz) signaling previous trial reward. Stimulation of PFC increased overall acceptance of offers in addition to increasing the impact of reward on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, as well as supporting a causal role of PFC for such choices.

Exploring the Impact of Smartphone Addiction on Risk Decision-Making Behavior among College Students Based on fNIRS Technology

Smartphone Addiction is a social issue caused by excessive smartphone use, affecting decision-making processes. Current research on the risky decision-making abilities of smartphone addicts is limited. This study used the functional Near-Infrared Spectroscopy (fNIRS) brain imaging technique and a Sequential Risk-Taking Task experimental paradigm to investigate the decision-making behavior and brain activity of smartphone addicts under varying risk levels. Using a mixed experimental design, the research assessed decision-making ability and brain activation levels as dependent variables across two groups (addiction and control), two risk amounts (high and low), and two outcomes (gain and loss). The study included 42 participants, with 25 in the addiction group and 17 in the control group. Results indicated that risk level significantly impacted the decision-making ability of smartphone addicts, with high-risk levels leading to weaker decision-making ability and increased risk-taking. However, at low-risk levels, decision-making abilities between addicts and healthy individuals showed no significant difference. Furthermore, brain imaging results using fNIRS revealed stronger brain activation in the dorsolateral Prefrontal Cortex (dlPFC) region for smartphone addicts under loss outcome conditions, with no significant differences between the two groups in terms of brain activation at varying risk volumes. These findings are critical in promoting healthy smartphone use, guiding clinical treatment, and advancing brain mechanism research.

Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus

For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.