Anatomical dissociation of intracerebral signals for reward and punishment prediction errors in humans

Neural evidence for attentional capture by salient distractors

Salient objects often capture our attention, serving as distractors and hindering our current goals. It remains unclear when and how salient distractors interact with our goals, and our knowledge on the neural mechanisms responsible for attentional capture is limited to a few brain regions recorded from non-human primates. Here we conducted a multivariate analysis on human intracranial signals covering most brain regions and successfully dissociated distractor-specific representations from target-arousal signals in the high-frequency (60-100 Hz) activity. We found that salient distractors were processed rapidly around 220 ms, while target-tuning attention was attenuated simultaneously, supporting initial capture by distractors. Notably, neuronal activity specific to the distractor representation was strongest in the superior and middle temporal gyrus, amygdala and anterior cingulate cortex, while there were smaller contributions from the parietal and frontal cortices. These results provide neural evidence for attentional capture by salient distractors engaging a much larger network than previously appreciated.

More widespread and rigid neuronal representation of reward expectation underlies impulsive choices

Impulsive choices prioritize smaller, more immediate rewards over larger, delayed, or potentially uncertain rewards. Impulsive choices are a critical aspect of substance use disorders and maladaptive decision-making across the lifespan. Here, we sought to understand the neuronal underpinnings of expected reward and risk estimation on a trial-by-trial basis during impulsive choices. To do so, we acquired electrical recordings from the human brain while participants carried out a risky decision-making task designed to measure choice impulsivity. Behaviorally, we found a reward-accuracy tradeoff, whereby more impulsive choosers were more accurate at the task, opting for a more immediate reward while compromising overall task performance. We then examined how neuronal populations across frontal, temporal, and limbic brain regions parametrically encoded reinforcement learning model variables, namely reward and risk expectation and surprise, across trials. We found more widespread representations of reward value expectation and prediction error in more impulsive choosers, whereas less impulsive choosers preferentially represented risk expectation. A regional analysis of reward and risk encoding highlighted the anterior cingulate cortex for value expectation, the anterior insula for risk expectation and surprise, and distinct regional encoding between impulsivity groups. Beyond describing trial-by-trial population neuronal representations of reward and risk variables, these results suggest impaired inhibitory control and model-free learning underpinnings of impulsive choice. These findings shed light on neural processes underlying reinforced learning and decision-making in uncertain environments and how these processes may function in psychiatric disorders.

Working memory components modulation of attentional disengagement from evaluative distractor

It is important for people to disengage attention from a distraction, which can help them complete the task at hand as quickly as possible. Recent studies have shown that people's attention stays longer on reward-distractors than on loss-distractors, and a delay in attentional disengagement is noted when reward-distractors are present. However, few studies have examined whether attentional disengagement from an evaluative distractor relies upon working memory (WM) components. In the present study, we used an attentional disengagement paradigm in which reward- or loss-distractors were presented at a central location and the target was presented at a peripheral location, in combination with different WM tasks. The results from Experiment 1 showed that participants were slower to disengage their attention from a central reward-distractor than a loss-distractor regardless of cognitive load when the phonological loop component of WM was involved. The results from Experiment 2 revealed that people had difficulty in shifting their attention away from a reward-distractor in comparison to a loss-distractor when spatial WM was low, whereas no such difference was observed when spatial WM was high. We conclude that WM components differently modulate attentional disengagement from evaluative distractors. That is, the processing of evaluative (reward and loss) distractors may rely on the same cognitive resources as the spatial WM component, but not the phonological loop component.

Temporally organized representations of reward and risk in the human brain

The value and uncertainty associated with choice alternatives constitute critical features relevant for decisions. However, the manner in which reward and risk representations are temporally organized in the brain remains elusive. Here we leverage the spatiotemporal precision of intracranial electroencephalography, along with a simple card game designed to elicit the unfolding computation of a set of reward and risk variables, to uncover this temporal organization. Reward outcome representations across wide-spread regions follow a sequential order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We further highlight the role of the anterior insula in generalizing between reward prediction error and risk prediction error codes. Together our results emphasize the importance of neural dynamics for understanding value-based decisions under uncertainty.

Punishment Leads to Greater Sensorimotor Learning But Less Movement Variability Compared to Reward

When a musician practices a new song, hitting a correct note sounds pleasant while striking an incorrect note sounds unpleasant. Such reward and punishment feedback has been shown to differentially influence the ability to learn a new motor skill. Recent work has suggested that punishment leads to greater movement variability, which causes greater exploration and faster learning. To further test this idea, we collected 102 participants over two experiments. Unlike previous work, in Experiment 1 we found that punishment did not lead to faster learning compared to reward (n = 68), but did lead to a greater extent of learning. Surprisingly, we also found evidence to suggest that punishment led to less movement variability, which was related to the extent of learning. We then designed a second experiment that did not involve adaptation, allowing us to further isolate the influence of punishment feedback on movement variability. In Experiment 2, we again found that punishment led to significantly less movement variability compared to reward (n = 34). Collectively our results suggest that punishment feedback leads to less movement variability. Future work should investigate whether punishment feedback leads to a greater knowledge of movement variability and or increases the sensitivity of updating motor actions.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

The parietal cortex has a causal role in ambiguity computations in humans

Humans often face the challenge of making decisions between ambiguous options. The level of ambiguity in decision-making has been linked to activity in the parietal cortex, but its exact computational role remains elusive. To test the hypothesis that the parietal cortex plays a causal role in computing ambiguous probabilities, we conducted consecutive fMRI and TMS-EEG studies. We found that participants assigned unknown probabilities to objective probabilities, elevating the uncertainty of their decisions. Parietal cortex activity correlated with the objective degree of ambiguity and with a process that underestimates the uncertainty during decision-making. Conversely, the midcingulate cortex (MCC) encodes prediction errors and increases its connectivity with the parietal cortex during outcome processing. Disruption of the parietal activity increased the uncertainty evaluation of the options, decreasing cingulate cortex oscillations during outcome evaluation and lateral frontal oscillations related to value ambiguous probability. These results provide evidence for a causal role of the parietal cortex in computing uncertainty during ambiguous decisions made by humans.

Time course of EEG power during creative problem-solving with insight or remote thinking

Problem-solving often requires creativity and is critical in everyday life. However, the neurocognitive mechanisms underlying creative problem-solving remain poorly understood. Two mechanisms have been highlighted: the formation of new connections among problem elements and insight solving, characterized by sudden realization of a solution. In this study, we investigated EEG activity during a modified version of the remote associates test, a classical insight problem task that requires finding a word connecting three unrelated words. This allowed us to explore the brain correlates associated with the semantic remoteness of connections (by varying the remoteness of the solution word across trials) and with insight solving (identified as a Eurêka moment reported by the participants). Semantic remoteness was associated with power increase in the alpha band (8-12 Hz) in a left parieto-temporal cluster, the beta band (13-30 Hz) in a right fronto-temporal cluster in the early phase of the task, and the theta band (3-7 Hz) in a bilateral frontal cluster just prior to participants' responses. Insight solving was associated with power increase preceding participants' responses in the alpha and gamma (31-60 Hz) bands in a left temporal cluster and the theta band in a frontal cluster. Source reconstructions revealed the brain regions associated with these clusters. Overall, our findings shed new light on some of the mechanisms involved in creative problem-solving.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Beta and theta oscillations track effort and previous reward in human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we investigate this with chronic intracranial recordings from prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12-20 Hz) oscillations tracking subjective effort on a single trial basis and PFC theta (4-7 Hz) signaling previous trial reward. Stimulation of PFC increased overall acceptance of offers in addition to increasing the impact of reward on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, as well as supporting a causal role of PFC for such choices.

Reward and punishment learning deficits among bipolar disorder subtypes

BACKGROUND

Reward sensitivity is an essential dimension related to mood fluctuations in bipolar disorder (BD), but there is currently a debate around hypersensitivity or hyposensitivity hypotheses to reward in BD during remission, probably related to a heterogeneous population within the BD spectrum and a lack of reward bias evaluation. Here, we examine reward maximization vs. punishment avoidance learning within the BD spectrum during remission.

METHODS

Patients with BD-I (n = 45), BD-II (n = 34) and matched (n = 30) healthy controls (HC) were included. They performed an instrumental learning task designed to dissociate reward-based from punishment-based reinforcement learning. Computational modeling was used to identify the mechanisms underlying reinforcement learning performances.

RESULTS

Behavioral results showed a significant reward learning deficit across BD subtypes compared to HC, captured at the computational level by a lower sensitivity to rewards compared to punishments in both BD subtypes. Computational modeling also revealed a higher choice randomness in BD-II compared to BD-I that reflected a tendency of BD-I to perform better during punishment avoidance learning than BD-II.

LIMITATIONS

Our patients were not naive to antipsychotic treatment and were not euthymic (but in syndromic remission) according to the International Society for Bipolar Disorder definition.

CONCLUSIONS

Our results are consistent with the reward hyposensitivity theory in BD. Computational modeling suggests distinct underlying mechanisms that produce similar observable behaviors, making it a useful tool for distinguishing how symptoms interact in BD versus other disorders. In the long run, a better understanding of these processes could contribute to better prevention and management of BD.

Human thalamic low-frequency oscillations correlate with expected value and outcomes during reinforcement learning

Reinforcement-based adaptive decision-making is believed to recruit fronto-striatal circuits. A critical node of the fronto-striatal circuit is the thalamus. However, direct evidence of its involvement in human reinforcement learning is lacking. We address this gap by analyzing intra-thalamic electrophysiological recordings from eight participants while they performed a reinforcement learning task. We found that in both the anterior thalamus (ATN) and dorsomedial thalamus (DMTN), low frequency oscillations (LFO, 4-12 Hz) correlated positively with expected value estimated from computational modeling during reward-based learning (after outcome delivery) or punishment-based learning (during the choice process). Furthermore, LFO recorded from ATN/DMTN were also negatively correlated with outcomes so that both components of reward prediction errors were signaled in the human thalamus. The observed differences in the prediction signals between rewarding and punishing conditions shed light on the neural mechanisms underlying action inhibition in punishment avoidance learning. Our results provide insight into the role of thalamus in reinforcement-based decision-making in humans.

Electrophysiological signatures of inequity-dependent reward encoding in the human OFC

Social decision making requires the integration of reward valuation and social cognition systems, both dependent on the orbitofrontal cortex (OFC). How these two OFC functions interact is largely unknown. We recorded intracranial activity from the OFC of ten patients making choices in a social context where reward inequity with a social counterpart varied and could be either advantageous or disadvantageous. We find that OFC high-frequency activity (HFA; 70-150 Hz) encodes self-reward, consistent with previous reports. We also observe encoding of the social counterpart's reward, as well as the type of inequity being experienced. Additionally, we find evidence of inequity-dependent reward encoding: depending on the type of inequity, electrodes rapidly and reversibly switch between different reward-encoding profiles. These results provide direct evidence for encoding of self- and other rewards in the human OFC and highlight the dynamic nature of encoding in the OFC as a function of social context.

Causal role of the dorsolateral prefrontal cortex in modulating the balance between Pavlovian and instrumental systems in the punishment domain

Previous literature suggests that a balance between Pavlovian and instrumental decision-making systems is critical for optimal decision-making. Pavlovian bias (i.e., approach toward reward-predictive stimuli and avoid punishment-predictive stimuli) often contrasts with the instrumental response. Although recent neuroimaging studies have identified brain regions that may be related to Pavlovian bias, including the dorsolateral prefrontal cortex (dlPFC), it is unclear whether a causal relationship exists. Therefore, we investigated whether upregulation of the dlPFC using transcranial current direct stimulation (tDCS) would reduce Pavlovian bias. In this double-blind study, participants were assigned to the anodal or the sham group; they received stimulation over the right dlPFC for 3 successive days. On the last day, participants performed a reinforcement learning task known as the orthogonalized go/no-go task; this was used to assess each participant's degree of Pavlovian bias in reward and punishment domains. We used computational modeling and hierarchical Bayesian analysis to estimate model parameters reflecting latent cognitive processes, including Pavlovian bias, go bias, and choice randomness. Several computational models were compared; the model with separate Pavlovian bias parameters for reward and punishment domains demonstrated the best model fit. When using a behavioral index of Pavlovian bias, the anodal group showed significantly lower Pavlovian bias in the punishment domain, but not in the reward domain, compared with the sham group. In addition, computational modeling showed that Pavlovian bias parameter in the punishment domain was lower in the anodal group than in the sham group, which is consistent with the behavioral findings. The anodal group also showed a lower go bias and choice randomness, compared with the sham group. These findings suggest that anodal tDCS may lead to behavioral suppression or change in Pavlovian bias in the punishment domain, which will help to improve comprehension of the causal neural mechanism.

Temporally organized representations of reward and risk in the human brain

The value and uncertainty associated with choice alternatives constitute critical features along which decisions are made. While the neural substrates supporting reward and risk processing have been investigated, the temporal organization by which these computations are encoded remains elusive. Here we leverage the high spatiotemporal precision of intracranial electroencephalography (iEEG) to uncover how representations of decision-related computations unfold in time. We present evidence of locally distributed representations of reward and risk variables that are temporally organized across multiple regions of interest. Reward outcome representations across wide-spread regions follow a temporally cascading order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We highlight the role of the anterior insula in generalizing between reward prediction error (RePE) and risk prediction error (RiPE), within which the encoding of RePE in the distributed iEEG signal predicts RiPE. Together our results emphasize the utility of uncovering temporal dynamics in the human brain for understanding how computational processes critical for value-based decisions under uncertainty unfold.

A parahippocampal-sensory Bayesian vicious circle generates pain or tinnitus: a source-localized EEG study

Pain and tinnitus share common pathophysiological mechanisms, clinical features, and treatment approaches. A source-localized resting-state EEG study was conducted in 150 participants: 50 healthy controls, 50 pain, and 50 tinnitus patients. Resting-state activity as well as functional and effective connectivity was computed in source space. Pain and tinnitus were characterized by increased theta activity in the pregenual anterior cingulate cortex, extending to the lateral prefrontal cortex and medial anterior temporal lobe. Gamma-band activity was increased in both auditory and somatosensory cortex, irrespective of the pathology, and extended to the dorsal anterior cingulate cortex and parahippocampus. Functional and effective connectivity were largely similar in pain and tinnitus, except for a parahippocampal-sensory loop that distinguished pain from tinnitus. In tinnitus, the effective connectivity between parahippocampus and auditory cortex is bidirectional, whereas the effective connectivity between parahippocampus and somatosensory cortex is unidirectional. In pain, the parahippocampal-somatosensory cortex is bidirectional, but parahippocampal auditory cortex unidirectional. These modality-specific loops exhibited theta-gamma nesting. Applying a Bayesian brain model of brain functioning, these findings suggest that the phenomenological difference between auditory and somatosensory phantom percepts result from a vicious circle of belief updating in the context of missing sensory information. This finding may further our understanding of multisensory integration and speaks to a universal treatment for pain and tinnitus-by selectively disrupting parahippocampal-somatosensory and parahippocampal-auditory theta-gamma activity and connectivity.

Invasive Computational Psychiatry

Computational psychiatry, a relatively new yet prolific field that aims to understand psychiatric disorders with formal theories about the brain, has seen tremendous growth in the past decade. Despite initial excitement, actual progress made by computational psychiatry seems stagnant. Meanwhile, understanding of the human brain has benefited tremendously from recent progress in intracranial neuroscience. Specifically, invasive techniques such as stereotactic electroencephalography, electrocorticography, and deep brain stimulation have provided a unique opportunity to precisely measure and causally modulate neurophysiological activity in the living human brain. In this review, we summarize progress and drawbacks in both computational psychiatry and invasive electrophysiology and propose that their combination presents a highly promising new direction-invasive computational psychiatry. The value of this approach is at least twofold. First, it advances our mechanistic understanding of the neural computations of mental states by providing a spatiotemporally precise depiction of neural activity that is traditionally unattainable using noninvasive techniques with human subjects. Second, it offers a direct and immediate way to modulate brain states through stimulation of algorithmically defined neural regions and circuits (i.e., algorithmic targeting), thus providing both causal and therapeutic insights. We then present depression as a use case where the combination of computational and invasive approaches has already shown initial success. We conclude by outlining future directions as a road map for this exciting new field as well as presenting cautions about issues such as ethical concerns and generalizability of findings.

An orbitofrontal cortex-anterior insular cortex circuit gates compulsive cocaine use

Compulsive drug use, a cardinal symptom of drug addiction, is characterized by persistent substance use despite adverse consequences. However, little is known about the neural circuit mechanisms behind this behavior. Using a footshock-punished cocaine self-administration procedure, we found individual variability of rats in the process of drug addiction, and rats with compulsive cocaine use presented increased neural activity of the anterior insular cortex (aIC) compared with noncompulsive rats. Chemogenetic manipulating activity of aIC neurons, especially aIC glutamatergic neurons, bidirectionally regulated compulsive cocaine intake. Furthermore, the aIC received inputs from the orbitofrontal cortex (OFC), and the OFC-aIC circuit was enhanced in rats with compulsive cocaine use. Suppression of the OFC-aIC circuit switched rats from punishment resistance to sensitivity, while potentiation of this circuit increased compulsive cocaine use. In conclusion, our results found that aIC glutamatergic neurons and the OFC-aIC circuit gated the shift from controlled to compulsive cocaine use, which could serve as potential therapeutic targets for drug addiction.

Neural activations to loss anticipation mediates the association between difficulties in emotion regulation and screen media activities among early adolescent youth: A moderating role for depression

BACKGROUND

Screen media activities (SMAs; e.g., watching videos, playing videogames) have become increasingly prevalent among youth as ways to alleviate or escape from negative emotional states. However, neural mechanisms underlying these processes in youth are incompletely understood.

METHOD

Seventy-nine youth aged 11-15 years completed a monetary incentive delay task during fMRI scanning. Neural correlates of reward/loss processing and their associations with SMAs were explored. Next, brain activations during reward/loss processing in regions implicated in the processing of emotions were examined as potential mediating factors between difficulties in emotion regulation (DER) and engagement in SMAs. Finally, a moderated mediation model tested the effects of depressive symptoms in such relationships.

RESULT

The emotional components associated with SMAs in reward/loss processing included activations in the left anterior insula (AI) and right dorsolateral prefrontal cortex (DLPFC) during anticipation of working to avoid losses. Activations in both the AI and DLPFC mediated the relationship between DER and SMAs. Moreover, depressive symptoms moderated the relationship between AI activation in response to loss anticipation and SMAs.

CONCLUSION

The current findings suggest that DER link to SMAs through loss-related brain activations implicated in the processing of emotions and motivational avoidance, particularly in youth with greater levels of depressive symptoms. The findings suggest the importance of enhancing emotion-regulation tendencies/abilities in youth and, in particular, their regulatory responses to negative emotional situations in order to guide moderate engagement in SMAs.

Intracerebral mechanisms explaining the impact of incidental feedback on mood state and risky choice

Identifying factors whose fluctuations are associated with choice inconsistency is a major issue for rational decision theory. Here, we investigated the neuro-computational mechanisms through which mood fluctuations may bias human choice behavior. Intracerebral EEG data were collected in a large group of subjects (n=30) while they were performing interleaved quiz and choice tasks that were designed to examine how a series of unrelated feedbacks affect decisions between safe and risky options. Neural baseline activity preceding choice onset was confronted first to mood level, estimated by a computational model integrating the feedbacks received in the quiz task, and then to the weighting of option attributes, in a computational model predicting risk attitude in the choice task. Results showed that (1) elevated broadband gamma activity (BGA) in the ventromedial prefrontal cortex (vmPFC) and dorsal anterior insula (daIns) was respectively signaling periods of high and low mood, (2) increased vmPFC and daIns BGA respectively promoted and tempered risk taking by overweighting gain vs. loss prospects. Thus, incidental feedbacks induce brain states that correspond to different moods and bias the evaluation of risky options. More generally, these findings might explain why people experiencing positive (or negative) outcome in some part of their life tend to expect success (or failure) in any other.

Group-level inference of information-based measures for the analyses of cognitive brain networks from neurophysiological data

The reproducibility crisis in neuroimaging and in particular in the case of underpowered studies has introduced doubts on our ability to reproduce, replicate and generalize findings. As a response, we have seen the emergence of suggested guidelines and principles for neuroscientists known as Good Scientific Practice for conducting more reliable research. Still, every study remains almost unique in its combination of analytical and statistical approaches. While it is understandable considering the diversity of designs and brain data recording, it also represents a striking point against reproducibility. Here, we propose a non-parametric permutation-based statistical framework, primarily designed for neurophysiological data, in order to perform group-level inferences on non-negative measures of information encompassing metrics from information-theory, machine-learning or measures of distances. The framework supports both fixed- and random-effect models to adapt to inter-individuals and inter-sessions variability. Using numerical simulations, we compared the accuracy in ground-truth retrieving of both group models, such as test- and cluster-wise corrections for multiple comparisons. We then reproduced and extended existing results using both spatially uniform MEG and non-uniform intracranial neurophysiological data. We showed how the framework can be used to extract stereotypical task- and behavior-related effects across the population covering scales from the local level of brain regions, inter-areal functional connectivity to measures summarizing network properties. We also present an open-source Python toolbox called Frites¹ that includes the proposed statistical pipeline using information-theoretic metrics such as single-trial functional connectivity estimations for the extraction of cognitive brain networks. Taken together, we believe that this framework deserves careful attention as its robustness and flexibility could be the starting point toward the uniformization of statistical approaches.

The computational roots of positivity and confirmation biases in reinforcement learning

Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.

The human insula processes both modality-independent and pain-selective learning signals

Prediction errors (PEs) are generated when there are differences between an expected and an actual event or sensory input. The insula is a key brain region involved in pain processing, and studies have shown that the insula encodes the magnitude of an unexpected outcome (unsigned PEs). In addition to signaling this general magnitude information, PEs can give specific information on the direction of this deviation-i.e., whether an event is better or worse than expected. It is unclear whether the unsigned PE responses in the insula are selective for pain or reflective of a more general processing of aversive events irrespective of modality. It is also unknown whether the insula can process signed PEs at all. Understanding these specific mechanisms has implications for understanding how pain is processed in the brain in both health and in chronic pain conditions. In this study, 47 participants learned associations between 2 conditioned stimuli (CS) with 4 unconditioned stimuli (US; painful heat or loud sound, of one low and one high intensity each) while undergoing functional magnetic resonance imaging (fMRI) and skin conductance response (SCR) measurements. We demonstrate that activation in the anterior insula correlated with unsigned intensity PEs, irrespective of modality, indicating an unspecific aversive surprise signal. Conversely, signed intensity PE signals were modality specific, with signed PEs following pain but not sound located in the dorsal posterior insula, an area implicated in pain intensity processing. Previous studies have identified abnormal insula function and abnormal learning as potential causes of pain chronification. Our findings link these results and suggest that a misrepresentation of learning relevant PEs in the insular cortex may serve as an underlying factor in chronic pain.