Cardiac Concomitants of Feedback and Prediction Error Processing in Reinforcement Learning

Neurocomputational mechanisms underlying differential reinforcement learning from wins and losses in obesity with and without binge eating

BACKGROUND

Binge Eating Disorder (BED) is thought of as a disorder of cognitive control but evidence regarding its neurocognitive mechanisms is inconclusive. Key limitations in prior research are a lack of consistent separation between effects of BED and obesity, and a disregard for self-report evidence suggesting that neurocognitive alterations may emerge primarily in loss- or harm-avoidance contexts.

METHODS

Addressing these gaps, this longitudinal study investigated behavioral flexibility and its underlying neuro-computational processes in reward-seeking and loss-avoidance contexts. Obese participants with BED (BED), without BED (OB), and healthy normal-weight participants (NW) (Ntotal=96) performed a probabilistic reversal learning task during functional imaging, with different blocks focused on obtaining wins or avoiding losses. They were reinvited for a 6-months follow-up.

RESULTS

Analyses informed by computational models of reinforcement learning showed that unlike BED, OB performed worse in the win than the loss condition. Computationally, this was explained by differential learning sensitivities in the win vs loss conditions between groups. In the brain, this was echoed in differential neural learning signals in the ventromedial prefrontal cortex (vmPFC) per condition. The differences were subtle, but scaled with BED symptoms, such that more severe BED symptoms were associated with increasing bias towards improved learning from wins vs losses. Across conditions, OB switched more between choice options than NW. This was reflected in diminished representation of choice certainty in the vmPFC.

CONCLUSIONS

Our study highlights the importance of distinguishing between obesity with and without BED to identify unique neuro-computational alterations underlying different styles of maladaptive eating behavior.

How are overweight and obesity associated with reinforcement learning deficits? A systematic review

Reinforcement learning (RL) refers to the ability to learn stimulus-response or response-outcome associations relevant to the acquisition of behavioral repertoire and adaptation to the environment. Research data from correlational and case-control studies have shown that obesity is associated with impairments in RL. The aim of the present study was to systematically review how obesity and overweight are associated with RL performance. More specifically, the relationship between high body mass index (BMI) and task performance was explored through the analysis of specific RL processes associated with different physiological, computational, and behavioral manifestations. Our systematic analyses indicate that obesity might be associated with impairments in the use of aversive outcomes to change ongoing behavior, as revealed by results involving instrumental negative reinforcement and extinction/reversal learning, but further research needs to be conducted to confirm this association. Hypotheses regarding how obesity might be associated with altered RL were discussed.

Awareness is required for autonomic performance monitoring in instrumental learning: Evidence from cardiac activity

Performance monitoring is a vital aspect of successful learning and decision‐making. Performance errors are reflected in the autonomic nervous system, indicating the need for behavioral adjustment. As part of this response, errors cause a pronounced deceleration in heart rate, compared to correct decisions, and precede explicit awareness of stimulus–response outcome contingencies. However, it is unknown whether those signals are present and able to inform instrumental learning without stimulus awareness, where explicit performance monitoring is disabled. With mixed evidence for unconscious instrumental learning, determining the presence or absence of autonomic signatures of performance monitoring can shed light on its feasibility. Here, we employed an unconscious instrumental conditioning task, where successful learning is evidenced by increased approach responses to visually masked rewarding stimuli, and avoidance of punishing stimuli. An electrocardiogram (ECG) assessed cardiac activity throughout the learning process. Natural fluctuations of awareness under masking permitted us to contrast learning and cardiac deceleration for trials with, versus without, conscious stimulus awareness. Our results demonstrate that on trials where participants did not consciously perceive the stimulus, there was no differentiation in cardiac response between rewarding and punishing feedback, indicating an absence of performance monitoring. In contrast, consciously perceived stimuli elicited the expected error‐related deceleration. This result suggests that, in unconscious instrumental learning, the brain might be unable to acquire knowledge of stimulus values to guide correct instrumental choices. This evidence provides support for the notion that consciousness might be required for flexible adaptive behavior, and that this may be mediated through bodily signals.

Performance monitoring is vital to successful learning. We provide novel evidence that autonomic performance monitoring, indexed by cardiac deceleration, is only engaged in conscious, but not unconscious, instrumental learning. This result provides support for the notion that consciousness is required for flexible adaptive behaviour, and that this relationship may be mediated by bodily signals.

Cardiac deceleration following positive and negative feedback is influenced by competence-based social status

Previous studies indicate that neurophysiological signatures of feedback processing might be enhanced when participants are assigned a low-status position. Error commission and negative feedback can evoke responses in the peripheral (autonomic) nervous system including heart rate deceleration. We conducted an exploratory study to investigate whether such activity can be modulated by the participant's social status in a competence-based hierarchy. Participants were engaged in a cooperative time estimation task with two same-gender confederates. On each trial, they were provided with positive or negative feedback depending on their time estimation performance. Their social status varied during the task, so that they were either at the top (high-status) or at the bottom (low-status) of the hierarchy in different blocks. Results showed that cardiac deceleration was significantly modulated by feedback valence in the high-status but not in the low-status condition. We interpret this result as an increased activation of the performance monitoring system elicited by the desire to maintain a high-status position in an unstable hierarchy. In this vein, negative feedback might be processed as an aversive stimulus that signals a threat to the acquired status.

Graph Analysis of EEG Functional Connectivity Networks During a Letter-Speech Sound Binding Task in Adult Dyslexics

We performed an EEG graph analysis on data from 31 typical readers (22.27 ± 2.53 y/o) and 24 dyslexics (22.99 ± 2.29 y/o), recorded while they were engaged in an audiovisual task and during resting-state. The task simulates reading acquisition as participants learned new letter-sound mappings via feedback. EEG data was filtered for the delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), and beta (13–30 Hz) bands. We computed the Phase Lag Index (PLI) to provide an estimate of the functional connectivity between all pairs of electrodes per band. Then, networks were constructed using a Minimum Spanning Tree (MST), a unique sub-graph connecting all nodes (electrodes) without loops, aimed at minimizing bias in between groups and conditions comparisons. Both groups showed a comparable accuracy increase during task blocks, indicating that they correctly learned the new associations. The EEG results revealed lower task-specific theta connectivity, and lower theta degree correlation over both rest and task recordings, indicating less network integration in dyslexics compared to typical readers. This pattern suggests a role of theta oscillations in dyslexia and may reflect differences in task engagement between the groups, although robust correlations between MST metrics and performance indices were lacking.

Development of a Classical Conditioning Task for Humans Examining Phasic Heart Rate Responses to Signaled Appetitive Stimuli: A Pilot Study

Cardiac responses to appetitive stimuli have been studied as indices of motivational states and attentional processes, the former being associated with cardiac acceleration and latter deceleration. Very few studies have examined heart rate changes in appetitive classical conditioning in humans. The current study describes the development and pilot testing of a classical conditioning task to assess cardiac responses to appetitive stimuli and cues that reliably precede them. Data from 18 adults were examined. They were shown initially neutral visual stimuli (putative CS) on a computer screen followed by pictures of high-caloric food (US). Phasic cardiac deceleration to food images was observed, consistent with an orienting response to motivationally significant stimuli. Similar responses were observed to non-appetitive stimuli when they were preceded by the cue associated with the food images, suggesting that attentional processes were engaged by conditioned stimuli. These autonomic changes provide significant information about classical conditioning effects in humans.

Autonomic responses to choice outcomes: Links to task performance and reinforcement-learning parameters

Previous work has shown that autonomic responses to choice feedback can predict subsequent decision-making. In this study, we tested whether skin conductance responses (SCRs) and heart rate (HR) decelerations following the presentation of choice outcomes predict Iowa Gambling Task performance in nonclinical participants (n = 64). We also examined how these signals related to parameters of a reinforcement-learning (RL) model. Feedback SCRs and HR decelerations were greater following outcomes that included losses and choices from the bad decks defined by their negative expected value. In addition, SCRs predicted task performance. A hierarchical Bayesian RL model indicated that greater feedback SCR for the bad decks compared to good decks was associated with stronger loss aversion and a lower learning rate, both of which predicted higher performance. These results suggest that feedback-related SCRs are linked to individual differences in outcome evaluation and learning processes that guide reinforcement-learning.

Enhanced Go and NoGo Learning in Individuals With Obesity

Overeating in individuals with obesity is hypothesized to be partly caused by automatic action tendencies to food cues that have the potential to override goal-directed dietary restriction. Individuals with obesity are often characterized by alterations in the processing of such rewarding food, but also of non-food stimuli, and previous research has suggested a stronger impact on the execution of goal-directed actions in obesity. Here, we investigated whether Pavlovian cues can also corrupt the learning of new approach or withdrawal behavior in individuals with obesity. We employed a probabilistic Pavlovian-instrumental learning paradigm in which participants (29 normal-weight and 29 obese) learned to actively respond (Go learning) or withhold a response (NoGo learning) in order to gain monetary rewards or avoid losses. Participants were better at learning active approach responses (Go) in the light of anticipated rewards and at learning to withhold a response (NoGo) in the light of imminent punishments. Importantly, there was no evidence for a stronger corruption of instrumental learning in individuals with obesity. Instead, they showed better learning across conditions than normal-weight participants. Using a computational reinforcement learning model, we additionally found an increased learning rate in individuals with obesity. Previous studies have mostly reported a lower reinforcement learning performance in individuals with obesity. Our results contradict this and suggest that their performance is not universally impaired: Instead, while previous studies found reduced stimulus-value learning, individuals with obesity may show better action-value learning. Our findings highlight the need for a broader investigation of behavioral adaptation in obesity across different task designs and types of reinforcement learning.

Probability learning and feedback processing in dyslexia: A performance and heart rate analysis

Recent studies suggest that individuals with dyslexia may be impaired in probability learning and performance monitoring. These observations are consistent with findings indicating atypical neural activations in frontostriatal circuits in the brain, which are important for associative learning. The current study further examined probability learning and performance monitoring in adult individuals with dyslexia (n = 23) and typical readers (n = 31) using two varieties of a typical probabilistic learning task. In addition to performance measures, we measured heart rate, focusing on cardiac slowing with negative feedback as a manifestation of the automatic performance monitoring system. One task required participants to learn associations between artificial script and speech sounds and the other task required them to learn associations between geometric forms and bird sounds. Corrective feedback (informative or random) was provided in both tasks. Performance results indicated that individuals with dyslexia and typical readers learned the associations equally well in contrast to expectations. We found the typical cardiac response associated with feedback processing consisting of a heart rate slowing with the presentation of the feedback and a return to baseline thereafter. Interestingly, the heart rate slowing associated with feedback was less pronounced and the return to baseline was delayed in individuals with dyslexia relative to typical readers. These findings were interpreted in relation to current theorizing of performance monitoring linking the salience network in the brain to autonomic functioning.