Reliance on model-based and model-free control in obesity

Working memory gating in obesity is moderated by striatal dopaminergic gene variants

Everyday life requires an adaptive balance between distraction-resistant maintenance of information and the flexibility to update this information when needed. These opposing mechanisms are proposed to be balanced through a working memory gating mechanism. Prior research indicates that obesity may elevate the risk of working memory deficits, yet the underlying mechanisms remain elusive. Dopaminergic alterations have emerged as a potential mediator. However, current models suggest these alterations should only shift the balance in working memory tasks, not produce overall deficits. The empirical support for this notion is currently lacking, however. To address this gap, we pooled data from three studies (N = 320) where participants performed a working memory gating task. Higher BMI was associated with overall poorer working memory, irrespective of whether there was a need to maintain or update information. However, when participants, in addition to BMI level, were categorized based on certain putative dopamine-signaling characteristics (single-nucleotide polymorphisms [SNPs]; specifically, Taq1A and DARPP-32), distinct working memory gating effects emerged. These SNPs, primarily associated with striatal dopamine transmission, appear to be linked with differences in updating, specifically, among high-BMI individuals. Moreover, blood amino acid ratio, which indicates central dopamine synthesis capacity, combined with BMI shifted the balance between distractor-resistant maintenance and updating. These findings suggest that both dopamine-dependent and dopamine-independent cognitive effects exist in obesity. Understanding these effects is crucial if we aim to modify maladaptive cognitive profiles in individuals with obesity.

Disentangling negative reinforcement, working memory, and deductive reasoning deficits in elevated BMI

Neuropsychological data suggest that being overweight or obese is associated with a tendency to perseverate behavior despite negative feedback. This deficit might be observed due to other cognitive factors, such as working memory (WM) deficits or decreased ability to deduce model-based strategies when learning by trial-and-error. In the present study, a group of subjects with overweight or obesity (Ow/Ob, n = 30) was compared to normal-weight individuals (n = 42) in a modified Reinforcement Learning (RL) task. The task was designed to control WM effects on learning by manipulating cognitive load and to foster model-based learning via deductive reasoning. Computational modelling and analysis were conducted to isolate parameters related to RL mechanisms, WM use, and model-based learning (deduction parameter). Results showed that subjects with Ow/Ob had a higher number of perseverative errors and used a weaker deduction mechanism in their performance than control individuals, indicating impairments in negative reinforcement and model-based learning, whereas WM impairments were not responsible for deficits in RL. The present data suggests that obesity is associated with impairments in negative reinforcement and model-based learning.

Greater reliance on model-free learning in adolescent anorexia nervosa: An examination of dual-system reinforcement learning

Alterations in learning and decision-making systems are thought to contribute to core features of anorexia nervosa (AN), a psychiatric disorder characterized by persistent dietary restriction and weight loss. Instrumental learning theory identifies a dual-system of habit and goal-directed decision-making, linked to model-free and model-based reinforcement learning algorithms. Difficulty arbitrating between these systems, resulting in an over-reliance on one strategy over the other, has been implicated in compulsivity and extreme goal pursuit, both of which are observed in AN. Characterizing alterations in model-free and model-based systems, and their neural correlates, in AN may clarify mechanisms contributing to symptom heterogeneity (e.g., binge/purge symptoms). This study tested whether adolescents with restricting AN (AN-R; n = 36) and binge/purge AN (AN-BP; n = 20) differentially utilized model-based and model-free learning systems compared to a healthy control group (HC; n = 28) during a Markov two-step decision-making task under conditions of reward and punishment. Associations between model-free and model-based learning and resting-state functional connectivity between neural regions of interest, including orbitofrontal cortex (OFC), nucleus accumbens (NAcc), putamen, and sensory motor cortex (SMC) were examined. AN-R showed higher utilization of model-free learning compared to HC for reward, but attenuated model-free and model-based learning for punishment. In AN-R only, higher model-based learning was associated with stronger OFC-to-left NAcc functional connectivity, regions linked to goal-directed behavior. Greater utilization of model-free learning for reward in AN-R may differentiate this group, particularly during adolescence, and facilitate dietary restriction by prioritizing habitual control in rewarding contexts.

How are overweight and obesity associated with reinforcement learning deficits? A systematic review

Reinforcement learning (RL) refers to the ability to learn stimulus-response or response-outcome associations relevant to the acquisition of behavioral repertoire and adaptation to the environment. Research data from correlational and case-control studies have shown that obesity is associated with impairments in RL. The aim of the present study was to systematically review how obesity and overweight are associated with RL performance. More specifically, the relationship between high body mass index (BMI) and task performance was explored through the analysis of specific RL processes associated with different physiological, computational, and behavioral manifestations. Our systematic analyses indicate that obesity might be associated with impairments in the use of aversive outcomes to change ongoing behavior, as revealed by results involving instrumental negative reinforcement and extinction/reversal learning, but further research needs to be conducted to confirm this association. Hypotheses regarding how obesity might be associated with altered RL were discussed.

Reduced sensitivity but intact motivation to monetary rewards and reversal learning in obesity

BACKGROUND

Obesity has been linked to altered reward processing but little is known about which components of reward processing including motivation, sensitivity and learning are impaired in obesity. We examined whether obesity compared to healthy weight controls is associated with differences in distinct subdomains of reward processing. To this end, we used two established paradigms, namely the Effort Expenditure for Rewards task (EEfRT) and the Probabilistic Reversal Learning Task (PRLT).

METHODS

30 individuals with obesity (OBS) and 30 healthy weight control subjects (HC) were included in the study. Generalized estimating equation models were used to analyze EEfRT choice behavior. PRLT data was analyzed using both conventional behavioral variables of choices and computational models.

RESULTS

Our findings from the different tasks speak in favor of a hyposensitivity to non-food rewards in obesity. OBS did not make fewer overall hard task selections compared to HC in the EEfRT suggesting generally intact non-food reward motivation. However, in highly rewarding trials (i.e.,trials with high reward magnitude and high reward probability),OBSmadefewer hard task selections compared to normal weight subjects suggesting decreased sensitivity to highly rewarding non-food reinforcers. Hyposensitivity to non-food rewards was also evident in OBS in the PRLT as evidenced by lower win-stay probability compared to HC. Our computational modelling analyses revealed decreased stochasticity but intact reward and punishment learning rates in OBS.

CONCLUSIONS

Our findings provide evidence for intact reward motivation and learning in OBS but lower reward sensitivity which is linked to stochasticity of choices in a non-food context. These findings might provide further insight into the mechanism underlying dysfunctional choices in obesity.

A Neuroeconomics Approach to Obesity

Obesity is a heterogeneous condition that is affected by physiological, behavioral, and environmental factors. Value-based decision making is a useful framework for integrating these factors at the individual level. The disciplines of behavioral economics and reinforcement learning provide tools for identifying specific cognitive and motivational processes that may contribute to the development and maintenance of obesity. Neuroeconomics complements these disciplines by studying the neural mechanisms underlying these processes. We surveyed recent literature on individual decision characteristics that are most frequently implicated in obesity: discounting the value of future outcomes, attitudes toward uncertainty, and learning from rewards and punishments. Our survey highlighted both consistent and inconsistent behavioral findings. These findings underscore the need to examine multiple processes within individuals to identify unique behavioral profiles associated with obesity. Such individual characterization will inform future studies on the neurobiology of obesity as well as the design of effective interventions that are individually tailored.

Is the habit system altered in individuals with obesity? A systematic review

Habit-like eating behavior is repeatedly pointed to as a key cognitive mechanism contributing to the emergence and maintenance of obesity. Here, we conducted a systematic review of the literature to assess the existent behavioral evidence for the Habit Hypothesis for Overeating (HHO) which states that obesity is the consequence of an imbalance between the habit and goal-directed reward learning systems, leading to overconsumption of food. We found a total of 19 studies implementing a variety of experimental protocols (i.e., free operant paradigm, slips-of-action test, two-step task, Pavlovian-to-Instrumental paradigm, probabilistic learning task) and manipulations. Taken together, the studies on clinical (binge eating disorder) and non-clinical individuals with overweight or obesity do not support the HHO conclusively. While the scientific literature on HHO is still in its infancy, the heterogeneity of the extant studies makes it difficult to evaluate the degree of convergence of these findings. Uncovering the role of reward learning systems in eating behaviors might have a transformative impact on public health.