A Reward-Based Framework of Perceived Control

Sensitivity to the Instrumental Value of Choice Increases Across Development

Across development, people tend to demonstrate a preference for contexts in which they have the opportunity to make choices. However, it is not clear how children, adolescents, and adults learn to calibrate this preference based on the costs and benefits of agentic choice. Here, in both a primary, in-person, reinforcement-learning experiment (N = 92; age range = 10-25 years) and a preregistered online replication study (N = 150; age range = 8-25 years), we found that participants overvalued agentic choice but also calibrated their agency decisions to the reward structure of the environment, increasingly selecting agentic choice when choice had greater instrumental value. Regression analyses and computational modeling of participant choices revealed that participants' bias toward agentic choice-reflecting its intrinsic value-remained consistent across age, whereas sensitivity to the instrumental value of agentic choice increased from childhood to early adulthood.

My choice, my actions: self-determination, not instrumental value of outcomes enhances outcome monitoring during learning

Freedom of choice enhances our sense of agency. During goal-directed behavior, the freedom to choose between different response options increases the neural processing of positive and negative feedback, indicating enhanced outcome monitoring under conditions of high agency experience. However, it is unclear whether this enhancement is predominantly driven by an increased salience of self- compared to externally determined action outcomes or whether differences in the perceived instrumental value of outcomes contribute to outcome monitoring in goal-directed tasks. To test this, we recorded electroencephalography while participants performed a reinforcement learning task involving free choices, action-relevant forced choices, and action-irrelevant forced choices. We observed larger midfrontal theta power and N100 amplitudes for feedback following free choices compared with action-relevant and action-irrelevant forced choices. In addition, a Reward Positivity was only present for free but not forced choice outcomes. Crucially, our results indicate that enhanced outcome processing is not driven by the relevance of outcomes for future actions but rather stems from the association of outcomes with recent self-determined choice. Our findings highlight the pivotal role of self-determination in tracking the consequences of our actions and contribute to an understanding of the cognitive processes underlying the choice-induced facilitation in outcome monitoring.

Determinants for positive and negative experiences of interpersonal touch: context matters

The goal of the study was to determine which aspects of interpersonal touch interactions lead to a positive or negative experience. Previous research has focused primarily on physical characteristics. We suggest that this may not be sufficient to fully capture the complexity of the experience. Specifically, we examined how fulfilment of psychological needs influences touch experiences and how this relates to physical touch characteristics and situational factors.In two mixed-method studies, participants described their most positive and most negative interpersonal touch experience within a specific time frame. They reported fulfilment of nine needs, affect, intention, and reason for positivity/negativity, as well as the body part(s) touched, location, type of touch, interaction partner, and particular touch characteristics (e.g. humidity).Positive and negative touch experiences shared similar touch types, locations, and body parts touched, but differed in intended purpose and reasons. Overall, the valence of a touch experience could be predicted from fulfilment of relatedness, the interaction partner and initiator, and physical touch characteristics. Positive affect increased with need fulfilment, and negative affect decreased.The results highlight the importance of relatedness and reciprocity for the valence of touch, and emphasise the need to incorporate psychological needs in touch research.

Lived experiences: Growing up with a seriously mentally ill parent

INTRODUCTION

Individuals with serious mental illness often have persistent and disruptive symptoms. These can profoundly affect their children's lives, exposing them to adverse social and psychological conditions. Such conditions can result in traumatic lived experiences during childhood, which can carry over into adulthood, influencing their self-perceptions and shaping their attitudes toward themselves and society. To gain insights into this phenomenon, this study explored the lived experiences of adults who grew up with a parent with serious mental illness and their perceptions of their lives in adulthood.

DESIGN

This study used an interpretive phenomenological design.

METHODS

Participants were invited to voluntarily participate in the study through a call posted on social media. Semi-structured interviews were conducted with 30 adults (age range, 20-55 years) who grew up with a parent with serious mental illness. The interviews were recorded and transcribed, and inductive thematic analysis was used to identify main and overarching themes.

RESULTS

The overarching theme of transition from childhood survival to adulthood survival emerged and included four main themes: (1) a traumatic childhood, (2) perceived control, (3) resilience and general self-efficacy, and (4) adult quality of life. A traumatic childhood consisted of experiences of neglect and abuse, while participants used perceived control to achieve personal growth, self-care, and care of others. Resilience and general self-efficacy emerged during the transition to adulthood and helped participants further their social status and strengthen family bonds. Lastly, adult quality of life was described as being disturbed by feelings of loneliness and being burdensome, stemming from an inherent tendency to rely solely on themselves, leading to trust issues and mental health complications. Therefore, these adults found it difficult to reach out and get help or treatment for their concerns, as they initially did not want to appear dysfunctional or in need.

CONCLUSION

This study has illuminated the lived experiences of a specific, vulnerable population that has not been intentionally explored until now. To delve into these experiences, we employed a distinctive qualitative approach, merging the interpretive phenomenological perspective with an inductive thematic analysis. This allowed for rich insight with a relatively large group of participants and enabled an in-depth exploration within this methodological framework. Consequently, this study constitutes a notable contribution to the extant body of knowledge, exploring the intricacies of personal growth and its impact on participants' quality of life. It uncovers the essence of resilience and general self-efficacy, revealing how these elements intertwine with the negative results observed. However, the study findings emphasize the need for healthcare professionals, including nurses and other caregivers, to be mindful of the long-lasting effects of the adverse experiences of children of patients with serious mental illness. Prioritizing active clinical assessment and implementing tailored interventions to address such children's specific needs and difficulties across different developmental stages is imperative. Such comprehensive and targeted approaches are crucial in providing appropriate support and promoting the well-being of these individuals.

CLINICAL RELEVANCE

Enhanced clinical attention in holistic psychiatric care is crucial for individuals and their relatives, especially children. Comprehensive assessments of children and adults raised by seriously mentally ill parents can enable tailored and preventive interventions, positively impacting overall quality of life.

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Freedom of choice boosts midfrontal theta power during affective feedback processing of goal-directed actions

Sense of agency, the feeling of being in control of one's actions and their effects, is particularly relevant during goal-directed actions. During feedback learning, action effects provide information about the best course of action to reinforce positive and prevent negative outcomes. However, it is unclear whether agency experience selectively affects the processing of negative or positive feedback during the performance of goal-directed actions. As an important marker of feedback processing, we examined agency-related changes in midfrontal oscillatory activity in response to performance feedback using electroencephalography. Thirty-three participants completed a reinforcement learning task during which they received positive (monetary gain) or negative (monetary loss) feedback following item choices made either by themselves (free-choice) or by the computer (forced-choice). Independent of choice context, midfrontal theta activity was more enhanced for negative than positive feedback. In addition, free, compared to forced choices increased midfrontal theta power for both gain and loss feedback. These results indicate that freedom of choice in a motivationally salient learning task leads to a general enhancement in the processing of affective action outcomes. Our findings contribute to an understanding of the neuronal mechanisms underlying agency-related changes during action regulation and indicate midfrontal theta activity as a neurophysiological marker important for the monitoring of affective action outcomes, irrespective of feedback valence.

Perceived Neighborhood Safety and Depressive Symptoms: The Role of Perceived Neighborhood Cohesion and Perceived Control

Few have examined mechanisms explaining the link between perceived neighborhood unsafety, neighborhood social processes, and depressive symptoms for Black adolescents. The goal of this study was to examine the role of perceived control as a mechanism linking perceptions of neighborhood unsafety and depressive symptoms, and neighborhood cohesion as a protective factor. Participants were 412 Black adolescents living in a major Mid-Atlantic urban center in the United States (49% female, Mage = 15.80, SD = 0.36). Participants reported perceptions of neighborhood unsafety at grade 10, neighborhood cohesion at grade 10, perceived control at grades 10 and 11 and depressive symptoms in grades 10 and 12. High neighborhood unsafety was associated with low perceived control and in turn high depressive symptoms only when neighborhood cohesion was high. The results highlight the role of neighborhood unsafety and perceived control in the development of depressive symptom and the possible downsides of neighborhood social factors.

Modeling intentionality in the human brain

This paper is focusing on a rather neglected issue that concerns both aspects of philosophy and neurobiology in relation to the concept of intentionality. Intentionality is concerned with the 'directedness' or 'aboutness' of mental phenomena towards an object. Despite the fact that in philosophy both concepts of aboutness and directedness are conceptually identical with intentionality, a careful neuroscientific approach can demonstrate that these two phenomena represent two distinct conceptual and neurobiological aspects of intentionality with complementary functions. We described the interaction between a series of intentionality and pathogenetic psychobiological factors, the corresponding brain topography, and the resulting clinical manifestation and psychopathology. A permanent failure of intentionality dominates in psychosis, which includes an inappropriateness of the intentional object or connection, from the outset, or even from the prodromal phase of the disorder. Affective disorders may result from imprecise interoceptive prediction error signals, due to a confused identification of the intentional object. In suicidal patients there is an emotional intentionality failure, characterized by an absence of intentional object or a loss of conscious access to normal intentional objects. We may model an 'intentional system' as a higher order system, with a monitoring and regulatory role attributed to the brain and behavior. Also, we may consider mental disorders as the result of a radical disruption of intentionality, due to an inappropriateness or lack of the intentional object or due to an inappropriate connection in some points of the suggested brain pathways of intentionality.

Intrinsic motivation for choice varies with individual risk attitudes and the controllability of the environment

When deciding between options that do or do not lead to future choices, humans often choose to choose. We studied choice seeking by asking subjects to first decide between a choice opportunity or performing a computer-selected action, after which they either chose freely or performed the forced action. Subjects preferred choice when these options were equally rewarded, even deterministically, and traded extrinsic rewards for opportunities to choose. We explained individual variability in choice seeking using reinforcement learning models incorporating risk sensitivity and overvaluation of rewards obtained through choice. Model fits revealed that 28% of subjects were sensitive to the worst possible outcome associated with free choice, and this pessimism reduced their choice preference with increasing risk. Moreover, outcome overvaluation was necessary to explain patterns of individual choice preference across levels of risk. We also manipulated the degree to which subjects controlled stimulus outcomes. We found that degrading coherence between their actions and stimulus outcomes diminished choice preference following forced actions, although willingness to repeat selection of choice opportunities remained high. When subjects chose freely during these repeats, they were sensitive to rewards when actions were controllable but ignored outcomes-even positive ones-associated with reduced controllability. Our results show that preference for choice can be modulated by extrinsic reward properties including reward probability and risk as well as by controllability of the environment.

Transcranial direct-current stimulation enhances Pavlovian tendencies during intermittent loss of control

Introduction

Pavlovian bias is an innate motivational tendency to approach rewards and remain passive in the face of punishment. The relative reliance on Pavlovian valuation has been found to increase when the perceived control over environmental reinforcers is compromised, leading to behavior resembling learned helplessness (LH).

Methods

Sixty healthy young adults underwent a Go-NoGo reinforcement learning task and received anodal high-definition transcranial direct current stimulation (HD-tDCS) over the medial prefrontal/dorsal anterior cingulate cortex in our randomized, double-blind, sham- controlled study. Furthermore, we evaluated changes in cue-locked mid-frontal theta power derived from simultaneous electroencephalography (EEG). We hypothesized that active stimulation would reduce Pavlovian bias during manipulation of outcome controllability, and the effect would be accompanied by stronger mid-frontal theta activity, representing arbitration between choice strategies in favor of instrumental relative to Pavlovian valuation.

Results

We found a progressive decrease in Pavlovian bias during and after loss of control over feedback. Active HD-tDCS counteracted this effect while not affecting the mid-frontal theta signal.

Discussion

The results were at odds with our hypotheses but also with previous findings reporting LH-like patterns during and after loss of control without brain stimulation. The discrepancy may be related to different protocols for the controllability manipulation. We argue that the subjective evaluation of task controllability is crucial in mediating the balance between Pavlovian and instrumental valuation during reinforcement learning and that the medial prefrontal/dorsal anterior cingulate cortex is a key region in this respect. These findings have implications for understanding the behavioral and neural underpinnings of LH in humans.

Sensitive caregiving and reward responsivity: A novel mechanism linking parenting and executive functions development in early childhood

Sensitive caregiving is an essential aspect of positive parenting that influences executive functions development, but the mechanisms underlying this association are less clear. Using data from the Family Life Project, a large prospective longitudinal sample of 1292 families residing in rural, predominately low-income communities, the current study examined whether sensitive caregiving impacts executive functions development by shaping behavioral reward processing systems in early postnatal life. Results indicated that higher levels of sensitive caregiving during infancy were associated with heightened reward responsivity at age 4, which in turn predicted superior executive functions ability at age 5. Notably, children's reward responsivity partially mediated the relationship between sensitive caregiving in infancy and executive functions ability at school entry. These findings add to prior work on early experience and children's executive functions and highlight caregiver scaffolding of developing reward processing systems as a potential foundational mechanism for supporting adaptive behavior and self-regulation across the lifespan.

Adaption and validation of the Perceived Control of Internal States Scale (PCOISS) in Chinese adults: a cross-sectional study

BACKGROUND

Perceived control of internal states is important for disease prevention, stress buffering and life adaptability. However, there is no psychometric scale to measure control beliefs over internal states in China. This study aimed to adapt and validate the Perceived Control of Internal States Scale (PCOISS) in a large sample of Chinese adults.

METHODS

Data was collected through a big project, in which a cross-sectional online survey was conducted nationwide in China using a powerful Chinese online survey platform named WenJuanXing ( https://www.wjx.cn/ ). We translated the PCOISS into Chinese (C-PCOISS) with the forward-backward translation procedure. For the first time of the survey, a sample of 2709 participants (Sample 1) was valid for final analysis. Sample 1 was split into two datasets for principal component analysis (PCA) (n_A = 1355) and confirmatory factor analyses (CFA) (n_B = 1354) to determine potential factor structure. The scale's validity (i.e., discriminant validity, convergent validity, criterion validity) and internal consistency reliability were evaluated. Among the 1354 respondents (n_B), 761 (n_C = 761) participated in the follow-up second wave of the survey to assess a cross-sectional test-retest reliability.

RESULTS

The C-PCOISS retained 14 items. PCA yielded a three-factor model which was supported with the best fit indices in CFA. The C-PCOISS had satisfactory internal consistency with Cronbach's alpha coefficients of 0.86, 0.78 and 0.72 for three subscales, respectively. The scale also showed adequate test-retest reliability (Pearson correlations coefficient of 0.64, 0.62 and 0.54 with p < 0.001 for three subscales, respectively). Three factors of the C-PCOISS were positively associated with positive affect, and negatively associated with negative affect, depression, compulsion-anxiety and perceived stress.

CONCLUSIONS

The C-PCOISS is reliable and valid for measuring control beliefs over internal states in Chinese adults.

Trauma-Informed Design of Supported Housing: A Scoping Review through the Lens of Neuroscience

There is growing recognition of the importance of the design of the built environment in supporting mental health. In this context, trauma-informed design has emerged as a new field of practice targeting the design of the built environment to support wellbeing and ameliorate the physical, psychological and emotional impacts of trauma and related pathologies such as Post Traumatic Stress Disorder (PTSD). With high levels of prevalence of PTSD among people escaping homelessness and domestic violence, a priority area is the identification and application of evidence-based design solutions for trauma-informed supported housing. This study sought to examine the scope of existing evidence on the relationship between trauma, housing and design and the correlation of this evidence with trauma-informed design principles, and to identify gaps and opportunities for future research. In response to the commonly articulated limitations of the evidence-base in built environment design research, we combined a scoping review of literature on trauma, housing and design with insights from neuroscience to focus and extend understanding of the opportunities of trauma-informed design. We found that while limited in scope, there is strong alignment between existing evidence and the principles of trauma-informed design. We also identify three areas of future research related to the key domains of safety and security; control; and enriched environments.

Learned helplessness: expanding on a goal-directed perspective

This commentary reviews a novel model of learned helplessness proposed by Boddez et al. in this issue of Cognition and Emotion. Combining operant and goal-directed perspectives, Boddez et al. suggest that helplessness stems from a lack of reinforcement when striving toward a goal, with the degree of generalisation dependent on subjective perceptions of goal similarity. We begin by reviewing the theoretical model, describe possible expansions from a cognitive perspective, and discuss several considerations. We finish with a brief discussion of possible directions for future work.

More engagement in inefficient avoidance through partial reinforcement

BACKGROUND AND OBJECTIVES

In anxiety-related disorders, excessive avoidance often coexists with an impaired sense of control over external threats. In contrast, lab studies have shown that avoidance responding increase with higher objective controllability over threat, accompanied with more confidence in the effectiveness of the avoidance response. One reason for this divergence could be that those lab studies are overly simplistic with a single, avoidable threat.

METHODS

We conducted an experiment that additionally included a completely uncontrollable threat, and we manipulated the reinforcement rate of the avoidance response to the (semi-)controllable threat (75% versus 100%).

RESULTS

The 100% group showed increased avoidance to the controllable threat and decreased avoidance to the unavoidable threat over learning. Interestingly, compared to the 100% group, the 75% group displayed less confidence in their avoidance to the controllable threat and they avoided the uncontrollable threat more often.

LIMITATIONS

Only two reinforcement rates of effective avoidance were included, which may limit the generalizability of the current findings. Perceived control was not directly measured.

CONCLUSIONS

Lower reinforcement rates create ambiguity between effective and ineffective situations of avoidance, which engenders generalization of unpredictability from effective to ineffective situation, thereby driving up ineffective avoidance rates. Partially reinforced effective avoidance responses and elevated ineffective avoidance responses together lead to more exposure to uncontrollable threat, weakening the sense of control over the threat, which could further increase avoidance behaviors. Controllability is often overlooked in avoidance research but can be key to understanding the development of maladaptive avoidance behaviors.

The effect of obstructed action efficacy on reward-based decision-making in healthy adolescents: a novel functional MRI task to assay frustration

Frustration is common in adolescence and often interferes with executive functioning, particularly reward-based decision-making, and yet very little is known about how incidental frustrating events (independent of task-based feedback) disrupt the neural circuitry of reward processing in this important age group. While undergoing functional magnetic resonance imaging (fMRI), 45 healthy adolescents played a card game in which they had to guess between two options to earn points, in low- and high-stake conditions. Functioning of button presses through which they made decisions was intermittently blocked, thereby increasing frustration potential. Neural deactivation of the precuneus, a Default Mode Network region, was observed during obstructed action blocks across stake conditions, but less so on high- relative to low-stake trials. Moreover, less deactivation in goal-directed reward processing regions (i.e., caudate), frontoparietal "task control" regions, and interoceptive processing regions (i.e., somatosensory cortex, thalamus) were observed on high-stake relative to low-stake trials. These findings are consistent with less disruption of goal-directed reward seeking during blocked action efficacy in high-stake conditions among healthy adolescents. These results provide a roadmap of neural systems critical to the processing of frustrating events during reward-based decision-making in youths and could help to characterize how frustration regulation is altered in a range of pediatric psychopathologies.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

Stress-sensitive inference of task controllability

Estimating the controllability of the environment enables agents to better predict upcoming events and decide when to engage controlled action selection. How does the human brain estimate controllability? Trial-by-trial analysis of choices, decision times and neural activity in an explore-and-predict task demonstrate that humans solve this problem by comparing the predictions of an 'actor' model with those of a reduced 'spectator' model of their environment. Neural blood oxygen level-dependent responses within striatal and medial prefrontal areas tracked the instantaneous difference in the prediction errors generated by these two statistical learning models. Blood oxygen level-dependent activity in the posterior cingulate, temporoparietal and prefrontal cortices covaried with changes in estimated controllability. Exposure to inescapable stressors biased controllability estimates downward and increased reliance on the spectator model in an anxiety-dependent fashion. Taken together, these findings provide a mechanistic account of controllability inference and its distortion by stress exposure.

Relative deprivation and social anxiety among Chinese migrant children: Testing a moderated mediation model of perceived control and belief in a just world

To examine the relationship between relative deprivation and social anxiety, which affects mental health, and investigate the mediating role of perceived control and the moderating role of belief in a just world (BJW) in an understudied population in Asia, we surveyed 1573 rural-to-urban migrant children (48% female; M_age = 12.3, SD = 1.7) in southeast China. Relative deprivation was positively correlated with social anxiety; perceived control partially mediated this connection. Moreover, BJW moderated the indirect effect, which was stronger for male migrant children with lower levels of BJW. The limitations and practical implications of this study are discussed.

Neurocomputational mechanism of controllability inference under a multi-agent setting

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.

How we perceive controllability over an outcome if there are multiple other agents who can simultaneously influence that outcome? Previous ‘single-agent’ studies showed that an agents’ inferred controllability depends on contingency between its own action and following outcome and this inference involves striatum. Here, we show that in a multi-agent setting, other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which was explained as a biased Bayesian inference. Notably, bias in inference played an adaptive role under volatile controllability and was associated with a perception of guilt. Striatum and temporoparietal junction (TPJ) were involved in this multi-agent Bayesian controllability inference and this controllability information was leveraged to increase motivated behavior in the vmPFC. Our results first provide a neurocomputational mechanism of multi-agent controllability inference.

Reward anticipation buffers neuroendocrine and cardiovascular responses to acute psychosocial stress in healthy young adults

Research over the last 10 years suggests that the brain's reward system plays a crucial role in stress resilience. Notably, reward processing includes both an anticipatory (cue-triggered "wanting") phase and a consummatory ("liking") phase. However, previous studies manipulated rewards via direct reward administration, which makes it difficult to isolate the buffering effect of anticipating the reward stimulus. In the current study, we designed a paradigm to manipulate participants into generating reward anticipation or not and investigated whether reward anticipation can buffer psychological, neuroendocrine, and cardiovascular responses to psychosocial stress. A sample of 78 healthy young adults underwent the Trier Social Stress Test or placebo-Trier Social Stress Test after a reward anticipation task. Results showed that reward anticipation relieved subjective stress feelings, as well as the overall cortisol secretion and the increased heart rate induced by psychosocial stress. Taken together, these findings expanded our understanding of the role the reward system plays in stress resilience, and the possible psychological mechanism of the buffering effect for future stress study was also discussed.HIGHLIGHTSReward processing includes both an anticipatory and consummatory phasesThe buffering effect of anticipating the reward stimulus requires elucidationWe examined if said anticipation buffers varied responses to psychosocial stressReward anticipation relieved subjective stress, cortisol secretion, and heart rateWe clarified the role of the reward system in stress resilience.

The Interplay Between Affective Processing and Sense of Agency During Action Regulation: A Review

Sense of agency is the feeling of being in control of one's actions and their perceivable effects. Most previous research identified cognitive or sensory determinants of agency experience. However, it has been proposed that sense of agency is also bound to the processing of affective information. For example, during goal-directed actions or instrumental learning we often rely on positive feedback (e.g., rewards) or negative feedback (e.g., error messages) to determine our level of control over the current task. Nevertheless, we still lack a scientific model which adequately explains the relation between affective processing and sense of agency. In this article, we review current empirical findings on how affective information modulates agency experience, and, conversely, how sense of agency changes the processing of affective action outcomes. Furthermore, we discuss in how far agency-related changes in affective processing might influence the ability to enact cognitive control and action regulation during goal-directed behavior. A preliminary model is presented for describing the interplay between sense of agency, affective processing, and action regulation. We propose that affective processing could play a role in mediating the influence between subjective sense of agency and the objective ability to regulate one's behavior. Thus, determining the interrelation between affective processing and sense of agency will help us to understand the potential mechanistic basis of agency experience, as well as its functional significance for goal-directed behavior.

Transcranial Direct Current Stimulation above the Medial Prefrontal Cortex Facilitates Decision-Making following Periods of Low Outcome Controllability

Recent studies suggest that choice behavior in reinforcement learning tasks is shaped by the level of outcome controllability. In particular, Pavlovian bias (PB) seems to be enhanced under low levels of control, manifesting in approach tendencies toward rewards and response inhibition when facing potential losses. The medial prefrontal cortex (mPFC) has been implicated both in evaluating outcome controllability and in the recruitment of cognitive control (CC) to suppress maladaptive PB during reinforcement learning. The current study tested whether high-definition transcranial direct current stimulation (HD-tDCS) above the mPFC of healthy humans can influence PB, and counteract the previously documented, deleterious behavioral effects of low outcome controllability on decision-making. In a preregistered, between-group, double-blind study (N = 103 adults, both sexes), we tested the interaction between controllability and HD-tDCS on parameters of choice behavior in a Go/NoGo task. Relative to sham stimulation, HD-tDCS resulted in more robust performance improvement following reduced control, an effect that was more pronounced in appetitive trials. In addition, we found evidence for weaker PB when HD-tDCS was administered during low controllability over outcomes. Computational modeling revealed that parameter estimates of learning rate and choice randomness were modulated by controllability, HD-tDCS and their interaction. Overall, these results highlight the potential of our HD-tDCS protocol for interfering with choice arbitration under low levels of control, resulting in more adaptive behavior.

Decomposing the motivation to exert mental effort

Achieving most goals demands cognitive control, yet people vary widely in their success at meeting these demands. While motivation is known to be fundamental to determining these successes, what determines one's motivation to perform a given task remains poorly understood. Here, we describe recent efforts towards addressing this question using the Expected Value of Control model, which simulates the process by which people weigh the costs and benefits of exerting mental effort. By functionally decomposing this cost-benefit analysis, this model has been used to fill gaps in our understanding of the mechanisms of mental effort and to generate novel predictions about the sources of variability in real-world performance. We discuss the opportunities the model provides for formalizing hypotheses about why people vary in their motivation to perform tasks, as well as for understanding limitations in our ability to test these hypotheses based on a given measure of performance.

Early life stress moderated the influence of reward anticipation on acute psychosocial stress responses

Recent studies suggest that reward anticipation decreases individuals' acute stress responses. However, individuals who have experienced early life stress (ELS) may have a blunted capacity for reward anticipation, which reduces its buffering effect on psychosocial stress responses. To investigate this phenomenon, 66 young adults completed the Trier Social Stress Test following a reward anticipation task, and their ELS levels were measured using the Childhood Trauma Questionnaire (CTQ). Meanwhile, the current study collected biological and psychological measures of stress by analysing cortisol levels, heart rates, heart rate variability (HRV) as well as subjective stress levels, in response to the Trier Social Stress test. Results showed that reward anticipation successfully decreased acute stress responses in general, and it also improved participants' HRV. However, this effect was more evident in individuals with low ELS than those with high ELS. These findings help us deepen understanding of the role of reward anticipation in fostering resilience under stress and the potentially important implications for individuals who have been exposed to ELS are also discussed.

Anticipating control over aversive stimuli is mediated by the medial prefrontal cortex: An fMRI study with healthy adults

The anticipation of control over aversive events in life is relevant for our mental health. Insights on the underlying neural mechanisms remain limited. We developed a new functional magnetic resonance imaging (fMRI) task that uses auditory stimuli to explore the neural correlates of (1) the anticipation of control over aversion and (2) the processing of aversion. In a sample of 25 healthy adults, we observed increased neural activation in the medial prefrontal cortex (ventromedial prefrontal cortex and rostral anterior cingulate cortex), other brain areas relevant for reward anticipation (ventral striatum, brainstem [ventral tegmental area], midcingulate cortex), and the posterior cingulate cortex when they anticipated control over aversion compared with anticipating no control (1). The processing of aversive sounds compared to neutral sounds (2) was associated with increased neural activation in the bilateral posterior insula. Our findings provide evidence for the important role of medial prefrontal regions in control anticipation and highlight the relevance of conceiving the neural mechanisms involved within a reward-based framework.

The Protective Effects of Perceived Control During Repeated Exposure to Aversive Stimuli

The ability to perceive and exercise control is a major contributor to our mental and physical wellbeing. When faced with uncontrollable aversive stimuli, organisms develop heightened anxiety and become unwilling to exert effort to avoid the stimuli. In contrast, when faced with controllable aversive stimuli, organisms demonstrate behavioral vigor via avoidance attempts toward trying to seek and exercise control over the environment. As such, controllability confers protective effects against reduced avoidance motivation trigged by aversive environments. These observations beg the question of whether controllability can be potent enough to reverse passivity following repeated exposure to uncontrollable aversive stimuli and how this protective effect is encoded neurally. Human participants performed a Control in Aversive Domain (CAD) task where they were first subjected to a series of repeated uncontrollable aversive stimuli (i.e., aversive tones) across several contexts that were followed by a series of controllable aversive stimuli in a novel context. Faced with persistent uncontrollability, participants significantly reduced their avoidance attempts over time and biased toward giving up. However, the subsequent presence of controllability rescued participants' avoidance behavior. Strikingly, participants who responded more strongly to the protective effects of control also had greater ventromedial prefrontal cortical (vmPFC) activation-a region previously observed to be associated with encoding the subjective value of control. Taken together, these findings highlighted the protective effect conferred by perceived control against passivity and offered insights into the potential role of the vmPFC in controllable environments, with implications for understanding the beneficial influence of perceived control on adaptive behavior.

A Research Domain Criteria Approach to Gambling Disorder and Behavioral Addictions: Decision-Making, Response Inhibition, and the Role of Cannabidiol

Gambling Disorder (GD) has been recently re-classified in the DSM-5 under the "substance-related and addictive disorders," in light of its genetic, endophenotypic, and phenotypic resemblances to substance dependence. Diminished control is a core defining concept of psychoactive substance dependence or addiction and has given rise to the concept of "behavioral" addictions, which are syndromes analogous to substance addiction, but with a behavioral focus other than ingestion of a psychoactive substance. The main symptom clusters are represented by loss of control, craving/withdrawal, and neglect of other areas of life, whereas in a Research Domain Criteria (RDoC) perspective, GD patients exhibit deficits in the domain of "Positive valence systems," particularly in the "Approach motivation" and "Reward learning" constructs, as well as in the "Cognitive systems," primarily in the "Cognitive control" construct. In the Addictions Neuroclinical Assessment (ANA), three relevant domains for addictions emerge: "Incentive salience," "Negative Emotionality," and "Executive Function." The endocannabinoid system (ECS) may largely modulate these circuits, presenting a promising pharmaceutical avenue for treating addictions. Up to now, research on cannabidiol has shown some efficacy in Attention Deficit/Hyperactivity Disorder (ADHD), whereas in behavioral addictions its role has not been fully elucidated, as well as its precise action on RDoC domains. Herein, we review available evidence on RDoC domains affected in GD and behavioral addictions and summarize insights on the use of cannabidiol in those disorders and its potential mechanisms of action on reward, decisional, and sensorimotor processes.

Blunted neural effects of perceived control on reward feedback in major depressive disorder

BACKGROUND

Blunted reward processing has emerged as an endophenotype of major depressive disorder (MDD), but mechanistic understanding for this deficit remains elusive. The current event-related potential study examined whether this aberration is driven by the blunted effect of perceived control on reward processing.

METHODS

We adapted a well-validated gambling task in which perceived control was exercised by choice in 29 individuals with current MDD and 31 healthy controls. We examined the reward positivity in response to personally chosen versus passively received rewards.

RESULTS

We found that MDD patients relative to healthy controls exhibited a blunted reward positivity when rewards were delivered following voluntary choices but not when they were delivered following passive choices. This pattern was not observed during the relatively late stage, as indexed by the P300, of feedback processing.

LIMITATION

The current findings may be confounded with medication and anxiety.

CONCLUSIONS

These findings suggest that deficient reward processing in MDD is attributable to the deficiency in boosting reward responsivity by perceived control exercised by choice.

Linguistic and Psychometric Validation of the Chinese Version of the Control Attitudes Scale-Revised in Patients With Chronic Heart Failure

BACKGROUND

The concept of perceived control reflects the belief that one has resources needed to cope with negative events and the ability to positively influence consequences of those negative events. In patients with heart failure, perceived control is associated with a variety of health outcomes. Perceived control is commonly measured using the Control Attitudes Scale-Revised (CASR). There is no Chinese version of the CAS-R (CCAS-R).

OBJECTIVE

The purpose of this article was to perform linguistic validation and psychometric evaluation of the CCAS-R.

METHODS

The CAS-R was translated into Chinese according to Brislin's model. Then, a multicenter observational study was performed. Floor and ceiling effects, internal consistency, structural validity, and hypothesis testing were all assessed for psychometric validation of the CCAS-R.

RESULTS

A total of 227 patients with chronic heart failure were included. There were no ceiling or floor effects detected. Cronbach α was 0.94, indicating a high reliability. The results of the confirmatory factor analysis showed that the 1-factor structure as proposed by the original CAS-R fits the data well. The results of the principal component analysis suggested that the 1-factor structure was optimal as well, accounting for 71.6% of the total variance. The a priori hypothesis was supported by a statistically significant correlation between the CCAS-R and 3 theoretically related variables.

CONCLUSION

We developed a semantically equivalent version of the CAS-R in Chinese. The evaluation of the instruments' psychometric properties demonstrated that the CCAS-R has good reliability and validity for use in Chinese patients with chronic heart failure.

Functional heterogeneity of perceived control in feedback processing

Perceived control is a fundamental psychological function that can either boost positive affect or buffer negative affect. The current study addressed the electrophysiological correlates underlying perceived control, as exercised by choice, in the processing of feedback valence. Thirty-six participants performed an EEG choice task during which they received positive or negative feedback following choices made either by themselves or by a computer. Perceived control resulted in an enhanced reward positivity for positive feedback but increased theta power for negative feedback. Further, perceived control led to greater feedback P3 amplitude and delta power, regardless of feedback valence. These results suggest functional heterogeneity of perceived control in feedback processing as diverse as magnifying the reward signal, enhancing the need for control and increasing the motivational salience of outcome irrespective of valence.

Internal control beliefs shape positive affect and associated neural dynamics during outcome valuation

Experiencing events as controllable is essential for human well-being. Based on classic psychological theory, we test how internal control beliefs impact the affective valuation of task outcomes, neural dynamics and ensuing behavioral preferences. In three consecutive studies we show that dynamics in positive affect increase, with a qualitative shift towards self-evaluative pride, when agents believe they caused a given outcome. We demonstrate that these outcomes engage brain networks processing self-referential information in the cortical midline. Here, activity in the ventromedial prefrontal cortex tracks outcome valence regarding both success as well as internal control, and covaries with positive affect in response to outcomes. These affective dynamics also relate to increased functional coupling between the ventral striatum and cortical midline structures. Finally, we show that pride predicts preferences for control, even at monetary costs. Our investigations extend recent models of positive affect and well-being, and emphasize that control beliefs drive intrinsic motivation.

Human affect is shaped in part by whether desirable states are achieved under personal control. Here, the authors show that control beliefs affect happiness and pride, and how those effects relate to neural responses in the prefrontal cortex and behavioral preferences for exerting control.

Anxiety, avoidance, and sequential evaluation

Anxiety disorders are characterized by a range of aberrations in the processing of and response to threat, but there is little clarity what core pathogenesis might underlie these symptoms. Here we propose that a particular set of unrealistically pessimistic assumptions can distort an agent's behavior and underlie a host of seemingly disparate anxiety symptoms. We formalize this hypothesis in a decision theoretic analysis of maladaptive avoidance and a reinforcement learning model, which shows how a localized bias in beliefs can formally explain a range of phenomena related to anxiety. The core observation, implicit in standard decision theoretic accounts of sequential evaluation, is that the potential for avoidance should be protective: if danger can be avoided later, it poses less threat now. We show how a violation of this assumption - via a pessimistic, false belief that later avoidance will be unsuccessful - leads to a characteristic, excessive propagation of fear and avoidance to situations far antecedent of threat. This single deviation can explain a range of features of anxious behavior, including exaggerated threat appraisals, fear generalization, and persistent avoidance. Simulations of the model reproduce laboratory demonstrations of abnormal decision making in anxiety, including in situations of approach-avoid conflict and planning to avoid losses. The model also ties together a number of other seemingly disjoint phenomena in anxious disorders. For instance, learning under the pessimistic bias captures a hypothesis about the role of anxiety in the later development of depression. The bias itself offers a new formalization of classic insights from the psychiatric literature about the central role of maladaptive beliefs about control and self-efficacy in anxiety. This perspective also extends previous computational accounts of beliefs about control in mood disorders, which neglected the sequential aspects of choice.

Intermittent Absence of Control during Reinforcement Learning Interferes with Pavlovian Bias in Action Selection

The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss or to rely more on slowly accumulated stimulus-action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies. This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; "yoking") would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands. We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.

Corticostriatal Circuits Encode the Subjective Value of Perceived Control

The ability to perceive and exercise control over an outcome is both desirable and beneficial to our well-being. It has been shown that animals and humans alike exhibit behavioral bias towards seeking control and that such bias recruits the ventromedial prefrontal cortex (vmPFC) and striatum. Yet, this bias remains to be quantitatively captured and studied neurally. Here, we employed a behavioral task to measure the preference for control and characterize its neural underpinnings. Participants made a series of binary choices between having control and no-control over a game for monetary reward. The mere presence of the control option evoked activity in the ventral striatum. Importantly, we manipulated the expected value (EV) of each choice pair to extract the pairing where participants were equally likely to choose either option. The difference in EV between the options at this point of equivalence was inferred as the subjective value of control. Strikingly, perceiving control inflated the reward value of the associated option by 30% and this value inflation was tracked by the vmPFC. Altogether, these results capture the subjective value of perceived control inherent in decision making and highlight the role of corticostriatal circuitry in the perception of control.