Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

The affective gradient hypothesis: an affect-centered account of motivated behavior

Everyone agrees that feelings and actions are intertwined, but cannot agree how. According to dominant models, actions are directed by estimates of value and these values shape or are shaped by affect. I propose instead that affect is the only form of value that drives actions. Our mind constantly represents potential future states and how they would make us feel. These states collectively form a gradient reflecting feelings we could experience depending on actions we take. Motivated behavior reflects the process of traversing this affective gradient, towards desirable states and away from undesirable ones. This affective gradient hypothesis solves the puzzle of where values and goals come from, and offers a parsimonious account of apparent conflicts between emotion and cognition.

Common neural choice signals can emerge artefactually amid multiple distinct value signals

Previous work has identified characteristic neural signatures of value-based decision-making, including neural dynamics that closely resemble the ramping evidence accumulation process believed to underpin choice. Here we test whether these signatures of the choice process can be temporally dissociated from additional, choice-'independent' value signals. Indeed, EEG activity during value-based choice revealed distinct spatiotemporal clusters, with a stimulus-locked cluster reflecting affective reactions to choice sets and a response-locked cluster reflecting choice difficulty. Surprisingly, 'neither' of these clusters met the criteria for an evidence accumulation signal. Instead, we found that stimulus-locked activity can 'mimic' an evidence accumulation process when aligned to the response. Re-analysing four previous studies, including three perceptual decision-making studies, we show that response-locked signatures of evidence accumulation disappear when stimulus-locked and response-locked activity are modelled jointly. Collectively, our findings show that neural signatures of value can reflect choice-independent processes and look deceptively like evidence accumulation.

Mutual inclusivity improves decision-making by smoothing out choice's competitive edge

Decisions form a central bottleneck to most tasks, one that people often experience as costly. Past work proposes mitigating those costs by lowering one's threshold for deciding. Here, we test an alternative solution, one that targets the basis for most choice costs: that choosing one option sacrifices others (mutual exclusivity). Across 5 studies (N = 462), we test whether this tension can be relieved by framing choices as inclusive (allowing selection of more than one option, as in buffets). We find that inclusivity makes choices more efficient, by selectively reducing competition between potential responses as participants accumulate information for each of their options. Inclusivity also made participants feel less conflicted, especially when they couldn't decide which good option to keep or which bad option to get rid of. These inclusivity benefits were also distinguishable from the effects of manipulating decision threshold (increased urgency), which improved choices but not experiences thereof.

Reactive and proactive control processes in voluntary task choice

Deciding which task to perform when multiple tasks are available can be influenced by external influences in the environment. In the present study, we demonstrate that such external biases on task-choice behavior reflect reactive control adjustments instead of a failure in control to internally select a task goal. Specifically, in two experiments we delayed the onset of one of two task stimuli by a short (50 ms), medium (300 ms), or long (1,000 ms) stimulus-onset asynchrony (SOA) within blocks while also varying the relative frequencies of short versus long SOAs across blocks (i.e., short SOA frequent vs. long SOA frequent). Participants' task choices were increasingly biased towards selecting the task associated with the first stimulus with increasing SOAs. Critically, both experiments also revealed that the short-to-medium SOA bias was larger in blocks with more frequent long SOAs when participants had limited time to prepare for an upcoming trial. When time to select an upcoming task was extended in Experiment 2, this interaction was not significant, suggesting that the extent to which people rely on reactive control adjustments is additionally modulated by proactive control processes. Thus, the present findings also suggest that voluntary task choices are jointly guided by both proactive and reactive processes, which are likely to adjust the relative activation of different task goals in working memory.

Multiple timescales of temporal context in risky choice: Behavioral identification and relationships to physiological arousal

Context-dependence is fundamental to risky monetary decision-making. A growing body of evidence suggests that temporal context, or recent events, alters risk-taking at a minimum of three timescales: immediate (e.g. trial-by-trial), neighborhood (e.g. a group of consecutive trials), and global (e.g. task-level). To examine context effects, we created a novel monetary choice set with intentional temporal structure in which option values shifted between multiple levels of value magnitude ("contexts") several times over the course of the task. This structure allowed us to examine whether effects of each timescale were simultaneously present in risky choice behavior and the potential mechanistic role of arousal, an established correlate of risk-taking, in context-dependency. We found that risk-taking was sensitive to immediate, neighborhood, and global timescales: risk-taking decreased following large (vs. small) outcome amounts, increased following large positive (but not negative) shifts in context, and increased when cumulative earnings exceeded expectations. We quantified arousal with skin conductance responses, which were related to the global timescale, increasing with cumulative earnings, suggesting that physiological arousal captures a task-level assessment of performance. Our results both replicate and extend prior research by demonstrating that risky decision-making is consistently dynamic at multiple timescales and that the role of arousal in risk-taking extends to some, but not all timescales of context-dependence.

Reinforcement learning of adaptive control strategies

Humans can up- or downregulate the degree to which they rely on task information for goal-directed behaviour, a process often referred to as cognitive control. Adjustments in cognitive control are traditionally studied in response to experienced or expected task-rule conflict. However, recent theories suggest that people can also learn to adapt control settings through reinforcement. Across three preregistered task switching experiments (n = 415), we selectively rewarded correct performance on trials with either more (incongruent) or less (congruent) task-rule conflict. Results confirmed the hypothesis that people rewarded more on incongruent trials showed smaller task-rule congruency effects, thus optimally adapting their control settings to the reward scheme. Using drift diffusion modelling, we further show that this reinforcement of cognitive control may occur through conflict-dependent within-trial adjustments of response thresholds after conflict detection. Together, our findings suggest that, while people remain more efficient at learning stimulus-response associations through reinforcement, they can similarly learn cognitive control strategies through reinforcement.

Learning how to reason and deciding when to decide

Research on human reasoning has both popularized and struggled with the idea that intuitive and deliberate thoughts stem from two different systems, raising the question how people switch between them. Inspired by research on cognitive control and conflict monitoring, we argue that detecting the need for further thought relies on an intuitive, context-sensitive process that is learned in itself.

Task state representations in vmPFC mediate relevant and irrelevant value signals and their behavioral influence

The ventromedial prefrontal-cortex (vmPFC) is known to contain expected value signals that inform our choices. But expected values even for the same stimulus can differ by task. In this study, we asked how the brain flexibly switches between such value representations in a task-dependent manner. Thirty-five participants alternated between tasks in which either stimulus color or motion predicted rewards. We show that multivariate vmPFC signals contain a rich representation that includes the current task state or context (motion/color), the associated expected value, and crucially, the irrelevant value of the alternative context. We also find that irrelevant value representations in vmPFC compete with relevant value signals, interact with task-state representations and relate to behavioral signs of value competition. Our results shed light on vmPFC's role in decision making, bridging between its role in mapping observations onto the task states of a mental map, and computing expected values for multiple states.

Conscious cognitive effort in cognitive control

Cognitive effort is thought to be familiar in everyday life, ubiquitous across multiple variations of task and circumstance, and integral to cost/benefit computations that are themselves central to the proper functioning of cognitive control. In particular, cognitive effort is thought to be closely related to the assessment of cognitive control's costs. I argue here that the construct of cognitive effort, as it is deployed in cognitive psychology and neuroscience, is problematically unclear. The result is that talk of cognitive effort may paper over significant disagreement regarding the nature of cognitive effort, and its key functions for cognitive control. I highlight key points of disagreement, and several open questions regarding what causes cognitive effort, what cognitive effort represents, cognitive effort's relationship to action, and cognitive effort's relationship to consciousness. I also suggest that pluralism about cognitive effort-that cognitive effort may manifest as a range of intentional or non-intentional actions the function of which is to promote greater success at paradigmatic cognitive control tasks-may be a fruitful and irenic way to conceive of cognitive effort. Finally, I suggest that recent trends in work on cognitive control suggests that we might fruitfully conceive of cognitive effort as one key node in a complex network of mental value, and that studying this complex network may illuminate the nature of cognitive control, and the role of consciousness in cognitive control's proper functioning. This article is categorized under: Philosophy > Consciousness Philosophy > Psychological Capacities Neuroscience > Cognition.

Phantom controllers: Misspecified models create the false appearance of adaptive control during value-based choice

Decision scientists have grown increasingly interested in how people adaptively control their decision making. Researchers have demonstrated that parameters governing the accumulation of evidence towards a choice, such as the decision threshold, are shaped by information available prior to or in parallel with one's evaluation of an option set (e.g., recent outcomes or choice conflict). A recent account has taken a bold leap forward in this approach, suggesting that adjustments in decision parameters can be motivated by the value of the options under consideration. This motivated control account predicts that when faced with difficult choices (similarly valued options) under time pressure, people will adaptively lower their decision threshold to ensure that they make a choice in time. This account was supported by drift diffusion modeling of a deadlined choice task, demonstrating that decision thresholds decrease for difficult relative to easy choices. Here, we reanalyze the data from this experiment, and show that evidence for this novel account does not hold up to further scrutiny. Using a more systematic and comprehensive modeling approach, we show that this previously observed threshold adjustment disappears (or even reverses) under a more complete model of the data. Importantly, we further show how this and other apparent evidence for motivated control arises as an artifact of model (mis)specification, where one model's putatively controlled decision process (e.g., value-driven threshold adjustments) can mimic another model's stimulus-driven decision processes (e.g., accumulator competition or collapsing bounds). Collectively, this work reveals crucial insights and constraints in the pursuit of understanding how control guides decision-making, and when it doesn't.

Goals, usefulness and abstraction in value-based choice

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.