Choice-induced preference change in the free-choice paradigm: a critical methodological review

Do adolescents use choice to learn about their preferences? Development of value refinement and its associations with depressive symptoms in adolescence

Independent decision making requires forming stable estimates of one's preferences. We assessed whether adolescents learn about their preferences through choice deliberation and whether depressive symptoms disrupt this process. Adolescents aged 11-18 (N = 214; participated 2021-22; Female: 53.9%; White/Black/Asian/Mixed/Arab or Latin American: 26/21/19/9/8%) rated multiple activities, chose between pairs of activities and re-rated those activities. As expected, overall, participants uprated chosen and downrated unchosen activities (dz = .20). This value refinement through choice was not evident in younger participants but emerged across adolescence. Contrary to our predictions, depressive symptoms were associated with greater value refinement. Despite this, more depressed adolescents reported lower value certainty and choice confidence. The cognitive processes through which choice deliberation shapes preference develop over adolescence, and are disrupted in depression.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Characterizing Human Habits in the Lab

Habits pose a fundamental puzzle for those aiming to understand human behavior. They pervade our everyday lives and dominate some forms of psychopathology but are extremely hard to elicit in the lab. In this Registered Report, we developed novel experimental paradigms grounded in computational models, which suggest that habit strength should be proportional to the frequency of behavior and, in contrast to previous research, independent of value. Specifically, we manipulated how often participants performed responses in two tasks varying action repetition without, or separately from, variations in value. Moreover, we asked how this frequency-based habitization related to value-based operationalizations of habit and self-reported propensities for habitual behavior in real life. We find that choice frequency during training increases habit strength at test and that this form of habit shows little relation to value-based operationalizations of habit. Our findings empirically ground a novel perspective on the constituents of habits and suggest that habits may arise in the absence of external reinforcement. We further find no evidence for an overlap between different experimental approaches to measuring habits and no associations with self-reported real-life habits. Thus, our findings call for a rigorous reassessment of our understanding and measurement of human habitual behavior in the lab.

Neuromodulation of choice-induced preference changes: the tDCS study of cognitive dissonance

Introduction

Difficult choices between two equally attractive options result in a cognitive discrepancy between dissonant cognitions such as preferences and actions often followed by a sense of psychological discomfort known as cognitive dissonance. It can lead to changes in the desirability of options: the chosen option becomes more desirable, whereas the rejected option is devalued. Despite the ample experimental evidence to show this effect, the neural mechanisms and timing of such choice-induced preference changes are not fully understood.

Methods

In this study, we used transcranial direct current stimulation (tDCS) to modulate the activity of the posterior medial frontal cortex (pMFC), which has been associated with conflict monitoring and choice-induced preference changes in neuroimaging studies. Prior to a revised version of Brehm's free-choice paradigm, participants in two experiments underwent cathodal (inhibitory) or anodal (excitatory) tDCS of the pMFC compared to sham (placebo) stimulation prior to the choice phase.

Results

Our results showed that cathodal tDCS significantly decreased the choice-induced preference change relative to a sham, but only in direct comparisons of rejected options. No significant effect of anodal tDCS in comparison with sham was observed.

Discussion

This study replicates the general behavioral effect of cognitive dissonance and provide partial support for the theory of the pMFC contribution to choice-related cognitive dissonance and subsequent preference changes, with possible limitations of an under-sampling for the obtained effect size and an asymmetry in the inhibitory-excitatory effects of non-invasive tDCS.

Choice-induced tasting. Evaluating the effect of choice on children's acceptance of an unfamiliar vegetable

Children's vegetable intake is in general below recommendations. It is known that self-selection of vegetables out of multiple options can increase intake in children. However, it is not clear if this effect is driven by a pre-existing preference for the selected food, or purely by having a choice. Allowing children to choose could create a positive bias and endorse feelings of autonomy, which could increase acceptance of the food and promote intake. The aim of the present pre-registered study was to investigate the effect of choice in promoting the acceptance of an unfamiliar vegetable during a blind-choice task. Children aged 4-5 years old (n = 161) were randomly assigned to the choice or no-choice condition. Each child was presented with three closed cups, containing a vegetable. The children were unaware that the cups contained the same vegetable, a piece of raw celeriac. In the choice condition, the children were able to choose a cup themselves whereas in the no-choice condition, the experimenter made the decision. We hypothesized that children in the choice condition would show more acceptance of the vegetable compared to children in the no-choice condition and that higher levels of picky eating would lead to less vegetable acceptance. The results demonstrated that choice indeed significantly increased vegetable acceptance (4.7 ± 1.7 versus 4.0 ± 1.9 on a 6-point scale), irrespective of pickiness in eating. In addition, picky children showed less acceptance of the vegetable compared to non-picky children. The results of this study imply that choice is an important factor in promoting unfamiliar vegetable intake in young children. Including children in the decision process may be a practical strategy for educators and caregivers to increase children's vegetable intake.

"Choose it, and remember it": The impact of choice on destination memory

Destination memory can be defined as the capacity to remember to whom we transmit information. It is measured through the accuracy of retrieving the association between the information we transmit and the person to whom we transmit it. A destination memory procedure aims to emulate human interaction by sharing facts with celebrities (i.e., familiar faces) since we often communicate with people we know. However, the role of the choice about who we intend to transmit the information to has not been evaluated before. This paper investigated whether deciding with whom to share a piece of information benefits destination memory. We designed two experiments with different levels of cognitive load, increasing it from Experiment 1 to Experiment 2. The experiments included two conditions: the choice condition, in which participants chose from two options to whom they desired to share a fact, and the no-choice condition, in which participants simply shared facts with celebrities without the possibility of a choice. Experiment 1 suggested that a choice component did not affect destination memory. However, when in Experiment 2 we raised the cognitive load by increasing the number of stimuli, we found that selecting the recipient during the more challenging task provided an advantage in destination memory. This result is congruent with the explanation that the shift of the participants' attentional resources to the recipient, caused by the choice component, leads to a destination memory improvement. In sum, it seems that a choice component can improve destination memory only under demanding attentional conditions.

The interactive effect of external rewards and self-determined choice on memory

Both external motivational incentives (e.g., monetary reward) and internal motivational incentives (e.g., self-determined choice) have been found to promote memory, but much less is known about how these two types of incentives interact with each other to affect memory. The current study (N = 108) examined how performance-dependent monetary rewards affected the role of self-determined choice in memory performance, also known as the choice effect. Using a modified and better controlled version of the choice paradigm and manipulating levels of reward, we demonstrated an interactive effect between monetary reward and self-determined choice on 1-day delayed memory performance. Specifically, the choice effect on memory decreased when we introduced the performance-dependent external rewards. These results are discussed in terms of understanding how external and internal motivators interact to impact learning and memory.

Choice Reminder Modulates Choice-Induced Preference Change in Older Adults

OBJECTIVES

Choices not only reflect preference but also shape preference. The choice-induced preference change (CIPC) occurs when making a decision modifies people's attitudes about the options. When people rate a series of items and then must choose between 2 items rated as equally attractive, they later rate the unchosen item as less attractive than before. One explanation is that the choice and the equal preference for 2 options cause a psychological discomfort known as cognitive dissonance, which can be reduced by changing the preference. The current study aims to investigate the age-related differences in the CIPC effect, and how an explicit reminder of the previous choice modulates this effect.

METHODS

Using an artifact-controlled free-choice paradigm, with a sample of 79 younger and 76 older participants, we manipulated the choice reminder in 2 experiments.

RESULTS

We found that compared with young adults, older adults are less susceptible to CIPC when their previous choices were not explicitly reminded. After boosting the salience of choice-preference incongruency by reminding participants of their previous choices, older adults showed comparable CIPC as young adults.

DISCUSSION

Our results suggest that older adults tend to downweigh the information that leads to cognitive dissonance and use this strategy only when such information is relatively implicit. The diminished CIPC in older adults could be one of the emotional regulation strategies that older adults engage in to maintain positive emotional states when making difficult decisions.

Value certainty and choice confidence are multidimensional constructs that guide decision-making

The degree of certainty that decision-makers have about their evaluations of available choice alternatives and their confidence about selecting the subjectively best alternative are important factors that affect current and future value-based choices. Assessments of the alternatives in a given choice set are rarely unidimensional; their values are usually derived from a combination of multiple distinct attributes. For example, the taste, texture, quantity, and nutritional content of a snack food may all be considered when determining whether to consume it. We examined how certainty about the levels of individual attributes of an option relates to certainty about the overall value of that option as a whole and/or to confidence in having chosen the subjectively best available option. We found that certainty and confidence are derived from unequally weighted combinations of attribute certainties rather than simple, equal combinations of all sources of uncertainty. Attributes that matter more in determining choice outcomes also are weighted more in metacognitive evaluations of certainty or confidence. Moreover, we found that the process of deciding between two alternatives leads to refinements in both attribute estimations and the degree of certainty in those estimates. Attributes that are more important in determining choice outcomes are refined more during the decision process in terms of both estimates and certainty. Although certainty and confidence are typically treated as unidimensional, our results indicate that they, like value estimates, are subjective, multidimensional constructs.

Choice perseverance underlies pursuing a hard-to-get target in an avatar choice task

People sometimes persistently pursue hard-to-get targets. Why people pursue such targets is unclear. Here, we hypothesized that choice perseverance, which is the tendency to repeat the same choice independent of the obtained outcomes, leads individuals to repeatedly choose a hard-to-get target, which consequently increases their preference for the target. To investigate this hypothesis, we conducted an online experiment involving an avatar choice task in which the participants repeatedly selected one avatar, and the selected avatar expressed their valence reactions through facial expressions and voice. We defined “hard-to-get” and “easy-to-get” avatars by manipulating the outcome probability such that the hard-to-get avatars rarely provided a positive reaction when selected, while the easy-to-get avatars frequently did. We found that some participants repeatedly selected hard-to-get avatars (Pursuit group). Based on a simulation, we found that higher choice perseverance accounted for the pursuit of hard-to-get avatars and that the Pursuit group had significantly higher choice perseverance than the No-pursuit group. Model fitting to the choice data also supported that choice perseverance can account for the pursuit of hard-to-get avatars in the Pursuit group. Moreover, we found that although baseline attractiveness was comparable among all avatars used in the choice task, the attractiveness of the hard-to-get avatars was significantly increased only in the Pursuit group. Taken together, we conclude that people with high choice perseverance pursue hard-to-get targets, rendering such targets more attractive. The tolerance for negative outcomes might be an important factor for succeeding in our lives but sometimes triggers problematic behavior, such as stalking. The present findings may contribute to understanding the psychological mechanisms of passion and perseverance for one’s long-term goals, which are more general than the romantic context imitated in avatar choice.

Choosing to choose or not

To what degree do people prefer to choose for themselves and what drives this preference? Is it memory-based and results from a life-long association between choices and better outcomes, or is the process of choice itself reinforcing? In a new paradigm, across 6 experiments, participants experienced both ’Own Choice’ and ’Computer Picks’ conditions with identical outcomes before selecting which condition to re-experience in the final part of the experiment. Consistent with previous work, an overwhelming majority ( 83%) preferred own-choice. Several variations of the paradigm reveal that (1) Preference For Choice (PFC) is reduced when thinking about the task without actually choosing in it, (2) PFC is substantially reduced by choice-unrelated cognitive load, and (3) Preference For Choice is further diminished when selection is based on criteria other than one’s preferences. Across experiments, participants’ self-rated enjoyment predicted a significant portion of their PFC, while their perceived gains had little to no predictive value. If PFC stems solely from past reinforcement learning (i.e., memory) then neither performing another few scores of choices nor adding cognitive load to that sequence of choices would be expected to dramatically affect it. Hence, our findings suggest that a significant part of this preference stems from the process of choice itself, and that the experience it confers can itself be reinforcing. We discuss the implications of the proposed mechanism for PFC, which leads us to the prediction that PFC may be muted or even reversed under specific conditions and what this means for when the ‘opposite’ effect – sticking with the default – will occur.

Does choice change preferences? An incentivized test of the mere choice effect

Widespread evidence from psychology and neuroscience documents that previous choices unconditionally increase the later desirability of chosen objects, even if those choices were uninformative. This is problematic for economists who use choice data to estimate latent preferences, demand functions, and social welfare. The evidence on this mere choice effect, however, exhibits serious shortcomings which prevent evaluating its possible relevance for economics. In this paper, we present a novel, parsimonious experimental design to test for the economic validity of the mere choice effect addressing these shortcomings. Our design uses well-defined, monetary lotteries, all decisions are incentivized, and we effectively randomize participants' initial choices without relying on deception. Results from a large, pre-registered online experiment find no support for the mere choice effect. Our results challenge conventional wisdom outside economics. The mere choice effect does not seem to be a concern for economics, at least in the domain of decision making under risk.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10683-021-09728-5.

Trading mental effort for confidence in the metacognitive control of value-based decision-making

Why do we sometimes opt for actions or items that we do not value the most? Under current neurocomputational theories, such preference reversals are typically interpreted in terms of errors that arise from the unreliable signaling of value to brain decision systems. But, an alternative explanation is that people may change their mind because they are reassessing the value of alternative options while pondering the decision. So, why do we carefully ponder some decisions, but not others? In this work, we derive a computational model of the metacognitive control of decisions or MCD. In brief, we assume that fast and automatic processes first provide initial (and largely uncertain) representations of options' values, yielding prior estimates of decision difficulty. These uncertain value representations are then refined by deploying cognitive (e.g., attentional, mnesic) resources, the allocation of which is controlled by an effort-confidence tradeoff. Importantly, the anticipated benefit of allocating resources varies in a decision-by-decision manner according to the prior estimate of decision difficulty. The ensuing MCD model predicts response time, subjective feeling of effort, choice confidence, changes of mind, as well as choice-induced preference change and certainty gain. We test these predictions in a systematic manner, using a dedicated behavioral paradigm. Our results provide a quantitative link between mental effort, choice confidence, and preference reversals, which could inform interpretations of related neuroimaging findings.

Computational modeling of choice-induced preference change: A Reinforcement-Learning-based approach

The value learning process has been investigated using decision-making tasks with a correct answer specified by the external environment (externally guided decision-making, EDM). In EDM, people are required to adjust their choices based on feedback, and the learning process is generally explained by the reinforcement learning (RL) model. In addition to EDM, value is learned through internally guided decision-making (IDM), in which no correct answer defined by external circumstances is available, such as preference judgment. In IDM, it has been believed that the value of the chosen item is increased and that of the rejected item is decreased (choice-induced preference change; CIPC). An RL-based model called the choice-based learning (CBL) model had been proposed to describe CIPC, in which the values of chosen and/or rejected items are updated as if own choice were the correct answer. However, the validity of the CBL model has not been confirmed by fitting the model to IDM behavioral data. The present study aims to examine the CBL model in IDM. We conducted simulations, a preference judgment task for novel contour shapes, and applied computational model analyses to the behavioral data. The results showed that the CBL model with both the chosen and rejected value’s updated were a good fit for the IDM behavioral data compared to the other candidate models. Although previous studies using subjective preference ratings had repeatedly reported changes only in one of the values of either the chosen or rejected items, we demonstrated for the first time both items’ value changes were based solely on IDM choice behavioral data with computational model analyses.

Effective brain connectivity at rest is associated with choice-induced preference formation

Preferences can change as a consequence of making hard decisions whereby the value of chosen options increases and the value of rejected options decreases. Such choice-induced preference changes have been associated with brain areas detecting choice conflict (anterior cingulate cortex, ACC), updating stimulus value (dorsolateral prefrontal cortex, dlPFC) and supporting memory of stimulus value (hippocampus and ventromedial prefrontal cortex, vmPFC). Here we investigated whether resting-state neuronal activity within these regions is associated with the magnitude of individuals' preference updates. We fitted a dynamic causal model (DCM) to resting-state neuronal activity in the spectral domain (spDCM) and estimated the causal connectivity among core regions involved in preference formation following hard choices. The extent of individuals' choice-induced preference changes were found to be associated with a diminished resting-state excitation between the left dlPFC and the vmPFC, whereas preference consistency was related to a higher resting-state excitation from the ACC to the left hippocampus and vmPFC. Our results point to a model of preference formation during which the dynamic network configurations between left dlPFC, ACC, vmPFC and left hippocampus at rest are linked to preference change or stability.

How decisions and the desire for coherency shape subjective preferences over time

Recent findings suggest a bidirectional relationship between preferences and choices such that what is chosen can become preferred. Yet, it is still commonly held that preferences for individual items are maintained, such as caching a separate value estimate for each experienced option. Instead, we propose that all possible choice options and preferences are represented in a shared, continuous, multidimensional space that supports generalization. Decision making is cast as a learning process that seeks to align choices and preferences to maintain coherency. We formalized an error-driven learning model that updates preferences to align with past choices, which makes repeating those and related choices more likely in the future. The model correctly predicts that making a free choice increases preferences along related attributes. For example, after choosing a political candidate based on trivial information (e.g., they like cats), voters' views on abortion, immigration, and trade subsequently shifted to match their chosen candidate.

Choosing what we like vs liking what we choose: How choice-induced preference change might actually be instrumental to decision-making

For more than 60 years, it has been known that people report higher (lower) subjective values for items after having selected (rejected) them during a choice task. This phenomenon is coined "choice-induced preference change" or CIPC, and its established interpretation is that of "cognitive dissonance" theory. In brief, if people feel uneasy about their choice, they later convince themselves, albeit not always consciously, that the chosen (rejected) item was actually better (worse) than they had originally estimated. While this might make sense from an intuitive psychological standpoint, it is challenging from a theoretical evolutionary perspective. This is because such a cognitive mechanism might yield irrational biases, whose adaptive fitness would be unclear. In this work, we consider an alternative possibility, namely that CIPC is -at least partially- due to the refinement of option value representations that occurs while people are pondering about choice options. For example, contemplating competing possibilities during a choice may highlight aspects of the alternative options that were not considered before. In the context of difficult decisions, this would enable people to reassess option values until they reach a satisfactory level of confidence. This makes CIPC the epiphenomenal outcome of a cognitive process that is instrumental to the decision. Critically, our hypothesis implies novel predictions about how observed CIPC should relate to two specific meta-cognitive processes, namely: choice confidence and subjective certainty regarding pre-choice value judgments. We test these predictions in a behavioral experiment where participants rate the subjective value of food items both before and after choosing between equally valued items; we augment this traditional design with both reports of choice confidence and subjective certainty about value judgments. The results confirm our predictions and provide evidence that many quantitative features of CIPC (in particular: its relationship with metacognitive judgments) may be explained without ever invoking post-choice cognitive dissonance reduction explanation. We then discuss the relevance of our work in the context of the existing debate regarding the putative cognitive mechanisms underlying CIPC.

Increased Preference and Value of Consumer Products by Attentional Selection

It is usually assumed that individuals base their preferences for products or other items on the utility or value associated with the items. However, there is evidence that the attentional selection of an item alone already modulates the preference for that item. This has been shown, for instance, in preference choice tasks with unknown consumer products. Products that served as targets in a preceding visual search task were preferred to former distractor products. However, it is unclear whether such effects can also be observed when individuals have pre-existing attitudes toward products and whether attentional selection can change the perceived value of products. Hence, the aim of the present research was to replicate the attentional-selection effect on choice with known products and examine whether selective attention affects the perceived value of products beyond choosing the items. In two experiments, we replicated the attentional-selection effect on item preference in a choice task. Items that had served as targets in the search task were preferred to previous distractors. Introducing a response deadline in the preference-choice task in Experiment 2 did not further increase this effect. However, the value of former targets was rated higher than that of former distractors. Hence, the present results indicate that attentional selection not only affects preference choices but can also increase the value of attended and selected items.

Elucidating the role of the posterior medial frontal cortex in social conflict processing

A fundamental function of the brain is learning via new information. Studies investigating the neural basis of information-based learning processes indicate an important role played by the posterior medial frontal cortex (pMFC) in representing conflict between an individual's expectation and new information. However, specific function of the pMFC in this process remains relatively indistinct. Particularly, it's unclear whether the pMFC plays a role in the detection of conflict of incoming information, or the update of their belief after new information is provided. In an fMRI scanner, twenty-eight Japanese students viewed scenarios depicting various pro-social/anti-social behaviors. Participants rated how likely Japanese and South Korean students would perform each behavior, followed by feedback of the actual likelihood. They were then asked to rerate the scenarios after the fMRI session. Participants updated their second estimates based on feedback, with estimate changes more pronounced for favorable feedback (e.g., higher likelihood of pro-social behavior than expected) despite nationality, indicating participants were willing to view other people favorably. The fMRI results demonstrated activity in a part of the pMFC, the dorsomedial prefrontal cortex (dmPFC), was correlated with social conflict (difference between participant's estimate and actual likelihood), but not the corresponding belief update. Importantly, activity in a different part within the dmPFC was more sensitive to unfavorable trials compared to favorable trials. These results indicate sensitivity in the pMFC (at least within the dmPFC) relates to conflict between desirable outcomes versus reality, as opposed to the associated update of belief.

Effect of aging on choice-induced cognitive conflict

When individuals are forced to choose between similarly preferable alternatives, a negatively arousing cognitive conflict occurs, and the preference attitudes toward the chosen and rejected alternatives diverge. This phenomenon, often referred to as "cognitive dissonance", is of interest in psychological and decision neuroscience research. The dorsal anterior cingulate cortex (dACC) is involved in representing the cognitive conflict induced by difficult-choice tasks. Previous studies have shown age-related decline of the dACC function. However, whether the heightened activity of the dACC regarding cognitive conflict, and choice-induced preference change that behaviorally occur in young subjects also occur in the elderly is unclear. Furthermore, recent studies have noted substantial methodological flaw with the free-choice paradigm that often used in studies focusing on cognitive dissonance. Here, we used functional magnetic resonance imaging (fMRI) and a modified free-choice paradigm to formally test the effect of aging on choice-induced cognitive conflict. In the young participants, behavioral data confirmed the existence of cognitive conflict and preference change for the alternatives that they rejected in the difficult-choice trials. The imaging data revealed that the right dACC displayed an interaction effect associated with cognitive conflict. In contrast, we did not observe such effects in the elderly participants. These suggest a possibility that elderly people likely feel less cognitive dissonance.

Residual Information of Previous Decision Affects Evidence Accumulation in Current Decision

Bias in perceptual decisions can be generally defined as an effect which is controlled by factors other than the decision-relevant information (e.g., perceptual information in a perceptual task, when trials are independent). The literature on decision-making suggests two main hypotheses to account for this kind of bias: internal bias signals are derived from (a) the residual of motor signals generated to report a decision in the past, and (b) the residual of sensory information extracted from the stimulus in the past. Beside these hypotheses, this study suggests that making a decision in the past per se may bias the next decision. We demonstrate the validity of this assumption, first, by performing behavioral experiments based on the two-alternative forced-choice (TAFC) discrimination of motion direction paradigms and, then, we modified the pure drift-diffusion model (DDM) based on the accumulation-to-bound mechanism to account for the sequential effect. In both cases, the trace of the previous trial influences the current decision. Results indicate that the probability of being correct in the current decision increases if it is in line with the previously made decision even in the presence of feedback. Moreover, a modified model that keeps the previous decision information in the starting point of evidence accumulation provides a better fit to the behavioral data. Our findings suggest that the accumulated evidence in the decision-making process after crossing the bound in the previous decision can affect the parameters of information accumulation for the current decision in consecutive trials.

Sour grapes and sweet victories: How actions shape preferences

Classical decision theory postulates that choices proceed from subjective values assigned to the probable outcomes of alternative actions. Some authors have argued that opposite causality should also be envisaged, with choices influencing subsequent values expressed in desirability ratings. The idea is that agents may increase their ratings of items that they have chosen in the first place, which has been typically explained by the need to reduce cognitive dissonance. However, evidence in favor of this reverse causality has been the topic of intense debates that have not reached consensus so far. Here, we take a novel approach using Bayesian techniques to compare models in which choices arise from stable (but noisy) underlying values (one-way causality) versus models in which values are in turn influenced by choices (two-way causality). Moreover, we examined whether in addition to choices, other components of previous actions, such as the effort invested and the eventual action outcome (success or failure), could also impact subsequent values. Finally, we assessed whether the putative changes in values were only expressed in explicit ratings, or whether they would also affect other value-related behaviors such as subsequent choices. Behavioral data were obtained from healthy participants in a rating-choice-rating-choice-rating paradigm, where the choice task involves deciding whether or not to exert a given physical effort to obtain a particular food item. Bayesian selection favored two-way causality models, where changes in value due to previous actions affected subsequent ratings, choices and action outcomes. Altogether, these findings may help explain how values and actions drift when several decisions are made successively, hence highlighting some shortcomings of classical decision theory.

The standard way to explain decisions is the so-called valuation/selection model, which includes 1) a value function that calculates desirability for every possible outcome of alternative actions and 2) a choice function that integrates outcome values and generates selection probability for every action. In this classical view, choices are therefore determined (in a probabilistic sense) by hidden values. However, some authors have argued that causality could also be reversed, meaning that values may in turn be influenced by choices. Yet existing demonstrations of reverse causality have been criticized because pseudo-effects may arise from statistical artifacts. Here, we take a novel computational approach that directly compares models with and without the existence of reverse causality, on the basis of behavioral data obtained from volunteers in a new task. The winning model is a generalization of the reverse causality hypothesis, showing that people tend to like more the items that they previously chose to pursue, and even more if they did obtain these items. These effects were manifest not only in desirability ratings but also in subsequent actions, showing that value changes were more profound than just verbal statements. Altogether, our results invite reconsideration of decision theory, showing that actions are not neutral to the values driving them, hence suggesting that the history of actions should be taken into account.

Hard Decisions Shape the Neural Coding of Preferences

Hard decisions between equally valued alternatives can result in preference changes, meaning that subsequent valuations for chosen items increase and decrease for rejected items. Previous research suggests that this phenomenon is a consequence of cognitive dissonance reduction after the decision, induced by the mismatch between initial preferences and decision outcomes. In contrast, this functional magnetic resonance imaging and eye-tracking study with male and female human participants found that preferences are already updated online during the process of decision-making. Preference changes were predicted from activity in left dorsolateral prefrontal cortex and precuneus while making hard decisions. Fixation durations during this phase predicted both choice outcomes and subsequent preference changes. These preference adjustments became behaviorally relevant only for choices that were remembered and were in turn associated with hippocampus activity. Our results suggest that preferences evolve dynamically as decisions arise, potentially as a mechanism to prevent stalemate situations in underdetermined decision scenarios.SIGNIFICANCE STATEMENT Most theories of decision-making assume that we always choose the best option available, based on a set of stable preferences. However, what happens for hard decisions when the available options are preferred equally? We show that in such stalemate situations, decision-makers adjust their preferences dynamically during the process of decision-making, and these preference adjustments are predicted by a left prefrontal-parietal network. We also show that eye movements during decision-making are predictive of the magnitude of the upcoming value change. Our results suggest that preferences are dynamic, adjusted every time a hard decision is made, prompting a re-evaluation of existing frameworks of decision-making.

Methods for trustworthy nutritional recommendations NutriRECS (Nutritional Recommendations and accessible Evidence summaries Composed of Systematic reviews): a protocol

Background

Recent systematic reviews and editorials suggest that many organizations that produce nutritional guideline recommendations do not adhere to internationally recognized standards set forth by the Institute of Medicine (IoM), Guidelines International Network (GIN), Appraisal of Guidelines Research and Evaluation (AGREE), and Grading Recommendations, Assessment, Development and Evaluation (GRADE).

Methods

The potential solution is an independent group with content expertise and skilled in the methodology of systematic reviews and practice guidelines to produce trustworthy guideline recommendations, recommendations that are supported by publication in a top tier journal. The BMJ Rapid Recommendations project has recently demonstrated the feasibility and utility of this approach. Here, we are proposing trustworthy nutritional guideline recommendations based on internationally accepted guideline development standards, recommendations that will be informed by rigorous and novel systematic reviews of the benefits and harms associated with nutritional exposures, as well as studies on the values and preferences related to dietary behaviors among members of the international community.

Discussion

Adhering to international guideline standards, conducting high quality systematic reviews, and actively assessing the values and preferences of key stakeholders is expected to improve the quality of nutritional guidelines and their relevance to end-users, particularly patients and community members. We will send our work for peer review, and if found acceptable, we will publish our nutritional recommendations in top-tier general medicine journals.

Decision-making Increases Episodic Memory via Postencoding Consolidation

The ability for individuals to actively make decisions engages regions within the mesolimbic system and enhances memory for chosen items. In other behavioral contexts, mesolimbic engagement has been shown to enhance episodic memory by supporting consolidation. However, research has yet to investigate how consolidation may support interactions between decision-making and episodic memory. Across two studies, participants encoded items that were covered by occluder screens and could either actively decide which of two items to uncover or an item was preselected by the experimenter. In Study 1, we show that active decision-making reduces forgetting rates across an immediate and 24-hr memory test, a behavioral marker of consolidation. In Study 2, we use functional neuroimaging to characterize putative neural markers of memory consolidation by measuring interactions between the hippocampus and perirhinal cortex (PRC) during a postencoding period that reexposed participants to elements of the decision-making context without exposing them to memoranda. We show that choice-related striatal engagement is associated with increased postencoding hippocampal-PRC interactions. Finally, we show that a previous reported relationship between choice-related striatal engagement and long-term memory is accounted for by these postencoding hippocampal-PRC interactions. Together, these findings support a model by which actively deciding to encode information enhances memory consolidation to preserve episodic memory for outcomes, a process that may be facilitated by reexposure to the original decision-making context.

Post-decision biases reveal a self-consistency principle in perceptual inference

Making a categorical judgment can systematically bias our subsequent perception of the world. We show that these biases are well explained by a self-consistent Bayesian observer whose perceptual inference process is causally conditioned on the preceding choice. We quantitatively validated the model and its key assumptions with a targeted set of three psychophysical experiments, focusing on a task sequence where subjects first had to make a categorical orientation judgment before estimating the actual orientation of a visual stimulus. Subjects exhibited a high degree of consistency between categorical judgment and estimate, which is difficult to reconcile with alternative models in the face of late, memory related noise. The observed bias patterns resemble the well-known changes in subjective preferences associated with cognitive dissonance, which suggests that the brain's inference processes may be governed by a universal self-consistency constraint that avoids entertaining 'dissonant' interpretations of the evidence.

Ventrolateral Prefrontal Cortex Updates Chosen Value According to Choice Set Size

Having chosen an item typically increases the subjective value of the chosen item, and people generally enjoy making choices from larger choice sets. However, having too many items to choose from can reduce the value of chosen items-for example, because of conflict or choice difficulty. In this study, we investigated the effects of choice set size on behavioral and neural value updating (revaluation) of the chosen item. In the scanner, participants selected items from choice sets of various sizes (one, two, four, or eight items). After they chose an item, participants rerated the chosen item, and we quantified revaluation by taking the difference of postchoice minus prechoice ratings. Revaluation of chosen items increased up to choice sets of four alternatives but then decreased again for items chosen from choice sets of eight alternatives, revealing both a linear and a quadratic effect of choice set size. At the time of postchoice rating, activation of the ventrolateral pFC (VLPFC) reflected the influence of choice set size on parametric revaluation, without significant relation to either prechoice or postchoice ratings tested separately. Additional analyses revealed relations of choice set size to anterior cingulate and insula activity during actual choice and increased coupling of both regions to revaluation-related VLPFC during postchoice rating. These data suggest that the VLPFC plays a central role in a network that relates choice set size to updating the value of chosen items and integrates choice overload with value-enhancing effects of larger choice sets.

Relation between choice-induced preference change and depression

Most experimental studies of depressive symptom effects on decision-making have examined situations in which a single correct answer exists based on external circumstances (externally guided decision-making, e.g., gambling task). In addition to such decision-making, for decision-making of other types, no correct answer exists based on external circumstances (internally guided decision-making, e.g., preference judgment). For internally guided decision-making, a phenomenon is known by which preference for the chosen item increases and preference for the rejected item is decreased after choosing between two equally preferred items which is designated as choice-induced preference change. Recent reports suggest that this phenomenon is explainable by reinforcement learning theory just as it is with externally guided decision-making. Although many earlier studies have revealed the effects of depression in externally guided decision-making, the relation between depressive symptoms and choice-induced preference change remains unclear. This study investigated the relation between depressive symptoms and choice-induced preference change using the blind choice paradigm. Results show that depressive symptoms are correlated with change in preference of rejected items (Spearman's r = .28, p = .04): depressed individuals tend to show less decreased preference of rejected items. These results indicate that individual differences of depressive symptoms affect choice-induced preference change. We discuss the mechanisms underlying the relation between depression and choice-induced preference change.

Neural Mechanisms of Cognitive Dissonance (Revised): An EEG Study

Cognitive dissonance theory suggests that our preferences are modulated by the mere act of choosing. A choice between two similarly valued alternatives creates psychological tension (cognitive dissonance) that is reduced by a postdecisional reevaluation of the alternatives. We measured EEG of human subjects during rest and free-choice paradigm. Our study demonstrates that choices associated with stronger cognitive dissonance trigger a larger negative frontocentral evoked response similar to error-related negativity, which has in turn been implicated in general performance monitoring. Furthermore, the amplitude of the evoked response is correlated with the reevaluation of the alternatives. We also found a link between individual neural dynamics (long-range temporal correlations) of the frontocentral cortices during rest and follow-up neural and behavioral effects of cognitive dissonance. Individuals with stronger resting-state long-range temporal correlations demonstrated a greater postdecisional reevaluation of the alternatives and larger evoked brain responses associated with stronger cognitive dissonance. Thus, our results suggest that cognitive dissonance is reflected in both resting-state and choice-related activity of the prefrontal cortex as part of the general performance-monitoring circuitry.

SIGNIFICANCE STATEMENT Contrary to traditional decision theory, behavioral studies repeatedly demonstrate that our preferences are modulated by the mere act of choosing. Difficult choices generate psychological (cognitive) dissonance, which is reduced by the postdecisional devaluation of unchosen options. We found that decisions associated with a higher level of cognitive dissonance elicited a stronger negative frontocentral deflection that peaked ∼60 ms after the response. This activity shares similar spatial and temporal features as error-related negativity, the electrophysiological correlate of performance monitoring. Furthermore, the frontocentral resting-state activity predicted the individual magnitude of preference change and the strength of cognitive dissonance-related neural activity.

Cognitive dissonance resolution depends on episodic memory

The notion that past choices affect preferences is one of the most influential concepts of social psychology since its first report in the 50 s, and its theorization within the cognitive dissonance framework. In the free-choice paradigm (FCP) after choosing between two similarly rated items, subjects reevaluate chosen items as more attractive and rejected items as less attractive. However the relations prevailing between episodic memory and choice-induced preference change (CIPC) remain highly debated: is this phenomenon dependent or independent from memory of past choices? We solve this theoretical debate by demonstrating that CIPC occurs exclusively for items which were correctly remembered as chosen or rejected during the choice stage. We used a combination of fMRI and intra-cranial electrophysiological recordings to reveal a modulation of left hippocampus activity, a hub of episodic memory retrieval, immediately before the occurrence of CIPC during item reevaluation. Finally, we show that contrarily to a previous influential report flawed by a statistical artifact, this phenomenon is absent in amnesic patients for forgotten items. These results demonstrate the dependence of cognitive dissonance on conscious episodic memory. This link between current preferences and previous choices suggests a homeostatic function of this regulative process, aiming at preserving subjective coherence.

Tolerant Paternalism: Pro-ethical Design as a Resolution of the Dilemma of Toleration

Toleration is one of the fundamental principles that inform the design of a democratic and liberal society. Unfortunately, its adoption seems inconsistent with the adoption of paternalistically benevolent policies, which represent a valuable mechanism to improve individuals' well-being. In this paper, I refer to this tension as the dilemma of toleration. The dilemma is not new. It arises when an agent A would like to be tolerant and respectful towards another agent B's choices but, at the same time, A is altruistically concerned that a particular course of action would harm, or at least not improve, B's well-being, so A would also like to be helpful and seeks to ensure that B does not pursue such course of action, for B's sake and even against B's consent. In the article, I clarify the specific nature of the dilemma and show that several forms of paternalism, including those based on ethics by design and structural nudging, may not be suitable to resolve it. I then argue that one form of paternalism, based on pro-ethical design, can be compatible with toleration and hence with the respect for B's choices, by operating only at the informational and not at the structural level of a choice architecture. This provides a successful resolution of the dilemma, showing that tolerant paternalism is not an oxymoron but a viable approach to the design of a democratic and liberal society.

Post-response βγ power predicts the degree of choice-based learning in internally guided decision-making

Choosing an option increases a person’s preference for that option. This phenomenon, called choice-based learning (CBL), has been investigated separately in the contexts of internally guided decision-making (IDM, e.g., preference judgment), for which no objectively correct answer exists, and externally guided decision making (EDM, e.g., perceptual decision making), for which one objectively correct answer exists. For the present study, we compared decision making of these two types to examine differences of underlying neural processes of CBL. As IDM and EDM tasks, occupation preference judgment and salary judgment were used, respectively. To compare CBL for the two types of decision making, we developed a novel measurement of CBL: decision consistency. When CBL occurs, decision consistency is higher in the last-half trials than in first-half trials. Electroencephalography (EEG) data have demonstrated that the change of decision consistency is positively correlated with the fronto-central beta–gamma power after response in the first-half trials for IDM, but not for EDM. Those results demonstrate for the first time the difference of CBL between IDM and EDM. The fronto-central beta–gamma power is expected to reflect a key process of CBL, specifically for IDM.

The simple act of choosing influences declarative memory

Individuals value the opportunity to make choices and exert control over their environment. This perceived sense of agency has been shown to have broad influences on cognition, including preference, decision-making, and valuation. However, it is unclear whether perceived control influences memory. Using a combined behavioral and functional magnetic resonance imaging approach, we investigated whether imbuing individuals with a sense of agency over their learning experience influences novel memory encoding. Participants encoded objects during a task that manipulated the opportunity to choose. Critically, unlike previous work on active learning, there was no relationship between individuals' choices and the content of memoranda. Despite this, we found that the opportunity to choose resulted in robust, reliable enhancements in declarative memory. Neuroimaging results revealed that anticipatory activation of the striatum, a region associated with decision-making, valuation, and exploration, correlated with choice-induced memory enhancements in behavior. These memory enhancements were further associated with interactions between the striatum and hippocampus. Specifically, anticipatory signals in the striatum when participants are alerted to the fact that they will have to choose one of two memoranda were associated with encoding success effects in the hippocampus on a trial-by-trial basis. The precedence of the striatal signal in these interactions suggests a modulatory relationship of the striatum over the hippocampus. These findings not only demonstrate enhanced declarative memory when individuals have perceived control over their learning but also support a novel mechanism by which these enhancements emerge. Furthermore, they demonstrate a novel context in which mesolimbic and declarative memory systems interact.

A causal role for posterior medial frontal cortex in choice-induced preference change

After a person chooses between two items, preference for the chosen item will increase and preference for the unchosen item will decrease because of the choice made. In other words, we tend to justify or rationalize our past behavior by changing our attitude. This phenomenon of choice-induced preference change has been traditionally explained by cognitive dissonance theory. Choosing something that is disliked or not choosing something that is liked are both cognitively inconsistent and, to reduce this inconsistency, people tend to change their subsequently stated preference in accordance with their past choices. Previously, human neuroimaging studies identified posterior medial frontal cortex (pMFC) as a key brain region involved in cognitive dissonance. However, it remains unknown whether the pMFC plays a causal role in inducing preference change after cognitive dissonance. Here, we demonstrate that 25 min, 1 Hz repetitive transcranial magnetic stimulation applied over the pMFC significantly reduces choice-induced preference change compared with sham stimulation or control stimulation over a different brain region, demonstrating a causal role for the pMFC.

Action and valence modulate choice and choice-induced preference change

Choices are not only communicated via explicit actions but also passively through inaction. In this study we investigated how active or passive choice impacts upon the choice process itself as well as a preference change induced by choice. Subjects were tasked to select a preference for unfamiliar photographs by action or inaction, before and after they gave valuation ratings for all photographs. We replicate a finding that valuation increases for chosen items and decreases for unchosen items compared to a control condition in which the choice was made post re-evaluation. Whether choice was expressed actively or passively affected the dynamics of revaluation differently for positive and negatively valenced items. Additionally, the choice itself was biased towards action such that subjects tended to choose a photograph obtained by action more often than a photographed obtained through inaction. These results highlight intrinsic biases consistent with a tight coupling of action and reward and add to an emerging understanding of how the mode of action itself, and not just an associated outcome, modulates the decision making process.

Choice-induced preference change and the free-choice paradigm: A clarification

Positive spreading of ratings or rankings in the classical free-choice paradigm is commonly taken to indicate choice-induced change in preferences and has motivated influential theories as cognitive dissonance theory and self-perception theory. Chen and Risen [2010] argued by means of a mathematical proof that positive spreading is merely a statistical consequence of a flawed design. However, positive spreading has also been observed in blind choice and other designs where the alleged flaw should be absent. We show that the result in Chen and Risen [2010] is mathematically incorrect, although it can be recovered in a particular case. Specifically, we present a formal model of decision making that satisfies all assumptions in that article but implies that spreading need not be positive in the absence of choice-induced preference change. Hence, although the free-choice paradigm is flawed, the present research shows that reasonable models of human behavior need not predict consistent positive spreading. As a consequence, taken as a whole, previous experimental results remain informative.

Cognitive dissonance resolution is related to episodic memory

The notion that our past choices affect our future behavior is certainly one of the most influential concepts of social psychology since its first experimental report in the 50 s, and its initial theorization by Festinger within the “cognitive dissonance” framework. Using the free choice paradigm (FCP), it was shown that choosing between two similarly rated items made subjects reevaluate the chosen items as more attractive and the rejected items as less attractive. However, in 2010 a major work by Chen and Risen revealed a severe statistical flaw casting doubt on most previous studies. Izuma and colleagues (2010) supplemented the traditional FCP with original control conditions and concluded that the effect observed could not be solely attributed to this methodological flaw. In the present work we aimed at establishing the existence of genuine choice-induced preference change and characterizing this effect. To do so, we replicated Izuma et al.’ study and added a new important control condition which was absent from the original study. Moreover, we added a memory test in order to measure the possible relation between episodic memory of choices and observed behavioral effects. In two experiments we provide experimental evidence supporting genuine choice-induced preference change obtained with FCP. We also contribute to the understanding of the phenomenon by showing that choice-induced preference change effects are strongly correlated with episodic memory.

Model-observer similarity, error modeling and social learning in rhesus macaques

Monkeys readily learn to discriminate between rewarded and unrewarded items or actions by observing their conspecifics. However, they do not systematically learn from humans. Understanding what makes human-to-monkey transmission of knowledge work or fail could help identify mediators and moderators of social learning that operate regardless of language or culture, and transcend inter-species differences. Do monkeys fail to learn when human models show a behavior too dissimilar from the animals’ own, or when they show a faultless performance devoid of error? To address this question, six rhesus macaques trained to find which object within a pair concealed a food reward were successively tested with three models: a familiar conspecific, a ‘stimulus-enhancing’ human actively drawing the animal’s attention to one object of the pair without actually performing the task, and a ‘monkey-like’ human performing the task in the same way as the monkey model did. Reward was manipulated to ensure that all models showed equal proportions of errors and successes. The ‘monkey-like’ human model improved the animals’ subsequent object discrimination learning as much as a conspecific did, whereas the ‘stimulus-enhancing’ human model tended on the contrary to retard learning. Modeling errors rather than successes optimized learning from the monkey and ‘monkey-like’ models, while exacerbating the adverse effect of the ‘stimulus-enhancing’ model. These findings identify error modeling as a moderator of social learning in monkeys that amplifies the models’ influence, whether beneficial or detrimental. By contrast, model-observer similarity in behavior emerged as a mediator of social learning, that is, a prerequisite for a model to work in the first place. The latter finding suggests that, as preverbal infants, macaques need to perceive the model as ‘like-me’ and that, once this condition is fulfilled, any agent can become an effective model.