Memory and decision making interact to shape the value of unchosen options

Different learning aberrations relate to delusion-like beliefs with different contents

The prediction error account of delusions has had success. However, its explanation of delusions with different contents has been lacking. Persecutory delusions and paranoia are the common unfounded beliefs that others have harmful intentions towards us. Other delusions include believing that one's thoughts or actions are under external control or that events in the world have specific personal meaning. We compare learning in two different cognitive tasks, probabilistic reversal learning and Kamin blocking, that have relationships to paranoid and non-paranoid delusion-like beliefs, respectively. We find that clinical high-risk status alone does not result in different behavioural results in the probabilistic reversal learning task but that an individual's level of paranoia is associated with excessive switching behaviour. During the Kamin blocking task, paranoid individuals learned inappropriately about the blocked cue. However, they also had decreased learning about the control cue, suggesting more general learning impairments. Non-paranoid delusion-like belief conviction (but not paranoia) was associated with aberrant learning about the blocked cue but intact learning about the control cue, suggesting specific impairments in learning related to cue combination. We fit task-specific computational models separately to behavioural data to explore how latent parameters vary within individuals between tasks and how they can explain symptom-specific effects. We find that paranoia is associated with low learning rates in the probabilistic reversal learning task and the blocking task. Non-paranoid delusion-like belief conviction is instead related to parameters controlling the degree and direction of similarity between cue updating during simultaneous cue presentation. These results suggest that paranoia and other delusion-like beliefs involve dissociable deficits in learning and belief updating, which, given the transdiagnostic status of paranoia, might have differential utility in predicting psychosis.

Memory for rewards guides retrieval

Rewards paid out for successful retrieval motivate the formation of long-term memory. However, it has been argued that the Motivated Learning Task does not measure reward effects on memory strength but decision-making during retrieval. We report three large-scale online experiments in healthy participants (N = 200, N = 205, N = 187) that inform this debate. In experiment 1, we found that explicit stimulus-reward associations formed during encoding influence response strategies at retrieval. In experiment 2, reward affected memory strength and decision-making strategies. In experiment 3, reward affected decision-making strategies only. These data support a theoretical framework that assumes that promised rewards not only increase memory strength, but additionally lead to the formation of stimulus-reward associations that influence decisions at retrieval.

Invariable distribution of co-evolutionary complex adaptive systems with agent's behavior and local topological configuration

In this study, we developed a dynamical Multi-Local-Worlds (MLW) complex adaptive system with co-evolution of agent's behavior and local topological configuration to predict whether agents' behavior would converge to a certain invariable distribution and derive the conditions that should be satisfied by the invariable distribution of the optimal strategies in a dynamical system structure. To this end, a Markov process controlled by agent's behavior and local graphic topology configuration was constructed to describe the dynamic case's interaction property. After analysis, the invariable distribution of the system was obtained using the stochastic process method. Then, three kinds of agent's behavior (smart, normal, and irrational) coupled with corresponding behaviors, were introduced as an example to prove that their strategies converge to a certain invariable distribution. The results showed that an agent selected his/her behavior according to the evolution of random complex networks driven by preferential attachment and a volatility mechanism with its payment, which made the complex adaptive system evolve. We conclude that the corresponding invariable distribution was determined by agent's behavior, the system's topology configuration, the agent's behavior noise, and the system population. The invariable distribution with agent's behavior noise tending to zero differed from that with the population tending to infinity. The universal conclusion, corresponding to the properties of both dynamical MLW complex adaptive system and cooperative/non-cooperative game that are much closer to the common property of actual economic and management events that have not been analyzed before, is instrumental in substantiating managers' decision-making in the development of traffic systems, urban models, industrial clusters, technology innovation centers, and other applications.

Option similarity modulates the link between choice and memory

Choices made in everyday life are highly variable. Sometimes, you may find yourself choosing between two similar options (e.g., breakfast foods to eat) and other times between two dissimilar options (e.g., what to buy with a gift certificate). The goal of the present study was to understand how the similarity of choice options affects our ability to remember what we choose and what we did not choose. We hypothesized that choosing between similar as compared to dissimilar options would evoke a comparison-based strategy (evaluating options with respect to one another), fostering a relational form of encoding and leading to better memory for both the chosen and unchosen options. In Experiment 1, participants reported their strategy when choosing between pairs of similar or dissimilar options, revealing that participants were more likely to use a comparison-based strategy when faced with similar options. In Experiment 2, we tested memory after participants made choices between similar or dissimilar options, finding improved memory for both chosen and unchosen options from the similar compared to dissimilar choice trials. In Experiment 3, we examined strategy use when choosing between pairs of similar or dissimilar options and memory for these options. Replicating and extending the results of the first two experiments, we found that participants were more likely to use a comparison-based strategy when choosing between similar than dissimilar options, and that the positive effect of similarity on memory was stronger for unchosen than chosen options when controlling for strategy use. We interpret our results as evidence that option similarity impacts the mnemonic processes used during choice, altering what we encode and ultimately remember about our choices.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

Computational mechanisms underlying latent value updating of unchosen actions

Current studies suggest that individuals estimate the value of their choices based on observed feedback. Here, we ask whether individuals also update the value of their unchosen actions, even when the associated feedback remains unknown. One hundred seventy-eight individuals completed a multi-armed bandit task, making choices to gain rewards. We found robust evidence suggesting latent value updating of unchosen actions based on the chosen action's outcome. Computational modeling results suggested that this effect is mainly explained by a value updating mechanism whereby individuals integrate the outcome history for choosing an option with that of rejecting the alternative. Properties of the deliberation (i.e., duration/difficulty) did not moderate the latent value updating of unchosen actions, suggesting that memory traces generated during deliberation might take a smaller role in this specific phenomenon than previously thought. We discuss the mechanisms facilitating credit assignment to unchosen actions and their implications for human decision-making.

Memory precision and age differentially predict the use of decision-making strategies across the lifespan

Memory function declines in normal aging, in a relatively continuous fashion following middle-age. The effect of aging on decision-making is less well-understood, with seemingly conflicting results on both the nature and direction of these age effects. One route for clarifying these mixed findings is to understand how age-related differences in memory affect decisions. Recent work has proposed memory sampling as a specific computational role for memory in decision-making, alongside well-studied mechanisms of reinforcement learning (RL). Here, we tested the hypothesis that age-related declines in episodic memory alter memory sampling. Participants (total N = 361; ages 18-77) performed one of two variants of a standard reward-guided decision experiment with additional trial-unique mnemonic content and a separately-administered task for assessing memory precision. When we fit participants' choices with a hybrid computational model implementing both memory-based and RL-driven valuation side-by-side, we found that memory precision tracked the contribution of memory sampling to choice. At the same time, age corresponded to decreasing influence of RL and increasing perseveration. A second experiment confirmed these results and further revealed that memory precision tracked the specificity of memories selected for sampling. Together, these findings suggest that differences in decision-making across the lifespan may be related to memory function, and that interventions which aim to improve the former may benefit from targeting the latter.

The intersection of the retrieval state and internal attention

Large-scale brain states or distributed patterns of brain activity modulate downstream processing and behavior. Sustained attention and memory retrieval states impact subsequent memory, yet how these states relate to one another is unclear. I hypothesize that internal attention is a central process of the retrieval state. The alternative is that the retrieval state specifically reflects a controlled, episodic retrieval mode, engaged only when intentionally accessing events situated within a spatiotemporal context. To test my hypothesis, I developed a mnemonic state classifier independently trained to measure retrieval state evidence and applied this classifier to a spatial attention task. I find that retrieval state evidence increases during delay and response intervals when participants are maintaining spatial information. Critically, retrieval state evidence is positively related to the amount of maintained spatial location information and predicts target detection reaction times. Together, these findings support the hypothesis that internal attention is a central process of the retrieval state.

The role of memory in counterfactual valuation

Value-based decisions are often guided by past experience. If a choice led to a good outcome, we are more likely to repeat it. This basic idea is well-captured by reinforcement-learning models. However, open questions remain about how we assign value to options we did not choose and which we therefore never had the chance to learn about directly. One solution to this problem is proposed by policy gradient reinforcement-learning models; these do not require direct learning of value, instead optimizing choices according to a behavioral policy. For example, a logistic policy predicts that if a chosen option was rewarded, the unchosen option would be deemed less desirable. Here, we test the relevance of these models to human behavior and explore the role of memory in this phenomenon. We hypothesize that a policy may emerge from an associative memory trace formed during deliberation between choice options. In a preregistered study (n = 315) we show that people tend to invert the value of unchosen options relative to the outcome of chosen options, a phenomenon we term inverse decision bias. The inverse decision bias is correlated with memory for the association between choice options; moreover, it is reduced when memory formation is experimentally interfered with. Finally, we present a new memory-based policy gradient model that predicts both the inverse decision bias and its dependence on memory. Our findings point to a significant role of associative memory in valuation of unchosen options and introduce a new perspective on the interaction between decision-making, memory, and counterfactual reasoning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Dual-Functional Triboelectric Nanogenerator Based on the Comprehensive Integration and Synergetic Utilization of Triboelectrification, Electrostatic Induction, and Electrostatic Discharge to Achieve Alternating Current/Direct Current Convertible Outputs

Traditional alternating current (AC) and direct current (DC) triboelectric nanogenerators (TENGs), which are implemented via the pairwise coupling of triboelectrification, electrostatic induction, and electrostatic discharge, have been widely explored in various fields. In this work, the comprehensive integration and synergetic utilization of triboelectrification, electrostatic induction, and electrostatic discharge in a single device for the first time is realized, achieving a dual-functional TENG (DF-TENG) to produce an AC/DC convertible output. Distinguishing from the conventional TENGs, the coupling of triboelectrification and electrostatic discharge enables charge circulation between the dielectric tribo-layers, while electrostatic induction realizes charge transfer in the external circuit. This novel energy conversion mechanism has been proven to be applicable to a variety of materials, including polymers, fabrics, and semiconductors. The output mode of the DF-TENG can be tuned by adjusting the slider motion state, and its constant output current and power density can reach 1.51 mA m^-2 Hz^-1 and 398 mW m^-2 Hz^-1 , respectively, which are the highest records reported for constant DC-TENGs to date. This work not only provides a paradigm shift to achieve AC/DC convertible output, but it also exhibits high potential for extending the TENG design philosophy.

Counterfactual curiosity: motivated thinking about what might have been

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate 'counterfactual curiosity': they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally-the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Training diversity promotes absolute-value-guided choice

Many decision-making studies have demonstrated that humans learn either expected values or relative preferences among choice options, yet little is known about what environmental conditions promote one strategy over the other. Here, we test the novel hypothesis that humans adapt the degree to which they form absolute values to the diversity of the learning environment. Since absolute values generalize better to new sets of options, we predicted that the more options a person learns about the more likely they would be to form absolute values. To test this, we designed a multi-day learning experiment comprising twenty learning sessions in which subjects chose among pairs of images each associated with a different probability of reward. We assessed the degree to which subjects formed absolute values and relative preferences by asking them to choose between images they learned about in separate sessions. We found that concurrently learning about more images within a session enhanced absolute-value, and suppressed relative-preference, learning. Conversely, cumulatively pitting each image against a larger number of other images across multiple sessions did not impact the form of learning. These results show that the way humans encode preferences is adapted to the diversity of experiences offered by the immediate learning context.

Lessons learned about willingness to adopt various protective measures during the early COVID-19 pandemic in three countries

BACKGROUND

Regarding the COVID-19 pandemic, concerted efforts have been invested in research to investigate and communicate the importance of complying with protective behaviors, such as handwashing and mask wearing. Protective measures vary in how effective they are in protecting the individual against infection, how much experience people have with them, whether they provide individual or societal protection, and how they are perceived on these dimensions.

METHODS

This study assessed the willingness to follow recommended measures, depending on these features, among participants from Germany (n = 333), Hong Kong (n = 367), and the U.S. (n = 495). From April 24th to May 1st, 2020, individuals completed an online survey that assessed the antecedents of interest.

RESULTS

It was shown that assumed effectiveness, previous experience, and intended self- and other-protection positively predicted willingness to comply across countries. When measures were mainly perceived as protecting others (vs. the self), individuals were less prone to adopt them. When a measure's effectiveness to protect the individual was perceived as lower, willingness to adopt the measure increased with higher levels of prior experience and collectivism. Moreover, protecting others was more strongly related to adoption when individuals had higher levels of collectivism and lower levels of individualism.

CONCLUSIONS

Emphasizing the benefit for others could be a means to lower the potential detrimental effects of low assumed effectiveness for individual protection.

How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective

Reinforcement learning and working memory are two core processes of human cognition and are often considered cognitively, neuroscientifically, and algorithmically distinct. Here, we show that the brain networks that support them actually overlap significantly and that they are less distinct cognitive processes than often assumed. We review literature demonstrating the benefits of considering each process to explain properties of the other and highlight recent work investigating their more complex interactions. We discuss how future research in both computational and cognitive sciences can benefit from one another, suggesting that a key missing piece for artificial agents to learn to behave with more human-like efficiency is taking working memory's role in learning seriously. This review highlights the risks of neglecting the interplay between different processes when studying human behavior (in particular when considering individual differences). We emphasize the importance of investigating these dynamics to build a comprehensive understanding of human cognition.

Why unchosen options linger in our minds

In the 1930s, philosopher John Dewey stated: “We do not learn from experience… we learn from reflecting on experience.” The question of how we learn from the consequences of our actions has been investigated for decades. When deliberating between options, it is assumed that the outcome of our choice is used as a feedback signal to learn the value of the chosen option. But what about the forgone alternative? In a recent paper, Biderman and Shohamy show that we also revise the valuation of forgone options, assuming them to be inversely related to that of chosen ones.

Memory and decision making interact to shape the value of unchosen options

The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants' memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other.