Uncertainty alters the balance between incremental learning and episodic memory

The Role of the Cerebellum in Learning to Predict Reward: Evidence from Cerebellar Ataxia

Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.

Two routes to value-based decisions in Parkinson's disease: differentiating incremental reinforcement learning from episodic memory

Patients with Parkinson's disease are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in Parkinson's. Recent studies have found that a combination of different cognitive and computational strategies contribute even to simple reward-based learning tasks, suggesting a possible role for episodic memory. These findings raise critical questions about how incremental learning and episodic memory interact to support learning from past experience and what their relative contributions are to impaired decision-making in Parkinson's disease. Here we addressed these questions by asking patients with Parkinson's disease (n=26) both on and off their dopamine replacement medication and age- and education-matched healthy controls (n=26) to complete a task designed to isolate the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that Parkinson's patients performed as well as healthy controls when using episodic memory, but were impaired at incremental reward-based learning. Dopamine replacement medication remediated this deficit while enhancing subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that Parkinson's patients are impaired at learning about reward from trial-and-error when episodic memory is properly controlled for, and that learning based on the value of single experiences remains intact in patients with Parkinson's disease.

Naturalistic reinforcement learning

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

Back to the future: Progressing memory research in eating disorders

OBJECTIVE

Human behaviors, thoughts, and emotions are guided by memories of the past. Thus, there can be little doubt that memory plays a fundamental role in the behaviors (e.g., binging), thoughts (e.g., body-image concerns), and emotions (e.g., guilt) that characterize eating disorders (EDs). Although a growing body of research has begun to investigate the role of memory in EDs, this literature is limited in numerous ways and has yet to be integrated into an overarching framework.

METHODS

In the present article, we provide an operational framework for characterizing different domains of memory, briefly review existing ED memory research within this framework, and highlight crucial gaps in the literature.

RESULTS

We distinguish between three domains of memory-episodic, procedural, and working-which differ based on functional attributes and underlying neural systems. Most recent ED memory research has focused on procedural memory broadly defined (e.g., reinforcement learning), and findings within all three memory domains are highly mixed. Further, few studies have attempted to assess these different domains simultaneously, though most behavior is achieved through coordination and competition between memory systems. We, therefore, offer recommendations for how to move ED research forward within each domain of memory and how to study the interactions between memory systems, using illustrative examples from other areas of basic and clinical research.

DISCUSSION

A stronger and more integrated understanding of the mechanisms that connect memory of past experiences to present ED behavior may yield more comprehensive theoretical models of EDs that guide novel treatment approaches.

PUBLIC SIGNIFICANCE

Memories of previous eating-related experiences may contribute to the onset and maintenance of eating disorders (EDs). However, research on the role of memory in EDs is limited, and distinct domains of ED memory research are rarely connected. We, therefore, offer a framework for organizing, progressing, and integrating ED memory research, to provide a better foundation for improving ED treatment and intervention going forward.

Principles of cognitive control over task focus and task switching

Adaptive behaviour requires the ability to focus on a task and protect it from distraction (cognitive stability) and to rapidly switch tasks when circumstances change (cognitive flexibility). Burgeoning research literatures have aimed to understand how people achieve task focus and task switch readiness. In this Perspective, I integrate these literatures to derive a cognitive architecture and functional rules underlying the regulation of cognitive stability and flexibility. I propose that task focus and task switch readiness are supported by independent mechanisms. However, I also suggest that the strategic regulation of both mechanisms is governed by shared learning principles: an incremental, online learner that nudges control up or down based on the recent history of task demands (a recency heuristic) and episodic reinstatement when the current context matches a past experience (a recognition heuristic). Finally, I discuss algorithmic and neural implementations of these processes, as well as clinical implications.

Memory precision and age differentially predict the use of decision-making strategies across the lifespan

Memory function declines in normal aging, in a relatively continuous fashion following middle-age. The effect of aging on decision-making is less well-understood, with seemingly conflicting results on both the nature and direction of these age effects. One route for clarifying these mixed findings is to understand how age-related differences in memory affect decisions. Recent work has proposed memory sampling as a specific computational role for memory in decision-making, alongside well-studied mechanisms of reinforcement learning (RL). Here, we tested the hypothesis that age-related declines in episodic memory alter memory sampling. Participants (total N = 361; ages 18-77) performed one of two variants of a standard reward-guided decision experiment with additional trial-unique mnemonic content and a separately-administered task for assessing memory precision. When we fit participants' choices with a hybrid computational model implementing both memory-based and RL-driven valuation side-by-side, we found that memory precision tracked the contribution of memory sampling to choice. At the same time, age corresponded to decreasing influence of RL and increasing perseveration. A second experiment confirmed these results and further revealed that memory precision tracked the specificity of memories selected for sampling. Together, these findings suggest that differences in decision-making across the lifespan may be related to memory function, and that interventions which aim to improve the former may benefit from targeting the latter.

Predictions transform memories: How expected versus unexpected events are integrated or separated in memory

Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.