Negative reinforcement impairs overnight memory consolidation

Enhancing and advancing the understanding and study of dreaming and memory consolidation: Reflections, challenges, theoretical clarity, and methodological considerations

Empirical investigations that search for a link between dreaming and sleep-dependent memory consolidation have focused on testing for an association between dreaming of what was learned, and improved memory performance for learned material. Empirical support for this is mixed, perhaps owing to the inherent challenges presented by the nature of dreams, and methodological inconsistencies. The purpose of this paper is to address critically prevalent assumptions and practices, with the aim of clarifying and enhancing research on this topic, chiefly by providing a theoretical synthesis of existing models and evidence. Also, it recommends the method of Targeted Memory Reactivation (TMR) as a means for investigating if dream content can be linked to specific cued activations. Other recommendations to enhance research practice and enquiry on this subject are also provided, focusing on the HOW and WHY we search for memory sources in dreams, and what purpose (if any) they might serve.

Impact of Pre-Sleep Visual Media Exposure on Dreams: A Scoping Review

A body of experimental research has aimed to investigate processes underlying dream formation by examining the effects of a range of pre-sleep stimuli and events on subsequent dream content. Given its ever-growing presence and salience in people's everyday lives, pre-sleep media consumption stands out as a key variable that could influence people's dreams. We conducted a scoping review to evaluate the experimental evidence of the effects of pre-sleep exposure to visual media on dream content. A systematic search on PubMed, PsycInfo, and Web of Science using terms related to moving visual media and dreams yielded 29 studies meeting the inclusion criteria. Overall, we found modest yet varied effects of pre-sleep exposure to visual media on dream content, with rates of stimulus-related incorporation ranging from 3% to 43% for REM dream reports, 4% to 30% for NREM sleep mentation reports, and between 11% and 35% for home dream reports. Our review highlights the large methodological heterogeneity and gaps across studies, the general difficulty in influencing dream content using pre-sleep exposure to visual media, and suggests promising venues for future research to advance our understanding of how and why digital media may impact people's dreams.

The effects of reward and punishment on the performance of ping-pong ball bouncing

Introduction

Reward and punishment modulate behavior. In real-world motor skill learning, reward and punishment have been found to have dissociable effects on optimizing motor skill learning, but the scientific basis for these effects is largely unknown.

Methods

In the present study, we investigated the effects of reward and punishment on the performance of real-world motor skill learning. Specifically, three groups of participants were trained and tested on a ping-pong ball bouncing task for three consecutive days. The training and testing sessions were identical across the three days: participants were trained with their right (dominant) hand each day under conditions of either reward, punishment, or a neutral control condition (neither). Before and after the training session, all participants were tested with their right and left hands without any feedback.

Results

We found that punishment promoted early learning, while reward promoted late learning. Reward facilitated short-term memory, while punishment impaired long-term memory. Both reward and punishment interfered with long-term memory gains. Interestingly, the effects of reward and punishment transferred to the left hand.

Discussion

The results show that reward and punishment have different effects on real-world motor skill learning. The effects change with training and transfer readily to novel contexts. The results suggest that reward and punishment may act on different learning processes and engage different neural mechanisms during real-world motor skill learning. In addition, high-level metacognitive processes may be enabled by the additional reinforcement feedback during real-world motor skill learning. Our findings provide new insights into the mechanisms underlying motor learning, and may have important implications for practical applications such as sports training and motor rehabilitation.

A meta-analysis of the relation between dream content and memory consolidation

The frequent appearance of newly learned information in dreams suggests that dream content is influenced by memory consolidation. Many studies have tested this hypothesis by asking whether dreaming about a learning task is associated with improved memory, but results have been inconsistent. We conducted a meta-analysis to determine the strength of the association between learning-related dreams and post-sleep memory improvement. We searched the literature for studies that (1) trained participants on a pre-sleep learning task and then tested their memory after sleep, and (2) associated post-sleep memory improvement with the extent to which dreams incorporated learning task content. Sixteen studies qualified for inclusion, which together reported 45 effects. Integrating across effects, we report a strong and statistically significant association between task-related dreaming and memory performance (SMD = 0.51 [95% CI 0.28, 0.74], p < 0.001). Among studies using polysomnography, this relationship was statistically significant for dreams collected from non-rapid eye movement (NREM) sleep (n = 10) but not for dreams collected from rapid eye movement (REM) sleep (n = 12). There was a significant association between dreaming and memory for all types of learning tasks studied. This meta-analysis provides further evidence that dreaming about a learning task is associated with improved memory performance, suggesting that dream content may be an indication of memory consolidation. Furthermore, we report preliminary evidence that the relationship between dreaming and memory may be stronger in NREM sleep compared to REM.

Differential effects of reward and punishment on reinforcement-based motor learning and generalization

Reward and punishment have long been recognized as potent modulators of human behavior. Although reinforcement learning is a significant motor learning process, the exact mechanisms underlying how the brain learns movements through reward and punishment are not yet fully understood. Beyond the memory of specific examples, investigating the ability to generalize to new situations offers a better understanding of motor learning. This study hypothesizes that reward and punishment engage qualitatively different motivational systems with different neurochemical and neuroanatomical substrates, which would have differential effects on reinforcement-based motor learning and generalization. To test this hypothesis, two groups of participants learn a motor task in one direction and then relearn the same task in a new direction, receiving only performance-based reward or punishment score feedback. Our findings support our hypothesis, showing that reward led to slower learning but promoted generalization. On the other hand, punishment led to faster learning but impaired generalization. These behavioral differences may be due to different tendencies of movement variability in each group. The punishment group tended to explore more actively than the reward group during the initial learning phase, possibly due to loss aversion. In contrast, the reward group tended to explore more actively than the initial learning phase during the generalization test phase, seemingly recalling the strategy that led to the reward. These results suggest that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization, with important implications for practical applications such as sports training and motor rehabilitation.NEW & NOTEWORTHY Although reinforcement learning is a significant motor learning process, the mechanisms underlying how the brain learns movements through reward and punishment are not fully understood. We modified a well-established motor adaptation task and used savings (faster relearning) to measure generalization. We found reward led to slower learning but promoted generalization, whereas punishment led to faster learning but impaired generalization, suggesting that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization.

Memory reactivations during sleep: a neural basis of dream experiences?

Newly encoded memory traces are spontaneously reactivated during sleep. Since their discovery in the 1990s, these memory reactivations have been discussed as a potential neural basis for dream experiences. New results from animal and human research, as well as from the rapidly growing field of sleep and dream engineering, provide essential insights into this question, and reveal both strong parallels and disparities between the two phenomena. We suggest that, although memory reactivations may contribute to subjective experiences across different states of consciousness, they are not likely to be the primary neural basis of dreaming. We identify important limitations in current research paradigms and suggest novel strategies to address this question empirically.

Future-relevant memories are not selectively strengthened during sleep

Overnight consolidation processes are thought to operate in a selective manner, such that important (i.e. future-relevant) memories are strengthened ahead of irrelevant information. Using an online protocol, we sought to replicate the seminal finding that the memory benefits of sleep are enhanced when people expect a future test [Wilhelm et al., 2011]. Participants memorised verbal paired associates to a criterion of 60 percent (Experiment 1) or 40 percent correct (Experiment 2) before a 12-hour delay containing overnight sleep (sleep group) or daytime wakefulness (wake group). Critically, half of the participants were informed that they would be tested again the following day, whereas the other half were told that they would carry out a different set of tasks. We observed a robust memory benefit of overnight consolidation, with the sleep group outperforming the wake group in both experiments. However, knowledge of an upcoming test had no impact on sleep-associated consolidation in either experiment, suggesting that overnight memory processes were not enhanced for future-relevant information. These findings, together with other failed replication attempts, show that sleep does not provide selective support to memories that are deemed relevant for the future.

Punishments and rewards both modestly impair visuomotor memory retention

While the effects of rewards on memory appear well documented, the effects of punishments remain uncertain. Based on neuroimaging data, this study tested the hypothesis that, as compared to a neutral condition, a context allowing successful punishment avoidance would enhance memory to a similar extent as rewards. In a fully within-subject and counter-balanced design, participants (n = 18) took part in 3 distinct learning sessions during which the delivery of performance-contingent monetary punishments and rewards was manipulated. Specifically, participants had to reach towards visual targets while compensating for a gradually introduced visual deviation. Accuracy at achieving targets was either punished (Hit: "+0$"; Miss: "-0.5$), rewarded (Hit: "+0.5$"; Miss: "-0$"), or associated with neutral binary feedback (Hit: "Hit"; Miss: "Miss"). Retention was assessed through reach aftereffects both immediately and 24 h after initial acquisition. The results disconfirmed the hypothesis by showing that the punishment and reward learning sessions both impaired retention as compared to the neutral session, suggesting that both types of incentives similarly impaired memory formation and consolidation. Two alternative but complementary interpretations are discussed. One interpretation is that the presence of punishments and rewards induced a negative learning context, which - based on neurobiological data - could have been sufficient to interfere with memory formation and consolidation. Another interpretation is that punishments and rewards emphasized the disrupting effects of target hits on implicit learning processes, therefore yielding retention impairments. Altogether, these results suggest that incentives can have counter-productive effects on memory.

Whole-body procedural learning benefits from targeted memory reactivation in REM sleep and task-related dreaming

Sleep facilitates memory consolidation through offline reactivations of memory traces. Dreaming may play a role in memory improvement and may reflect these memory reactivations. To experimentally address this question, we used targeted memory reactivation (TMR), i.e., application, during sleep, of a stimulus that was previously associated with learning, to assess whether it influences task-related dream imagery (or task-dream reactivations). Specifically, we asked if TMR or task-dream reactivations in either slow-wave (SWS) or rapid eye movement (REM) sleep benefit whole-body procedural learning. Healthy participants completed a virtual reality (VR) flying task prior to and following a morning nap or rest period during which task-associated tones were readministered in either SWS, REM sleep, wake or not at all. Findings indicate that learning benefits most from TMR when applied in REM sleep compared to a Control-sleep group. REM dreams that reactivated kinesthetic elements of the VR task (e.g., flying, accelerating) were also associated with higher improvement on the task than were dreams that reactivated visual elements (e.g., landscapes) or that had no reactivations. TMR did not itself influence dream content but its effects on performance were greater when coexisting with task-dream reactivations in REM sleep. Findings may help explain the mechanistic relationships between dream and memory reactivations and may contribute to the development of sleep-based methods to optimize complex skill learning.

Does Sleep Selectively Strengthen Certain Memories Over Others Based on Emotion and Perceived Future Relevance?

Sleep has been found to have a beneficial effect on memory consolidation. It has furthermore frequently been suggested that sleep does not strengthen all memories equally. The first aim of this review paper was to examine whether sleep selectively strengthens emotional declarative memories more than neutral ones. We examined this first by reviewing the literature focusing on sleep/wake contrasts, and then the literature on whether any specific factors during sleep preferentially benefit emotional memories, with a special focus on the often-suggested claim that rapid eye movement sleep primarily consolidates emotional memories. A second aim was to examine if sleep preferentially benefits memories based on other cues of future relevance such as reward, test-expectancy or different instructions during encoding. Once again, we first focused on studies comparing sleep and wake groups, and then on studies examining the contributions of specific factors during sleep (for each future relevance paradigm, respectively). The review revealed that although some support exists that sleep is more beneficial for certain kinds of memories based on emotion or other cues of future relevance, the majority of studies does not support such an effect. Regarding specific factors during sleep, our review revealed that no sleep variable has reliably been found to be specifically associated with the consolidation of certain kinds of memories over others based on emotion or other cues of future relevance.

Neurochemical mechanisms for memory processing during sleep: basic findings in humans and neuropsychiatric implications

Sleep is essential for memory formation. Active systems consolidation maintains that memory traces that are initially stored in a transient store such as the hippocampus are gradually redistributed towards more permanent storage sites such as the cortex during sleep replay. The complementary synaptic homeostasis theory posits that weak memory traces are erased during sleep through a competitive down-selection mechanism, ensuring the brain's capability to learn new information. We discuss evidence from neuropharmacological experiments in humans to show how major neurotransmitters and neuromodulators are implicated in these memory processes. As to the major excitatory neurotransmitter glutamate that plays a prominent role in inducing synaptic consolidation, we show that these processes, while strengthening cortical memory traces during sleep, are insufficient to explain the consolidation of hippocampus-dependent declarative memories. In the inhibitory GABAergic system, we will offer insights how drugs may alter the intricate interplay of sleep oscillations that have been identified to be crucial for strengthening memories during sleep. Regarding the dopaminergic reward system, we will show how it is engaged during sleep replay, but that dopaminergic neuromodulation likely plays a side role for enhancing relevant memories during sleep. Also, we briefly go into basic evidence on acetylcholine and cortisol whose low tone during slow wave sleep (SWS) is crucial in supporting hippocampal-to-neocortical memory transmission. Finally, we will outline how these insights can be used to improve treatment of neuropsychiatric disorders focusing mainly on anxiety disorders, depression, and addiction that are strongly related to memory processing.

Incorporation of fragmented visuo-olfactory episodic memory into dreams and its association with memory performance

The question of a possible link between dream content and memory consolidation remains open. After a comprehensive review of the literature, we present novel findings from an experiment testing whether the incorporation of recently learned stimuli into dream reports is associated with improved post-sleep memory performance. Thirty-two high dream recallers freely explored new visuo-olfactory episodes for 3 consecutive days. During the nights following each non-explicit encoding, participants wore a wrist actimeter, and woke up at 5am and their usual waking time to record their dreams (intensity of all oneiric sensory perception was assessed using scales). A total of 120 dreams were reported and elements related to the encoding phase were identified in 37 of them, either learning-related (mainly visual- and rarely olfactory-related elements), or experiment-related (lab- or experimenters-related elements). On the 4^th day, we found that participants with learning-related (n = 16) and participants with learning-related and/or experiment-related dreams (n = 21) had similar odor recognition and odor-evoked episodic memory with the other participants. However, they had significantly better visuo-spatial memory of the episodes in comparison to the other participants. Our results support the hypothesis that the learning phase is loosely incorporated into dreams and that this incorporation is associated with sleep related memory consolidation.

Dreaming of a learning task is associated with enhanced memory consolidation: Replication in an overnight sleep study

Sleep following learning benefits memory. One model attributes this effect to the iterative "reactivation" of memory traces in the sleeping brain, demonstrated in animal models. Although technical limitations prohibit using the same methods to observe memory reactivation in the human brain, the study of mental activity during sleep provides an alternative method of observing memory activation during sleep. In fact, the content of dream experience may reflect the process of memory reactivation and consolidation in the sleeping brain. In line with this hypothesis, we previously reported that dreaming about a spatial learning task during a nap strongly predicts subsequent performance improvements. Here, we replicate this observation in an overnight sleep study, for the first time demonstrating that pre-sleep training on a virtual maze navigation task is reflected in dreams reported from all phases of sleep, with unambiguous representation of the task in dream content associated with improved next-morning performance. These observations are consistent with reactivation-based models of memory consolidation in sleep, confirming our earlier finding that the cognitive-level activation of recent experience during sleep is associated with subsequent performance gains.

Sculpting memory during sleep: concurrent consolidation and forgetting

There is compelling evidence that sleep actively supports the formation of long-lasting memory representations. Experimental cuing of memories proved that neural replay of representations during sleep plays a causal role for this consolidation, which has also been shown to promote neocortical synaptic plasticity and spine formation. Concurrently, sleep has been proposed to facilitate forgetting through processes of synaptic renormalisation. This view received indirect support by findings in humans of sleep enhancing TMS-evoked plasticity and capabilities for encoding new information. First direct behavioural evidence of sleep inducing forgetting has only recently emerged after encoding large amounts of stimuli in adults. We propose forgetting complements sleep-dependent consolidation and facilitates gist abstraction especially at high memory loads, when reactivation-based consolidation reaches capacity limits.

Test Expectation Enhances Memory Consolidation across Both Sleep and Wake

Memory consolidation benefits from post-training sleep. However, recent studies suggest that sleep does not uniformly benefit all memory, but instead prioritizes information that is important to the individual. Here, we examined the effect of test expectation on memory consolidation across sleep and wakefulness. Following reports that information with strong “future relevance” is preferentially consolidated during sleep, we hypothesized that test expectation would enhance memory consolidation across a period of sleep, but not across wakefulness. To the contrary, we found that expectation of a future test enhanced memory for both spatial and motor learning, but that this effect was equivalent across both wake and sleep retention intervals. These observations differ from those of least two prior studies, and fail to support the hypothesis that the “future relevance” of learned material moderates its consolidation selectively during sleep.

Sleep enhances a spatially mediated generalization of learned values

Sleep is thought to play an important role in memory consolidation. Here we tested whether sleep alters the subjective value associated with objects located in spatial clusters that were navigated to in a large-scale virtual town. We found that sleep enhances a generalization of the value of high-value objects to the value of locally clustered objects, resulting in an impaired memory for the value of high-valued objects. Our results are consistent with (a) spatial context helping to bind items together in long-term memory and serve as a basis for generalizing across memories and (b) sleep mediating memory effects on salient/reward-related items.