Electrophysiological analysis of the role of novelty in the von Restorff effect

From silos to synergy: Integrating approaches to investigate the role of prior knowledge and expectations on episodic memory

Significant progress in the investigation of how prior knowledge influences episodic memory has been made using three sometimes isolated (but not mutually exclusive) approaches: strictly adult behavioral investigations, computational models, and investigations into the development of the system. Here we point out that these approaches are complementary, each approach informs and is informed by the other. Thus, a natural next step for research is to combine all three approaches to further our understanding of the role of prior knowledge in episodic memory. Here we use studies of memory for expectation-congruent and incongruent information from each of these often disparate approaches to illustrate how combining approaches can be used to test and revise theories from the other. This domain is particularly advantageous because it highlights important features of more general memory processes, further differentiates models of memory, and can shed light on developmental change in the memory system. We then present a case study to illustrate the progress that can be made from integrating all three approaches and highlight the need for more endeavors in this vein. As a first step, we also propose a new computational model of memory that takes into account behavioral and developmental factors that can influence prior knowledge and episodic memory interactions. This integrated approach has great potential for offering novel insights into the relationship between prior knowledge and episodic memory, and cognition more broadly.

Observing memory encoding while it unfolds: Functional interpretation and current debates regarding ERP subsequent memory effects

Our ability to remember the past depends on neural processes set in train in the moment an event is experienced. These processes can be studied by segregating brain activity according to whether an event is later remembered or forgotten. The present review integrates a large number of studies examining this differential brain activity, labeled subsequent memory effect (SME), with the ERP technique, into a functional organization and discusses routes for further research. Based on the reviewed literature, we suggest that memory encoding is implemented by multiple processes, typically reflected in three functionally different subcomponents of the ERP SME elicited by study stimuli, which presumably interact with preparatory SME activity preceding the to be encoded event. We argue that ERPs are a valuable method in the SME paradigm because they have a sufficiently high temporal resolution to disclose the subcomponents of encoding-related brain activity. Implications of the proposed functional organization for future studies using the SME procedure in basic and applied settings will be discussed.

A predictive account of how novelty influences declarative memory

A rich body of studies in the human and non-human literature has examined the question how novelty influences memory. For a variety of different stimuli, ranging from simple objects and words to vastly complex scenarios, the literature reports that novelty improves memory in some cases, but impairs memory in other cases. In recent attempts to reconcile these conflicting findings, novelty has been divided into different subtypes, such as relative versus absolute novelty, or stimulus versus contextual novelty. Nevertheless, a single overarching theory of novelty and memory has been difficult to attain, probably due to the complexities in the interactions among stimuli, environmental factors (e.g., spatial and temporal context) and level of prior knowledge (but see Duszkiewicz et al., 2019; Kafkas & Montaldi, 2018b; Schomaker & Meeter, 2015). Here we describe how a predictive coding framework might be able to shed new light on different types of novelty and how they affect declarative memory in humans. More precisely, we consider how prior expectations modulate the influence of novelty on encoding episodes into memory, e.g., in terms of surprise, and how novelty/surprise affect memory for surrounding information. By reviewing a range of behavioural findings and their possible underlying neurobiological mechanisms, we highlight where a predictive coding framework succeeds and where it appears to struggle.

Encoding-linked pupil response is modulated by expected and unexpected novelty: Implications for memory formation and neurotransmission

Whether a novel stimulus is expected or unexpected may have implications for the kind of ensuing encoding and the type of subsequent memory. Pupil response was used in the present study to explore the way expected and unexpected stimuli are encoded and whether encoding-linked pupil response is modulated by expectation. Participants first established a contingency relationship between a series of symbols and the type of stimulus (man-made or natural) that followed each one. At encoding, some of the target stimuli violated the previously established relationship (i.e., unexpected), while the majority conformed to this relationship (i.e., expected). Expectation at encoding had opposite effects on familiarity and recollection, the two types of memory that support recognition, and modulated differently the way pupil response predicted subsequent memory. Encoding of unexpected novel stimuli was associated with increased pupil dilation as a predictor of subsequent memory type and strength. In contrast, encoding of expected novel stimuli was associated with decreased pupil response (constriction), which was predictive of subsequent memory type and strength. The findings support the close link between pupil response and memory formation, but critically indicate that this is modulated by the type of novelty as defined by expectation. These novel findings have important implications for the encoding mechanisms involved when different types of novelty are detected and is proposed to indicate the operation of different neurotransmitters during memory formation.

A context-change account of temporal distinctiveness

The distinctiveness effect refers to the finding that items that stand out from other items in a learning set are more likely to be remembered later. Traditionally, distinctiveness has been defined based on item features; specifically, an item is deemed to be distinctive if its features are different from the features of other to-be-learned items. We propose that distinctiveness can be redefined based on context change-distinctive items are those with features that deviate from the others in the current temporal context, a recency-weighted running average of experience-and that this context change modulates learning. We test this account with two novel experiments and introduce a formal mathematical model that instantiates our proposed theory. In the experiments, participants studied lists of words, with each word appearing on one of two background colors. Within each list, each color was used for 50% of the words, but the sequence of the colors was controlled so that runs of the same color for that list were common in Experiment 1 and common, rare, or random in Experiment 2. In both experiments, participants' source memory for background color was enhanced for items where the color changed, especially if the change occurred after a stable run without color changes. Conversely, source memory was not significantly better for nonchanges after runs of alternating colors with each item. This pattern is inconsistent with theories of learning based on prediction error, but is consistent with our context-change account.

Effects of distinctiveness and memory on lateralized and unlateralized brain-electrical activity during visual word encoding

The present study investigated a recently introduced left-lateralized component in the event-related potential (ERP), the posterior semantic asymmetry (PSA), in the context of an isolation paradigm. The PSA is a relative negativity that is most pronounced at temporoparietal electrodes, peaks around 300 ms, and is assumed to reflect early semantic processing of visual words. A free-recall, word-list-learning paradigm was conducted. The learning list comprised two stimuli which were physically isolated from the other stimuli (by different font size or different typeface). The typical behavioral isolation effect with higher recall for isolated stimuli was observed. Furthermore, ERP effects of stimulus type and subsequent memory were analyzed. A left-lateralized negativity that matched the topography of the PSA but occurred somewhat later showed an effect of stimulus distinctiveness, with increased amplitudes for isolates, thus suggesting their deeper semantic processing. However, PSA amplitude did not predict subsequent recall. Unlateralized ERPs replicated previous findings of a greater late frontal positivity during elaborated encoding of both isolated stimuli and subsequently recalled stimuli. This recall effect was greater for isolated than standard stimuli. We argue that physical distinctiveness during encoding facilitates recall to the extent that it promotes the frontally-mediated processes that predict better recall in general.

Expectation affects learning and modulates memory experience at retrieval

Our ability to make predictions and monitor regularities has a profound impact on the way we perceive the environment, but the effect this mechanism has on memory is not well understood. In four experiments, we explored the effects on memory of the expectation status of information at encoding or at retrieval. In a rule-learning task participants learned a contingency relationship between 6 different symbols and the type of stimulus that followed each one. Either at encoding (Experiments 1a and 1b) or at retrieval (Experiments 2a and 2b), the established relationship was violated for a subset of stimuli resulting in the presentation of both expected and unexpected stimuli. The expectation status of the stimuli was found to have opposite effects on familiarity and recollection performance, the two kinds of memory that support recognition memory. At encoding (Experiments 1a and 1b), the presentation of expected stimuli selectively enhanced subsequent familiarity performance, while unexpected stimuli selectively enhanced subsequent recollection. Similarly, at retrieval (Experiments 2a and 2b), expected stimuli were more likely to be deemed familiar than unexpected stimuli, whereas unexpected stimuli were more likely to be recollected than were expected stimuli. These findings suggest that two separate memory enhancement mechanisms exist; one sensitive and modulating the accuracy of memory for the contextually distinctive or unexpected, and the other sensitive to and modulating the accuracy of memory for the expected. Therefore, the degree to which information fits with expectation has critical implications for the type of computational mechanism that will be engaged to support memory.

Novelty's effect on memory encoding

It is often thought that novelty benefits memory formation. However, support for this idea mostly comes from paradigms that are open to alternative explanations. In the present study we manipulated novelty in a word-learning task through task-irrelevant background images. These background images were either standard (presented repeatedly), or novel (presented only once). Two types of background images were used: Landscape pictures and fractals. EEG was also recorded during encoding. Contrary to the idea that novelty aids memory formation, memory performance was not affected by the novelty of the background. In the evoked response potentials, we found evidence of distracting effects of novelty: both the N1 and P3b components were smaller to words studied with novel backgrounds, and the amplitude of the N2b component correlated negatively with subsequent retrieval. We conclude that although evidence from other studies does suggest benefits on a longer time scale, novelty has no instantaneous benefits for learning.

Exploring a novel environment improves motivation and promotes recall of words

Active exploration of novel environments is known to increase plasticity in animals, promoting long-term potentiation in the hippocampus and enhancing memory formation. These effects can occur during as well as after exploration. In humans novelty’s effects on memory have been investigated with other methods, but never in an active exploration paradigm. We therefore investigated whether active spatial exploration of a novel compared to a previously familiarized virtual environment promotes performance on an unrelated word learning task. Exploration of the novel environment enhanced recall, generally thought to be hippocampus-dependent, but not recognition, believed to rely less on the hippocampus. Recall was better for participants that gave higher presence ratings for their experience in the virtual environment. These ratings were higher for the novel compared to the familiar virtual environment, suggesting that novelty increased attention for the virtual rather than real environment; however, this did not explain the effect of novelty on recall.

The detection of novelty relies on dopaminergic signaling: evidence from apomorphine's impact on the novelty N2

Despite much research, it remains unclear if dopamine is directly involved in novelty detection or plays a role in orchestrating the subsequent cognitive response. This ambiguity stems in part from a reliance on experimental designs where novelty is manipulated and dopaminergic activity is subsequently observed. Here we adopt the alternative approach: we manipulate dopamine activity using apomorphine (D1/D2 agonist) and measure the change in neurological indices of novelty processing. In separate drug and placebo sessions, participants completed a von Restorff task. Apomorphine speeded and potentiated the novelty-elicited N2, an Event-Related Potential (ERP) component thought to index early aspects of novelty detection, and caused novel-font words to be better recalled. Apomorphine also decreased the amplitude of the novelty-P3a. An increase in D1/D2 receptor activation thus appears to potentiate neural sensitivity to novel stimuli, causing this content to be better encoded.