Multiple ways to the prior occurrence of an event: an electrophysiological dissociation of experimental and conceptually driven familiarity in recognition memory

Schema-congruency supports the formation of unitized representations: Evidence from event-related potentials

The main goal of the present study was to investigate whether schema-based encoding of novel word pairs (i.e., novel compound words) supports the formation of unitized representations and thus, associative familiarity-based recognition. We report two experiments that both comprise an incidental learning task, in which novel noun-noun compound words were presented in semantically congruent contexts, enabling schema-supported processing of both constituents, contrasted with a schema-neutral condition. In Experiment 1, the effects of schema congruency on memory performance were larger for associative memory performance than for item memory performance in a memory test in which intact, recombined, and new compound words had to be discriminated. This supports the view that schema congruency boosts associative memory by promoting unitization. When contrasting event-related potentials (ERPs) for hits with correct rejections or associative misses, an N400 attenuation effect (520-676 ms) indicating absolute familiarity was present in the congruent condition, but not in the neutral condition. In line with this, a direct comparison of ERPs on hits across conditions revealed more positive waveforms in the congruent than in the neutral condition. This suggests that absolute familiarity contributes to associative recognition memory when schema-supported processing is established. In Experiment 2, we tested whether schema congruency enables the formation of semantically overlapping representations. Therefore, we included semantically similar lure compound words in the test phase and compared false alarm rates to these lures across conditions. In line with our hypothesis, we found higher false alarm rates in the congruent as compared to the neutral condition. In conclusion, we provide converging evidence for the view that schema congruency enables the formation of unitized representations and supports familiarity-based memory retrieval.

Differentiation of the Contribution of Familiarity and Recollection to the Old/New Effects in Associative Recognition: Insight from Semantic Relation

Previous research has revealed two different old/new effects, the early mid-frontal old/new effect (a.k.a., FN400) and the late parietal old/new effect (a.k.a., LPC), which relate to familiarity and recollection processes, respectively. Although associative recognition is thought to be more based on recollection, recent studies have confirmed that familiarity can make a great contribution when the items of a pair are unitized. However, it remains unclear whether the old/new effects are sensitive to the nature of different semantic relations. The current ERP (event-related potentials) study aimed to address this, where picture pairs of thematic, taxonomic, and unrelated relations served as stimuli and participants were required to discriminate the pair type: intact, rearranged, “old + new”, or new. We confirmed both FN400 and LPC. Our findings, by comparing the occurrence and the amplitudes of these two components, implicate that the neural activity of associative recognition is sensitive to the semantic relation of stimuli and depends more on stimulus properties, that the familiarity of a single item can impact the neural activities in discriminating associative pairs, and that the interval length between encoding and test modulates the familiarity of unrelated pairs. In addition, the dissociation between FN400 and LPC reinforces the dual-process models.

The more you know: Schema-congruency supports associative encoding of novel compound words. Evidence from event-related potentials

We aimed to investigate the neurocognitive mechanisms of event congruency with prior (schema) knowledge for the learning of novel compound words. Event-related potentials (ERPs) were recorded during an incidental learning task, in which novel noun-noun compounds were presented in a semantically congruent context, enabling schema-supported processing, or in a neutral context. As expected, associative memory performance was better for compounds preceded by a congruent context. Although the N400 was attenuated in the congruent condition, subsequent memory effects (SMEs) in the N400 time interval did not differ across conditions, suggesting that the processes reflected in the N400 cannot account for the memory advantage in the congruent condition. However, a parietal SME was obtained for compounds preceded by a congruent context, only, which we interpret as reflecting the schema-supported formation of a conceptual compound representation. A late frontal SME was obtained in both conditions, presumably reflecting the more general inter-item associative encoding of compound constituents.

The N400 indexes acquisition of novel emotion concepts via conceptual combination

The ability to learn new emotion concepts is adaptive and socially valuable as it communicates culturally held understandings about values, goals, and experiences. Yet, little work has examined the underlying mechanisms that allow for new emotion concepts and words to be integrated into the conceptual system. One such mechanism may be conceptual combination, or the ability to form novel concepts by dynamically combining previously acquired conceptual knowledge. In this study, we used event-related potentials (ERPs) to investigate the electrophysiological correlates of novel emotion concept acquisition via conceptual combination. Participants were briefly trained on 30 novel emotion combinations, each consisting of two English emotion words (the components; e.g., "sadness + fatigue") and a pseudoword (the target; e.g., "despip"). Participants then completed a semantic congruency task while ERPs were recorded. On each trial, two components were presented serially, followed by a target; participants judged whether the target was a valid combination of the preceding components. Targets could be correct or incorrect trained pseudowords, or new untrained pseudowords. Furthermore, components could be presented in reversed order (e.g., "fatigue" then "sadness") or as synonyms (e.g., "exhaustion" for "fatigue"). Consistent with our main hypotheses, we found a main effect of target, such that the correct combinations showed reduced N400 amplitudes when compared to both incorrect and untrained pseudowords. Critically, this effect held regardless of how the preceding components were presented, suggesting deeper semantic learning. These results extend prior findings on conceptual combination and novel word learning, and are congruent with predictive processing accounts of brain function.

Usefulness of familiarity signals during recognition depends on test format: Neurocognitive evidence for a core assumption of the CLS framework

Familiarity-based discrimination between studied items and similar foils in yes/no recognition memory tests is relatively poor. The complementary learning systems (CLS) framework explains this with the small difference in familiarity strength between targets and foils. The framework, however, also predicts that familiarity values of targets and corresponding similar foils are directly comparable - as long as they are presented side by side in a forced-choice corresponding (FCC) test. This is because in each trial, targets tend to be more familiar than their corresponding foils. In contrast, when forced-choice displays contain non-corresponding foils (FCNC) which are similar to other studied items, familiarity values are not directly comparable (as in yes/no-tasks). In a recognition memory task with pictures of objects, we found that the putative ERP correlate of familiarity, the mid-frontal old/new effect for targets vs. foils, was significantly larger in FCC compared to FCNC displays. Moreover, single-trial target-foil amplitude differences predicted the accuracy of the recognition judgment. This study supports the assumption of the CLS framework that the test format can influence the diagnostic reliability of familiarity. Moreover, it implies that the mid-frontal old/new effect does not reflect the difference in the familiarity signal between studied and non-studied items but the task-adequate assessment of this signal.

Unitization modulates recognition of within-domain and cross-domain associations: Evidence from event-related potentials

Although it is often assumed that memory of episodic associations requires recollection, it has been suggested that, when stimuli are experienced as a unit, earlier memory processes might contribute to their subsequent associative recognition. We investigated the effects of associative relations and perceptual domain during episodic encoding on the ability to utilize early memory processes to retrieve associative information. During the study phase, participants encoded compound and noncompound words pairs, presented either to the same sensory modality (visual presentation) or to different sensory modalities (audiovisual presentation). At the test phase, they discriminated between old, rearranged, and new pairs while ERPs were recorded. In an early ERP component, differences related to associative memory emerged only for compounds, regardless of their encoding modality. These findings indicate that episodic retrieval of compound words can be supported by early-onset recognition processes regardless of whether both words were presented to the same or different sensory modalities, and suggests that unitization can operate at an abstract level, across a broad range of materials.

The discovery of processing stages: Extension of Sternberg's method

We introduce a method for measuring the number and durations of processing stages from the electroencephalographic signal and apply it to the study of associative recognition. Using an extension of past research that combines multivariate pattern analysis with hidden semi-Markov models, the approach identifies on a trial-by-trial basis where brief sinusoidal peaks (called bumps) are added to the ongoing electroencephalographic signal. We propose that these bumps mark the onset of critical cognitive stages in processing. The results of the analysis can be used to guide the development of detailed process models. Applied to the associative recognition task, the hidden semi-Markov models multivariate pattern analysis method indicates that the effects of associative strength and probe type are localized to a memory retrieval stage and a decision stage. This is in line with a previously developed the adaptive control of thought-rational process model, called ACT-R, of the task. As a test of the generalization of our method we also apply it to a data set on the Sternberg working memory task collected by Jacobs, Hwang, Curran, and Kahana (2006). The analysis generalizes robustly, and localizes the typical set size effect in a late comparison/decision stage. In addition to providing information about the number and durations of stages in associative recognition, our analysis sheds light on the event-related potential components implicated in the study of recognition memory. (PsycINFO Database Record

Electrophysiological evidence for the effects of unitization on associative recognition memory in older adults

Normal aging is associated with greater decline in associative memory relative to item memory due to impaired recollection. Familiarity may also contribute to associative recognition when stimuli are perceived as a 'unitized' representation. Given that familiarity is relatively preserved in older adults, we explored whether age-related associative memory deficits could be attenuated when associations were unitized (i.e., compounds) compared with those non-unitized (i.e., unrelated word pairs). Young and older adults performed an associative recognition task while electroencephalogram (EEG) was recorded. Behavioral results showed that age differences were smaller for recognition of compounds than for unrelated word pairs. ERP results indicated that only compounds evoked an early frontal old/new effect in older adults. Moreover, the early frontal old/new effect was positively correlated with associative discrimination accuracy. These findings suggest that reduced age-related associative deficits under unitized condition may be associated with the presence of familiarity-based retrieval of compounds in older adults.

Sometimes we have to intentionally focus on the details: Incidental encoding and perceptual change decrease recognition memory performance and the ERP correlate of recollection

Prior studies suggest that memory retrieval is based on two independent processes: Recollection and familiarity. Here, we investigated the role of incidental and intentional encoding, and specifically whether perceptual changes between study and test affects behavioral and electrophysiological correlates of both retrieval processes. During retrieval, participants distinguished between identical and changed exemplars as well as novel distractors. Following incidental encoding, participants had difficulty identifying changed exemplars; item and feature recognition increased after intentional encoding, in particular for changed exemplars. Reflecting this increase in memory performance, the ERP correlate of recollection was larger after intentional encoding and for identical item repetitions, whereas the ERP correlate for familiarity was largely unaffected. Pre-response old/new effects corresponding to later aspects of recollection (700-1000 ms relative to stimulus onset) were larger in response-compared to stimulus-locked averages, but also of similar magnitude for identical and changed exemplars. These results corroborate previous findings suggesting that the electrophysiological signature of recollection is modulated as a function of memory performance. The role of task characteristics and material retrieved from memory for modulations in familiarity-based retrieval processes is discussed.

Associative recognition processes are modulated by the semantic unitizability of memoranda

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity processes might contribute to their subsequent associative recognition. To investigate the effect of semantic relatedness during episodic encoding on the processes of retrieval of associative information, we had participants interactively encode pairs of object pictures, vertically arranged so as to suggest a functional or configural relationship between them. Half the pairs were independently judged to be of related objects (e.g., a lamp over a table) and half of unrelated objects (e.g., a key-ring over an apple). At test, participants discriminated between intact, recombined, and new pairs while event related potentials (ERPs) were recorded. In an early ERP marker of retrieval success generally associated with familiarity processes, differences related to associative memory only emerged for related pairs, while differences associated with item memory emerged for both related and unrelated pairs. In contrast, in a later ERP effect associated with recollection, differences related to associative memory emerged for both related and unrelated pairs. These findings may indicate that retrieval of episodic associations formed between two semantically related visual stimuli can be supported by familiarity-related processes.

Associative Recognition Processes Are Modulated by Modality Relations

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity-related processes might contribute to their subsequent associative recognition. Furthermore, intradomain associations are believed to be unitized more readily than interdomain associations. To assess these claims, we tested associative recognition following two types of pair associate learning. In the unimodal task, stimulus pairs were pictures of common objects, whereas in the cross-modal task, stimulus pairs consisted of an object picture and an unrelated environmental sound. At test, participants discriminated intact from recombined pairs while ERPs were recorded. In the unimodal task only, associative recognition was accompanied by a robust frontal deflection reminiscent of a component commonly interpreted as related to familiarity processes. In contrast, ERP correlates of associative recognition observed at more posterior sites, akin to a component that has been related to recollection, were apparent in both tasks. These findings indicate that retrieval of unimodal associations can be supported by familiarity-related processes that are dissociable from recollective processes required for the retrieval of cross-modal associations.

First come, last primed: FN400 reflects post-encoding editing of the memory trace

Explicit associative memory relies on different neural substrates depending on similarity of the items associated. Extant literature dissociates old/new effects elicited by homogeneous and heterogeneous pairs. However, current results scarcely address potential priming effects induced by the presentation sequence. In the present ERP study, participants learned associations between two pictures, which either belonged to the same semantic category (e.g., animal-animal), or to different categories (e.g., animal-scene). Pictures forming a pair were shown sequentially, allowing for investigation of the different neural processes related to presentation of the first and the second item. After the study phase, participants performed a recognition judgment. After recognition, participants were asked to recall the associated picture. During retrieval, between 260 and 350 ms post-stimulus there was a significant frontal effect of category (i.e., same-category versus different-category), but only for items shown first within a pair. In the 350-600 ms time window the parietal old/new effect was unaffected by semantic category, but was modulated by presentation order. Exploratory analyses revealed even earlier effects in the time windows 40-90 ms and 150-200 ms. This evidence supports the priming account of the FN400 and highlights the importance of sequence effects in electrophysiological activity during episodic retrieval.

Semantic relations differentially impact associative recognition memory: electrophysiological evidence

Though associative recognition memory is thought to rely primarily on recollection, recent research indicates that familiarity might also make a substantial contribution when to-be-learned items are integrated into a coherent structure by means of an existing semantic relation. It remains unclear how different types of semantic relations, such as categorical (e.g., dancer-singer) and thematic (e.g., dancer-stage) relations might affect associative recognition, however. Using event-related potentials (ERPs), we addressed this question by manipulating the type of semantic link between paired words in an associative recognition memory experiment. An early midfrontal old/new effect, typically linked to familiarity, was observed across the relation types. In contrast, a robust left parietal old/new effect was found in the categorical condition only, suggesting a clear contribution of recollection to associative recognition for this kind of pairs. One interpretation of this pattern is that familiarity was sufficiently diagnostic for associative recognition of thematic relations, which could result from the integrative nature of the thematic relatedness compared to the similarity-based nature of categorical pairs. The present study suggests that the extent to which recollection and familiarity are involved in associative recognition is at least in part determined by the properties of semantic relations between the paired associates.

Memory asymmetry of forward and backward associations in recognition tasks

There is an intensive debate on whether memory for serial order is symmetric. The objective of this study was to explore whether associative asymmetry is modulated by memory task (recognition vs. cued recall). Participants were asked to memorize word triples (Experiments 1-2) or pairs (Experiments 3-6) during the study phase. They then recalled the word by a cue during a cued recall task (Experiments 1-4) and judged whether the presented 2 words were in the same or in a different order compared with the study phase during a recognition task (Experiments 1-6). To control for perceptual matching between the study and test phase, participants were presented with vertical test pairs when they made directional judgment in Experiment 5. In Experiment 6, participants also made associative recognition judgments for word pairs presented at the same or the reversed position. The results showed that forward associations were recalled at similar levels as backward associations and that the correlations between forward and backward associations were high in the cued recall tasks. On the other hand, the direction of forward associations was recognized more accurately (and more quickly) than backward associations, and their correlations were comparable to the control condition in the recognition tasks. This forward advantage was also obtained for the associative recognition task. Diminishing positional information did not change the pattern of associative asymmetry. These results suggest that associative asymmetry is modulated by cued recall and recognition manipulations and that direction as a constituent part of a memory trace can facilitate associative memory.

Neurophysiological evidence for a recollection impairment in amnesia patients that leaves familiarity intact

In several previous behavioral studies, we have identified a group of amnestic patients that, behaviorally, appear to exhibit severe deficits in recollection with relative preservation of familiarity-based recognition. However, these studies have relied exclusively on behavioral measures, rather than direct measures of physiology. Event-related potentials (ERPs) have been used to identify putative neural correlates of familiarity- and recollection-based recognition memory, but little work has been done to determine the extent to which these ERP correlates are spared in patients with relatively specific memory disorders. ERP studies of recognition in healthy subjects have indicated that recollection and familiarity are related to a parietal old-new effect characterized as a late positive component (LPC) and an earlier mid-frontal old-new effect referred to as an 'FN400', respectively. Here, we sought to determine the extent to which the putative ERP correlates of recollection and familiarity are intact or impaired in these patients. We recorded ERPs in three amnestic patients and six age matched controls while they made item recognition and source recognition judgments. The current patients were able to discriminate between old and new items fairly well, but showed nearly chance-level performance at source recognition. Moreover, whereas control subjects exhibited ERP correlates of memory that have been linked to recollection and familiarity, the patients only exhibited the mid-frontal FN400 ERP effect related to familiarity-based recognition. The results show that recollection can be severely impaired in amnesia even when familiarity-related processing is relatively spared, and they also provide further evidence that ERPs can be used to distinguish between neural correlates of familiarity and recollection.

Examining ERP correlates of recognition memory: evidence of accurate source recognition without recollection

Recollection is typically associated with high recognition confidence and accurate source memory. However, subjects sometimes make accurate source memory judgments even for items that are not confidently recognized, and it is not known whether these responses are based on recollection or some other memory process. In the current study, we measured event related potentials (ERPs) while subjects made item and source memory confidence judgments in order to determine whether recollection supported accurate source recognition responses for items that were not confidently recognized. In line with previous studies, we found that recognition memory was associated with two ERP effects: an early on-setting FN400 effect, and a later parietal old-new effect [late positive component (LPC)], which have been associated with familiarity and recollection, respectively. The FN400 increased gradually with item recognition confidence, whereas the LPC was only observed for highly confident recognition responses. The LPC was also related to source accuracy, but only for items that had received a high confidence item recognition response; accurate source judgments to items that were less confidently recognized did not exhibit the typical ERP correlate of recollection or familiarity, but rather showed a late, broadly distributed negative ERP difference. The results indicate that accurate source judgments of episodic context can occur even when recollection fails.