Unitization mitigates interference by intrinsic negative emotion in familiarity and recollection of associative memory: Electrophysiological evidence

Improving associative memory in younger and older adults with unitization: evidence from meta-analysis and behavioral studies

Introduction

The finding that familiarity can support associative memory by unitizing the to -be-learned items into a novel representation has been widely accepted, but its effects on overall performance of associative memory and recollection are still controversial.

Methods

The current study aims to elucidate these discrepancies by identifying potential moderating factors through a combined approach of meta-analysis and behavioral experiment.

Results

Results consistently showed that changes in the level of unitization and age groups were two important moderators. Specifically, unitization enhanced younger and older adults' associative memory and its supporting processes (i.e., familiarity and recollection) when the level of unitization between studied and rearranged pairs was changed. However, when this level remained constant, unitization exhibited no impact on associative memory and familiarity in younger adults, but showed an enhanced effect in older adults. Furthermore, results revealed a marked group difference between younger and older adults in associative memory when the unitization level of noncompound words remained unaltered. Upon breaking this condition, the group difference was reduced by enhancing familiarity or recollection.

Discussion

These findings not only clarify some of the inconsistencies in the literature concerning the impact of unitization on associative memory, but also suggest that unitization is a beneficial strategy for reducing group difference in associative memory, with its effectiveness varying according to the level of unitization changes.

The effect of target detection task on memory encoding varies in different stimulus onset asynchronies

The attentional boost effect (ABE) and action-induced memory enhancement (AIME) suggest that memory performance for target-paired items is superior to that for distractor-paired items when participants performed a target detection task and a memory encoding task simultaneously. Though the memory enhancement has been well established, the temporal dynamics of how the target detection task influenced memory encoding remains unclear. To investigate this, we manipulated the stimulus onset asynchrony (SOA) between detection stimuli and the words to be memorized using a remember/know study-test paradigm, and we focused primarily on memory performance for the words that appeared after the detection response. The results showed that target-paired memory enhancement was robust from SOA = 0 s to SOA = 0.75 s, but was not significant when examined by itself in Experiment 1A or weakened in Experiment 2 and the conjoint analysis when SOA = 1 s, which were only observed in R responses. The post-response memory enhancement still existed when there was no temporal overlap between the word and target, similar to the magnitude of memory enhancement observed with temporal overlap. These results supported the view that target-paired memory enhancement (recollection rather than familiarity) occurred irrespective of whether the items appeared simultaneously with the targets or within a short period after the response, and the temporal overlap of the word and target was not necessary for post-response memory enhancement.

The effect of part-list cuing on associative recognition

The modulation of part-list cuing on item memory has been well-documented, whereas its impact on associative memory remains largely unknown. The present study explored the effect of part-list cuing on associative recognition and, more specifically, whether this forgetting effect caused by part-list cuing is more sensitive to recollection or familiarity in recognition memory. Experiments 1a and 1b combined the intact/rearranged/new judgement task of associative recognition with the classical part-list cuing paradigm, and the result showed that part-list cuing impaired the recognition accuracy of "intact" and "rearranged" face-scene pairs. Moreover, the discriminability score of relational recognition and item recognition was significantly decreased in the part-list cuing condition compared to the no-part-list cuing condition. Experiments 2a and 2b further used the Remember/Know/Guess task to explore which recognition processes (recollection vs. familiarity) were sensitive to the presentation of part-list cuing. The results showed that part-list cuing reduced the familiarity of relational recognition and the recollection and familiarity of item recognition. These findings suggest that part-list cuing was harmful to the recognition of relationships (familiarity) and items (recollection and familiarity) in associative memory.

The beneficial effects of concept definition and interactive imagery tasks on associative memory: Evidence from event-related potentials

It is widely accepted that familiarity can support associative memory when the to-be-remember items are unitized into a new representation. However, there has been relatively little attention devoted to investigating the effects of different unitization manipulations on associative memory. The present study aimed to address this gap by examining the effects of varying levels of unitization through three tasks: Concept definition, interactive imagery, and sentence frame tasks. The behavioral results revealed that associative memory was significantly enhanced in the interactive imagery task compared to the sentence frame task. However, no significant differences were found between the sentence frame and concept definition tasks, or between the concept definition and interactive imagery tasks. In terms of the neural correlates, the event-related potential (ERP) results revealed that the sentence frame task only elicited a significant recollection-related LPC old/new effect, while the concept definition task only elicited a significant familiarity-related FN400 old/new effect. However, the interactive representation task elicited both of these distinct effects. These findings suggest that both the concept definition and interactive imagery tasks can enhance familiarity for supporting associative memory, but their beneficial effects on associative memory or LPC old/new effects may be different.

A meta-analysis of event-related potential correlates of recognition memory

A longstanding question in memory research is whether recognition is supported by more than one mnemonic process. Dual-process models distinguish recollection of episodic detail from familiarity, while single-process models explain recognition in terms of one process that varies in strength. Dual process models have drawn support from findings that recollection and familiarity elicit distinct electroencephalographic event-related potentials (ERPs): a mid-frontal ERP effect that occurs at around 300-500 ms post-stimulus onset and is often larger for familiarity than recollection contrasts, and a parietal ERP effect that occurs at around 500-800 ms and is larger for recollection than familiarity contrasts. We sought to adjudicate between dual- and single-process models by investigating whether the dissociation between these two ERP effects is reliable over studies. We extracted effect sizes from 41 experiments that had used Remember-Know, source memory, and associative memory paradigms (1,000 participants). Meta-analysis revealed a strong interaction between ERP effect and mnemonic process of the form predicted by dual-process models. Although neither ERP effect was significantly process-selective taken alone, a moderator analysis revealed a larger mid-frontal effect for familiarity than recollection contrasts in studies using the Remember-Know paradigm. Mega-analysis of raw data from six studies further showed significant process-selectivity for both mid-frontal and parietal ERPs in the predicted time windows. On balance, the findings favor dual- over single-process theories of recognition memory, but point to a need to promote sharing of raw data.

Are multiple types of associative memory differently impacted by emotion?

The effect of emotion on associative memory is still an open question. Our aim was to test whether discrepant findings are due to differential impact of emotion on different types of associative memory or to differences in the way participants encoded stimuli across studies. We examined the effect of negative content on multiple forms of associative memory, using the same encoding task. Two registered experiments were conducted in parallel with random allocation of participants to experiments. Each experiment included 4 encoding blocks, in which participants read a neutral text comprised of 6 paragraphs, which were interleaved with neutral or negative images. Images were controlled for visual properties and semantic similarity. Memory tests included recognition memory, Remember/Know, order memory, temporal source memory and contextual memory. Analyses showed that emotion decreased contextual memory but not order memory or temporal source memory. We also found that temporal source memory and contextual memory were correlated. Recognition accuracy and subjective recollection were not impacted by emotion. In agreement with previous work, participants self-reported a reduced ability to integrate blocks containing negative images with paragraphs. In contrast to our hypothesis, results suggest that emotion does not impact all types of associative memory when stimuli are controlled.

The Effects of Reward on Associative Memory Depend on Unitization Depths

Previous studies have found that reward effect is stronger for more difficult to retrieve items, but whether this effect holds true for the associative memory remains unclear too. We investigated the effects and neural mechanisms of the different unitization depths and reward sets on encoding associative memory using event-related potentials (ERPs), which were recorded through a Neuroscan system with a 64-channel electrode cap according to the international 10-20 system, and five electrodes (Fz, FCz, Cz, CPz, and Pz) were selected for analysis. Thirty healthy college students took part in this study. During encoding, participants were carried out two encoding tasks, a congruity-judgment task with high unitization and a color-judgment task with low unitization, with half of the items rewarded. The test phase was conducted immediately after the encoding phase. The results for false alarm rates and Prs (i.e., hit rates for old pairs minus false alarm rates for new pairs) in relational retrieval revealed that the reward differences in the color-judgment task were greater than those in the congruity-judgment task. The ERP results further showed significant reward effects (i.e., the reward significantly improved the average amplitudes compared to no reward) at P300 (300-500 ms) and LPP (500-800 ms) in the color-judgment task both for intact and rearranged items, and the reward effects at LPP (electrodes Fz, FCz, Cz, CPz, and Pz) were distributed more widely than the reward effects at P300 (electrodes Fz and FCz) in the color-judgment task. These results suggest that reward provided a greater boost when retrieving associative memory of low unitized items.

The different effects of concept definition and interactive imagery encoding on associative recognition for word and picture stimuli

Traditional view holds that associative recognition require recollection while familiarity can't support associative recognition. However, recent research indicate that familiarity can also contribute to associative recognition when the stimuli are unitized in encoding. Here, we investigated the electrophysiological correlates of retrieval of word and picture stimuli in three encoding conditions. Semantically unrelated word pairs or picture pairs were encoded in concept definition, interactive imagery, and item comparison conditions, separately. In test, the participants were required to discriminate between old pairs that appeared in the same pairing as in study, rearranged pairs that appeared in different pairings in study, or completely new pairs. The behavioral results revealed that higher associative recognition was observed in interactive imagery condition than in concept definition condition, with item comparison condition eliciting the worst recognition, regardless of word or picture stimuli. ERP results of word stimuli revealed that the FN400 old/new effect was solely elicited in concept definition and interactive imagery conditions, but not in item comparison condition. However, ERP results of picture stimuli revealed that the late FN400 old/new effect was observed in three encoding conditions and that larger magnitude of old/new effect was elicited in item comparison condition than in interactive imagery condition. There may be different neural mechanisms of unitization on associative recognition for word and picture stimuli. These findings suggested that the pattern of engagement of familiarity during successful retrieval was dependent on the stimulating properties and the encoding conditions. We will discuss the possibility that top-down unitization which manipulates two unrelated stimuli through instructions may lead to the engagement of specific forms of familiarity-association familiarity and item familiarity.

Is color source retrieval sensitive to emotion? Electrophysiological evidence from old/new effects

It has been proved that item memory and source memory are two dissociable processes, as reflected by differential influence at behavioral and electrophysiological levels, the latter being evidenced by the ERP old/new effects. Specially for source memory, the retrieval of color source may be unique from recollecting other types of contextual information, which can be seen from the late posterior negativity (LPN). However, the mediation of emotional valence on the old/new effects for verbal stimuli encoded in colors remains unknown. Adopting words of three emotional valences (i.e., positive, neutral, and negative), with their displayed colors serving as sources, the current experiment aimed to explore the sensitivity of old/new effects to emotion for both item memory and source memory. Results demonstrated that: the FN400 that reflects familiarity was recorded and it was sensitive to emotional valence across both memory tasks; the mediation of emotional valence was absent in recollection-reflected LPC, neither for item memory nor for source memory; an association between LPN and color source retrieval was confirmed, with reliable amplitudes for neutral words but not for emotional words. These data were discussed in terms of the dual-process model and other accounts. Future research directions were recommended.

Unitization of internal and external features contributes to associative recognition for faces: Evidence from modulations of the FN400

Associative recognition requires discriminating between old items and conjunction lures constructed by recombining elements from two different study items. This task can be solved not only by recollection but also by familiarity if the to-be-remembered stimuli are perceived as a unitized representation. In two event-related potential (ERP) studies, we provide evidence for the integration of internal and external facial features by showing that the early frontal old-new effect (considered a correlate of familiarity) is modulated by the specific combination of facial features. Participants studied faces consisting of internal features (eyes, eyebrows, nose, and mouth) paired with external features (hair, head shape, and ears). During the testing phase, intact, recombined, and new faces were presented. Recombined faces consisted of internal and external features taken from two different studied faces. The results showed that at the frontal sites, during the time window from 300 to 500 ms, ERPs to intact faces were more positive than those to new and recombined faces; the latter two did not differ from one another. The late parietal effect was observed only after a more extended study phase in Experiment 2. We take the modulation of the early frontal old-new effect as evidence for the contribution of familiarity to associative recognition for combinations of internal and external facial features.

Under the condition of unitization at encoding rather than unitization at retrieval, familiarity could support associative recognition and the relationship between unitization and recollection was moderated by unitization-congruence

It is widely accepted that associative recognition can be supported by familiarity through integrating more than two stimuli into a unit, but there are still three unsolved questions: (1) how unitization affects recollection-based associative recognition; (2) whether it is necessary to match the level of unitization (LOU) between original and rearranged pairs, which was term as unitization-congruence (UC); (3) whether unitization can occur at encoding or at retrieval. The purposes of this study are to try to answer these questions. During the encoding phase, the participants were asked to learn compound words and unrelated word pairs, and during the retrieval phase, they needed to distinguish intact pairs from rearranged consistent and rearranged inconsistent pairs with "remember/know" paradigm. The results showed that (1) the role of unitization in recollection was moderated by UC; (2) Under the consistent UC condition, unitization could improve familiarity-based associative recognition without affecting recollection-based associative recognition, while under the inconsistent UC condition, unitization could improve familiarity-based and recollection-based associative recognition simultaneously, these results indicated that it was necessary to match the LOU between original and rearranged pairs; (3) unitization at encoding could support familiarity-based associative recognition, while unitization at retrieval did not. In briefly, unitization at encoding could improve associative recognition and this effect was moderated by UC, while unitization at retrieval did not affect associative recognition.

Is internal source memory recognition modulated by emotional encoding contexts?

The influence of emotion on memory has been mainly examined by manipulating the emotional valence and/or arousal of critical items. Few studies probed how emotional information presented during the encoding of critical neutral items modulates memory recognition, particularly when considering source memory features. In this study, we specified the role of emotional encoding contexts in internal source memory performance (discrimination between encoding tasks) using a mixed (Experiment 1) and a blocked design (Experiment 2). During the study phase, participants were required to evaluate a set of neutral words, using either a self-referential or a semantic (common judgment) encoding strategy. Prior and concomitantly with each word, negative, neutral or positive pictures were presented in the background. The beneficial effect of self-referential encoding was observed for both item and internal source memory in both experiments. Remarkably, item and internal source memory recognition was not modulated by emotion, even though a secondary analysis indicated that the consistent exposure to negative (vs. positive) information led to worse source memory performance. These findings suggest that internal source memory of neutral items is not always affected by changing or repetitive emotional encoding contexts.

Neurophysiological evidence for the retrieval practice effect under emotional context

Testing has been demonstrated to enhance memory retention compared to restudy, which is known as the retrieval practice effect. Although much evidence has shown that emotional arousal has an impact on memory retention, little is known about how emotional arousal would influence the retrieval practice effect. To answer this question, the current study used emotional pictures as a context to induce the emotional arousal for restudy or retrieval practice. A Remember/Know paradigm was used in the final test with EEG recordings to examine how the impact of emotional arousal on retrieval practice effect would be reflected in familiarity or recollection. Behavioral results showed that the remembrance was enhanced by the retrieval practice compared to the restudy under both neutral and emotional contexts. Consistently, a parietal old/new effect from 500 to 800 ms was only found for the retrieval practice but not for the restudy condition under both neutral and emotional contexts. These findings indicated that retrieval practice could enhance later recollection when compared to restudy irrespective of emotional or neutral contexts.

The effect of unitization on associative recognition was not moderated by the unitization-congruence between original and rearranged picture pairs (UC) for picture stimuli

It is widely accepted that associative recognition can be supported by familiarity through integrating more than two stimuli into a unit, but the role of unitization in recollection-based associative recognition remains unclear. In this study, we aimed to illustrate how the level of unitization (LOU) affected recollection-based associative recognition and to examine whether the unitization-congruence between original and rearranged picture pairs (UC) could have effect on the relationship between LOU and associative recognition. In encoding, participants were asked to learn related and unrelated picture pairs, and in retrieval, they needed to distinguish intact pairs from rearranged pairs. We also distinguished the LOU of the pairs based on its status at encoding or retrieval separately. The results showed that: (1) LOU-at-encoding could improve associative recognition through increasing recollection-based associative recognition selectively; (2) LOU-at-retrieval could improve associative recognition through increasing familiarity-based and recollection-based associative recognitions; (3) UC did not moderate the relationship between LOU and associative recognition. Hence, in future studies, researchers do not need to pay much attention to the construction of rearranged pairs to ensure that the LOU between original and rearranged picture pairs is matched. It greatly reduces the difficulty of materials selection.