Recognition memory for faces: when familiarity supports associative recognition judgments

Face naming and recollection represent key memory deficits in developmental prosopagnosia

Previous studies have found that face perception deficits do not fully account for the severity of face recognition deficits in developmental prosopagnosia (DP). Researchers have begun identifying deficient memory mechanisms such as impaired face recollection, but these findings require replication, and further characterization of additional memory deficits is necessary. Our goals were to replicate prior findings of face recollection impairment in DP and extend these findings to assess different types of face associative memory. We had 69 DPs and 99 controls perform a face perception battery as well as three face memory tasks: 1) Old/New task with confidence ratings to calculate recollection and familiarity using ROC analysis, 2) Face/Scene task to examine remember-know judgments and contextual memory for faces, and 3) Face-Name/Occupation task to assess the ability to learn semantic associations with faces. Compared to controls, DPs showed poorer recollection and familiarity across both Old/New and Face/Scene tasks as well as reduced scene accuracy for correct faces. Of these differences, only Old/New recollection remained significant after controlling for group differences in face perception abilities. In the Face-Name/Occupation task, after controlling for face perception, DPs showed poorer recall of names than controls but performed similarly in recalling occupations. Finally, we found that DPs with major, mild, and no face perception deficits showed consistent impairments in Old/New recollection and face-naming, and larger perceptual deficits were associated with larger memory deficits. Together, these results provide several mechanistic insights into the nature of memory deficits in DPs and have diagnostic and treatment implications.

Comparing two unitisation manipulations: effects on familiarity, recollection-based recognition, and semantic interference

The recognition of associative memory can be significantly influenced by the use of an encoding strategy known as unitisation, which has been implemented through various manipulations. However, [Shao, H., Opitz, B., Yang, J., & Weng, X. (2016). Recollection reduces unitised familiarity effect. Memory (Hove, England), 24(4), 535-547. https://doi.org/10.1080/09658211.2015.1021258] found intriguing distinctions between two common manipulations, the compound task and the imagery task, leading to a dispute. We propose that differences in levels of processing in the imagery task may account for these discrepancies. This study tested our hypothesis using two approaches. The first two experiments utilised the R/K paradigm to investigate the effects of these methods on familiarity-based and recollection-based recognition. The results demonstrated that familiarity was increased in the compound task, while recollection was increased in the imagery task. In the subsequent two experiments, an interference paradigm was employed to examine differences in semantic processing within the two tasks. The results showed that the compound task did not impact participants' inclination towards lures, while the imagery task led to a bias towards semantic lures over episodic lures, suggesting that the two encodings in the imagery task involve different levels of semantic processing. These results support our hypothesis and underscore the importance of carefully choosing comparisons that account for other variables in the study of unitisation.

Neural distinctiveness and discriminability underlying unitization and associative memory in aging

Previous work has suggested unitized pairs behave as a single unit and more critically, are processed neurally different than those of associative memories. The current works examines the neural differences between unitization and non-unitized memory using fMRI and multivoxel analyses. Specifically, we examined the differences across face-occupation pairings as a function of whether the pairing was viewed as a person performing the given job (unitized binding) or a person saying they knew someone who had a particular job (non-unitized binding). The results show that at encoding and retrieval, the angular gyrus can discriminate between unitized and non-unitized target trials. Additionally, during encoding, the medial temporal lobe (hippocampus and perirhinal cortex), frontal parietal regions (angular gyrus and medial frontal gyrus) and visual regions (middle occipital cortex) exhibit distinct neural patterns to recollected unitized and non-unitized targets. Furthermore, the perirhinal cortex and medial frontal gyrus show greater neural similarity within subsequently recollected unitized trials compared to non-unitized trials. We conclude that an encoding based strategy to elicit unitization can produce greater associative memory compared to non-unitized trials in older adults. Additionally, when unitized trials are subsequently recollected in the perirhinal cortex older adults show greater neural similarity within unitized trials compared to non-unitized trials.

Chinese character unitization enhances item memory in addition to associative memory: Evidence from ERP and TFR

While the effect of unitization on associative memory has been established, its effect on item memory remains debated. This study aimed to investigate the influence of unitization on item memory using Chinese characters to manipulate unitization and recording scalp EEG to elucidate the underlying neural mechanisms. In the learning phase, participants were asked to determine whether the character pairs presented could form a Chinese compound character. In the subsequent testing phase, participants performed item recognition and associative recognition tasks. Behavioral results revealed that unitization not only improved associative memory but also facilitated item memory. Event-related potential analysis indicated there were FN400 effect (related to familiarity) and LPC effect (related to recollection) during associative recognition after unitization, however, only the LPC effect was observed for the item recognition. More importantly, time-frequency analysis demonstrated stronger θ oscillations (associated with recollection) in the unitized condition compared to the non-unitized condition, which further partially mediated the reduction in RT during the item recognition. These results suggest that unitization enhances item memory through recollection, thereby leading to more confident recognition judgments, and that unitization does not impair item processing within an association but rather enables more precise and accurate processing.

Chinese character unitization enhances recollection-based associative recognition: Evidence from fMRI

Previous research has suggested that familiarity can enhance associative memory after unitization, but the cognitive mechanisms underlying unitization remain debated. To explore the neural mechanisms of associative memory after unitization in the absence of semantic relations, we used Chinese characters as stimuli and recorded participants' blood oxygen level-dependent signals during recognition. Behavioral results showed that after Chinese character unitization, not only the associative performance of recognition (Pr, hit rate minus false alarm rate) and general Pr but also the hit rate and correct rejection rate increased. Neuroimaging results revealed activation of the hippocampus and parahippocampal gyrus during associative recognition in both the unitized and the non-unitized condition, and hippocampal activation increased after unitization. However, activation of the perirhinal cortex was not observed in either condition. These findings, in contrast to those from previous studies on unitization, suggest that Chinese character unitization enhances recollection-based, rather than familiarity-based, associative recognition. This suggests that the encoding of semantic relations during unitization is critical for subsequent familiarity-based associative recognition.

Detecting Dementia from Face-Related Features with Automated Computational Methods

Alzheimer's disease (AD) is a type of dementia that is more likely to occur as people age. It currently has no known cure. As the world's population is aging quickly, early screening for AD has become increasingly important. Traditional screening methods such as brain scans or psychiatric tests are stressful and costly. The patients are likely to feel reluctant to such screenings and fail to receive timely intervention. While researchers have been exploring the use of language in dementia detection, less attention has been given to face-related features. The paper focuses on investigating how face-related features can aid in detecting dementia by exploring the PROMPT dataset that contains video data collected from patients with dementia during interviews. In this work, we extracted three types of features from the videos, including face mesh, Histogram of Oriented Gradients (HOG) features, and Action Units (AU). We trained traditional machine learning models and deep learning models on the extracted features and investigated their effectiveness in dementia detection. Our experiments show that the use of HOG features achieved the highest accuracy of 79% in dementia detection, followed by AU features with 71% accuracy, and face mesh features with 66% accuracy. Our results show that face-related features have the potential to be a crucial indicator in automated computational dementia detection.

Learning facilitates dual-process face recognition regardless of holistic processing

Much evidence suggests that faces are recognized based on their global familiarity in a signal-detection manner. However, experiments drawing this conclusion typically present study lists of faces only once or twice, and the nature of face recognition at higher levels of learning remains unclear. Here, three experiments are reported in which participants studied some faces eight times and others twice and then took a recognition test containing previously viewed faces, entirely new faces, and faces which recombined the parts of previously viewed faces. Three measures converged to suggest that study list repetition increased the likelihood of participants rejecting recombined faces as new by recollecting that their parts were studied but in a different combination, and that manipulating holistic or Gestalt-like processing-a hallmark of face perception-qualitatively preserved its effect on how memory judgments are made. This suggests that face learning causes a shift from the use of a signal-detection strategy to the use of a dual-process strategy of face recognition regardless of holistic processing.

From encoding to retrieval: Change in level of unitization resolves debate about Unitization's effect on associative recognition

Although it is widely accepted that familiarity could support associative recognition when the to-be-learn items are 'unitized' into a new representation, the effects of unitization on associative recognition and recollection remain much debated. The current study aimed to explain these debates by exploring when and how unitization benefits associative recognition using event-related potentials (ERPs). During the encoding phase, participants learned compound words and unrelated word pairs (i.e., High vs. Low level of unitization). At retrieval, the compound words were rearranged into new compound words (i.e., no-change) and unrelated word pairs (i.e., change). Similarly, the unrelated word pairs were rearranged into new unrelated word pairs (i.e., no-change) and compound words (i.e., change). Results showed that under the no-change condition, unitization did not affect associative recognition, nor its underlying processes. In contrast, under the change condition, unitization improved associative recognition by increasing both familiarity-related FN400 effect and recollection-related LPC effect. In addition, a planned comparison between the compound-change and unrelated-no change conditions-a common index for unitization effect in past studies-revealed that unitization could not only elicit significant FN400 effect, but also improve associative recognition by increasing LPC effect. Collectively, these results not only allowed to explain the current discrepancies in the literature concerning the effect of unitization on associative recognition, but also emphasized the importance of matching the level of unitization between the studied and rearranged word pairs.

Predicting visual memory across images and within individuals

We only remember a fraction of what we see-including images that are highly memorable and those that we encounter during highly attentive states. However, most models of human memory disregard both an image's memorability and an individual's fluctuating attentional states. Here, we build the first model of memory synthesizing these two disparate factors to predict subsequent image recognition. We combine memorability scores of 1100 images (Experiment 1, n = 706) and attentional state indexed by response time on a continuous performance task (Experiments 2 and 3, n = 57 total). Image memorability and sustained attentional state explained significant variance in image memory, and a joint model of memory including both factors outperformed models including either factor alone. Furthermore, models including both factors successfully predicted memory in an out-of-sample group. Thus, building models based on individual- and image-specific factors allows for directed forecasting of our memories. SIGNIFICANCE STATEMENT: Although memory is a fundamental cognitive process, much of the time memory failures cannot be predicted until it is too late. However, in this study, we show that much of memory is surprisingly pre-determined ahead of time, by factors shared across the population and highly specific to each individual. Specifically, we build a new multidimensional model that predicts memory based just on the images a person sees and when they see them. This research synthesizes findings from disparate domains ranging from computer vision, attention, and memory into a predictive model. These findings have resounding implications for domains such as education, business, and marketing, where it is a top priority to predict (and even manipulate) what information people will remember.

A multistage retrieval account of associative recognition ROC curves

Despite its name, associative recognition is a paradigm thought to rely on memory recall. However, it remains unclear how associative information may be represented and retrieved from memory and what its relationship to other information, such as item memory, is. Here, we propose a computational model of associative recognition, where relational information is accessed in a generic, multistage retrieval process. The model explains the relative difficulty of associative recognition compared with item recognition, the difference in experimental outcomes when different types of lures are used, as well as the conditions leading to the emergence of associative ROC curves with different shapes.

Attempted recall of biographical information influences face attractiveness

We examined affective consequences arising from the kinds of memory retrieval failures that often accompany social interaction. To do so, we measured the influence of cued-recall outcomes for biographical information on the rated attractiveness of faces. The data demonstrate that retrieval of names (Experiment 1a) and professions (Experiment 1b) increases the rated attractiveness of target faces relative to faces that failed to produce recall of associative information. This was predicted by a confirmation of search (COS) model originally developed on verbal memoranda, which assumes that confirmation bias during memory search leads to affective consequences depending upon retrieval's success or failure. The current study extends this model, showing that evaluative judgments of individuals are in part contingent upon the memory retrieval skills of their assessors. We conclude by discussing potential extensions of the COS paradigm to the measurement of implicit attitudes and special populations.

The interaction of relational encoding and unitization: Effects on medial temporal lobe processing during retrieval

In retrieval of typical episodic memories, recollection leads to retrieval of context details whereas familiarity is only diagnostic for item memory. Unitization is an encoding strategy that allows context details to be processed as item features and therefore increases the involvement of familiarity-based recognition in retrieval of these context details. Relational encoding is a hippocampally-dependent process that stores items and contexts independently. Our previous study Tu and Diana [1] concluded that mixing unitized and non-unitized context details in the same episode reduced the contribution of familiarity to retrieval of any one detail. In the current study, we modified the paradigm by removing visual cues to the context details and the condition-specific blocking during test. Surprisingly, the behavioral data diverged from our 2016 study and indicated that the two manipulated context details in the modified paradigm were processed independently of one another. Neuroimaging data further revealed anterior hippocampal activation was associated with unitization of source information as compared to relational encoding. We also found the predicted increase in bilateral perirhinal cortex activation and decrease in parahippocampal cortex activation during retrieval of unitized color information when compared to relationally-encoded color information. We did not find that same predicted pattern of differences due to unitization of size information.

Object recognition is enabled by an experience-dependent appraisal of visual features in the brain's value system

This paper addresses perceptual synthesis by comparing responses evoked by visual stimuli before and after they are recognized, depending on prior exposure. Using magnetoencephalography, we analyzed distributed patterns of neuronal activity - evoked by Mooney figures - before and after they were recognized as meaningful objects. Recognition induced changes were first seen at 100-120 ​ms, for both faces and tools. These early effects - in right inferior and middle occipital regions - were characterized by an increase in power in the absence of any changes in spatial patterns of activity. Within a later 210-230 ​ms window, a quite different type of recognition effect appeared. Regions of the brain's value system (insula, entorhinal cortex and cingulate of the right hemisphere for faces and right orbitofrontal cortex for tools) evinced a reorganization of their neuronal activity without an overall power increase in the region. Finally, we found that during the perception of disambiguated face stimuli, a face-specific response in the right fusiform gyrus emerged at 240-290 ​ms, with a much greater latency than the well-known N170m component, and, crucially, followed the recognition effect in the value system regions. These results can clarify one of the most intriguing issues of perceptual synthesis, namely, how a limited set of high-level predictions, which is required to reduce the uncertainty when resolving the ill-posed inverse problem of perception, can be available before category-specific processing in visual cortex. We suggest that a subset of local spatial features serves as partial cues for a fast re-activation of object-specific appraisal by the value system. The ensuing top-down feedback from value system to visual cortex, in particular, the fusiform gyrus enables high levels of processing to form category-specific predictions. This descending influence of the value system was more prominent for faces than for tools, the fact that reflects different dependence of these categories on value-related information.

Unitization of internal and external features contributes to associative recognition for faces: Evidence from modulations of the FN400

Associative recognition requires discriminating between old items and conjunction lures constructed by recombining elements from two different study items. This task can be solved not only by recollection but also by familiarity if the to-be-remembered stimuli are perceived as a unitized representation. In two event-related potential (ERP) studies, we provide evidence for the integration of internal and external facial features by showing that the early frontal old-new effect (considered a correlate of familiarity) is modulated by the specific combination of facial features. Participants studied faces consisting of internal features (eyes, eyebrows, nose, and mouth) paired with external features (hair, head shape, and ears). During the testing phase, intact, recombined, and new faces were presented. Recombined faces consisted of internal and external features taken from two different studied faces. The results showed that at the frontal sites, during the time window from 300 to 500 ms, ERPs to intact faces were more positive than those to new and recombined faces; the latter two did not differ from one another. The late parietal effect was observed only after a more extended study phase in Experiment 2. We take the modulation of the early frontal old-new effect as evidence for the contribution of familiarity to associative recognition for combinations of internal and external facial features.

Distance- rather than location-based temporal judgments are more accurate during episodic recall in a real-world task

Definitions of episodic memory typically emphasise the importance of spatiotemporal frameworks in the contextual reconstruction of episodic retrieval. However, our ability to retrieve specific temporal contexts of experienced episodes is poor. This has bearing on the prominence of temporal context in the definition and evaluation of episodic memory, particularly among non-human animals. Studies demonstrating that rats rely on elapsed time (distance) rather than specific timestamps (location) to disambiguate events have been used to suggest that human episodic memory is qualitatively different to other species. We examined whether humans were more accurate using a distance- or location-based method for judging when an event happened. Participants (n = 57) were exposed to a series of events and then asked either when (e.g., 1:03 pm) or how long ago (HLA; e.g., 33 min) a specific event took place. HLA judgements were significantly more accurate, particularly for the most recently experienced episode. Additionally, a significantly higher proportion of participants making HLA judgements accurately recalled non-temporal episodic features across all episodes. Finally, for participants given the choice of methods for making temporal judgements, a significantly higher proportion chose to use HLA judgements. These findings suggest that human and non-human temporal judgements are not qualitatively different.

Under the condition of unitization at encoding rather than unitization at retrieval, familiarity could support associative recognition and the relationship between unitization and recollection was moderated by unitization-congruence

It is widely accepted that associative recognition can be supported by familiarity through integrating more than two stimuli into a unit, but there are still three unsolved questions: (1) how unitization affects recollection-based associative recognition; (2) whether it is necessary to match the level of unitization (LOU) between original and rearranged pairs, which was term as unitization-congruence (UC); (3) whether unitization can occur at encoding or at retrieval. The purposes of this study are to try to answer these questions. During the encoding phase, the participants were asked to learn compound words and unrelated word pairs, and during the retrieval phase, they needed to distinguish intact pairs from rearranged consistent and rearranged inconsistent pairs with "remember/know" paradigm. The results showed that (1) the role of unitization in recollection was moderated by UC; (2) Under the consistent UC condition, unitization could improve familiarity-based associative recognition without affecting recollection-based associative recognition, while under the inconsistent UC condition, unitization could improve familiarity-based and recollection-based associative recognition simultaneously, these results indicated that it was necessary to match the LOU between original and rearranged pairs; (3) unitization at encoding could support familiarity-based associative recognition, while unitization at retrieval did not. In briefly, unitization at encoding could improve associative recognition and this effect was moderated by UC, while unitization at retrieval did not affect associative recognition.

Composite Face Effect Predicts Configural Encoding in Visual Short-Term Memory

In natural vision, visual scenes consist of individual items (e.g., trees) and global properties of items as a whole (e.g., forest). These different levels of representations can all contribute to perception, natural scene understanding, sensory memory, working memory, and long-term memory. Despite these various hierarchical representations across perception and cognition, the nature of the global representations has received considerably less attention in empirical research on working memory than item representations. The present study aimed to understand the perceptual root of the configural information retained in Visual Short-term Memory (VSTM). Specifically, we assessed whether configural VSTM was related to holistic face processing across participants using an individual differences approach. Configural versus item encoding in VSTM was assessed using Xie and Zhang's (2017) dual-trace Signal Detection Theory model in a change detection task for orientation. Configural face processing was assessed using Le Grand composite face effect (CFE). In addition, overall face recognition was assessed using Glasgow Face Matching Test (GFMT). Across participants, holistic face encoding, but not face recognition accuracy, predicted configural information, but not item information, retained in VSTM. Together, these findings suggest that configural encoding in VSTM may have a perceptual root.

Unitization modulates recognition of within-domain and cross-domain associations: Evidence from event-related potentials

Although it is often assumed that memory of episodic associations requires recollection, it has been suggested that, when stimuli are experienced as a unit, earlier memory processes might contribute to their subsequent associative recognition. We investigated the effects of associative relations and perceptual domain during episodic encoding on the ability to utilize early memory processes to retrieve associative information. During the study phase, participants encoded compound and noncompound words pairs, presented either to the same sensory modality (visual presentation) or to different sensory modalities (audiovisual presentation). At the test phase, they discriminated between old, rearranged, and new pairs while ERPs were recorded. In an early ERP component, differences related to associative memory emerged only for compounds, regardless of their encoding modality. These findings indicate that episodic retrieval of compound words can be supported by early-onset recognition processes regardless of whether both words were presented to the same or different sensory modalities, and suggests that unitization can operate at an abstract level, across a broad range of materials.

The missing link? Testing a schema account of unitization

Unitization refers to the creation of a new unit from previously distinct items. The concept of unitization has been used to explain how novel pairings between items can be remembered without requiring recollection, by virtue of new, item-like representations that enable familiarity-based retrieval. We tested an alternative account of unitization - a schema account - which suggests that associations between items can be rapidly assimilated into a schema. We used a common operationalization of "unitization" as the difference between two unrelated words being linked by a definition, relative to two words being linked by a sentence, during an initial study phase. During the following relearning phase, a studied word was re-paired with a new word, either related or unrelated to the original associate from study. In a final test phase, memory for the relearned associations was tested. We hypothesized that, if unitized representations act like schemas, then we would observe some generalization to related words, such that memory would be better in the definition than sentence condition for related words, but not for unrelated words. Contrary to the schema hypothesis, evidence favored the null hypothesis of no difference between definition and sentence conditions for related words (Experiment 1), even when each cue was associated with multiple associates, indicating that the associations can be generalized (Experiment 2), or when the schematic information was explicitly re-activated during Relearning (Experiment 3). These results suggest that unitized associations do not generalize to accommodate new information, and therefore provide evidence against the schema account.

Misrecollection prevents older adults from benefitting from semantic relatedness of the memoranda in associative memory

Memory for episodic associations declines in aging, ostensibly due to decreased recollection abilities. Accordingly, associative unitization - the encoding of associated items as one integrated entity - may potentially attenuate age-related associative deficits by enabling familiarity-based retrieval, which is relatively preserved in aging. To test this hypothesis, we induced bottom-up unitization by manipulating semantic relatedness between memoranda. Twenty-four young and 24 older adults studied pairs of object pictures that were either semantically related or unrelated. Participants subsequently discriminated between intact, recombined and new pairs. We found that semantic relatedness increased the contributions of both familiarity and recollection in young adults, but did not improve older adults' performance. Instead, they showed associative deficits, driven by increased recollection-based false recognition. This may reflect a "misrecollection" phenomenon, in which older adults make more false alarms to recombined pairs with particularly high confidence, due to poorer retrieval monitoring regarding semantically-related associative probes.

Can implicit measures detect source information in crime-related amnesia?

Participants who are asked to simulate amnesia for a mock crime have a weaker memory for this event when they have to give up their role as a feigner, than those who are not asked to feign memory loss. According to the source monitoring framework (SMF), this memory-undermining effect of simulating amnesia for a crime would be due to misattribution of the right source of information. However, we know that the content of self-generated information (e.g., feigned version of the crime) might be preserved and recognised over time as a result of elaborative cognitive processing. In the present study, we aimed to contrast these two explanations. We showed participants a mock crime video and we instructed them to either feign amnesia (simulators) or confess the mock crime (confessors). Next, a free recall memory test was administered. After one week, participants were asked to perform a personalised source monitoring task using the autobiographical Implicit Association Test (aIAT). As predicted, we found that simulators were able to discriminate the content of their self-generated feigned story of the crime from the original version. Moreover, simulators were quicker than confessors at the aIAT task. Practical and theoretical implications of our results are discussed.

Increasing relational memory in childhood with unitization strategies

Young children often experience relational memory failures, which are thought to result from immaturity of the recollection processes presumed to be required for these tasks. However, research in adults has suggested that relational memory tasks can be accomplished using familiarity, a process thought to be mature by the end of early childhood. The goal of the present study was to determine whether relational memory performance could be improved in childhood by teaching young children memory strategies that have been shown to increase the contribution of familiarity in adults (i.e., unitization). Groups of 6- and 8-year-old children were taught to use visualization strategies that either unitized or did not unitize pictures and colored borders. Estimates of familiarity and recollection were extracted by fitting receiver operator characteristic curves (Yonelinas, Journal of Experimental Psychology: Learning, Memory, and Cognition 20, 1341-1354, 1994, Yonelinas, Memory & Cognition 25, 747-763, 1997) based on dual-process models of recognition. Bayesian analysis revealed that strategies involving unitization improved memory performance and increased the contribution of familiarity in both age groups.

The influence of recollection and familiarity in the formation and updating of associative representations

Prior representations affect future learning. Little is known, however, about the effects of recollective or familiarity-based representations on such learning. We investigate the ability to reuse or reassociate elements from recollection- and familiarity-based associations to form new associations. Past neuropsychological research suggests that hippocampal, and presumably recollective, representations are more flexible than extra-hippocampal, presumably familiarity-based, representations. We therefore hypothesize that the elements of recollective associations, as opposed to familiarity-based representations, may be more easily manipulated and decoupled from each other, and facilitate the formation of new associations. To investigate this hypothesis we used the AB/AC learning paradigm. Across two recall studies we observed an advantage in learning AC word pairs if AB word pairs were initially recollected. Furthermore, AB word pairs were more likely to intrude during a final AC test if those AB word pairs were initially familiarity-based. A third experiment using a recognition version of the AB/AC paradigm ruled out the possibility that our findings were due to memory strength. Our results support the idea that elements in recollective associative traces may be more discretely coded, leading to their flexible use, whereas elements in familiarity-based associative traces are less flexible.

Event-related potentials during encoding: Comparing unitization to relational processing

Context details are typically encoded into episodic memory via arbitrary associations to the relevant item, known as relational binding. Subsequent retrieval of those context details is primarily supported by recollection. Research suggests that context retrieval can rely on familiarity if the context details are "unitized" and thereby encoded as features of the item itself in a single new representation. With most investigations into unitization focusing on the contributions of familiarity and recollection during retrieval, little is known about unitization during encoding. In an effort to begin understanding unitization as an encoding process, we used event-related potentials to monitor brain activity while participants were instructed to encode words with color information using relational association or unitization. Results showed that unitization-based encoding elicited significantly more negative potentials in the left parietal region than relational encoding during presentation of the second segment of strategically-specific sentences. This difference continued through presentation of the third sentence segment, becoming less lateralized, and ended before the final two segments were presented. During the mental imagery period, unitization-based encoding elicited significantly more positive potentials than relational encoding in the first 200ms centrally and from 400 through 1000ms in left fronto-temporal and parieto-occipital regions. Our findings indicate that unitization and relational processing diverged at approximately the time that the context item was presented in the relational condition. During mental imagery, unitization diverged from relational processing immediately, suggesting that unitization affected the nature of the item representation, and possibly the brain regions involved, during memory encoding.

The Effect of Unitizing Word Pairs on Recollection Versus Familiarity-Based Retrieval- Further Evidence From ERPs

We investigated the contribution of familiarity and recollection to associative retrieval of word pairs depending on the extent to which the pairs have been unitized through task instructions in the encoding phase. Participants in the unitization condition encoded word pairs in the context of a definition that tied them together such that they were treated as a coherent new item, while in the control condition word pairs were inserted into a sentence frame in which each word remained an individual unit. Contrasting event-related potentials (ERERPs) elicited in a subsequent recognition test by old (intact) and recombined (a new combination of two words from different study pairs) word pairs, an early frontal effect, the putative ER P correlate of familiarity-based retrieval, was apparent in the unitization condition. The left parietal old/new effect, reflecting recollection-based retrieval, was elicited only in the control condition. This suggests that in the unitization condition only, familiarity was sufficiently diagnostic to distinguish old from recombined pairs, while in the control condition, recollection contributed to associative recognition. Our findings add to a body of literature suggesting that unitization of associations increases the relative contribution of familiarity to subsequent associative retrieval.

Familiarity influences on direct and indirect associative memory for objects in scenes

Remembering arbitrary associations, such as unrelated word pairs or object-background pairs, appears to depend on recollection. However, for cases in which the components of an association share pre-existing semantic relations, can familiarity support associative recognition? In two experiments with congruent object-background pairings, we found that participants were successful at direct and indirect associative recognition in both 1000 ms time restriction (speeded) and unlimited response time (non-speeded) test conditions. Because dual-process theory postulates that familiarity is less impacted by speeded responses, relative to recollection, these findings suggest that congruent object-background associations may not necessitate recollection when an arbitrary link is not constructed at encoding. Experiment 3 compared direct associative memory for congruent and incongruent object-background pairs in speeded and non-speeded test conditions. We found that participants in the non-speeded condition performed comparably with congruent and incongruent pairs, whereas those in the speeded condition performed significantly worse on the incongruent pairs than on the congruent pairs. Together, these findings suggest a differential role of familiarity and recollection depending on the types of associations. Implications for dual-process recognition memory models and levels of unitization framework are discussed.

The Effects of the Amount of Information on Episodic Memory Binding

The effects of increasing the number of items to be remembered on associative recognition and cued recall were examined. Thirty participants were asked during encoding to determine whether two- and three-item stimuli contained natural objects, artificial objects, or both. In an associative recognition task, the participants indicated whether the stimuli were identical to those presented during encoding, were rearranged by exchanging one of the two-item stimuli for one of the three-item stimuli, or represented a new stimulus. The correctly identified rearranged item pairs and triads were included in a subsequent cued-recall task in which participants verbally reported the missing item. As the number of items increased, the discrimination of rearranged stimuli diminished, but that of identical trials remained the same. Furthermore, the ability to retrieve the missing item was unaffected. It was concluded that the effect of the amount of information on binding depends on how the information must be retrieved.

Two are not better than one: Combining unitization and relational encoding strategies

In recognition memory, recollection is defined as retrieval of the context associated with an event, whereas familiarity is defined as retrieval based on item strength alone. Recent studies have shown that conventional recollection-based tasks, in which context details are manipulated for source memory assessment at test, can also rely on familiarity when context information is "unitized" with the relevant item information at encoding. Unlike naturalistic episodic memories that include many context details encoded in different ways simultaneously, previous studies have focused on unitization and its effect on the recognition of a single context detail. To further understand how various encoding strategies operate on item and context representations, we independently assigned unitization and relational association to 2 context details (size and color) of each item and tested the contribution of recollection and familiarity to source recognition of each detail. The influence of familiarity on retrieval of each context detail was compared as a function of the encoding strategy used for each detail. Receiver operating characteristic curves suggested that the unitization effect was not additive and that similar levels of familiarity occurred for 1 or multiple details when unitization was the only strategy applied during encoding. On the other hand, a detrimental effect was found when relational encoding and unitization were simultaneously applied to 1 item such that a salient nonunitized context detail interfered with the effortful processing required to unitize an accompanying context detail. However, this detrimental effect was not reciprocal and possibly dependent on the nature of individual context details. (PsycINFO Database Record

Electrophysiological evidence for the effects of unitization on associative recognition memory in older adults

Normal aging is associated with greater decline in associative memory relative to item memory due to impaired recollection. Familiarity may also contribute to associative recognition when stimuli are perceived as a 'unitized' representation. Given that familiarity is relatively preserved in older adults, we explored whether age-related associative memory deficits could be attenuated when associations were unitized (i.e., compounds) compared with those non-unitized (i.e., unrelated word pairs). Young and older adults performed an associative recognition task while electroencephalogram (EEG) was recorded. Behavioral results showed that age differences were smaller for recognition of compounds than for unrelated word pairs. ERP results indicated that only compounds evoked an early frontal old/new effect in older adults. Moreover, the early frontal old/new effect was positively correlated with associative discrimination accuracy. These findings suggest that reduced age-related associative deficits under unitized condition may be associated with the presence of familiarity-based retrieval of compounds in older adults.

The importance of unitization for familiarity-based learning

It is often assumed that recollection is necessary to support memory for novel associations, whereas familiarity supports memory for single items. However, the levels of unitization framework assumes that familiarity can support associative memory under conditions in which the components of an association are unitized (i.e., treated as a single coherent item). In the current study we tested two critical assumptions of this framework. First, does unitization reflect a specialized form of learning or is it simply a form of semantic or elaborative encoding, and, second, can the beneficial effects of unitization on familiarity be observed for across-domain associations or are they limited to creating new associations between items that are from the same stimulus domains? Unitization was found to increase associative recognition but not item recognition. It affected familiarity more than recollection, increased associative but not item priming, and was dissociable from levels of processing effects. Moreover, unitization effects were found to be particularly effective in supporting face-word and fractal-sound pairs. The current results indicate that unitization reflects a specialized form of learning that supports associative familiarity of within- and across-domain associations.

The role of familiarity in associative recognition of unitized compound word pairs

This study examined the effect of unitization and contribution of familiarity in the recognition of word pairs. Compound words were presented as word pairs and were contrasted with noncompound word pairs in an associative recognition task. In Experiments 1 and 2, yes–no recognition hit and false-alarm rates were significantly higher for compound than for noncompound word pairs, with no difference in discrimination in both within- and between-subject comparisons. Experiment 2 also showed that item recognition was reduced for words from compound compared to noncompound word pairs, providing evidence of the unitization of the compound pairs. A two-alternative forced-choice test used in Experiments 3A and 3B provided evidence that the concordant effect for compound word pairs was largely due to familiarity. A discrimination advantage for compound word pairs was also seen in these experiments. Experiment 4A showed that a different pattern of results is seen when repeated noncompound word pairs are compared to compound word pairs. Experiment 4B showed that memory for the individual items of compound word pairs was impaired relative to items in repeated and nonrepeated noncompound word pairs, and Experiment 5 demonstrated that this effect is eliminated when the elements of compound word pairs are not unitized. The concordant pattern seen in yes–no recognition and the discrimination advantage in forced-choice recognition for compound relative to noncompound word pairs is due to greater reliance on familiarity at test when pairs are unitized.

Hypergraph-based recognition memory model for lifelong experience

Cognitive agents are expected to interact with and adapt to a nonstationary dynamic environment. As an initial process of decision making in a real-world agent interaction, familiarity judgment leads the following processes for intelligence. Familiarity judgment includes knowing previously encoded data as well as completing original patterns from partial information, which are fundamental functions of recognition memory. Although previous computational memory models have attempted to reflect human behavioral properties on the recognition memory, they have been focused on static conditions without considering temporal changes in terms of lifelong learning. To provide temporal adaptability to an agent, in this paper, we suggest a computational model for recognition memory that enables lifelong learning. The proposed model is based on a hypergraph structure, and thus it allows a high-order relationship between contextual nodes and enables incremental learning. Through a simulated experiment, we investigate the optimal conditions of the memory model and validate the consistency of memory performance for lifelong learning.

Associative recognition processes are modulated by the semantic unitizability of memoranda

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity processes might contribute to their subsequent associative recognition. To investigate the effect of semantic relatedness during episodic encoding on the processes of retrieval of associative information, we had participants interactively encode pairs of object pictures, vertically arranged so as to suggest a functional or configural relationship between them. Half the pairs were independently judged to be of related objects (e.g., a lamp over a table) and half of unrelated objects (e.g., a key-ring over an apple). At test, participants discriminated between intact, recombined, and new pairs while event related potentials (ERPs) were recorded. In an early ERP marker of retrieval success generally associated with familiarity processes, differences related to associative memory only emerged for related pairs, while differences associated with item memory emerged for both related and unrelated pairs. In contrast, in a later ERP effect associated with recollection, differences related to associative memory emerged for both related and unrelated pairs. These findings may indicate that retrieval of episodic associations formed between two semantically related visual stimuli can be supported by familiarity-related processes.

Associative Recognition Processes Are Modulated by Modality Relations

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity-related processes might contribute to their subsequent associative recognition. Furthermore, intradomain associations are believed to be unitized more readily than interdomain associations. To assess these claims, we tested associative recognition following two types of pair associate learning. In the unimodal task, stimulus pairs were pictures of common objects, whereas in the cross-modal task, stimulus pairs consisted of an object picture and an unrelated environmental sound. At test, participants discriminated intact from recombined pairs while ERPs were recorded. In the unimodal task only, associative recognition was accompanied by a robust frontal deflection reminiscent of a component commonly interpreted as related to familiarity processes. In contrast, ERP correlates of associative recognition observed at more posterior sites, akin to a component that has been related to recollection, were apparent in both tasks. These findings indicate that retrieval of unimodal associations can be supported by familiarity-related processes that are dissociable from recollective processes required for the retrieval of cross-modal associations.

The hippocampus supports high-resolution binding in the service of perception, working memory and long-term memory

It is well established that the hippocampus plays a critical role in our ability to recollect past events. A number of recent studies have indicated that the hippocampus may also play a critical role in working memory and perception, but these results have been highly controversial because other similar studies have failed to find evidence for hippocampal involvement. Thus, the precise role that the hippocampus plays in cognition is still debated. In the current paper, I propose that the hippocampus supports the generation and utilization of complex high-resolution bindings that link together the qualitative aspects that make up an event; these bindings are essential for recollection, and they can also contribute to performance across a variety of tasks including perception and working memory. An examination of the existing patient literature provides support for this proposal by showing that hippocampal damage leads to impairments on perception and working memory tasks that require complex high-resolution bindings. Conversely, hippocampal damage is much less likely to lead to impairments on tasks that require only low-resolution or simple associations/relations. The current proposal can be distinguished from earlier accounts of hippocampal function, and it generates a number of novel predictions that can be tested in future studies.

Semantic relations differentially impact associative recognition memory: electrophysiological evidence

Though associative recognition memory is thought to rely primarily on recollection, recent research indicates that familiarity might also make a substantial contribution when to-be-learned items are integrated into a coherent structure by means of an existing semantic relation. It remains unclear how different types of semantic relations, such as categorical (e.g., dancer-singer) and thematic (e.g., dancer-stage) relations might affect associative recognition, however. Using event-related potentials (ERPs), we addressed this question by manipulating the type of semantic link between paired words in an associative recognition memory experiment. An early midfrontal old/new effect, typically linked to familiarity, was observed across the relation types. In contrast, a robust left parietal old/new effect was found in the categorical condition only, suggesting a clear contribution of recollection to associative recognition for this kind of pairs. One interpretation of this pattern is that familiarity was sufficiently diagnostic for associative recognition of thematic relations, which could result from the integrative nature of the thematic relatedness compared to the similarity-based nature of categorical pairs. The present study suggests that the extent to which recollection and familiarity are involved in associative recognition is at least in part determined by the properties of semantic relations between the paired associates.

Representational demands modulate involvement of perirhinal cortex in face processing

The classic view holds that the medial temporal lobes (MTL) are dedicated to declarative memory functioning. Recent evidence, however, suggests that perirhinal cortex (PrC), a structure within the anterior MTL, may also play a role in perceptual discriminations when representations of complex conjunctions of features, or of gestalt-characteristics of objects must be generated. Interestingly, neuroimaging and electrophysiological recordings in nonhuman primates have also revealed a face patch in the anterior collateral sulcus with preferential responses to face stimuli in various task contexts. In the present fMRI study, we investigated the representational demands that influence PrC involvement in different types of judgments on human faces. Holding stimulus complexity constant, we independently manipulated the nature of the task and the orientation of the stimuli presented (through face inversion). Aspects of right PrC showed increased responses in a forced-choice recognition-memory and a perceptual-oddity task, as compared to a feature-search task that was included to probe visual detection of an isolated face feature. Effects of stimulus orientation in right PrC were observed when the recognition-memory condition for upright faces was compared with all other experimental conditions, including recognition-memory for inverted faces-a result that can be related to past work on the role of PrC in object unitization. Notably, both effects in right PrC paralleled activity patterns in broader networks of regions that also included the right fusiform gyrus and the amygdala, regions frequently implicated in face processing in prior research. As such, the current findings do not support the view that reference to a prior study episode clearly distinguishes the role of PrC from that of more posterior ventral visual pathway regions. They add to a growing body of evidence suggesting that the functional role of specific MTL structures may be best understood in terms of the representations that are required by the task and the stimuli at hand.