Familiarity impairments after anterior temporal-lobe resection with hippocampal sparing: Lessons learned from case NB

The Multiple Dimensions of Familiarity: From Representations to Phenomenology

This article focuses on familiarity, the form of memory allowing humans to recognize stimuli that have been encountered before. We aim to emphasize its complex nature which includes representational and phenomenological dimensions. The former implies that its neural correlates depend on the type and complexity of the cue stimulus, as different classes of stimuli are represented in distributed ventral visual and medial temporal regions. The second dimension relates to the subjective feeling of familiarity, which results from a fluency signal that is attributed to past encounters with the stimulus. We review mnemonic and non-mnemonic sources of fluency that can induce a sense of familiarity, as well as cases where fluency is not attributed to memory, among which the phenomenological experience of déjà-vu. Across these two dimensions, we highlight key questions to be answered by future studies to improve our understanding of the underpinnings of this form of memory and contribute to building an integrative neurocognitive model of familiarity. Essential to this aim is the clarification of the computational, cognitive, and neural mechanisms involved, namely global matching, fluency attribution, and sharpening. Furthermore, future research is needed to unravel the relationships between these mechanisms. We argue that to achieve these goals, researchers must use appropriate behavioral paradigms and clearly define which dimension of familiarity they investigate.

A third kind of episodic memory: Context familiarity is distinct from item familiarity and recollection

Episodic memory is accounted for with two processes: 'familiarity' when generally recognizing an item and 'recollection' when retrieving the full contextual details bound with the item. Paradoxically, people sometimes report contextual information as familiar but without recollecting details, which is not easily accounted for by existing theories. We tested a combination of item recognition confidence and source memory, focusing upon 'item-only hits with source unknown' ('item familiarity'), 'low-confidence hits with correct source memory' ('context familiarity'), and 'high-confidence hits with correct source memory' ('recollection'). Results across multiple within-subjects (trial-wise) and between subjects (individual variability) levels indicated these were behaviorally and physiologically distinct. Behaviorally, a crossover interaction was evident in response times, with context familiarity being slower than each condition during item recognition, but faster during source memory. Electrophysiologically, a Condition x Time x Location triple dissociation was evident in event-related potentials (ERPs), which was then independently replicated. Context familiarity exhibited an independent negative central effect from 800-1200 ms, differentiated from positive ERPs for item-familiarity (400 to 600 ms) and recollection (600 to 900 ms). These three conditions thus reflect mutually exclusive, fundamentally different processes of episodic memory. Context familiarity is a third distinct process of episodic memory.

Summary

Memory for past events is widely believed to operate through two different processes: one called 'recollection' when retrieving confident, specific details of a memory, and another called 'familiarity' when only having an unsure but conscious awareness that an item was experienced before. When people successfully retrieve details such as the source or context of a prior event, it has been assumed to reflect recollection. We demonstrate that familiarity of context is functionally distinct from familiarity of items and recollection and offer a new, tri-component model of memory. The three memory responses were differentiated across multiple behavioral and brain wave measures. What has traditionally been thought to be two kinds of memory processes are actually three, becoming evident when using sensitive enough multi-measures. Results are independently replicated across studies from different labs. These data reveal that context familiarity is a third process of human episodic memory.

Shared neural codes of recognition memory

Recognition memory research has identified several electrophysiological indicators of successful memory retrieval, known as old-new effects. These effects have been observed in different sensory domains using various stimulus types, but little attention has been given to their similarity or distinctiveness and the underlying processes they may share. Here, a data-driven approach was taken to investigate the temporal evolution of shared information content between different memory conditions using openly available EEG data from healthy human participants of both sexes, taken from six experiments. A test dataset involving personally highly familiar and unfamiliar faces was used. The results show that neural signals of recognition memory for face stimuli were highly generalized starting from around 200 ms following stimulus onset. When training was performed on non-face datasets, an early (around 200-300 ms) to late (post-400 ms) differentiation was observed over most regions of interest. Successful cross-classification for non-face stimuli (music and object/scene associations) was most pronounced in late period. Additionally, a striking dissociation was observed between familiar and remembered objects, with shared signals present only in the late window for correctly remembered objects, while cross-classification for familiar objects was successful in the early period as well. These findings suggest that late neural signals of memory retrieval generalize across sensory modalities and stimulus types, and the dissociation between familiar and remembered objects may provide insight into the underlying processes.

The role of recollection, familiarity, and the hippocampus in episodic and working memory

The hippocampus plays an essential role in long-term episodic memory by supporting the recollection of contextual details, whereas surrounding regions such as the perirhinal cortex support familiarity-based recognition discriminations. Working memory - the ability to maintain information over very brief periods of time - is traditionally thought to rely heavily on frontoparietal attention networks, but recent work has shown that it can also rely on the hippocampus. However, the conditions in which the hippocampus becomes involved in working memory tasks are unclear and whether it contributes to recollection or familiarity-based responses in working memory is only beginning to be explored. In the current paper, we first review and contrast the existing amnesia literature examining recollection and familiarity in episodic and working memory. The results indicate that recollection and familiarity contribute to both episodic and working memory. However, in contrast to episodic memory, in working memory the hippocampus is particularly critical for familiarity-based rather than recollection-based discrimination. Moreover, the results indicate that the role of the hippocampus in working memory can be obscured due to 'criterion-induced process-masking' because it primarily supports intermediate-confidence recognition decisions. We then report results from a new working memory study examining the ability of amnesics to detect global and local changes in novel complex objects (i.e., fribbles), which indicates that the hippocampus plays an especially critical role in working memory when the task requires the detection of global rather than discrete changes. We conclude by considering the results in light of neurocomputational models and proposing a general framework for understanding the relationship between episodic and working memory.

Computational models can distinguish the contribution from different mechanisms to familiarity recognition

Familiarity is the strange feeling of knowing that something has already been seen in our past. Over the past decades, several attempts have been made to model familiarity using artificial neural networks. Recently, two learning algorithms successfully reproduced the functioning of the perirhinal cortex, a key structure involved during familiarity: Hebbian and anti-Hebbian learning. However, performance of these learning rules is very different from one to another thus raising the question of their complementarity. In this work, we designed two distinct computational models that combined Deep Learning and a Hebbian learning rule to reproduce familiarity on natural images, the Hebbian model and the anti-Hebbian model, respectively. We compared the performance of both models during different simulations to highlight the inner functioning of both learning rules. We showed that the anti-Hebbian model fits human behavioral data whereas the Hebbian model fails to fit the data under large training set sizes. Besides, we observed that only our Hebbian model is highly sensitive to homogeneity between images. Taken together, we interpreted these results considering the distinction between absolute and relative familiarity. With our framework, we proposed a novel way to distinguish the contribution of these familiarity mechanisms to the overall feeling of familiarity. By viewing them as complementary, our two models allow us to make new testable predictions that could be of interest to shed light on the familiarity phenomenon.

The amygdala is not necessary for the familiarity aspect of recognition memory

Dual-process accounts of item recognition posit two memory processes: slow but detailed recollection, and quick but vague familiarity. It has been proposed, based on prior rodent work, that the amygdala is critical for the familiarity aspect of item recognition. Here, we evaluated this proposal in male rhesus monkeys (Macaca mulatta) with selective bilateral excitotoxic amygdala damage. We used four established visual memory tests designed to assess different aspects of familiarity, all administered on touchscreen computers. Specifically, we assessed monkeys' tendencies to make low-latency false alarms, to make false alarms to recently seen lures, to produce curvilinear ROC curves, and to discriminate stimuli based on repetition across days. Three of the four tests showed no familiarity impairment and the fourth was explained by a deficit in reward processing. Consistent with this, amygdala damage did produce an anticipated deficit in reward processing in a three-arm-bandit gambling task, verifying the effectiveness of the lesions. Together, these results contradict prior rodent work and suggest that the amygdala is not critical for the familiarity aspect of item recognition.

Involuntary memory signals in the medial temporal lobe

We highlight recent progress in neuroimaging and neuropsychological research on memory mechanisms in the medial temporal lobe that speaks to the involuntary nature of memory retrieval processes. We suggest that evidence form these studies supports Barzykowski and Moulin's proposal that memory signals involved in experiences of familiarity and déjà vu can be generated in the absence of retrieval intentionality.

Perirhinal cortex automatically tracks multiple types of familiarity regardless of task-relevance

Perirhinal cortex (PrC) has long been implicated in familiarity assessment for objects and corresponding concepts. However, extant studies have focused mainly on changes in familiarity induced by recent exposure in laboratory settings. There is an increasing appreciation of other types of familiarity signals, in particular graded familiarity accumulated throughout one's lifetime. In prior work (Duke et al., 2017, Cortex, 89, 61-70), PrC has been shown to track lifetime familiarity ratings when participants make related judgements. A theoretically important characteristic of familiarity is its proposed automaticity. Support for automaticity comes from a documented impact of recent stimulus exposure on behavioral performance, and on PrC signals, under conditions in which this exposure is not task relevant. In the current fMRI study, we tested whether PrC also tracks lifetime familiarity of object concepts automatically, and whether this type of familiarity influences behavior even when it is not task relevant. During scanning, neurotypical participants (N = 30, age range 18-40, 7 males) provided animacy judgements about concrete object concepts presented at differing frequencies in an initial study phase. In a subsequent test phase, they made graded judgements of recent or lifetime familiarity. Behavioral performance showed sensitivity to lifetime familiarity even when it was not relevant for the task at hand. Across five sets of fMRI analyses, we found that PrC consistently tracked recent and lifetime familiarity of object concepts regardless of the task performed. Critically, while several other temporal-lobe regions also showed isolated familiarity effects, none of them tracked familiarity with the same consistency. These findings demonstrate that PrC automatically tracks multiple types of familiarity. They support models that assign a broad role in the representation of information about object concepts to this structure.

Context dissociations of the FN400 and N400 are evidence for recognition based on relative or absolute familiarity

Fluency, the ease of processing, can increase familiarity and the ability to recognize previously encountered information. Mixing or blocking the pre-experimental familiarity of test probes alters the pattern of recognition Event-Related Potentials (ERPs) for picture stimuli (Bruett & Leynes, 2015). The present study tested this idea further by mixing or blocking pre-experimental familiarity of word stimuli. A test with only pre-experimentally familiar words (Experiment 1) elicited the prototypical mid-frontal FN400 old/new ERP difference, which is a correlate of familiarity. However, tests with a mix of pre-experimentally familiar and unfamiliar words (Experiment 2) elicited posterior N400 old/new differences. Based on the Unexpected Fluency Attribution model (Mecklinger & Bader, 2020), this pattern of results suggests that pre-experimental fluency can influence the use of relative (changes from recent exposure) or absolute (baseline levels accrued from experience) familiarity to make a recognition judgment.

Category-sensitive incidental reinstatement in medial temporal lobe subregions during word recognition

During associative retrieval, the brain reinstates neural representations that were present during encoding. The human medial temporal lobe (MTL), with its subregions hippocampus (HC), perirhinal cortex (PRC), and parahippocampal cortex (PHC), plays a central role in neural reinstatement. Previous studies have given compelling evidence for reinstatement in the MTL during explicitly instructed associative retrieval. High-confident recognition may be similarly accompanied by recollection of associated information from the encoding context. It is unclear, however, whether high-confident recognition memory elicits reinstatement in the MTL even in the absence of an explicit instruction to retrieve associated information. Here, we addressed this open question using high-resolution fMRI. Twenty-eight male and female human volunteers engaged in a recognition memory task for words that they had previously encoded together with faces and scenes. Using complementary univariate and multivariate approaches, we show that MTL subregions including the PRC, PHC, and HC differentially reinstate category-sensitive representations during high-confident word recognition, even though no explicit instruction to retrieve the associated category was given. This constitutes novel evidence that high-confident recognition memory is accompanied by incidental reinstatement of associated category information in MTL subregions, and supports a functional model of the MTL that emphasizes content-sensitive representations during both encoding and retrieval.

Transcranial Direct Current Stimulation Over the Right Anterior Temporal Lobe Does Not Modulate False Recognition

Non-invasive transcranial direct current stimulation (tDCS) over the left anterior temporal lobe (ATL) has been shown to cause a reduction in the rate of false memories with semantically related words. Such a reduction seems to be specific to false memories induced by the study of associative lists, but is not observed when the studied lists are categorical in nature. These findings are interpreted as evidence that the left ATL functions as an integration hub that is crucial for the binding of semantic information into coherent representations of concepts. In order to investigate whether the right ATL might also contribute to semantic integration in the processing of verbal associative material, a follow-up tDCS study was conducted with the stimulation at study lateralized on the right ATL. A sample of 75 undergraduate students participated in an experiment in which they studied 8 associative lists and 8 categorical lists. One third of the participants studied all their word lists under anodal stimulation, another third studied under cathodal stimulation and the other third under sham stimulation. Results showed that stimulation of the right ATL by tDCS does not modulate false recognition for either association-related critical words or category-related critical words. These results provide preliminary support to views positing asymmetric connectivity between the anterior temporal lobes and the semantic representational network, and provide evidence for understanding bilateral brain dynamics and the nature of semantically induced memory distortions.

Lesion of the hippocampus selectively enhances LEC's activity during recognition memory based on familiarity

The sense of familiarity for events is crucial for successful recognition memory. However, the neural substrate and mechanisms supporting familiarity remain unclear. A major controversy in memory research is whether the parahippocampal areas, especially the lateral entorhinal (LEC) and the perirhinal (PER) cortices, support familiarity or whether the hippocampus (HIP) does. In addition, it is unclear if LEC, PER and HIP interact within this frame. Here, we especially investigate if LEC and PER's contribution to familiarity depends on hippocampal integrity. To do so, we compare LEC and PER neural activity between rats with intact hippocampus performing on a human to rat translational task relying on both recollection and familiarity and rats with hippocampal lesions that have been shown to then rely on familiarity to perform the same task. Using high resolution Immediate Early Gene imaging, we report that hippocampal lesions enhance activity in LEC during familiarity judgments but not PER’s. These findings suggest that different mechanisms support familiarity in LEC and PER and led to the hypothesis that HIP might exert a tonic inhibition on LEC during recognition memory that is released when HIP is compromised, possibly constituting a compensatory mechanism in aging and amnesic patients.

Functional Specialization of the Medial Temporal Lobes in Human Recognition Memory: Dissociating Effects of Hippocampal versus Parahippocampal Damage

A central debate in the systems neuroscience of memory concerns whether different medial temporal lobe (MTL) structures support different processes in recognition memory. Using two recognition memory paradigms, we tested a rare patient (MH) with a perirhinal lesion that appeared to spare the hippocampus. Consistent with a similar previous case, MH showed impaired familiarity and preserved recollection. When compared with patients with hippocampal lesions appearing to spare perirhinal cortex, MH showed greater impairment on familiarity and less on recollection. Nevertheless, the hippocampal patients also showed impaired familiarity compared with healthy controls. However, when replacing this traditional categorization of patients with analyses relating memory performance to continuous measures of damage across patients, hippocampal volume uniquely predicted recollection, whereas parahippocampal, rather than perirhinal, volume uniquely predicted familiarity. We consider whether the familiarity impairment in MH and our patients with hippocampal lesions arises from "subthreshold" damage to parahippocampal cortex (PHC). Our data provide the most compelling neuropsychological support yet for dual-process models of recognition memory, whereby recollection and familiarity depend on different MTL structures, and may support a role for PHC in familiarity. Our study highlights the value of supplementing single-case studies with examinations of continuous brain-behavior relationships across larger patient groups.

Recognition Receiver Operating Characteristic Curves: The Complex Influence of Input Statistics, Memory, and Decision-making

Receiver operating characteristic (ROC) analysis is the standard tool for studying recognition memory. In particular, the curvilinearity and the y-offset of recognition ROC curves have been interpreted as indicative of either memory strength (single-process models) or different memory processes (dual-process model). The distinction between familiarity and recollection has been widely studied in cognitive neuroscience in a variety of conditions, including lesions of different brain regions. We develop a computational model that explicitly shows how performance in recognition memory is affected by a complex and, as yet, underappreciated interplay of various factors, such as stimulus statistics, memory processing, and decision-making. We demonstrate that (1) the factors in the model affect recognition ROC curves in unexpected ways, (2) fitting R and F parameters according to the dual-process model is not particularly useful for understanding the underlying processes, and (3) the variability of recognition ROC curves and the controversies they have caused might be due to the uncontrolled variability in the contributing factors. Although our model is abstract, its functional components can be mapped onto brain regions, which are involved in corresponding functions. This enables us to reproduce and interpret in a coherent framework the diverse effects on recognition memory that have been reported in patients with frontal and hippocampal lesions. To conclude, our work highlights the importance of the rich interplay of a variety of factors in driving recognition memory performance, which has to be taken into account when interpreting recognition ROC curves.

Deficits in recent but not lifetime familiarity in amnestic mild cognitive impairment

People with amnestic mild cognitive impairment (aMCI) repeat questions, seemingly without any sense of familiarity (i.e., recognition of prior occurrence without recollection of episodic context). Accumulation of neurofibrillary tau in preclinical Alzheimer's disease begins in perirhinal cortex, a medial temporal lobe region linked to familiarity. Both observations would predict impaired familiarity assessment in aMCI; however, the extant evidence is mixed. To reveal familiarity impairments, it may be necessary to minimize the influence of recollection. In the current study, older adults with aMCI and healthy controls were administered two tasks on which a well-characterized patient (NB) with selective familiarity impairments due to surgical left temporal lobe excision sparing the hippocampus showed abnormal performance: frequency judgments for words exposed to in a recent study phase and judgments of cumulative lifetime familiarity for object concepts denoted by words. We also administered a process dissociation procedure (PDP) task that previously revealed spared familiarity in aMCI. We predicted that familiarity would be spared in aMCI on the PDP task, but impaired when assessed by frequency judgments for recent laboratory exposures and lifetime familiarity judgments. Familiarity was spared on the PDP task, but was impaired when probed with frequency judgments for recently exposed words in aMCI. Lifetime familiarity was also not impaired in aMCI. These results highlight the benefits of studying familiarity under conditions that minimize recollection and the value of frequency judgments in revealing familiarity deficits, and suggest that perirhinal cortex may not be necessary for accessing familiarity accumulated over a lifetime of experience.