The speed of visual recognition memory

Simultaneous invasive and non-invasive recordings in humans: A novel Rosetta stone for deciphering brain activity

Simultaneous noninvasive and invasive electrophysiological recordings provide a unique opportunity to achieve a comprehensive understanding of human brain activity, much like a Rosetta stone for human neuroscience. In this review we focus on the increasingly-used powerful combination of intracranial electroencephalography (iEEG) with scalp electroencephalography (EEG) or magnetoencephalography (MEG). We first provide practical insight on how to achieve these technically challenging recordings. We then provide examples from clinical research on how simultaneous recordings are advancing our understanding of epilepsy. This is followed by the illustration of how human neuroscience and methodological advances could benefit from these simultaneous recordings. We conclude with a call for open data sharing and collaboration, while ensuring neuroethical approaches and argue that only with a true collaborative approach the promises of simultaneous recordings will be fulfilled.

Identification of Early Hippocampal Dynamics during Recognition Memory with Independent Component Analysis

The hippocampus is generally considered to have relatively late involvement in recognition memory, its main electrophysiological signature being between 400 and 800 ms after stimulus onset. However, most electrophysiological studies have analyzed the hippocampus as a single responsive area, selecting only a single-site signal exhibiting the strongest effect in terms of amplitude. These classical approaches may not capture all the dynamics of this structure, hindering the contribution of other hippocampal sources that are not located in the vicinity of the selected site. We combined intracerebral electroencephalogram recordings from epileptic patients with independent component analysis during a recognition memory task involving the recognition of old and new images. We identified two sources with different responses emerging from the hippocampus: a fast one (maximal amplitude at ∼250 ms) that could not be directly identified from raw recordings and a latter one, peaking at ∼400 ms. The former component presented different amplitudes between old and new items in 6 out of 10 patients. The latter component had different delays for each condition, with a faster activation (∼290 ms after stimulus onset) for recognized items. We hypothesize that both sources represent two steps of hippocampal recognition memory, the faster reflecting the input from other structures and the latter the hippocampal internal processing. Recognized images evoking early activations would facilitate neural computation in the hippocampus, accelerating memory retrieval of complementary information. Overall, our results suggest that the hippocampal activity is composed of several sources with an early activation related to recognition memory.

Response-locked theta dissociations reveal potential feedback signal following successful retrieval

Successful memory retrieval relies on memory processes to access an internal representation and decision processes to evaluate and respond to the accessed representation, both of which are supported by fluctuations in theta (4-8Hz) activity. However, the extent to which decision making processes are engaged following a memory response is unclear. Here, we recorded scalp electroencephalography (EEG) while human participants performed a recognition memory task. We focused on response-locked data, allowing us to investigate the processes that occur prior to and following a memory response. We replicate previous work and find that prior to a memory response theta power is greater for identification of previously studied items (hits) relative to rejection of novel lures (correct rejections; CRs). Following the memory response, the theta power dissociation 'flips' whereby theta power is greater for CRs relative to hits. We find that the post-response 'flip' is more robust for hits that are committed quickly, potentially reflecting a positive feedback signal for strongly remembered experiences. Our findings suggest that there are potentially distinct processes occurring before and after a memory response that are modulated by successful memory retrieval.

Large-scale network dynamics underlying the first few hundred milliseconds after stimulus presentation: An investigation of visual recognition memory using iEEG

Recognition memory is the ability to recognize previously encountered objects. Even this relatively simple, yet extremely fast, ability requires the coordinated activity of large-scale brain networks. However, little is known about the sub-second dynamics of these networks. The majority of current studies into large-scale network dynamics is primarily based on imaging techniques suffering from either poor temporal or spatial resolution. We investigated the dynamics of large-scale functional brain networks underlying recognition memory at the millisecond scale. Specifically, we analyzed dynamic effective connectivity from intracranial electroencephalography while epileptic subjects (n = 18) performed a fast visual recognition memory task. Our data-driven investigation using Granger causality and the analysis of communities with the Louvain algorithm spotlighted a dynamic interplay of two large-scale networks associated with successful recognition. The first network involved the right visual ventral stream and bilateral frontal regions. It was characterized by early, predominantly bottom-up information flow peaking at 115 ms. It was followed by the involvement of another network with predominantly top-down connectivity peaking at 220 ms, mainly in the left anterior hemisphere. The transition between these two networks was associated with changes in network topology, evolving from a more segregated to a more integrated state. These results highlight that distinct large-scale brain networks involved in visual recognition memory unfold early and quickly, within the first 300 ms after stimulus onset. Our study extends the current understanding of the rapid network changes during rapid cognitive processes.

Incorporating Functional Response Time Effects into a Signal Detection Theory Model

Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401-409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4 R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.

Studying memory processes at different levels with simultaneous depth and surface EEG recordings

Investigating cognitive brain functions using non-invasive electrophysiology can be challenging due to the particularities of the task-related EEG activity, the depth of the activated brain areas, and the extent of the networks involved. Stereoelectroencephalographic (SEEG) investigations in patients with drug-resistant epilepsy offer an extraordinary opportunity to validate information derived from non-invasive recordings at macro-scales. The SEEG approach can provide brain activity with high spatial specificity during tasks that target specific cognitive processes (e.g., memory). Full validation is possible only when performing simultaneous scalp SEEG recordings, which allows recording signals in the exact same brain state. This is the approach we have taken in 12 subjects performing a visual memory task that requires the recognition of previously viewed objects. The intracranial signals on 965 contact pairs have been compared to 391 simultaneously recorded scalp signals at a regional and whole-brain level, using multivariate pattern analysis. The results show that the task conditions are best captured by intracranial sensors, despite the limited spatial coverage of SEEG electrodes, compared to the whole-brain non-invasive recordings. Applying beamformer source reconstruction or independent component analysis does not result in an improvement of the multivariate task decoding performance using surface sensor data. By analyzing a joint scalp and SEEG dataset, we investigated whether the two types of signals carry complementary information that might improve the machine-learning classifier performance. This joint analysis revealed that the results are driven by the modality exhibiting best individual performance, namely SEEG.

Are involuntary autobiographical memory and déjà vu natural products of memory retrieval?

Involuntary autobiographical memories (IAMs) and déjà vu are phenomena that occur spontaneously in daily life. IAMs are recollections of the personal past, whereas déjà vu is defined as an experience in which the person feels familiarity at the same time as knowing that the familiarity is false. We present and discuss the idea that both IAMs and déjà vu can be explained as natural phenomena resulting from memory processing and, importantly, are both based on the same memory retrieval processes. Briefly, we hypothesise that both can be described as "involuntary" or spontaneous cognitions, where IAMs deliver content and déjà vu delivers only the feeling of retrieval. We map out the similarities and differences between the two, making a theoretical and neuroscientific account for their integration into models of memory retrieval and how the autobiographical memory literature can explain these quirks of daily life and unusual but meaningful phenomena. We explain the emergence of the déjà vu phenomenon by relating it to well-known mechanisms of autobiographical memory retrieval, concluding that IAMs and déjà vu lie on a continuum.

The Efficiency of Demography in Face Perception

When we look at a face, we cannot help but "read" it: Beyond simply processing its identity, we also form robust impressions of both transient psychological states (e.g., surprise) and stable character traits (e.g., trustworthiness). But perhaps the most fundamental traits we extract from faces are their social demographics, for example, race, age, and gender. How much exposure is required to extract such properties? Curiously, despite extensive work on the temporal efficiency of extracting both higher-level social properties (such as competence and dominance) and more basic characteristics (such as identity and familiarity), this question remains largely unexplored for demography. We correlated observers' percepts of the race/age/gender of unfamiliar faces viewed at several brief durations (and then masked) with their judgments after unlimited exposure. Performance reached asymptote by 100 ms, was above chance by only 33.33 ms, and had a similar temporal profile to detecting faces in the first place. This was true even when the property to be reported wasn't revealed until after the face had disappeared, and when the faces were matched for several lower-level visual properties. Collectively, these results demonstrate that the extraction of demographic features from faces is highly efficient, and can truly be done at a glance.

Revisiting the remember-know task: Replications of Gardiner and Java (1990)

One of the most evidential behavioral results for two memory processes comes from Gardiner and Java (Memory & Cognition, 18, 23-30 1990). Participants provided more "remember" than "know" responses for old words but more know than remember responses for old nonwords. Moreover, there was no effect of word/nonword status for new items. The combination of a crossover interaction for old items with an invariance for new items provides strong evidence for two distinct processes while ruling out criteria or bias explanations. Here, we report a modern replication of this study. In three experiments, (Experiments 1, 2, and 4) with larger numbers of items and participants, we were unable to replicate the crossover. Instead, our data are more consistent with a single-process account. In a fourth experiment (Experiment 3), we were able to replicate Gardiner and Java's baseline results with a sure-unsure paradigm supporting a single-process explanation. It seems that Gardiner and Java's remarkable crossover result is not replicable.

A Fast Visual Recognition Memory System in Humans Identified Using Intracerebral ERP

One key item of information retrieved when surveying our visual world is whether or not objects are familiar. However, there is no consensus on the respective roles of medial temporal lobe structures, particularly the perirhinal cortex (PRC) and hippocampus. We considered whether the PRC could support a fast recognition memory system independently from the hippocampus. We recorded the intracerebral electroencephalograph activity of epileptic patients while they were performing a fast visual recognition memory task, constraining them to use their quickest strategy. We performed event-related potential (ERP) and classification analyses. The PRC was, by far, the earliest region involved in recognition memory. This activity occurred before the first behavioral responses and was found to be related to reaction times, unlike the hippocampus. Single-trial analyses showed that decoding power was equivalent in the PRC and hippocampus but occurred much earlier in the PRC. A critical finding was that recognition memory-related activity occurred in different frontal and parietal regions, including the supplementary motor area, before the hippocampus. These results, based on ERP analyses, suggest that the human brain is equipped with a fast recognition memory system, which may bypass the hippocampus and in which the PRC plays a critical role.

Improving the integrative memory model by integrating the temporal dynamics of memory

Despite highlighting the role of the attribution system and proposing a coherent large-scale architecture of declarative memory, the integrative memory model would be more “integrative” if the temporal dynamics of the interactions between its components was clarified. This is necessary to make predictions in patients with brain injury and hypothesize dissociations.

Superior explicit memory despite severe developmental amnesia: In-depth case study and neural correlates

The acquisition of new semantic memories is sometimes preserved in patients with hippocampal amnesia. Robust evidence for this comes from case reports of developmental amnesia suggesting that low-to-normal levels of semantic knowledge can be achieved despite compromised episodic learning. However, it is unclear whether this relative preservation of semantic memory results from normal acquisition and retrieval or from residual episodic memory, combined with effortful repetition. Furthermore, lesion studies have mainly focused on the hippocampus itself, and have seldom reported the state of structures in the extended hippocampal system. Preserved components of this system may therefore mediate residual episodic abilities, contributing to the apparent semantic preservation. We report an in-depth study of Patient KA, a 27-year-old man who had severe hypoxia at birth, in which we carefully explored his residual episodic learning abilities. We used novel speeded recognition paradigms to assess whether KA could explicitly acquire and retrieve new context-free memories. Despite a pattern of very severe amnesia, with a 44-point discrepancy between his intelligence and memory quotients, KA exhibited normal-to-superior levels of knowledge, even under strict time constraints. He also exhibited normal-to-superior recognition memory for new material, again under strict time constraints. Multimodal neuroimaging revealed an unusual pattern of selective atrophy within each component of the extended hippocampal system, contrasting with the preservation of anterior subhippocampal cortices. A cortical thickness analysis yielded a pattern of thinner but also thicker regional cortices, pointing toward specific temporal lobe reorganization following early injury. We thus report the first case of superior explicit learning and memory in a severe case of amnesia, raising important questions about how such knowledge can be acquired.

Object and proper name retrieval in temporal lobe epilepsy: a study of difficulties and latencies

PURPOSE

Retrieving a specific name is sometimes difficult and can be even harder when pathology affects the temporal lobes. Word finding difficulties have been well documented in temporal lobe epilepsy (TLE) but analyses have mostly concentrated on the study of accuracy. Our aim here was to go beyond simple accuracy and to provide both a quantitative and a qualitative assessment of naming difficulties and latencies in patients with TLE.

METHODS

Thirty-two patients with temporal lobe epilepsy (16 with epilepsy affecting the cerebral hemisphere dominant for language (D-TLE) and 16 with epilepsy affecting the cerebral hemisphere non-dominant for language (ND-TLE)) and 34 healthy matched control subjects were included in the study. The experiment involved naming 70 photographs of objects and 70 photographs of celebrities as fast as possible. Accuracy and naming reaction times were recorded. Following each trial, a questionnaire was used to determine the specific nature of each subject's difficulty in retrieving the name (e.g., no difficulty, paraphasia, tip of the tongue, feeling of knowing the name, etc). Reaction times were analysed both across subjects and across trials.

KEY FINDINGS

D-TLE patients showed consistent and quasi-systematic impairment compared to matched control subjects on both object and famous people naming. This impairment was characterized not only by lower accuracy but also by more qualitative errors and tip of the tongue phenomena. Furthermore, minimum reaction times were slowed down by about 70 ms for objects and 150 ms for famous people naming. In contrast, patients with ND-TLE were less impaired, and their impairment was limited to object naming.

SIGNIFICANCE

These results suggest that patients with TLE, in particular D-TLE, show a general impairment of lexical access. Furthermore, there was evidence of subtle difficulties (increased reaction times) in patients with TLE.

Recognition memory and the medial temporal lobe: from monkey research to human pathology

This review provides a historical overview of decades of research on recognition memory, the process that allows both humans and animals to tell familiar from novel items. The emphasis is put on how monkey research improved our understanding of the medial temporal lobe (MTL) role and how tasks designed for monkeys influenced research in humans. The story starts in the early 1950s. Back then, memory was not a fashionable scientific topic. It was viewed as a function of the whole brain and not of specialized brain areas. All that changed in 1957-1958 when Brenda Milner, a neuropsychologist from Montreal, described patient H.M. He forgot all events as he lived them despite a fully preserved intelligence. He had received a MTL resection to relieve epilepsy. H.M. (1926-2008) would become the most influential patient in brain science. Which structures among those included in H.M.'s large lesion were important for recognition memory could not be evaluated in humans. It was gradually understood only after the successful development of a monkey model of human amnesia by Mishkin in 1978. Selective lesions and two behavioral tasks, delayed nonmatching-to-sample and visual paired comparison, were used to distinguish the contribution of the hippocampus from that of adjacent cortical areas. Driven by findings in non-human primates, human research on recognition memory is now trying to solve the question of whether the different structures composing MTL contributes to familiarity and recollection, the two possible forms taken by recognition. We described in particular two French patients, FRG and JMG, whose deficits support the currently dominant model attributing to the perirhinal cortex a critical role in recognition memory. Research on recognition memory has implications for the clinician as it may help understanding the cognitive deficits observed in different diseases. An illustration of such approach, linking basic and applied research, is provided for Alzheimer's disease.