Plugging in to Human Memory: Advantages, Challenges, and Insights from Human Single-Neuron Recordings

Decoding dynamic visual scenes across the brain hierarchy

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Theta-phase locking of single neurons during human spatial memory

The precise timing of single-neuron activity in relation to local field potentials may support various cognitive functions. Extensive research in rodents, along with some evidence in humans, suggests that single-neuron activity at specific phases of theta oscillations plays a crucial role in memory processes. Our fundamental understanding of such theta-phase locking in humans and its dependency on basic electrophysiological properties of the local field potential is still limited, however. Here, using single-neuron recordings in epilepsy patients performing a spatial memory task, we thus aimed at improving our understanding of factors modulating theta-phase locking in the human brain. Combining a generalized-phase approach for frequency-adaptive theta-phase estimation with time-resolved spectral parameterization, our results show that theta-phase locking is a strong and prevalent phenomenon across human medial temporal lobe regions, both during spatial memory encoding and retrieval. Neuronal theta-phase locking increased during periods of elevated theta power, when clear theta oscillations were present, and when aperiodic activity exhibited steeper slopes. Theta-phase locking was similarly strong during successful and unsuccessful memory, and most neurons activated at similar theta phases between encoding and retrieval. Some neurons changed their preferred theta phases between encoding and retrieval, in line with the idea that different memory processes are separated within the theta cycle. Together, these results help disentangle how different properties of local field potentials and memory states influence theta-phase locking of human single neurons. This contributes to a better understanding of how interactions between single neurons and local field potentials may support human spatial memory.

Neuronal population representation of human emotional memory

Understanding how emotional processing modulates learning and memory is crucial for the treatment of neuropsychiatric disorders characterized by emotional memory dysfunction. We investigate how human medial temporal lobe (MTL) neurons support emotional memory by recording spiking activity from the hippocampus, amygdala, and entorhinal cortex during encoding and recognition sessions of an emotional memory task in patients with pharmaco-resistant epilepsy. Our findings reveal distinct representations for both remembered compared to forgotten and emotional compared to neutral scenes in single units and MTL population spiking activity. Additionally, we demonstrate that a distributed network of human MTL neurons exhibiting mixed selectivity on a single-unit level collectively processes emotion and memory as a network, with a small percentage of neurons responding conjointly to emotion and memory. Analyzing spiking activity enables a detailed understanding of the neurophysiological mechanisms underlying emotional memory and could provide insights into how emotion alters memory during healthy and maladaptive learning.

Ripple-locked coactivity of stimulus-specific neurons and human associative memory

Associative memory enables the encoding and retrieval of relations between different stimuli. To better understand its neural basis, we investigated whether associative memory involves temporally correlated spiking of medial temporal lobe (MTL) neurons that exhibit stimulus-specific tuning. Using single-neuron recordings from patients with epilepsy performing an associative object-location memory task, we identified the object-specific and place-specific neurons that represented the separate elements of each memory. When patients encoded and retrieved particular memories, the relevant object-specific and place-specific neurons activated together during hippocampal ripples. This ripple-locked coactivity of stimulus-specific neurons emerged over time as the patients' associative learning progressed. Between encoding and retrieval, the ripple-locked timing of coactivity shifted, suggesting flexibility in the interaction between MTL neurons and hippocampal ripples according to behavioral demands. Our results are consistent with a cellular account of associative memory, in which hippocampal ripples coordinate the activity of specialized cellular populations to facilitate links between stimuli.

A Computational Framework for Memory Engrams

Memory engrams in mice brains are potentially related to groups of concept cells in human brains. A single concept cell in human hippocampus responds, for example, not only to different images of the same object or person but also to its name written down in characters. Importantly, a single mental concept (object or person) is represented by several concept cells and each concept cell can respond to more than one concept. Computational work shows how mental concepts can be embedded in recurrent artificial neural networks as memory engrams and how neurons that are shared between different engrams can lead to associations between concepts. Therefore, observations at the level of neurons can be linked to cognitive notions of memory recall and association chains between memory items.

A dynamic attractor network model of memory formation, reinforcement and forgetting

Empirical evidence shows that memories that are frequently revisited are easy to recall, and that familiar items involve larger hippocampal representations than less familiar ones. In line with these observations, here we develop a modelling approach to provide a mechanistic understanding of how hippocampal neural assemblies evolve differently, depending on the frequency of presentation of the stimuli. For this, we added an online Hebbian learning rule, background firing activity, neural adaptation and heterosynaptic plasticity to a rate attractor network model, thus creating dynamic memory representations that can persist, increase or fade according to the frequency of presentation of the corresponding memory patterns. Specifically, we show that a dynamic interplay between Hebbian learning and background firing activity can explain the relationship between the memory assembly sizes and their frequency of stimulation. Frequently stimulated assemblies increase their size independently from each other (i.e. creating orthogonal representations that do not share neurons, thus avoiding interference). Importantly, connections between neurons of assemblies that are not further stimulated become labile so that these neurons can be recruited by other assemblies, providing a neuronal mechanism of forgetting.

Hippocampal neurons code individual episodic memories in humans

The hippocampus is an essential hub for episodic memory processing. However, how human hippocampal single neurons code multi-element associations remains unknown. In particular, it is debated whether each hippocampal neuron represents an invariant element within an episode or whether single neurons bind together all the elements of a discrete episodic memory. Here we provide evidence for the latter hypothesis. Using single-neuron recordings from a total of 30 participants, we show that individual neurons, which we term episode-specific neurons, code discrete episodic memories using either a rate code or a temporal firing code. These neurons were observed exclusively in the hippocampus. Importantly, these episode-specific neurons do not reflect the coding of a particular element in the episode (that is, concept or time). Instead, they code for the conjunction of the different elements that make up the episode.

Memory: Synaptic or Cellular, That Is the Question

According to the commonly accepted opinion, memory engrams are formed and stored at the level of neural networks due to a change in the strength of synaptic connections between neurons. This hypothesis of synaptic plasticity (HSP), formulated by Donald Hebb in the 1940s, continues to dominate the directions of experimental studies and the interpretations of experimental results in the field. The universal acceptance of the HSP has transformed it from a hypothesis into an incontrovertible theory. In this article, I show that the entire body of experimental and clinical data obtained in studies of long-term memory in mammals and humans is inconsistent with the HSP. Instead, these data suggest that long-term memory is formed and stored at the intracellular level where it is reliably protected from ongoing synaptic activity, including pathological epileptic activity. It seems that the generally accepted HSP became a serious obstacle to understanding the mechanisms of memory and that progress in this field requires rethinking this doctrine and shifting experimental efforts toward exploring the intracellular mechanisms.

Single neuron responses underlying face recognition in the human midfusiform face-selective cortex

Faces are critical for social interactions and their recognition constitutes one of the most important and challenging functions of the human brain. While neurons responding selectively to faces have been recorded for decades in the monkey brain, face-selective neural activations have been reported with neuroimaging primarily in the human midfusiform gyrus. Yet, the cellular mechanisms producing selective responses to faces in this hominoid neuroanatomical structure remain unknown. Here we report single neuron recordings performed in 5 human subjects (1 male, 4 females) implanted with intracerebral microelectrodes in the face-selective midfusiform gyrus, while they viewed pictures of familiar and unknown faces and places. We observed similar responses to faces and places at the single cell level, but a significantly higher number of neurons responding to faces, thus offering a mechanistic account for the face-selective activations observed in this region. Although individual neurons did not respond preferentially to familiar faces, a population level analysis could consistently determine whether or not the faces (but not the places) were familiar, only about 50 ms after the initial recognition of the stimuli as faces. These results provide insights into the neural mechanisms of face processing in the human brain.

Neural implants without brain surgery

Injectable bioprobes record single-neuron activity from within blood vessels.

An integrative view of human hippocampal function: Differences with other species and capacity considerations

We describe an integrative model that encodes associations between related concepts in the human hippocampal formation, constituting the skeleton of episodic memories. The model, based on partially overlapping assemblies of "concept cells," contrast markedly with the well-established notion of pattern separation, which relies on conjunctive, context dependent single neuron responses, instead of the invariant, context independent responses found in the human hippocampus. We argue that the model of partially overlapping assemblies is better suited to cope with memory capacity limitations, that the finding of different types of neurons and functions in this area is due to a flexible and temporary use of the extraordinary machinery of the hippocampus to deal with the task at hand, and that only information that is relevant and frequently revisited will consolidate into long-term hippocampal representations, using partially overlapping assemblies. Finally, we propose that concept cells are uniquely human and that they may constitute the neuronal underpinnings of cognitive abilities that are much further developed in humans compared to other species.

Transcranial magnetic stimulation to the angular gyrus modulates the temporal dynamics of the hippocampus and entorhinal cortex

Transcranial magnetic stimulation (TMS) delivered to the angular gyrus (AG) affects hippocampal function and associated behaviors (Thakral PP, Madore KP, Kalinowski SE, Schacter DL. Modulation of hippocampal brain networks produces changes in episodic simulation and divergent thinking. 2020a. Proc Natl Acad Sci U S A. 117:12729-12740). Here, we examine if functional magnetic resonance imaging (fMRI)-guided TMS disrupts the gradient organization of temporal signal properties, known as the temporal organization, in the hippocampus (HPC) and entorhinal cortex (ERC). For each of 2 TMS sessions, TMS was applied to either a control site (vertex) or to a left AG target region (N = 18; 14 females). Behavioral measures were then administered, and resting-state scans were acquired. Temporal dynamics were measured by tracking change in the fMRI signal (i) "within" single voxels over time, termed single-voxel autocorrelation and (ii) "between" different voxels over time, termed intervoxel similarity. TMS reduced AG connectivity with the hippocampal target and induced more rapid shifting of activity in single voxels between successive time points, lowering the single-voxel autocorrelation, within the left anteromedial HPC and posteromedial ERC. Intervoxel similarity was only marginally affected by TMS. Our findings suggest that hippocampal-targeted TMS disrupts the functional properties of the target site along the anterior/posterior axis. Further studies should examine the consequences of altering the temporal dynamics of these medial temporal areas to the successful processing of episodic information under task demand.

A wearable platform for closed-loop stimulation and recording of single-neuron and local field potential activity in freely moving humans

Advances in technologies that can record and stimulate deep brain activity in humans have led to impactful discoveries within the field of neuroscience and contributed to the development of novel therapies for neurological and psychiatric disorders. Further progress, however, has been hindered by device limitations in that recording of single-neuron activity during freely moving behaviors in humans has not been possible. Additionally, implantable neurostimulation devices, currently approved for human use, have limited stimulation programmability and restricted full-duplex bidirectional capability. In this study, we developed a wearable bidirectional closed-loop neuromodulation system (Neuro-stack) and used it to record single-neuron and local field potential activity during stationary and ambulatory behavior in humans. Together with a highly flexible and customizable stimulation capability, the Neuro-stack provides an opportunity to investigate the neurophysiological basis of disease, develop improved responsive neuromodulation therapies, explore brain function during naturalistic behaviors in humans and, consequently, bridge decades of neuroscientific findings across species.

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, the 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

When shared concept cells support associations: Theory of overlapping memory engrams

Assemblies of neurons, called concepts cells, encode acquired concepts in human Medial Temporal Lobe. Those concept cells that are shared between two assemblies have been hypothesized to encode associations between concepts. Here we test this hypothesis in a computational model of attractor neural networks. We find that for concepts encoded in sparse neural assemblies there is a minimal fraction c_min of neurons shared between assemblies below which associations cannot be reliably implemented; and a maximal fraction c_max of shared neurons above which single concepts can no longer be retrieved. In the presence of a periodically modulated background signal, such as hippocampal oscillations, recall takes the form of association chains reminiscent of those postulated by theories of free recall of words. Predictions of an iterative overlap-generating model match experimental data on the number of concepts to which a neuron responds.

Experimental evidence suggests that associations between concepts are encoded in the hippocampus by cells shared between neuronal assemblies (“overlap” of concepts). What is the necessary overlap that ensures a reliable encoding of associations? Under which conditions can associations induce a simultaneous or a chain-like activation of concepts? Our theoretical model shows that the ideal overlap presents a tradeoff: the overlap should be larger than a minimum value in order to reliably encode associations, but lower than a maximum value to prevent loss of individual memories. Our theory explains experimental data from human Medial Temporal Lobe and provides a mechanism for chain-like recall in presence of inhibition, while still allowing for simultaneous recall if inhibition is weak.

Retrospective memory integration accompanies reconfiguration of neural cell assemblies

Memory is a dynamic process that is based on and can be altered by experiences. Integrating memories of multiple experiences (memory integration) is the basis of flexible and complex decision-making. However, the mechanism of memory integration in neural networks of the brain remains poorly understood. In this study, we built a recurrent spiking network model and investigated the neural mechanism of memory integration before a decision is made (retrospective memory integration) at the neural circuit level. Our simulations suggest that retrospective memory integration accompanies reconfiguration of neural cell assemblies. Additionally, partially blocking neural network plasticity leads to failure of memory integration. These findings can potentially guide the experimental investigation of the neural mechanism of retrospective memory integration and serve as the basis for developing new artificial intelligence algorithms.

Drifting assemblies for persistent memory: Neuron transitions and unsupervised compensation

Change is ubiquitous in living beings. In particular, the connectome and neural representations can change. Nevertheless, behaviors and memories often persist over long times. In a standard model, associative memories are represented by assemblies of strongly interconnected neurons. For faithful storage these assemblies are assumed to consist of the same neurons over time. Here we propose a contrasting memory model with complete temporal remodeling of assemblies, based on experimentally observed changes of synapses and neural representations. The assemblies drift freely as noisy autonomous network activity and spontaneous synaptic turnover induce neuron exchange. The gradual exchange allows activity-dependent and homeostatic plasticity to conserve the representational structure and keep inputs, outputs, and assemblies consistent. This leads to persistent memory. Our findings explain recent experimental results on temporal evolution of fear memory representations and suggest that memory systems need to be understood in their completeness as individual parts may constantly change.

Biological constraints on neural network models of cognitive function

Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.

Hippocampal ripples and their coordinated dialogue with the default mode network during recent and remote recollection

Hippocampal ripples are prominent synchronization events generated by hippocampal neuronal assemblies. To date, ripples have been primarily associated with navigational memory in rodents and short-term episodic recollections in humans. Here, we uncover different profiles of ripple activity in the human hippocampus during the retrieval of recent and remote autobiographical events and semantic facts. We found that the ripple rate increased significantly before reported recall compared to control conditions. Patterns of ripple activity across multiple hippocampal sites demonstrated remarkable specificity for memory type. Intriguingly, these ripple patterns revealed a semantization dimension, in which patterns associated with autobiographical contents become similar to those of semantic memory as a function of memory age. Finally, widely distributed sites across the neocortex exhibited ripple-coupled activations during recollection, with the strongest activation found within the default mode network. Our results thus reveal a key role for hippocampal ripples in orchestrating hippocampal-cortical communication across large-scale networks involved in conscious recollection.

Heterogeneity across neural populations: Its significance for the dynamics and functions of neural circuits

Neural populations show patterns of synchronous activity, as they share common correlated inputs. Neurons in the cortex that are connected by strong synapses cause rapid firing explosions. In addition, areas that are connected by weaker synapses have a slower dynamics and they can contribute to asymmetries in the input distributions. The aim of this work is to develop a neural model to investigate how the heterogeneities in the synaptic input distributions affect different levels of organizational activity in the brain dynamics. We analytically show how small changes in the correlation inputs can cause large changes in the interactions of the outputs that lead to a phase transition, demonstrating that a simple variation in the direction of a biased skewed distribution in the neuronal inputs can generate a transition of states in the firing rate, passing from spontaneous silence ("down state") to an absolute spiking activity ("up state"). We present an exact quantification of the dynamics of the output variables, showing that when considering a biased skewed distribution in the inputs of neuronal population, the critical point is not in an asynchronous or synchronous state but rather at an intermediate value.

No consolidation without representation: Correspondence between neural and psychological representations in recent and remote memory

Memory systems consolidation is often conceived as the linear, time-dependent, neurobiological shift of memory from hippocampal-cortical to cortico-cortical dependency. We argue that contrary to this unidirectional view of memory reorganization, information about events may be retained in multiple forms (e.g., event-specific sensory-near episodic memory, event-specific gist information, event-general schematic information, or abstract semantic memory). These representations can all form at the time of the event and may continue to coexist for long durations. Their relative strength, composition, and dominance of expression change with time and experience, with task demands, and through their dynamic interaction with one another. These different psychological mnemonic representations depend on distinct functional and structural neurobiological substrates such that there is a neural-psychological representation correspondence (NPRC) among them. We discuss how the dynamics of psychological memory representations are reflected in multiple levels of neurobiological markers and their interactions. By this view, there are only variations of synaptic consolidation and memory dynamics without assuming a distinct systems consolidation process.

Cognitive and Emotional Mapping With SEEG

Mapping of cortical functions is critical for the best clinical care of patients undergoing epilepsy and tumor surgery, but also to better understand human brain function and connectivity. The purpose of this review is to explore existing and potential means of mapping higher cortical functions, including stimulation mapping, passive mapping, and connectivity analyses. We examine the history of mapping, differences between subdural and stereoelectroencephalographic approaches, and some risks and safety aspects, before examining different types of functional mapping. Much of this review explores the prospects for new mapping approaches to better understand other components of language, memory, spatial skills, executive, and socio-emotional functions. We also touch on brain-machine interfaces, philosophical aspects of aligning tasks to brain circuits, and the study of consciousness. We end by discussing multi-modal testing and virtual reality approaches to mapping higher cortical functions.

How Human Single-Neuron Recordings Can Help Us Understand Cognition: Insights from Memory Studies

Understanding human cognition is a key goal of contemporary neuroscience. Due to the complexity of the human brain, animal studies and noninvasive techniques, however valuable, are incapable of providing us with a full understanding of human cognition. In the light of existing cognitive theories, we describe findings obtained thanks to human single-neuron recordings, including the discovery of concept cells and novelty-dependent cells, or activity patterns behind working memory, such as persistent activity. We propose future directions for studies using human single-neuron recordings and we discuss possible opportunities of investigating pathological brain.

The Architecture of Human Memory: Insights from Human Single-Neuron Recordings

Deciphering the mechanisms of human memory is a central goal of neuroscience, both from the point of view of the fundamental biology of memory and for its translational relevance. Here, we review some contributions that recordings from neurons in humans implanted with electrodes for clinical purposes have made toward this goal. Recordings from the medial temporal lobe, including the hippocampus, reveal the existence of two classes of cells: those encoding highly selective and invariant representations of abstract concepts, and memory-selective cells whose activity is related to familiarity and episodic retrieval. Insights derived from observing these cells in behaving humans include that semantic representations are activated before episodic representations, that memory content and memory strength are segregated, and that the activity of both types of cells is related to subjective awareness as expected from a substrate for declarative memory. Visually selective cells can remain persistently active for several seconds, thereby revealing a cellular substrate for working memory in humans. An overarching insight is that the neural code of human memory is interpretable at the single-neuron level. Jointly, intracranial recording studies are starting to reveal aspects of the building blocks of human memory at the single-cell level. This work establishes a bridge to cellular-level work in animals on the one hand, and the extensive literature on noninvasive imaging in humans on the other hand. More broadly, this work is a step toward a detailed mechanistic understanding of human memory that is needed to develop therapies for human memory disorders.

Hippocampal Contribution to Ordinal Psychological Time in the Human Brain

The chronology of events in time-space is naturally available to the senses, and the spatial and temporal dimensions of events entangle in episodic memory when navigating the real world. The mapping of time-space during navigation in both animals and humans implicates the hippocampal formation. Yet, one arguably unique human trait is the capacity to imagine mental chronologies that have not been experienced but may involve real events-the foundation of causal reasoning. Herein, we asked whether the hippocampal formation is involved in mental navigation in time (and space), which requires internal manipulations of events in time and space from an egocentric perspective. To address this question, we reanalyzed a magnetoencephalography data set collected while participants self-projected in time or in space and ordered historical events as occurring before/after or west/east of the mental self [Gauthier, B., Pestke, K., & van Wassenhove, V. Building the arrow of time… Over time: A sequence of brain activity mapping imagined events in time and space. Cerebral Cortex, 29, 4398-4414, 2019]. Because of the limitations of source reconstruction algorithms in the previous study, the implication of hippocampus proper could not be explored. Here, we used a source reconstruction method accounting explicitly for the hippocampal volume to characterize the involvement of deep structures belonging to the hippocampal formation (bilateral hippocampi [hippocampi proper], entorhinal cortices, and parahippocampal cortex). We found selective involvement of the medial temporal lobes (MTLs) with a notable lateralization of the main effects: Whereas temporal ordinality engaged mostly the left MTL, spatial ordinality engaged mostly the right MTL. We discuss the possibility of a top-down control of activity in the human hippocampal formation during mental time (and space) travels.

Dynamic network interactions among distinct brain rhythms as a hallmark of physiologic state and function

Brain rhythms are associated with a range of physiologic states, and thus, studies have traditionally focused on neuronal origin, temporal dynamics and fundamental role of individual brain rhythms, and more recently on specific pair-wise interactions. Here, we aim to understand integrated physiologic function as an emergent phenomenon of dynamic network interactions among brain rhythms. We hypothesize that brain rhythms continuously coordinate their activations to facilitate physiologic states and functions. We analyze healthy subjects during sleep, and we demonstrate the presence of stable interaction patterns among brain rhythms. Probing transient modulations in brain wave activation, we discover three classes of interaction patterns that form an ensemble representative for each sleep stage, indicating an association of each state with a specific network of brain-rhythm communications. The observations are universal across subjects and identify networks of brain-rhythm interactions as a hallmark of physiologic state and function, providing new insights on neurophysiological regulation with broad clinical implications.