Single-cell responses to face adaptation in the human medial temporal lobe

Multimodal investigations of emotional face processing and social trait judgment of faces

Faces are among the most important visual stimuli that humans perceive in everyday life. While extensive literature has examined emotional processing and social evaluations of faces, most studies have examined either topic using unimodal approaches. In this review, we promote the use of multimodal cognitive neuroscience approaches to study these processes, using two lines of research as examples: ambiguity in facial expressions of emotion and social trait judgment of faces. In the first set of studies, we identified an event-related potential that signals emotion ambiguity using electroencephalography and we found convergent neural responses to emotion ambiguity using functional neuroimaging and single-neuron recordings. In the second set of studies, we discuss how different neuroimaging and personality-dimensional approaches together provide new insights into social trait judgments of faces. In both sets of studies, we provide an in-depth comparison between neurotypicals and people with autism spectrum disorder. We offer a computational account for the behavioral and neural markers of the different facial processing between the two groups. Finally, we suggest new practices for studying the emotional processing and social evaluations of faces. All data discussed in the case studies of this review are publicly available.

Consciousness: a neurosurgical perspective

Neurosurgeons are in a unique position to shed light on the neural basis for consciousness, not only by their clinical care of patients with compromised states of consciousness, but also by employing neurostimulation and neuronal recordings through intracranial electrodes in awake surgical patients, as well as during stages of sleep and anethesia. In this review, we discuss several aspects of consciousness, i.e., perception, memory, and willed actions, studied by electrical stimulation and single neuron recordings in the human brain. We demonstrate how specific neuronal activity underlie the emergence of concepts, memories, and intentions in human consciousness. We discuss the representation of specific conscious content by temporal lobe neurons and present the discovery of "concept cells" and the encoding and retrieval of memories by neurons in the medial temporal lobe. We review prefrontal and parietal neuronal activation that precedes conscious intentions to act. Taken together with other studies in the field, these findings suggest that specific conscious experience may arise from stochastic fluctuations of neuronal activity, reaching a dynamic threshold. Advances in brain recording and stimulation technology coupled with the rapid rise in artificial intelligence are likely to increase the amount and analysis capabilities of data obtained from the human brain, thereby improving the decoding of conscious and preconscious states and open new horizons for modulation of human cognitive functions such as memory and volition.

An integrative view of human hippocampal function: Differences with other species and capacity considerations

We describe an integrative model that encodes associations between related concepts in the human hippocampal formation, constituting the skeleton of episodic memories. The model, based on partially overlapping assemblies of "concept cells," contrast markedly with the well-established notion of pattern separation, which relies on conjunctive, context dependent single neuron responses, instead of the invariant, context independent responses found in the human hippocampus. We argue that the model of partially overlapping assemblies is better suited to cope with memory capacity limitations, that the finding of different types of neurons and functions in this area is due to a flexible and temporary use of the extraordinary machinery of the hippocampus to deal with the task at hand, and that only information that is relevant and frequently revisited will consolidate into long-term hippocampal representations, using partially overlapping assemblies. Finally, we propose that concept cells are uniquely human and that they may constitute the neuronal underpinnings of cognitive abilities that are much further developed in humans compared to other species.

Neurons as will and representation

Memory recollections and voluntary actions are often perceived as spontaneously generated irrespective of external stimuli. Although products of our neurons, they are only rarely accessible in humans at the neuronal level. Here I review insights gleaned from unique neurosurgical opportunities to record and stimulate single-neuron activity in people who can declare their thoughts, memories and wishes. I discuss evidence that the subjective experience of human recollection and that of voluntary action arise from the activity of two internal neuronal generators, the former from medial temporal lobe reactivation and the latter from frontoparietal preactivation. I characterize properties of these generators and their interaction, enabling flexible recruitment of memory-based choices for action as well as recruitment of action-based plans for the representation of conceptual knowledge in memories. Both internal generators operate on surprisingly explicit but different neuronal codes, which appear to arise with distinct single-neuron activity, often observed before participants' reports of conscious awareness. I discuss prediction of behaviour based on these codes, and the potential for their modulation. The prospects of editing human memories and volitions by enhancement, inception or deletion of specific, selected content raise therapeutic possibilities and ethical concerns.

The universal brain code a genetic mechanism for memory

We do not have an understanding of the fundamental mechanism of how information is stored and retrieved by the brain. A Universal Brain Code utilized for these functions is proposed here. The basic tenent of the Code is that a memory engram is propagated and guided through the connectome by specific proteins/peptides embedded within the pre-synaptic neuronal membrane corresponding to information provided by afferent electrical currents to the pre-synaptic neuron. It is intended to provide a working approach to this central brain activity and begin the process of investigation based on these ideas which are new and unexplored.

How Human Single-Neuron Recordings Can Help Us Understand Cognition: Insights from Memory Studies

Understanding human cognition is a key goal of contemporary neuroscience. Due to the complexity of the human brain, animal studies and noninvasive techniques, however valuable, are incapable of providing us with a full understanding of human cognition. In the light of existing cognitive theories, we describe findings obtained thanks to human single-neuron recordings, including the discovery of concept cells and novelty-dependent cells, or activity patterns behind working memory, such as persistent activity. We propose future directions for studies using human single-neuron recordings and we discuss possible opportunities of investigating pathological brain.

The Architecture of Human Memory: Insights from Human Single-Neuron Recordings

Deciphering the mechanisms of human memory is a central goal of neuroscience, both from the point of view of the fundamental biology of memory and for its translational relevance. Here, we review some contributions that recordings from neurons in humans implanted with electrodes for clinical purposes have made toward this goal. Recordings from the medial temporal lobe, including the hippocampus, reveal the existence of two classes of cells: those encoding highly selective and invariant representations of abstract concepts, and memory-selective cells whose activity is related to familiarity and episodic retrieval. Insights derived from observing these cells in behaving humans include that semantic representations are activated before episodic representations, that memory content and memory strength are segregated, and that the activity of both types of cells is related to subjective awareness as expected from a substrate for declarative memory. Visually selective cells can remain persistently active for several seconds, thereby revealing a cellular substrate for working memory in humans. An overarching insight is that the neural code of human memory is interpretable at the single-neuron level. Jointly, intracranial recording studies are starting to reveal aspects of the building blocks of human memory at the single-cell level. This work establishes a bridge to cellular-level work in animals on the one hand, and the extensive literature on noninvasive imaging in humans on the other hand. More broadly, this work is a step toward a detailed mechanistic understanding of human memory that is needed to develop therapies for human memory disorders.

Microelectrode recordings in human epilepsy: a case for clinical translation

With their 'all-or-none' action potential responses, single neurons (or units) are accepted as the basic computational unit of the brain. There is extensive animal literature to support the mechanistic importance of studying neuronal firing as a way to understand neuronal microcircuits and brain function. Although most studies have emphasized physiology, there is increasing recognition that studying single units provides novel insight into system-level mechanisms of disease. Microelectrode recordings are becoming more common in humans, paralleling the increasing use of intracranial electroencephalography recordings in the context of presurgical evaluation in focal epilepsy. In addition to single-unit data, microelectrode recordings also record local field potentials and high-frequency oscillations, some of which may be different to that recorded by clinical macroelectrodes. However, microelectrodes are being used almost exclusively in research contexts and there are currently no indications for incorporating microelectrode recordings into routine clinical care. In this review, we summarize the lessons learnt from 65 years of microelectrode recordings in human epilepsy patients. We cover the electrode constructs that can be utilized, principles of how to record and process microelectrode data and insights into ictal dynamics, interictal dynamics and cognition. We end with a critique on the possibilities of incorporating single-unit recordings into clinical care, with a focus on potential clinical indications, each with their specific evidence base and challenges.

The Value of Failure in Science: The Story of Grandmother Cells in Neuroscience

The annals of science are filled with successes. Only in footnotes do we hear about the failures, the cul-de-sacs, and the forgotten ideas. Failure is how research advances. Yet it hardly features in theoretical perspectives on science. That is a mistake. Failures, whether clear-cut or ambiguous, are heuristically fruitful in their own right. Thinking about failure questions our measures of success, including the conceptual foundations of current practice, that can only be transient in an experimental context. This article advances the heuristics of failure analysis, meaning the explicit treatment of certain ideas or models as failures. The value of failures qua being a failure is illustrated with the example of grandmother cells; the contested idea of a hypothetical neuron that encodes a highly specific but complex stimulus, such as the image of one's grandmother. Repeatedly evoked in popular science and maintained in textbooks, there is sufficient reason to critically review the theoretical and empirical background of this idea.

A hierarchical model of social perception: Psychophysical evidence suggests late rather than early integration of visual information from facial expression and body posture

Facial expressions are one of the most important sources of information about another’s emotional states. More recently, other cues such as body posture have been shown to influence how facial expressions are perceived. It has been argued that this biasing effect is underpinned by an early integration of visual information from facial expression and body posture. Here, we replicate this biasing effect, but, using a psychophysical procedure, show that adaptation to facial expressions is unaffected by body context. The integration of face and body information therefore occurs downstream of the sites of adaptation, known to be localised in high-level visual areas of the temporal lobe. Contrary to previous research, our findings thus provide direct evidence for late integration of information from facial expression and body posture. They are consistent with a hierarchical model of social perception, in which social signals from different sources are initially processed independently and in parallel by specialised visual mechanisms. Integration of these different inputs in later stages of the visual system then supports the emergence of the integrated whole-person percept that is consciously experienced.

An online adaptive screening procedure for selective neuronal responses

BACKGROUND

A common problem in neurophysiology is to identify stimuli that elicit neuronal responses in a given brain region. Particularly in situations where electrode positions are fixed, this can be a time-consuming task that requires presentation of a large number of stimuli. Such a screening for response-eliciting stimuli is employed, e.g., as a standard procedure to identify 'concept cells' in the human medial temporal lobe.

NEW METHOD

Our new method evaluates neuronal responses to stimuli online during a screening session, which allows us to successively exclude stimuli that do not evoke a response. Using this method, we can screen a larger number of stimuli which in turn increases the chances of finding responsive neurons and renders time-consuming offline analysis unnecessary.

RESULTS

Our method enabled us to present 30% more stimuli in the same period of time with additional presentations of the most promising candidate stimuli. Our online method ran smoothly on a standard computer and network.

COMPARISON WITH AN EXISTING METHOD

To analyze how our online screening procedure performs in comparison to an established offline method, we used the Wave_Clus software package. We did not observe any major drawbacks in our method, but a much higher efficiency and analysis speed.

CONCLUSIONS

By transitioning from a traditional offline screening procedure to our new online method, we substantially increased the number of visual stimuli presented in a given time period. This allows to identify more response-eliciting stimuli, which forms the basis to better address a great number of questions in cognitive neuroscience.

Intracranial markers of conscious face perception in humans

Investigations of the neural basis of consciousness have greatly benefited from protocols that involve the presentation of stimuli at perceptual threshold, enabling the assessment of the patterns of brain activity that correlate with conscious perception, independently of any changes in sensory input. However, the comparison between perceived and unperceived trials would be expected to reveal not only the core neural substrate of a particular conscious perception, but also aspects of brain activity that facilitate, hinder or tend to follow conscious perception. We take a step towards the resolution of these confounds by combining an analysis of neural responses observed during the presentation of faces partially masked by Continuous Flash Suppression, and those responses observed during the unmasked presentation of faces and other images in the same subjects. We employed multidimensional classifiers to decode physical properties of stimuli or perceptual states from spectrotemporal representations of electrocorticographic signals (1071 channels in 5 subjects). Neural activity in certain face responsive areas located in both the fusiform gyrus and in the lateral-temporal/inferior-parietal cortex discriminated seen vs. unseen faces in the masked paradigm and upright faces vs. other categories in the unmasked paradigm. However, only the former discriminated upright vs. inverted faces in the unmasked paradigm. Our results suggest a prominent role for the fusiform gyrus in the configural perception of faces, and possibly other objects that are holistically processed. More generally, we advocate comparative analysis of neural recordings obtained during different, but related, experimental protocols as a promising direction towards elucidating the functional specificities of the patterns of neural activation that accompany our conscious experiences.

The human amygdala parametrically encodes the intensity of specific facial emotions and their categorical ambiguity

The human amygdala is a key structure for processing emotional facial expressions, but it remains unclear what aspects of emotion are processed. We investigated this question with three different approaches: behavioural analysis of 3 amygdala lesion patients, neuroimaging of 19 healthy adults, and single-neuron recordings in 9 neurosurgical patients. The lesion patients showed a shift in behavioural sensitivity to fear, and amygdala BOLD responses were modulated by both fear and emotion ambiguity (the uncertainty that a facial expression is categorized as fearful or happy). We found two populations of neurons, one whose response correlated with increasing degree of fear, or happiness, and a second whose response primarily decreased as a linear function of emotion ambiguity. Together, our results indicate that the human amygdala processes both the degree of emotion in facial expressions and the categorical ambiguity of the emotion shown and that these two aspects of amygdala processing can be most clearly distinguished at the level of single neurons.

The amygdala processes emotional facial expressions, but its exact contributions are unclear. Wang. et al. use behavioural analysis of amygdala lesion patients, fMRI, and single-neuron recordings to show that both emotional intensity and ambiguity signals are processed in the human amygdala.

Hippocampal Attractor Dynamics Predict Memory-Based Decision Making

Memories are thought to be retrieved by attractor dynamics if a given input is sufficiently similar to a stored attractor state [1-5]. The hippocampus, a region crucial for spatial navigation [6-12] and episodic memory [13-18], has been associated with attractor-based computations [5, 9], receiving support from the way rodent place cells "remap" nonlinearly between spatial representations [19-22]. In humans, nonlinear response patterns have been reported in perceptual categorization tasks [23-25]; however, it remains elusive whether human memory retrieval is driven by attractor dynamics and what neural mechanisms might underpin them. To test this, we used a virtual reality [7, 11, 26-28] task where participants learned object-location associations within two distinct virtual reality environments. Participants were subsequently exposed to four novel intermediate environments, generated by linearly morphing the background landscapes of the familiar environments, while tracking fMRI activity. We show that linear changes in environmental context cause linear changes in activity patterns in sensory cortex but cause dynamic, nonlinear changes in both hippocampal activity pattern and remembered locations. Furthermore, the sigmoidal response in the hippocampus scaled with the strength of the sigmoidal pattern in spatial memory. These results indicate that mnemonic decisions in an ambiguous novel context relate to putative attractor dynamics in the hippocampus, which support the dynamic remapping of memories.

Network science for the identification of novel therapeutic targets in epilepsy

The quality of life of children with epilepsy is a function of seizures and associated cognitive and behavioral comorbidities. Current treatments are not successful at stopping seizures in approximately 30% of patients despite the introduction of multiple new antiepileptic drugs over the last decade. In addition, modification of seizures has only a modest impact on the comorbidities. Therefore, novel approaches to identify therapeutic targets that improve seizures and comorbidities are urgently required. The potential of network science as applied to genetic, local neural network, and global brain data is reviewed. Several examples of possible new therapeutic approaches defined using novel network tools are highlighted. Further study to translate the findings into clinical practice is now required.

Non-human Primates in Neuroscience Research: The Case against its Scientific Necessity

Public opposition to non-human primate (NHP) experiments is significant, yet those who defend them cite minimal harm to NHPs and substantial human benefit. Here we review these claims of benefit, specifically in neuroscience, and show that: a) there is a default assumption of their human relevance and benefit, rather than robust evidence; b) their human relevance and essential contribution and necessity are wholly overstated; c) the contribution and capacity of non-animal investigative methods are greatly understated; and d) confounding issues, such as species differences and the effects of stress and anaesthesia, are usually overlooked. This is the case in NHP research generally, but here we specifically focus on the development and interpretation of functional magnetic resonance imaging (fMRI), deep brain stimulation (DBS), the understanding of neural oscillations and memory, and investigation of the neural control of movement and of vision/binocular rivalry. The increasing power of human-specific methods, including advances in fMRI and invasive techniques such as electrocorticography and single-unit recordings, is discussed. These methods serve to render NHP approaches redundant. We conclude that the defence of NHP use is groundless, and that neuroscience would be more relevant and successful for humans, if it were conducted with a direct human focus. We have confidence in opposing NHP neuroscience, both on scientific as well as on ethical grounds.

Neuronal codes for visual perception and memory

In this review, I describe and contrast the representation of stimuli in visual cortical areas and in the medial temporal lobe (MTL). While cortex is characterized by a distributed and implicit coding that is optimal for recognition and storage of semantic information, the MTL shows a much sparser and explicit coding of specific concepts that is ideal for episodic memory. I will describe the main characteristics of the coding in the MTL by the so-called concept cells and will then propose a model of the formation and recall of episodic memory based on partially overlapping assemblies.

Visual Adaptation

Sensory systems continuously mold themselves to the widely varying contexts in which they must operate. Studies of these adaptations have played a long and central role in vision science. In part this is because the specific adaptations remain a powerful tool for dissecting vision, by exposing the mechanisms that are adapting. That is, "if it adapts, it's there." Many insights about vision have come from using adaptation in this way, as a method. A second important trend has been the realization that the processes of adaptation are themselves essential to how vision works, and thus are likely to operate at all levels. That is, "if it's there, it adapts." This has focused interest on the mechanisms of adaptation as the target rather than the probe. Together both approaches have led to an emerging insight of adaptation as a fundamental and ubiquitous coding strategy impacting all aspects of how we see.

Monocular and binocular mechanisms mediating flicker adaptation

Flicker adaptation reduces subsequent temporal contrast sensitivity. Recent studies show that this adaptation likely results from neural changes in the magnocellular visual pathway, but whether this adaptation occurs at a monocular or a binocular level, or both, is unclear. Here, two experiments address this question. The first experiment exploits the observation that flicker adaptation is stronger at higher than lower temporal frequencies. Observers' two eyes adapted to 3Hz flicker with an incremental pulse at 1/4 duty cycle, either in-phase or out-of-phase in the two eyes. At the binocular level, the flicker rate was 6Hz in the out-of-phase condition if the two eyes' pulse trains sum. Similar sensitivity reduction was found in both phase conditions, as expected for independent monocular adapting mechanisms. The second experiment tested for interocular transfer of adaptation between eyes. Results showed that (1) flicker adaptation was strongest with adapting and test fields in only the same eye, (2) adaptation can be partially transferred interocularly with adaptation in only the opposite eye, and (3) adaptation was weakened when both eyes were adapted simultaneously at different contrasts, compared to test-eye adaptation alone. Taken together, the findings are consistent with mechanisms of flicker adaptation at both the monocular and binocular level.

Mechanisms Responsible for Cognitive Impairment in Epilepsy

Epilepsy is often associated with cognitive and behavioral impairments that can have profound impact on the quality of life of patients. Although the mechanisms of cognitive impairment are not completely understood, we make an attempt to describe, from a systems perspective, how information processing is affected in epilepsy disorders. The aim of this review is to (1) define the nature of cognitive deficits associated with epilepsy, (2) review fundamental systems-level mechanisms underlying information processing, and (3) describe how information processing is dysfunctional in epilepsy and investigate the relative contributions of etiology, seizures, and interictal discharges (IDs). We conclude that these mechanisms are likely to be important and deserve more detailed scrutiny in the future.

The primate amygdala in social perception - insights from electrophysiological recordings and stimulation

The role of the amygdala in emotion and social perception has been intensively investigated primarily through studies using functional magnetic resonance imaging (fMRI). Recently, this topic has been examined using single-unit recordings in both humans and monkeys, with a focus on face processing. The findings provide novel insights, including several surprises: amygdala neurons have very long response latencies, show highly nonlinear responses to whole faces, and can be exquisitely selective for very specific parts of faces such as the eyes. In humans, the responses of amygdala neurons correlate with internal states evoked by faces, rather than with their objective features. Current and future studies extend the investigations to psychiatric illnesses such as autism, in which atypical face processing is a hallmark of social dysfunction.

Concept cells through associative learning of high-level representations

In this issue of Neuron, Quian Quiroga et al. (2014) show that neurons in the human medial temporal lobe (MTL) follow subjects' perceptual states rather than the features of the visual input. Patients with MTL damage however have intact perceptual abilities but suffer instead from extreme forgetfulness. Thus, the reported MTL neurons could create new memories of the current perceptual state.