Processing the Facial Image: A Brief History

Neuronal Circuits for Social Decision-Making and Their Clinical Implications

Social living facilitates individual access to rewards, cognitive resources, and objects that would not be otherwise accessible. There are, however, some drawbacks to social living, particularly when competing for scarce resources. Furthermore, variability in our ability to make social decisions can be associated with neuropsychiatric disorders. The neuronal mechanisms underlying social decision-making are beginning to be understood. The momentum to study this phenomenon has been partially carried over by the study of economic decision-making. Yet, because of the similarities between these different types of decision-making, it is unclear what is a social decision. Here, we propose a definition of social decision-making as choices taken in a context where one or more conspecifics are involved in the decision or the consequences of it. Social decisions can be conceptualized as complex economic decisions since they are based on the subjective preferences between different goods. During social decisions, individuals choose based on their internal value estimate of the different alternatives. These are complex decisions given that conspecifics beliefs or actions could modify the subject's internal valuations at every choice. Here, we first review recent developments in our collective understanding of the neuronal mechanisms and circuits of social decision-making in primates. We then review literature characterizing populations with neuropsychiatric disorders showing deficits in social decision-making and the underlying neuronal circuitries associated with these deficits.

What does a "face cell" want?'

In the 1970s Charlie Gross was among the first to identify neurons that respond selectively to faces, in the macaque inferior temporal (IT) cortex. This seminal finding has been followed by numerous studies quantifying the visual features that trigger a response from face cells in order to answer the question; what do face cells want? However, the connection between face-selective activity in IT cortex and visual perception remains only partially understood. Here we present fMRI results in the macaque showing that some face patches respond to illusory facial features in objects. We argue that to fully understand the functional role of face cells, we need to develop approaches that test the extent to which their response explains what we see.

An Annotated Journey through Modern Visual Neuroscience

Recent advances in microscopy, genetics, physiology, and data processing have expanded the scope and accelerated the pace of discovery in visual neuroscience. However, the pace of discovery and the ever increasing number of published articles can present a serious issue for both trainees and senior scientists alike: with each passing year the fog of progress thickens, making it easy to lose sight of important earlier advances. As part of this special issue of the Journal of Neuroscience commemorating the 50th anniversary of SfN, here, we provide a variation on Stephen Kuffler's Oldies but Goodies classic reading list, with the hope that by looking back at highlights in the field of visual neuroscience we can better define remaining gaps in our knowledge and thus guide future work. We also hope that this article can serve as a resource that will aid those new to the field to find their bearings.

Perirhinal circuits for memory processing

The perirhinal cortex (PRC) serves as the gateway to the hippocampus for episodic memory formation and plays a part in retrieval through its backward connectivity to various neocortical areas. First, I present the evidence suggesting that PRC neurons encode both experientially acquired object features and their associative relations. Recent studies have revealed circuit mechanisms in the PRC for the retrieval of cue-associated information, and have demonstrated that, in monkeys, PRC neuron-encoded information can be behaviourally read out. These studies, among others, support the theory that the PRC converts visual representations of an object into those of its associated features and initiates backward-propagating, interareal signalling for retrieval of nested associations of object features that, combined, extensionally represent the object meaning. I propose that the PRC works as the ventromedial hub of a 'two-hub model' at an apex of the hierarchy of a distributed memory network and integrates signals encoded in other downstream cortical areas that support diverse aspects of knowledge about an object.

Neural Mechanisms of High-Level Vision

The last three decades have seen major strides in our understanding of neural mechanisms of high-level vision, or visual cognition of the world around us. Vision has also served as a model system for the study of brain function. Several broad insights, as yet incomplete, have recently emerged. First, visual perception is best understood not as an end unto itself, but as a sensory process that subserves the animal's behavioral goal at hand. Visual perception is likely to be simply a side effect that reflects the readout of visual information processing that leads to behavior. Second, the brain is essentially a probabilistic computational system that produces behaviors by collectively evaluating, not necessarily consciously or always optimally, the available information about the outside world received from the senses, the behavioral goals, prior knowledge about the world, and possible risks and benefits of a given behavior. Vision plays a prominent role in the overall functioning of the brain providing the lion's share of information about the outside world. Third, the visual system does not function in isolation, but rather interacts actively and reciprocally with other brain systems, including other sensory faculties. Finally, various regions of the visual system process information not in a strict hierarchical manner, but as parts of various dynamic brain-wide networks, collectively referred to as the "connectome." Thus, a full understanding of vision will ultimately entail understanding, in granular, quantitative detail, various aspects of dynamic brain networks that use visual sensory information to produce behavior under real-world conditions. © 2017 American Physiological Society. Compr Physiol 8:903-953, 2018.

Brain Computation Is Organized via Power-of-Two-Based Permutation Logic

There is considerable scientific interest in understanding how cell assemblies—the long-presumed computational motif—are organized so that the brain can generate intelligent cognition and flexible behavior. The Theory of Connectivity proposes that the origin of intelligence is rooted in a power-of-two-based permutation logic (N = 2ⁱ–1), producing specific-to-general cell-assembly architecture capable of generating specific perceptions and memories, as well as generalized knowledge and flexible actions. We show that this power-of-two-based permutation logic is widely used in cortical and subcortical circuits across animal species and is conserved for the processing of a variety of cognitive modalities including appetitive, emotional and social information. However, modulatory neurons, such as dopaminergic (DA) neurons, use a simpler logic despite their distinct subtypes. Interestingly, this specific-to-general permutation logic remained largely intact although NMDA receptors—the synaptic switch for learning and memory—were deleted throughout adulthood, suggesting that the logic is developmentally pre-configured. Moreover, this computational logic is implemented in the cortex via combining a random-connectivity strategy in superficial layers 2/3 with nonrandom organizations in deep layers 5/6. This randomness of layers 2/3 cliques—which preferentially encode specific and low-combinatorial features and project inter-cortically—is ideal for maximizing cross-modality novel pattern-extraction, pattern-discrimination and pattern-categorization using sparse code, consequently explaining why it requires hippocampal offline-consolidation. In contrast, the nonrandomness in layers 5/6—which consists of few specific cliques but a higher portion of more general cliques projecting mostly to subcortical systems—is ideal for feedback-control of motivation, emotion, consciousness and behaviors. These observations suggest that the brain’s basic computational algorithm is indeed organized by the power-of-two-based permutation logic. This simple mathematical logic can account for brain computation across the entire evolutionary spectrum, ranging from the simplest neural networks to the most complex.

Neural mechanisms of face perception, their emergence over development, and their breakdown

Face perception is probably the most developed visual perceptual skill in humans, most likely as a result of its unique evolutionary and social significance. Much recent research has converged to identify a host of relevant psychological mechanisms that support face recognition. In parallel, there has been substantial progress in uncovering the neural mechanisms that mediate rapid and accurate face perception, with specific emphasis on a broadly distributed neural circuit, comprised of multiple nodes whose joint activity supports face perception. This article focuses specifically on the neural underpinnings of face recognition, and reviews recent structural and functional imaging studies that elucidate the neural basis of this ability. In addition, the article covers some of the recent investigations that characterize the emergence of the neural basis of face recognition over the course of development, and explores the relationship between these changes and increasing behavioural competence. This paper also describes studies that characterize the nature of the breakdown of face recognition in individuals who are impaired in face recognition, either as a result of brain damage acquired at some point or as a result of the failure to master face recognition over the course of development. Finally, information regarding similarities between the neural circuits for face perception in humans and in nonhuman primates is briefly covered, as is the contribution of subcortical regions to face perception. WIREs Cogn Sci 2016, 7:247-263. doi: 10.1002/wcs.1388 For further resources related to this article, please visit the WIREs website.

Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination

Neurons that respond more to images of faces over nonface objects were identified in the inferior temporal (IT) cortex of primates three decades ago. Although it is hypothesized that perceptual discrimination between faces depends on the neural activity of IT subregions enriched with "face neurons," such a causal link has not been directly established. Here, using optogenetic and pharmacological methods, we reversibly suppressed the neural activity in small subregions of IT cortex of macaque monkeys performing a facial gender-discrimination task. Each type of intervention independently demonstrated that suppression of IT subregions enriched in face neurons induced a contralateral deficit in face gender-discrimination behavior. The same neural suppression of other IT subregions produced no detectable change in behavior. These results establish a causal link between the neural activity in IT face neuron subregions and face gender-discrimination behavior. Also, the demonstration that brief neural suppression of specific spatial subregions of IT induces behavioral effects opens the door for applying the technical advantages of optogenetics to a systematic attack on the causal relationship between IT cortex and high-level visual perception.

Double attention bias for positive and negative emotional faces in clinical depression: evidence from an eye-tracking study

BACKGROUND AND OBJECTIVES

According to cognitive models, attentional biases in depression play key roles in the onset and subsequent maintenance of the disorder. The present study examines the processing of emotional facial expressions (happy, angry, and sad) in depressed and non-depressed adults.

METHODS

Sixteen unmedicated patients with Major Depressive Disorder (MDD) and 34 never-depressed controls (ND) completed an eye-tracking task to assess different components of visual attention (orienting attention and maintenance of attention) in the processing of emotional faces.

RESULTS

Compared to ND, participants with MDD showed a negative attentional bias in attentional maintenance indices (i.e. first fixation duration and total fixation time) for sad faces. This attentional bias was positively associated with the severity of depressive symptoms. Furthermore, the MDD group spent a marginally less amount of time viewing happy faces compared with the ND group. No differences were found between the groups with respect to angry faces and orienting attention indices.

LIMITATIONS

The current study is limited by its cross-sectional design.

CONCLUSIONS

These results support the notion that attentional biases in depression are specific to depression-related information and that they operate in later stages in the deployment of attention.

Gaze-fixation and pupil dilation in the processing of emotional faces: the role of rumination

Sustained attentional processing of negative information plays a significant role in the development and maintenance of depression. The present study examines the relationships between rumination, a relevant factor in information processing in depression, and the attentional mechanisms activated in individuals with different levels of depression severity when attending to emotional information (i.e., sad, angry and happy faces). Behavioural and physiological indicators of sustained processing were assessed in 126 participants (39 dysphoric and 87 non-dysphoric) using eye-tracking technology. Pupil dilation and total time attending to negative faces were correlated with a global ruminative style in the total sample once depression severity was controlled. Furthermore, in dysphoric participants the brooding component of rumination was specifically associated with the total time attending to sad faces. Finally, bootstrapping analyses showed that the relationships between global rumination and pupil diameter to emotional faces were accounted by total time attending to emotional faces, specifically for participants reporting lower levels of depression severity. The results support the idea that sustained processing of negative information is associated with a higher ruminative style and indicate differential associations between these factors at different levels of depressive symptomatology.

Neural mechanism of dynamic responses of neurons in inferior temporal cortex in face perception

Understanding the neural mechanisms of object and face recognition is one of the fundamental challenges of visual neuroscience. The neurons in inferior temporal (IT) cortex have been reported to exhibit dynamic responses to face stimuli. However, little is known about how the dynamic properties of IT neurons emerge in the face information processing. To address this issue, we made a model of IT cortex, which performs face perception via an interaction between different IT networks. The model was based on the face information processed by three resolution maps in early visual areas. The network model of IT cortex consists of four kinds of networks, in which the information about a whole face is combined with the information about its face parts and their arrangements. We show here that the learning of face stimuli makes the functional connections between these IT networks, causing a high spike correlation of IT neuron pairs. A dynamic property of subthreshold membrane potential of IT neuron, produced by Hodgkin-Huxley model, enables the coordination of temporal information without changing the firing rate, providing the basis of the mechanism underlying face perception. We show also that the hierarchical processing of face information allows IT cortex to perform a "coarse-to-fine" processing of face information. The results presented here seem to be compatible with experimental data about dynamic properties of IT neurons.

Spatial frequency components influence cell activity in the inferotemporal cortex

We studied the correlation between the spatial frequency of complex stimuli and neuronal activity in the monkey inferotemporal (IT) cortex while performing a task that required visual recognition. Single-cell activity was recorded from the right IT cortex. The frequency components of the images used as stimuli were analyzed by using a fast Fourier transform, and a modulus was obtained for 40 spatial frequency ranges from 0.3 to 11.1 cycles/deg. We recorded 82 cells showing statistically significant responses (analysis of variance, P < 0.05) to at least one of the images used as a stimulus. Seventy-eight percent of these cells (n = 64) showed significant responses to at least three images, and in two thirds of them (n = 42), we found a statistically significant correlation (P < 0.05) between cell response and the modulus amplitude of at least one frequency range present in the images. Our results suggest that information about spatial frequency of the visual images is present in the IT cortex.

Towards transgenic primates: What can we learn from mouse genetics?

Considering the great physiological and behavioral similarities with humans, monkeys represent the ideal models not only for the study of complex cognitive behavior but also for the preclinical research and development of novel therapeutics for treating human diseases. Various powerful genetic technologies initially developed for making mouse models are being explored for generating transgenic primate models. We review the latest genetic engineering technologies and discuss the potentials and limitations for systematic production of transgenic primates.

Retinotopy of the face aftereffect

Physiological results for the size of face-specific units in inferotemporal cortex (IT) support an extraordinarily large range of possible sizes--from 2.5 degrees to 30 degrees or more. We use a behavioral test of face-specific aftereffects to measure the face analysis regions and find a coarse retinotopy consistent with receptive fields of intermediate size (10 degrees -12 degrees at 3 degrees eccentricity). In the first experiment, observers were adapted to a single face at 3 degrees from fixation. A test (a morph of the face and its anti-face) was then presented at different locations around fixation and subjects classified it as face or anti-face. The face aftereffect (FAE) was not constant at all test locations--it dropped to half its maximum value for tests 5 degrees from the adapting location. Simultaneous adaptation to both a face and its anti-face, placed at opposite locations across fixation, produced two separate regions of opposite aftereffects. However, with four stimuli, faces alternating with anti-faces equally spaced around fixation, the FAE was greatly reduced at all locations, implying a fairly coarse localization of the aftereffect. In the second experiment, observers adapted to a face and its anti-face presented either simultaneously or in alternation. Results showed that the simultaneous presentation of a face and its anti-face leads to stronger FAEs than sequential presentation, suggesting that face processing has a dynamic nature and its region of analysis is sharpened when there is more than one face in the scene. In the final experiment, a face and two anti-face flankers with different spatial offsets were presented during adaptation and the FAE was measured at the face location. Results showed that FAE at the face location was inhibited more as the distance of anti-face flankers to the face stimulus was reduced. This confirms the spatial extent of face analysis regions in a test with a fixed number of stimuli where only distance varied.

Single neuron studies of inferior temporal cortex

This paper reviews our experiments on the response properties of single neurons in inferior temporal (IT) cortex in the monkey that were carried out starting in 1965. It describes situational factors that led us to find neurons sensitive to images of faces and hands and summarizes the basic sensory properties of IT neurons. Subsequent developments on the cognitive properties of IT neurons and on imaging the responses of human temporal cortex to facial images are outlined. Finally, this paper summarizes recent results on fMRI imaging of the responses of temporal cortex to facial images.

Subspace projection approaches to classification and visualization of neural network-level encoding patterns

Recent advances in large-scale ensemble recordings allow monitoring of activity patterns of several hundreds of neurons in freely behaving animals. The emergence of such high-dimensional datasets poses challenges for the identification and analysis of dynamical network patterns. While several types of multivariate statistical methods have been used for integrating responses from multiple neurons, their effectiveness in pattern classification and predictive power has not been compared in a direct and systematic manner. Here we systematically employed a series of projection methods, such as Multiple Discriminant Analysis (MDA), Principal Components Analysis (PCA) and Artificial Neural Networks (ANN), and compared them with non-projection multivariate statistical methods such as Multivariate Gaussian Distributions (MGD). Our analyses of hippocampal data recorded during episodic memory events and cortical data simulated during face perception or arm movements illustrate how low-dimensional encoding subspaces can reveal the existence of network-level ensemble representations. We show how the use of regularization methods can prevent these statistical methods from over-fitting of training data sets when the trial numbers are much smaller than the number of recorded units. Moreover, we investigated the extent to which the computations implemented by the projection methods reflect the underlying hierarchical properties of the neural populations. Based on their ability to extract the essential features for pattern classification, we conclude that the typical performance ranking of these methods on under-sampled neural data of large dimension is MDA>PCA>ANN>MGD.

Neural encoding of the concept of nest in the mouse brain

As important as memory is to our daily functions, the ability to extract fundamental features and commonalities from various episodic experiences and to then generalize them into abstract concepts is even more crucial for both humans and animals to adapt to novel and complex situations. Here, we report the neural correlates of the abstract concept of nests or beds in mice. Specifically, we find hippocampal neurons that selectively fire or cease to fire when the mouse perceives nests or beds, regardless of their locations and environments. Parametric analyses show that responses of nest cells remain invariant over changes in the nests' physical shape, style, color, odor, or construction materials; rather, their responses are driven by conscious awareness and physical determination of the categorical features that would functionally define nests. Such functionality-based abstraction and generalization of conceptual knowledge, emerging from episodic experiences, suggests that the hippocampus is an intrinsic part of the hierarchical structure for generating concepts and knowledge in the brain.

Microstimulation of inferotemporal cortex influences face categorization

The inferior temporal cortex (IT) of primates is thought to be the final visual area in the ventral stream of cortical areas responsible for object recognition. Consistent with this hypothesis, single IT neurons respond selectively to highly complex visual stimuli such as faces. However, a direct causal link between the activity of face-selective neurons and face perception has not been demonstrated. In the present study of macaque monkeys, we artificially activated small clusters of IT neurons by means of electrical microstimulation while the monkeys performed a categorization task, judging whether noisy visual images belonged to 'face' or 'non-face' categories. Here we show that microstimulation of face-selective sites, but not other sites, strongly biased the monkeys' decisions towards the face category. The magnitude of the effect depended upon the degree of face selectivity of the stimulation site, the size of the stimulated cluster of face-selective neurons, and the exact timing of microstimulation. Our results establish a causal relationship between the activity of face-selective neurons and face perception.

Olfactory neuronal dynamics in behaving animals

More than 50 years have passed since the first recording of neuronal responses to an odor stimulus from the primary olfactory brain area, the main olfactory bulb. During this time very little progress has been achieved in understanding neuronal dynamics in the olfactory bulb in awake behaving animals, which is very different from that in anesthetized preparations. In this paper we formulate a new framework containing the main reasons for studying olfactory neuronal dynamics in awake animals and review advances in the field within this new framework.