Predictive codes of familiarity and context during the perceptual learning of facial identities

Prediction error processing and sharpening of expected information across the face-processing hierarchy

The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.

Sense of Agency and Skills Learning in Virtual-Mediated Environment: A Systematic Review

Agency is central to remote actions, and it may enhance skills learning due to a partial overlap between brain structures and networks, the promotion of confidence towards a telemanipulator, and the feeling of congruence of the motor choice to the motor plan. We systematically reviewed studies aiming to verify the role of agency in improving learning. Fifteen studies were selected from MEDLINE and Scopus®. When a mismatch is introduced between observed and performed actions, the decrease in agency and learning is proportional to the intensity of the mismatch, which is due to greater interference with the motor programming. Thanks to multisensory integration, agency and learning benefit both from sensory and performance feedback and from the timing of feedback based on control at the goal level or the perceptual-motor level. This work constitutes a bedrock for professional teleoperation settings (e.g., robotic surgery), with particular reference to the role of agency in performing complex tasks with remote control.

Neural mechanisms of face familiarity and learning in the human amygdala and hippocampus

Recognizing familiar faces and learning new faces play an important role in social cognition. However, the underlying neural computational mechanisms remain unclear. Here, we record from single neurons in the human amygdala and hippocampus and find a greater neuronal representational distance between pairs of familiar faces than unfamiliar faces, suggesting that neural representations for familiar faces are more distinct. Representational distance increases with exposures to the same identity, suggesting that neural face representations are sharpened with learning and familiarization. Furthermore, representational distance is positively correlated with visual dissimilarity between faces, and exposure to visually similar faces increases representational distance, thus sharpening neural representations. Finally, we construct a computational model that demonstrates an increase in the representational distance of artificial units with training. Together, our results suggest that the neuronal population geometry, quantified by the representational distance, encodes face familiarity, similarity, and learning, forming the basis of face recognition and memory.

Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.

Modeling face recognition in the predictive coding framework: A combined computational modeling and functional imaging study

The learning of new facial identities and the recognition of familiar faces are crucial processes for social interactions. Recently, a combined computational modeling and functional magnetic resonance imaging (fMRI) study used predictive coding as a biologically plausible framework to model face identity learning and to relate specific model parameters with brain activity (Apps and Tsakiris, Nat Commun 4, 2698, 2013). On the one hand, it was shown that behavioral responses on a two-option face recognition task could be predicted by the level of contextual and facial familiarity in a computational model derived from predictive-coding principles. On the other hand, brain activity in specific brain regions was associated with these parameters. More specifically, brain activity in the superior temporal sulcus (STS) varied with contextual familiarity, whereas activity in the fusiform face area (FFA) covaried with the prediction error parameter that updated facial familiarity. Literature combining fMRI assessments and computational modeling in humans still needs to be expanded. Furthermore, prior results are largely not replicated. The present study was, therefore, specifically set up to replicate these previous findings. Our results support the original findings in two critical aspects. First, on a group level, the behavioral responses were modeled best by the same computational model reported by the original authors. Second, we showed that estimates of these model parameters covary with brain activity in specific, face-sensitive brain regions. Our results thus provide further evidence that the functional properties of the face perception network conform to central principles of predictive coding. However, our study yielded diverging findings on specific computational model parameters reflected in brain activity. On the one hand, we did not find any evidence of a computational involvement of the STS. On the other hand, our results showed that activity in the right FFA was associated with multiple computational model parameters. Our data do not provide evidence for functional segregation between particular face-sensitive brain regions, as previously proposed.

Another in need enhances prosociality and modulates frontal theta oscillations in young adults

Introduction

Decision-making is a process that can be strongly affected by social factors. Evidence has shown how people deviate from traditional rational-choice predictions under different levels of social interactions. The emergence of prosocial decision-making, defined as any action that is addressed to benefit another individual even at the expense of personal benefits, has been reported as an example of such social influence. Furthermore, brain evidence has shown the involvement of structures such as the prefrontal cortex, anterior insula, and midcingulate cortex during decision settings in which a decision maker interacts with others under physical pain or distress or while being observed by others.

Methods

Using a slightly modified version of the dictator game and EEG recordings, we tested the hypothesis that the inclusion of another person into the decision setting increases prosocial decisions in young adults and that this increase is higher when the other person is associated with others in need. At the brain level, we hypothesized that the increase in prosocial decisions correlates with frontal theta activity.

Results and Discussion

The results showed that including another person in the decision, setting increased prosocial behavior only when this presence was associated with someone in need. This effect was associated with an increase in frontocentral theta-oscillatory activity. These results suggest that the presence of someone in need enhances empathy concerns and norm compliance, raising the participants' prosocial decision-making.

Speaking and listening to inter-brain relationships

Studies of inter-brain relationships thrive, and yet many reservations regarding their scope and interpretation of these phenomena have been raised by the scientific community. It is thus essential to establish common ground on methodological and conceptual definitions related to this topic and to open debate about any remaining points of uncertainty. We here offer insights to improve the conceptual clarity and empirical standards offered by social neuroscience studies of inter-personal interaction using hyperscanning with a particular focus on verbal communication.

Functional imaging analyses reveal prototype and exemplar representations in a perceptual single-category task

Similarity-based categorization can be performed by memorizing category members as exemplars or by abstracting the central tendency of the category – the prototype. In similarity-based categorization of stimuli with clearly identifiable dimensions from two categories, prototype representations were previously located in the hippocampus and the ventromedial prefrontal cortex (vmPFC) and exemplar representations in areas supporting visual memory. However, the neural implementation of exemplar and prototype representations in perceptual similarity-based categorization of single categories is unclear. To investigate these representations, we applied model-based univariate and multivariate analyses of functional imaging data from a dot-pattern paradigm-based task. Univariate prototype and exemplar representations occurred bilaterally in visual areas. Multivariate analyses additionally identified prototype representations in parietal areas and exemplar representations in the hippocampus. Bayesian analyses supported the non-presence of prototype representations in the hippocampus and the vmPFC. We additionally demonstrate that some individuals form both representation types simultaneously, probably granting flexibility in categorization strategies.

Model-based univariate and multivariate analyses of fMRI data from 62 healthy participants in a dot-pattern paradigm-based task provide further insight into the neural basis of similarity-based categorization.

Predictive processing models and affective neuroscience

The neural bases of affective experience remain elusive. Early neuroscience models of affect searched for specific brain regions that uniquely carried out the computations that underlie dimensions of valence and arousal. However, a growing body of work has failed to identify these circuits. Research turned to multivariate analyses, but these strategies, too, have made limited progress. Predictive processing models offer exciting new directions to address this problem. Here, we use predictive processing models as a lens to critique prevailing functional neuroimaging research practices in affective neuroscience. Our review highlights how much work relies on rigid assumptions that are inconsistent with a predictive processing approach. We outline the central aspects of a predictive processing model and draw out their implications for research in affective and cognitive neuroscience. Predictive models motivate a reformulation of "reverse inference" in cognitive neuroscience, and placing a greater emphasis on external validity in experimental design.

Long term fMRI adaptation depends on adapter response in face-selective cortex

Repetition suppression (RS) reflects a neural attenuation during repeated stimulation. We used fMRI and the subsequent memory paradigm to test the predictive coding hypothesis for RS during visual memory processing by investigating the interaction between RS and differences due to memory in category-selective cortex (FFA, pSTS, PPA, and RSC). Fifty-six participants encoded face and house stimuli twice, followed by an immediate and delayed (48 h) recognition memory assessment. Linear Mixed Model analyses with repetition, subsequent recognition performance, and their interaction as fixed effects revealed that absolute RS during encoding interacts with probability of future remembrance in face-selective cortex. This effect was not observed for relative RS, i.e. when controlled for adapter-response. The findings also reveal an association between adapter response and RS, both for short and long term (48h) intervals, after controlling for the mathematical dependence between both measures. These combined findings are challenging for predictive coding models of visual memory and are more compatible with adapter-related and familiarity accounts.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Learning abilities

Learning abilities are present in infancy, as they are critical for adaptation. From simple habituation and novelty responses to stimuli, learning capacities evolve throughout the lifespan. During development, learning abilities become more flexible and integrated across sensory modalities, allowing the encoding of more complex information, and in larger amounts. In turn, an increasing knowledge base leads to adaptive changes in behavior, making responses and actions more precise and effective. The objective of this chapter is to review the main behavioral manifestations of human learning abilities in early development and their biologic underpinnings, ranging from the cellular level to neurocognitive systems and mechanisms. We first focus on the ability to learn from repetitions of stimuli and how years of research in this field have recently contributed to theories of fundamental brain mechanisms whose implications for cognitive development are under study. The ability to memorize associations between different items and events is addressed next as we review the variety of contexts in which this associative memory and its neurologic bases come into play. Together, repetition-based learning and associative memory provide powerful means of understanding the surrounding environment, not only through the gathering and consolidation of specific types of information, but also by continually testing and adjusting stored information to better adapt to changing conditions.

Two Sides of Face Learning: Improving Between-Identity Discrimination While Tolerating More Within-Person Variability in Appearance

Two photos of an unfamiliar face are often perceived as belonging to different people—an error that disappears when a face is familiar. Face learning has been characterized as increased tolerance of within-person variability in appearance and is facilitated by exposure to such variability (e.g., differences in expression, lighting, and aesthetics). We hypothesized that increased tolerance of variability in appearance might lead to reduced discrimination and that misidentifications would be reduced if a face was learned in the context of a similar-looking identity. After validating our stimuli (Experiments 1a and 1b), we conducted three experiments investigating face learning. In two of these, participants learned three faces (Experiment 2: 15 images/identity and Experiment 3: 5 images/identity), two of which were similar. In a recognition task, misidentifications did not change as a function of similarity, although participants recognized more images of the target in Experiment 2 (i.e., after learning 15 images). In Experiment 4, participants learned one identity and the number of images studied varied across groups. Recognition of new images increased with the number of images studied, with no changes in false alarms; sensitivity (A′) marginally increased. The results suggest that recognition and discrimination reflect separable processes with minimal influence of between-person similarity on discrimination.

Contributions of the Medial Prefrontal Cortex to Social Influence in Economic Decision-Making

Economic decisions are guided by highly subjective reward valuations (SVs). Often these SVs are over-ridden when individuals conform to social norms. Yet, the neural mechanisms that underpin the distinct processing of such normative reward valuations (NVs) are poorly understood. The dorsomedial and ventromedial portions of the prefrontal cortex (dmPFC/vmPFC) are putatively key regions for processing social and economic information respectively. However, the contribution of these regions to economic decisions guided by social norms is unclear. Using functional magnetic resonance imaging and computational modeling we examine the neural mechanisms underlying the processing of SVs and NVs. Subjects (n = 15) indicated either their own economic preferences or made similar choices based on a social norm-learnt during a training session. We found that that the vmPFC and dmPFC make dissociable contributions to the processing of SV and NV. Regions of the dmPFC processed "only" the value of rewards when making normative choices. In contrast, we identify a novel mechanism in the vmPFC for the coding of value. This region signaled both subjective and normative valuations, but activity was scaled positively for SV and negatively for NV. These results highlight some of the key mechanisms that underpin conformity and social influence in economic decision-making.

The Anterior Cingulate Gyrus and Social Cognition: Tracking the Motivation of Others

The anterior cingulate cortex (ACC) is implicated in a broad range of behaviors and cognitive processes, but it has been unclear what contribution, if any, the ACC makes to social behavior. We argue that anatomical and functional evidence suggests that a specific sub-region of ACC-in the gyrus (ACCg)-plays a crucial role in processing social information. We propose that the computational properties of the ACCg support a contribution to social cognition by estimating how motivated other individuals are and dynamically updating those estimates when further evidence suggests they have been erroneous. Notably this model, based on vicarious motivation and error processing, provides a unified account of neurophysiological and neuroimaging evidence that the ACCg is sensitive to costs, benefits, and errors during social interactions. Furthermore, it makes specific, testable predictions about a key mechanism that may underpin variability in socio-cognitive abilities in health and disease.

Information-Theoretic Evidence for Predictive Coding in the Face-Processing System

Predictive coding suggests that the brain infers the causes of its sensations by combining sensory evidence with internal predictions based on available prior knowledge. However, the neurophysiological correlates of (pre)activated prior knowledge serving these predictions are still unknown. Based on the idea that such preactivated prior knowledge must be maintained until needed, we measured the amount of maintained information in neural signals via the active information storage (AIS) measure. AIS was calculated on whole-brain beamformer-reconstructed source time courses from MEG recordings of 52 human subjects during the baseline of a Mooney face/house detection task. Preactivation of prior knowledge for faces showed as α-band-related and β-band-related AIS increases in content-specific areas; these AIS increases were behaviorally relevant in the brain's fusiform face area. Further, AIS allowed decoding of the cued category on a trial-by-trial basis. Our results support accounts indicating that activated prior knowledge and the corresponding predictions are signaled in low-frequency activity (<30 Hz).SIGNIFICANCE STATEMENT Our perception is not only determined by the information our eyes/retina and other sensory organs receive from the outside world, but strongly depends also on information already present in our brains, such as prior knowledge about specific situations or objects. A currently popular theory in neuroscience, predictive coding theory, suggests that this prior knowledge is used by the brain to form internal predictions about upcoming sensory information. However, neurophysiological evidence for this hypothesis is rare, mostly because this kind of evidence requires strong a priori assumptions about the specific predictions the brain makes and the brain areas involved. Using a novel, assumption-free approach, we find that face-related prior knowledge and the derived predictions are represented in low-frequency brain activity.

Interoceptive dimensions across cardiac and respiratory axes

Interoception refers to the sensing of signals concerning the internal state of the body. Individual differences in interoceptive sensitivity are proposed to account for differences in affective processing, including the expression of anxiety. The majority of investigations of interoceptive accuracy focus on cardiac signals, typically using heartbeat detection tests and self-report measures. Consequently, little is known about how different organ-specific axes of interoception relate to each other or to symptoms of anxiety. Here, we compare interoception for cardiac and respiratory signals. We demonstrate a dissociation between cardiac and respiratory measures of interoceptive accuracy (i.e. task performance), yet a positive relationship between cardiac and respiratory measures of interoceptive awareness (i.e. metacognitive insight into own interoceptive ability). Neither interoceptive accuracy nor metacognitive awareness for cardiac and respiratory measures was related to touch acuity, an exteroceptive sense. Specific measures of interoception were found to be predictive of anxiety symptoms. Poor respiratory accuracy was associated with heightened anxiety score, while good metacognitive awareness for cardiac interoception was associated with reduced anxiety. These findings highlight that detection accuracies across different sensory modalities are dissociable and future work can better delineate their relationship to affective and cognitive constructs.This article is part of the themed issue 'Interoception beyond homeostasis: affect, cognition and mental health'.

The anatomy of empathy: Vicarious experience and disorders of social cognition

Empathy, the ability to vicariously experience and to understand the affect of other people, is fundamental for successful social-cognitive ability and behaviour. Empathy is thought to be a critical facilitator of prosocial behaviour and is disrupted in a number of psychiatric and neurological disorders. Research has begun to uncover the neural basis of such 'vicarious experience', which has been studied as a proxy measure of empathy. Together, these studies have identified portions of the insula and anterior cingulate cortex as critically involved. A key debate is whether overlapping or non-overlapping brain areas respond to personal and vicarious experience. This review will highlight emerging evidence for both types of brain response. Importantly, animal models have suggested that there are central divisions between the anterior cingulate gyrus and anterior cingulate sulcus that may be crucial for understanding social behaviour. Attention to this specific anatomy of vicarious processing could therefore help shed light on the functional profile of empathy. Studies in individuals with psychopathy and autism spectrum disorders have found that vicarious experience is atypical. However, the precise nature of these atypicalities is mixed. Understanding the mechanisms of vicarious experience can enhance our knowledge of the neural basis of empathy and, ultimately, help those with disorders of social cognition and behaviour.

Am I seeing myself, my friend or a stranger? The role of personal familiarity in visual distinction of body identities in the human brain

Several brain regions appear to play a role in representing different body identities. The specific contribution of each of these regions is still unclear, however. Here we investigated which brain areas enable the visual distinction between self and other bodies of different familiarity, and between familiar and unfamiliar other individuals, and moreover, where identity-specific information on the three individuals was encoded. Participants were confronted with standardized headless human body stimuli either showing the participant's own, a personally familiar or an unfamiliar other body, while performing a luminance discrimination task. Employing multivariate pattern analysis, we were able to identify areas that allowed for the distinction of self from personal familiar other bodies within the medial prefrontal cortex (mPFC) and posterior cingulate cortex/precuneus. Successful distinction of self from unfamiliar others was possible in the left middle frontal gyrus, the right inferior frontal gyrus, the left pre-supplementary motor area and the right putamen. Personally familiar others could be distinguished from unfamiliar others in the right temporoparietal junction (TPJ). An analysis of identity-specific information revealed a spatial gradient ranging from inferior posterior to superior anterior portions of the mPFC that was associated with encoding identity-related information for self via familiar to unfamiliar other bodies, respectively. Furthermore, several midline and frontal regions encoded information on more than one identity. The TPJ's role in deviance detection was underlined, as only identity-specific information on unfamiliar others was encoded here. Together, our findings suggest substantial spatial overlap in neural correlates of self and other body representation and thus, support the hypothesis of a socially-related representation of the self.

The Effect of Self-Referential Expectation on Emotional Face Processing

The role of self-relevance has been somewhat neglected in static face processing paradigms but may be important in understanding how emotional faces impact on attention, cognition and affect. The aim of the current study was to investigate the effect of self-relevant primes on processing emotional composite faces. Sentence primes created an expectation of the emotion of the face before sad, happy, neutral or composite face photos were viewed. Eye movements were recorded and subsequent responses measured the cognitive and affective impact of the emotion expressed. Results indicated that primes did not guide attention, but impacted on judgments of valence intensity and self-esteem ratings. Negative self-relevant primes led to the most negative self-esteem ratings, although the effect of the prime was qualified by salient facial features. Self-relevant expectations about the emotion of a face and subsequent attention to a face that is congruent with these expectations strengthened the affective impact of viewing the face.

Electrophysiological correlates of self-specific prediction errors in the human brain

Recognising one's self, vs. others, is a key component of self-awareness, crucial for social interactions. Here we investigated whether processing self-face and self-body images can be explained by the brain's prediction of sensory events, based on regularities in the given context. We measured evoked cortical responses while participants observed alternating sequences of self-face or other-face images (experiment 1) and self-body or other-body images (experiment 2), which were embedded in an identity-irrelevant task. In experiment 1, the expected sequences were violated by deviant morphed images, which contained 33%, 66% or 100% of the self-face when the other's face was expected (and vice versa). In experiment 2, the anticipated sequences were violated by deviant images of the self when the other's image was expected (and vice versa), or by two deviant images composed of pictures of the self-face attached to the other's body, or the other's face attached to the self-body. This manipulation allowed control of the prediction error associated with the self or the other's image. Deviant self-images (but not deviant images of the other) elicited a visual mismatch response (vMMR)--a cortical index of violations of regularity. This was source localised to face and body related visual, sensorimotor and limbic areas and had amplitude proportional to the amount of deviance from the self-image. We provide novel evidence that self-processing can be described by the brain's prediction error system, which accounts for self-bias in visual processing. These findings are discussed in the light of recent predictive coding models of self-processing.