The role of long-term and short-term familiarity in visual and haptic face recognition

Developing cortex is functionally pluripotent: Evidence from blindness

How rigidly does innate architecture constrain function of developing cortex? What is the contribution of early experience? We review insights into these questions from visual cortex function in people born blind. In blindness, occipital cortices are active during auditory and tactile tasks. What 'cross-modal' plasticity tells us about cortical flexibility is debated. On the one hand, visual networks of blind people respond to higher cognitive information, such as sentence grammar, suggesting drastic repurposing. On the other, in line with 'metamodal' accounts, sighted and blind populations show shared domain preferences in ventral occipito-temporal cortex (vOTC), suggesting visual areas switch input modality but perform the same or similar perceptual functions (e.g., face recognition) in blindness. Here we bring these disparate literatures together, reviewing and synthesizing evidence that speaks to whether visual cortices have similar or different functions in blind and sighted people. Together, the evidence suggests that in blindness, visual cortices are incorporated into higher-cognitive (e.g., fronto-parietal) networks, which are a major source long-range input to the visual system. We propose the connectivity-constrained experience-dependent account. Functional development is constrained by innate anatomical connectivity, experience and behavioral needs. Infant cortex is pluripotent, the same anatomical constraints develop into different functional outcomes.

Haptic Recognition of Static and Dynamic Expressions of Emotion in the Live Face

If humans can detect the wealth of tactile and haptic information potentially available in live facial expressions of emotion (FEEs), they should be capable of haptically recognizing the six universal expressions of emotion (anger, disgust, fear, happiness, sadness, and surprise) at levels well above chance. We tested this hypothesis in the experiments reported here. With minimal training, subjects' overall mean accuracy was 51% for static FEEs (Experiment 1) and 74% for dynamic FEEs (Experiment 2). All FEEs except static fear were successfully recognized above the chance level of 16.7%. Complementing these findings, overall confidence and information transmission were higher for dynamic than for corresponding static faces. Our performance measures (accuracy and confidence ratings, plus response latency in Experiment 2 only) confirmed that happiness, sadness, and surprise were all highly recognizable, and anger, disgust, and fear less so.

Visuo-haptic multisensory object recognition, categorization, and representation

Visual and haptic unisensory object processing show many similarities in terms of categorization, recognition, and representation. In this review, we discuss how these similarities contribute to multisensory object processing. In particular, we show that similar unisensory visual and haptic representations lead to a shared multisensory representation underlying both cross-modal object recognition and view-independence. This shared representation suggests a common neural substrate and we review several candidate brain regions, previously thought to be specialized for aspects of visual processing, that are now known also to be involved in analogous haptic tasks. Finally, we lay out the evidence for a model of multisensory object recognition in which top-down and bottom-up pathways to the object-selective lateral occipital complex are modulated by object familiarity and individual differences in object and spatial imagery.

Does my face FIT?: a face image task reveals structure and distortions of facial feature representation

Despite extensive research on face perception, few studies have investigated individuals' knowledge about the physical features of their own face. In this study, 50 participants indicated the location of key features of their own face, relative to an anchor point corresponding to the tip of the nose, and the results were compared to the true location of the same individual's features from a standardised photograph. Horizontal and vertical errors were analysed separately. An overall bias to underestimate vertical distances revealed a distorted face representation, with reduced face height. Factor analyses were used to identify separable subconfigurations of facial features with correlated localisation errors. Independent representations of upper and lower facial features emerged from the data pattern. The major source of variation across individuals was in representation of face shape, with a spectrum from tall/thin to short/wide representation. Visual identification of one's own face is excellent, and facial features are routinely used for establishing personal identity. However, our results show that spatial knowledge of one's own face is remarkably poor, suggesting that face representation may not contribute strongly to self-awareness.

Tactual perception: a review of experimental variables and procedures

This paper reviews the literature on tactual perception. Throughout this review, we will highlight some of the most relevant aspects in the touch literature: type of stimuli; type of participants; type of tactile exploration; and finally, the interaction between touch and other senses. Regarding type of stimuli, we will analyse studies with abstract stimuli such as vibrations, with two- and three-dimensional stimuli, and also concrete stimuli, considering the relation between familiar and unfamiliar stimuli and the haptic perception of faces. Under the "type of participants" topic, we separated studies with blind participants, studies with children and adults, and also performed an overview of sex differences in performance. The type of tactile exploration is explored considering conditions of active and passive touch, the relevance of movement in touch and the relation between haptic exploration and time. Finally, interactions between touch and vision, touch and smell and touch and taste are explored in the last topic. The review ends with an overall conclusion on the state of the art for the tactual perception literature. With this work, we intend to present an organised overview of the main variables in touch experiments, compiling aspects reported in the tactual literature, and attempting to provide both a summary of previous findings, and a guide to the design of future works on tactual perception and memory, through a presentation of implications from previous studies.

Haptic Recognition of Emotions in Raised-Line Drawings by Congenitally Blind and Sighted Adults

15 sighted and 15 congenitally blind adults were to classify raised-line pictures of emotional faces through haptics. Whereas accuracy did not vary significantly between the two groups, the blind adults were faster at the task. These results suggest that raised-line pictures of emotional faces are intelligible to blind adults.

Representing human hands haptically or visually from first-person versus third-person perspectives

Humans can recognise human body parts haptically as well as visually. We employed a mental-rotation task to determine whether participants could adopt a third-person perspective when judging the laterality of life-like human hands. Female participants adopted either a first-person or a third-person perspective using vision (experiment 1) or haptics (experiment 2), with hands presented at various orientations within a horizontal plane. In the first-person perspective task, most participants responded more slowly as hand orientation increasingly deviated from the participant's upright orientation, regardless of modality. In the visual third-person perspective task, most participants responded more slowly as hand orientation increasingly deviated from the experimenter's upright orientation; in contrast, less than half of the participants produced this same inverted U-shaped response-time function haptically. In experiment 3, participants were explicitly instructed to adopt a third-person perspective haptically by mentally rotating the rubber hand to the experimenter's upright orientation. Most participants produced an inverted U-shaped function. Collectively, these results suggest that humans can accurately assume a third-person perspective when hands are explored haptically or visually. With less explicit instructions, however, the canonical orientation for hand representation may be more strongly influenced haptically than visually by body-based heuristics, and less easily modified by perspective instructions.

Haptic Classification of Facial Identity in 2D Displays: Configural versus Feature-Based Processing

Participants learned through feedback to haptically classify the identity of upright versus inverted versus scrambled faces depicted in simple 2D raised-line displays. We investigated whether identity classification would make use of a configural face representation, as is evidenced for vision and 3D haptic facial displays. Upright and scrambled faces produced equivalent accuracy, and both were identified more accurately than inverted faces. The mean magnitude of the haptic inversion effect for 2D facial identity was a sizable 26 percent, indicating that the upright orientation was ¿privileged¿ in the haptic representations of facial identity in these 2D displays, as with other facial modalities. However, given the effect of scrambling, we conclude that configural processing was not employed; rather, only local information about the features was used, the features being treated as oriented objects within a body-centered frame of reference. The results indicate a fundamental difference between haptic identification of 2D facial depictions and 3D faces, paralleling a corresponding difference in recognition of nonface objects.

Canonical views in haptic object perception

Previous investigations of visual object recognition have found that some views of both familiar and unfamiliar objects promote more efficient recognition performance than other views. These views are considered as canonical and are often the views that present the most information about an object's 3-D structure and features in the image. Although objects can also be efficiently recognised with touch alone, little is known whether some views promote more efficient recognition than others. This may seem unlikely, given that the object structure and features are readily available to the hand during object exploration. We conducted two experiments to investigate whether canonical views existed in haptic object recognition. In the first, participants were required to position each object in a way that would present the best view for learning the object with touch alone. We found a large degree of consistency of viewpoint position across participants for both familiar and unfamiliar objects. In a second experiment, we found that these consistent, or canonical, views promoted better haptic recognition performance than other random views of the objects. Interestingly, these haptic canonical views were not necessarily the same as the canonical views normally found in visual perception. Nevertheless, our findings provide support for the idea that both the visual and the tactile systems are functionally equivalent in terms of how objects are represented in memory and subsequently recognised.

Cross-Modal Transfer in Visual and Haptic Face Recognition

We report four psychophysical experiments investigating cross-modal transfer in visual and haptic face recognition. We found surprisingly good haptic performance and cross-modal transfer for both modalities. Interestingly, transfer was asymmetric depending on which modality was learned first. These findings are discussed in relation to haptic object processing and face processing.

Haptic Processing of Facial Expressions of Emotion in 2D Raised-Line Drawings

Participants haptically (vs. visually) classified universal facial expressions of emotion (FEEs) depicted in simple 2D raised-line displays. Experiments 1 and 2 established that haptic classification was well above chance; face-inversion effects further indicated that the upright orientation was privileged. Experiment 2 added a third condition in which the normal configuration of the upright features was spatially scrambled. Results confirmed that configural processing played a critical role, since upright FEEs were classified more accurately and confidently than either scrambled or inverted FEEs, which did not differ. Because accuracy in both scrambled and inverted conditions was above chance, feature processing also played a role, as confirmed by commonalities across confusions for upright, inverted, and scrambled faces. Experiment 3 required participants to visually and haptically assign emotional valence (positive/negative) and magnitude to upright and inverted 2-D FEE displays. While emotional magnitude could be assigned using either modality, haptic presentation led to more variable valence judgments. We also documented a new face-inversion effect for emotional valence visually, but not haptically. These results suggest emotions can be interpreted from 2-D displays presented haptically as well as visually; however, emotional impact is judged more reliably by vision than by touch. Potential applications of this work are also considered.

Cross-modal object recognition is viewpoint-independent

BACKGROUND

Previous research suggests that visual and haptic object recognition are viewpoint-dependent both within- and cross-modally. However, this conclusion may not be generally valid as it was reached using objects oriented along their extended y-axis, resulting in differential surface processing in vision and touch. In the present study, we removed this differential by presenting objects along the z-axis, thus making all object surfaces more equally available to vision and touch.

METHODOLOGY/PRINCIPAL FINDINGS

Participants studied previously unfamiliar objects, in groups of four, using either vision or touch. Subsequently, they performed a four-alternative forced-choice object identification task with the studied objects presented in both unrotated and rotated (180 degrees about the x-, y-, and z-axes) orientations. Rotation impaired within-modal recognition accuracy in both vision and touch, but not cross-modal recognition accuracy. Within-modally, visual recognition accuracy was reduced by rotation about the x- and y-axes more than the z-axis, whilst haptic recognition was equally affected by rotation about all three axes. Cross-modal (but not within-modal) accuracy correlated with spatial (but not object) imagery scores.

CONCLUSIONS/SIGNIFICANCE

The viewpoint-independence of cross-modal object identification points to its mediation by a high-level abstract representation. The correlation between spatial imagery scores and cross-modal performance suggest that construction of this high-level representation is linked to the ability to perform spatial transformations. Within-modal viewpoint-dependence appears to have a different basis in vision than in touch, possibly due to surface occlusion being important in vision but not touch.

The role of familiarity in the recognition of static and dynamic objects

Although the perception of our world is experienced as effortless, the processes that underlie object recognition in the brain are often difficult to determine. In this chapter, we review the effects of familiarity on the recognition of moving or static objects. In particular, we concentrate on exemplar-level stimuli such as walking humans, unfamiliar objects and faces. We found that the perception of these objects can be affected by their familiarity; for example the learned view of an object or the learned dynamic pattern can influence object perception. Deviations in the viewpoint from the familiar viewpoint, or changes in the temporal pattern of the objects can result in some reduction of efficiency in the perception of the object. Furthermore, more efficient sex categorization and crossmodal matching were found for familiar than for unfamiliar faces. In sum, we find that our perceptual system is organized around familiar events and that perception is most efficient with these learned events.

Are representations of unfamiliar faces independent of encoding modality?

It is well documented that both featural and configural information are important in visual face recognition. Less is known, however, about the nature of the information underlying haptic face recognition and whether or not this information is the same as in vision. In our experiments we found better within modal than crossmodal face recognition performance suggesting that face representations are largely specific to each modality. Moreover, this cost in crossmodal performance was found to be independent of differences in exploratory procedures across the modalities during encoding. We found that crossmodal face perception was most efficient when configural information of the facial features was preserved suggesting that configural information is shared across modalities. Our findings suggest that face information is processed in a similar manner across vision and touch but that qualitative differences in the nature of the information encoded underlies efficient within modal relative to crossmodal recognition.

Haptic face identification activates ventral occipital and temporal areas: An fMRI study

Many studies in visual face recognition have supported a special role for the right fusiform gyrus. Despite the fact that faces can also be recognized haptically, little is known about the neural correlates of haptic face recognition. In the current fMRI study, neurologically intact participants were intensively trained to identify specific facemasks (molded from live faces) and specific control objects. When these stimuli were presented in the scanner, facemasks activated left fusiform and right hippocampal/parahippocampal areas (and other regions) more than control objects, whereas the latter produced no activity greater than the facemasks. We conclude that these ventral occipital and temporal areas may play an important role in the haptic identification of faces at the subordinate level. We further speculate that left fusiform gyrus may be recruited more for facemasks than for control objects because of the increased need for sequential processing by the haptic system.