1
|
Andrews TJ, Rogers D, Mileva M, Watson DM, Wang A, Burton AM. A narrow band of image dimensions is critical for face recognition. Vision Res 2023; 212:108297. [PMID: 37527594 DOI: 10.1016/j.visres.2023.108297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 07/07/2023] [Accepted: 07/12/2023] [Indexed: 08/03/2023]
Abstract
A key challenge in human and computer face recognition is to differentiate information that is diagnostic for identity from other sources of image variation. Here, we used a combined computational and behavioural approach to reveal critical image dimensions for face recognition. Behavioural data were collected using a sorting and matching task with unfamiliar faces and a recognition task with familiar faces. Principal components analysis was used to reveal the dimensions across which the shape and texture of faces in these tasks varied. We then asked which image dimensions were able to predict behavioural performance across these tasks. We found that the ability to predict behavioural responses in the unfamiliar face tasks increased when the early PCA dimensions (i.e. those accounting for most variance) of shape and texture were removed from the analysis. Image similarity also predicted the output of a computer model of face recognition, but again only when the early image dimensions were removed from the analysis. Finally, we found that recognition of familiar faces increased when the early image dimensions were removed, decreased when intermediate dimensions were removed, but then returned to baseline recognition when only later dimensions were removed. Together, these findings suggest that early image dimensions reflect ambient changes, such as changes in viewpoint or lighting, that do not contribute to face recognition. However, there is a narrow band of image dimensions for shape and texture that are critical for the recognition of identity in humans and computer models of face recognition.
Collapse
Affiliation(s)
| | - Daniel Rogers
- Department of Psychology, University of York, York YO10 5DD, UK
| | - Mila Mileva
- Department of Psychology, University of York, York YO10 5DD, UK
| | - David M Watson
- Department of Psychology, University of York, York YO10 5DD, UK
| | - Ao Wang
- Department of Psychology, University of York, York YO10 5DD, UK
| | - A Mike Burton
- Department of Psychology, University of York, York YO10 5DD, UK
| |
Collapse
|
2
|
White D, Wayne T, Varela VPL. Partitioning natural face image variability emphasises within-identity over between-identity representation for understanding accurate recognition. Cognition 2021; 219:104966. [PMID: 34861575 DOI: 10.1016/j.cognition.2021.104966] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 11/18/2021] [Accepted: 11/19/2021] [Indexed: 11/25/2022]
Abstract
Accurately recognising faces enables social interactions. In recent years it has become clear that people's accuracy differs markedly depending on viewer's familiarity with a face and their individual skill, but the cognitive and neural bases of these accuracy differences are not understood. We examined cognitive representations underlying these accuracy differences by measuring similarity ratings to natural facial image variation. Natural variation was sampled from uncontrolled images on the internet to reflect the appearance of faces as they are encountered in daily life. Using image averaging, and inspired by the computation of Analysis of Variance, we partitioned this variation into differences between faces (between-identity variation) and differences between photos of the same face (within-identity variation). This allowed us to compare modulation of these two sources of variation attributable to: (i) a person's familiarity with a face and, (ii) their face recognition ability. Contrary to prevailing accounts of human face recognition and perceptual learning, we found that modulation of within-identity variation - rather than between-identity variation - was associated with high accuracy. First, familiarity modulated similarity ratings to within-identity variation more than to between-face variation. Second, viewers that are extremely accurate in face recognition - 'super-recognisers' - differed from typical perceivers mostly in their ratings of within-identity variation, compared to between-identity variation. In a final computational analysis, we found evidence that transformations of between- and within-identity variation make separable contributions to perceptual expertise in face recognition. We conclude that inter- and intra-individual accuracy differences primarily arise from differences in the representation of within-identity image variation.
Collapse
Affiliation(s)
- David White
- School of Psychology, UNSW Sydney, Kensington, Australia.
| | - Tanya Wayne
- School of Psychology, UNSW Sydney, Kensington, Australia
| | | |
Collapse
|
3
|
Jin H, Oxner M, Corballis PM, Hayward WG. Holistic face processing is influenced by non-conscious visual information. Br J Psychol 2021; 113:300-326. [PMID: 34240413 DOI: 10.1111/bjop.12521] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 06/25/2021] [Indexed: 11/28/2022]
Abstract
Holistic face processing has been widely implicated in conscious face perception. Yet, little is known about whether holistic face processing occurs when faces are processed unconsciously. The present study used the composite face task and continuous flash suppression (CFS) to inspect whether the processing of target facial information (the top half of a face) is influenced by irrelevant information (the bottom half) that is presented unconsciously. Results of multiple experiments showed that the composite effect was observed in both monocular and CFS conditions, providing the first evidence that the processing of top facial halves is influenced by the aligned bottom halves no matter whether they are presented consciously or unconsciously. However, much of the composite effect for faces without masking was disrupted when bottom facial parts were rendered with CFS. These results suggest that holistic face processing can occur unconsciously, but also highlight the significance of holistic processing of consciously presented faces.
Collapse
Affiliation(s)
- Haiyang Jin
- School of Psychology, University of Auckland, New Zealand
| | - Matt Oxner
- Institute of Psychology, University of Leipzig, Germany
| | | | | |
Collapse
|
4
|
Noyes E, Parde CJ, Colón YI, Hill MQ, Castillo CD, Jenkins R, O'Toole AJ. Seeing through disguise: Getting to know you with a deep convolutional neural network. Cognition 2021; 211:104611. [PMID: 33592392 DOI: 10.1016/j.cognition.2021.104611] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2020] [Revised: 01/19/2021] [Accepted: 01/20/2021] [Indexed: 01/15/2023]
Abstract
People use disguise to look unlike themselves (evasion) or to look like someone else (impersonation). Evasion disguise challenges human ability to see an identity across variable images; Impersonation challenges human ability to tell people apart. Personal familiarity with an individual face helps humans to see through disguise. Here we propose a model of familiarity based on high-level visual learning mechanisms that we tested using a deep convolutional neural network (DCNN) trained for face identification. DCNNs generate a face space in which identities and images co-exist in a unified computational framework, that is categorically structured around identity, rather than retinotopy. This allows for simultaneous manipulation of mechanisms that contrast identities and cluster images. In Experiment 1, we measured the DCNN's baseline accuracy (unfamiliar condition) for identification of faces in no disguise and disguise conditions. Disguise affected DCNN performance in much the same way it affects human performance for unfamiliar faces in disguise (cf. Noyes & Jenkins, 2019). In Experiment 2, we simulated familiarity for individual identities by averaging the DCNN-generated representations from multiple images of each identity. Averaging improved DCNN recognition of faces in evasion disguise, but reduced the ability of the DCNN to differentiate identities of similar appearance. In Experiment 3, we implemented a contrast learning technique to simultaneously teach the DCNN appearance variation and identity contrasts between different individuals. This facilitated identification with both evasion and impersonation disguise. Familiar face recognition requires an ability to group images of the same identity together and separate different identities. The deep network provides a high-level visual representation for face recognition that supports both of these mechanisms of face learning simultaneously.
Collapse
Affiliation(s)
- Eilidh Noyes
- University of Huddersfield, Huddersfield, United Kingdom.
| | - Connor J Parde
- The University of Texas at Dallas, Richardson, TX, United States of America
| | - Y Ivette Colón
- The University of Texas at Dallas, Richardson, TX, United States of America
| | - Matthew Q Hill
- The University of Texas at Dallas, Richardson, TX, United States of America
| | | | | | - Alice J O'Toole
- The University of Texas at Dallas, Richardson, TX, United States of America
| |
Collapse
|
6
|
Mileva M, Young AW, Kramer RS, Burton AM. Understanding facial impressions between and within identities. Cognition 2019; 190:184-198. [DOI: 10.1016/j.cognition.2019.04.027] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2018] [Revised: 04/24/2019] [Accepted: 04/25/2019] [Indexed: 10/26/2022]
|
7
|
Vettori S, Dzhelyova M, Van der Donck S, Jacques C, Steyaert J, Rossion B, Boets B. Reduced neural sensitivity to rapid individual face discrimination in autism spectrum disorder. NEUROIMAGE-CLINICAL 2018; 21:101613. [PMID: 30522972 PMCID: PMC6411619 DOI: 10.1016/j.nicl.2018.101613] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/23/2018] [Revised: 11/07/2018] [Accepted: 11/20/2018] [Indexed: 12/18/2022]
Abstract
Background Individuals with autism spectrum disorder (ASD) are characterized by impairments in social communication and interaction. Although difficulties at processing social signals from the face in ASD have been observed and emphasized for many years, there is a lot of inconsistency across both behavioral and neural studies. Methods We recorded scalp electroencephalography (EEG) in 23 8-to-12 year old boys with ASD and 23 matched typically developing boys using a fast periodic visual stimulation (FPVS) paradigm, providing objective (i.e., frequency-tagged), fast (i.e., few minutes) and highly sensitive measures of rapid face categorization, without requiring any explicit face processing task. We tested both the sensitivity to rapidly (i.e., at a glance) categorize faces among other objects and to individuate unfamiliar faces. Outcomes While general neural synchronization to the visual stimulation and neural responses indexing generic face categorization were undistinguishable between children with ASD and typically developing controls, neural responses indexing individual face discrimination over the occipito-temporal cortex were substantially reduced in the individuals with ASD. This difference vanished when faces were presented upside-down, due to the lack of significant face inversion effect in ASD. Interpretation These data provide original evidence for a selective high-level impairment in individual face discrimination in ASD in an implicit task. The objective and rapid assessment of this function opens new perspectives for ASD diagnosis in clinical settings. We assess implicit face processing in ASD via Fast Periodic Visual Stimulation EEG. Rapid categorization of a face as a face is not impaired in children with ASD. Individual face discrimination is selectively impaired in ASD. Children with ASD show no face inversion effect. FPVS-EEG opens new perspectives for clinical settings.
Collapse
Affiliation(s)
- Sofie Vettori
- Center for Developmental Psychiatry, Department of Neurosciences, KU Leuven, Leuven, Belgium; Leuven Autism Research (LAuRes), KU Leuven, Leuven, Belgium.
| | - Milena Dzhelyova
- Institute of Research in Psychological Science, Institute of Neuroscience, University of Louvain, Louvain-La-Neuve, Belgium; Leuven Autism Research (LAuRes), KU Leuven, Leuven, Belgium
| | - Stephanie Van der Donck
- Center for Developmental Psychiatry, Department of Neurosciences, KU Leuven, Leuven, Belgium; Leuven Autism Research (LAuRes), KU Leuven, Leuven, Belgium
| | - Corentin Jacques
- Center for Developmental Psychiatry, Department of Neurosciences, KU Leuven, Leuven, Belgium; Institute of Research in Psychological Science, Institute of Neuroscience, University of Louvain, Louvain-La-Neuve, Belgium
| | - Jean Steyaert
- Center for Developmental Psychiatry, Department of Neurosciences, KU Leuven, Leuven, Belgium; Leuven Autism Research (LAuRes), KU Leuven, Leuven, Belgium
| | - Bruno Rossion
- Institute of Research in Psychological Science, Institute of Neuroscience, University of Louvain, Louvain-La-Neuve, Belgium; Université de Lorraine, CNRS, CRAN, F-54000 Nancy, France; Université de Lorraine, CHRU-Nancy, Service de Neurologie, F-5400, France
| | - Bart Boets
- Center for Developmental Psychiatry, Department of Neurosciences, KU Leuven, Leuven, Belgium; Leuven Autism Research (LAuRes), KU Leuven, Leuven, Belgium.
| |
Collapse
|