1
|
Bruera A, Poesio M. Electroencephalography Searchlight Decoding Reveals Person- and Place-specific Responses for Semantic Category and Familiarity. J Cogn Neurosci 2025; 37:135-154. [PMID: 38319891 DOI: 10.1162/jocn_a_02125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
Proper names are linguistic expressions referring to unique entities, such as individual people or places. This sets them apart from other words like common nouns, which refer to generic concepts. And yet, despite both being individual entities, one's closest friend and one's favorite city are intuitively associated with very different pieces of knowledge-face, voice, social relationship, autobiographical experiences for the former, and mostly visual and spatial information for the latter. Neuroimaging research has revealed the existence of both domain-general and domain-specific brain correlates of semantic processing of individual entities; however, it remains unclear how such commonalities and similarities operate over a fine-grained temporal scale. In this work, we tackle this question using EEG and multivariate (time-resolved and searchlight) decoding analyses. We look at when and where we can accurately decode the semantic category of a proper name and whether we can find person- or place-specific effects of familiarity, which is a modality-independent dimension and therefore avoids sensorimotor differences inherent among the two categories. Semantic category can be decoded in a time window and with spatial localization typically associated with lexical semantic processing. Regarding familiarity, our results reveal that it is easier to distinguish patterns of familiarity-related evoked activity for people, as opposed to places, in both early and late time windows. Second, we discover that within the early responses, both domain-general (left posterior-lateral) and domain-specific (right fronto-temporal, only for people) neural patterns can be individuated, suggesting the existence of person-specific processes.
Collapse
Affiliation(s)
- Andrea Bruera
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Queen Mary University of London
| | | |
Collapse
|
2
|
Bruera A, Poesio M. Family lexicon: Using language models to encode memories of personally familiar and famous people and places in the brain. PLoS One 2024; 19:e0291099. [PMID: 39576771 PMCID: PMC11584084 DOI: 10.1371/journal.pone.0291099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 09/15/2024] [Indexed: 11/24/2024] Open
Abstract
Knowledge about personally familiar people and places is extremely rich and varied, involving pieces of semantic information connected in unpredictable ways through past autobiographical memories. In this work, we investigate whether we can capture brain processing of personally familiar people and places using subject-specific memories, after transforming them into vectorial semantic representations using language models. First, we asked participants to provide us with the names of the closest people and places in their lives. Then we collected open-ended answers to a questionnaire, aimed at capturing various facets of declarative knowledge. We collected EEG data from the same participants while they were reading the names and subsequently mentally visualizing their referents. As a control set of stimuli, we also recorded evoked responses to a matched set of famous people and places. We then created original semantic representations for the individual entities using language models. For personally familiar entities, we used the text of the answers to the questionnaire. For famous entities, we employed their Wikipedia page, which reflects shared declarative knowledge about them. Through whole-scalp time-resolved and searchlight encoding analyses, we found that we could capture how the brain processes one's closest people and places using person-specific answers to questionnaires, as well as famous entities. Overall encoding performance was significant in a large time window (200-800ms). Using spatio-temporal EEG searchlight, we found that we could predict brain responses significantly better than chance earlier (200-500ms) in bilateral temporo-parietal electrodes and later (500-700ms) in frontal and posterior central electrodes. We also found that XLM, a contextualized (or large) language model, provided superior encoding scores when compared with a simpler static language model as word2vec. Overall, these results indicate that language models can capture subject-specific semantic representations as they are processed in the human brain, by exploiting small-scale distributional lexical data.
Collapse
Affiliation(s)
- Andrea Bruera
- Max Planck Institute for Human Cognitive and Brain Sciences, Cognition and Plasticity Research Group, Leipzig, Germany
- Queen Mary University of London, London, United Kingdom
| | - Massimo Poesio
- Max Planck Institute for Human Cognitive and Brain Sciences, Cognition and Plasticity Research Group, Leipzig, Germany
| |
Collapse
|
3
|
Li B, Zhang S. Registered Report Stage II: Decoding the category information from evoked potentials to visible and invisible visual objects. Int J Psychophysiol 2024; 205:112446. [PMID: 39389167 DOI: 10.1016/j.ijpsycho.2024.112446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2024] [Revised: 09/18/2024] [Accepted: 10/07/2024] [Indexed: 10/12/2024]
Abstract
Previous studies that use decoding methods and EEG to investigate the neural representation of the category information of visual objects focused mainly on consciously processed visual objects. It remains unclear whether the category information of unconsciously processed visual objects can be decoded and whether the decoding performance is different for consciously and unconsciously processed visual objects. The present study compared the neural decoding of the animacy category of visible and invisible visual objects via EEG and decoding methods. The results revealed that the animacy of visible visual objects could be decoded above the chance level by the P200, N300, and N400, but not by the early N/P100. However, the animacy of invisible visual objects could not be decoded above the chance level by neither early nor late ERP components. The decoding accuracy was greater for visible visual objects than that for invisible visual objects for the P200, N300 and N400. These results suggested that access to animacy category information for visual objects requires conscious processing.
Collapse
Affiliation(s)
- Bingbing Li
- School of Education Science, Jiangsu Normal University, Xuzhou, Jiangsu, China.
| | - Shuhui Zhang
- School of Education Science, Jiangsu Normal University, Xuzhou, Jiangsu, China.
| |
Collapse
|
4
|
Ventura M, Caffò AO, Manippa V, Rivolta D. Normative data of the Italian Famous Face Test. Sci Rep 2024; 14:15276. [PMID: 38961204 PMCID: PMC11222389 DOI: 10.1038/s41598-024-66252-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Accepted: 06/30/2024] [Indexed: 07/05/2024] Open
Abstract
The faces we see in daily life exist on a continuum of familiarity, ranging from personally familiar to famous to unfamiliar faces. Thus, when assessing face recognition abilities, adequate evaluation measures should be employed to discriminate between each of these processes and their relative impairments. We here developed the Italian Famous Face Test (IT-FFT), a novel assessment tool for famous face recognition in typical and clinical populations. Normative data on a large sample (N = 436) of Italian individuals were collected, assessing both familiarity (d') and recognition accuracy. Furthermore, this study explored whether individuals possess insights into their overall face recognition skills by correlating the Prosopagnosia Index-20 (PI-20) with the IT-FFT; a negative correlation between these measures suggests that people have a moderate insight into their face recognition skills. Overall, our study provides the first online-based Italian test for famous faces (IT-FFT), a test that could be used alongside other standard tests of face recognition because it complements them by evaluating real-world face familiarity, providing a more comprehensive assessment of face recognition abilities. Testing different aspects of face recognition is crucial for understanding both typical and atypical face recognition.
Collapse
Affiliation(s)
- Martina Ventura
- The MARCS Institute for Brain, Behaviour, and Development, Western Sydney University, Sydney, Australia
- Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy
| | - Alessandro Oronzo Caffò
- Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy
| | - Valerio Manippa
- Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy.
| | - Davide Rivolta
- Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy
| |
Collapse
|
5
|
Lutz CG, Coraj S, Fraga-González G, Brem S. The odd one out - Orthographic oddball processing in children with poor versus typical reading skills in a fast periodic visual stimulation EEG paradigm. Cortex 2024; 172:185-203. [PMID: 38354469 DOI: 10.1016/j.cortex.2023.12.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 09/17/2023] [Accepted: 12/12/2023] [Indexed: 02/16/2024]
Abstract
The specialization of left ventral occipitotemporal brain regions to automatically process word forms develops with reading acquisition and is diminished in children with poor reading skills (PR). Using a fast periodic visual oddball stimulation (FPVS) design during electroencephalography (EEG), we examined the level of sensitivity and familiarity to word form processing in ninety-two children in 2nd and 3rd grade with varying reading skills (n = 35 for PR, n = 40 for typical reading skills; TR). To test children's level of "sensitivity", false font (FF) and consonant string (CS) oddballs were embedded in base presentations of word (W) stimuli. "Familiarity" was examined by presenting letter string oddballs with increasing familiarity (CS, pseudoword - PW, W) in FF base stimuli. Overall, our results revealed stronger left-hemispheric coarse sensitivity effects ("FF in W" > "CS in W") in TR than in PR in both topographic and oddball frequency analyses. Further, children distinguished between orthographically legal and illegal ("W/PW in FF" > "CS in FF") but not yet between lexical and non-lexical ("W in FF" vs "PW in FF") word forms. Although both TR and PR exhibit visual sensitivity and can distinguish between orthographically legal and illegal letter strings, they still struggle with nuanced lexical distinctions. Moreover, the strength of sensitivity is linked to reading proficiency. Our work adds to established knowledge in the field to characterize the relationship between print tuning and reading skills and suggests differences in the developmental progress to automatically process word forms.
Collapse
Affiliation(s)
- Christina G Lutz
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric University Hospital Zurich, University of Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Switzerland
| | - Seline Coraj
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric University Hospital Zurich, University of Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Switzerland; Family Larsson-Rosenquist Foundation Center for Neurodevelopment, Growth, and Nutrition of the Newborn, Department of Neonatology, University Hospital Zurich, University of Zurich, Switzerland
| | - Gorka Fraga-González
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric University Hospital Zurich, University of Zurich, Switzerland
| | - Silvia Brem
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric University Hospital Zurich, University of Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Switzerland; University Research Priority Program (URPP), Adaptive Brain Circuits in Development and Learning (AdaBD), University of Zurich, Switzerland.
| |
Collapse
|
6
|
Cao R, Wang J, Brunner P, Willie JT, Li X, Rutishauser U, Brandmeir NJ, Wang S. Neural mechanisms of face familiarity and learning in the human amygdala and hippocampus. Cell Rep 2024; 43:113520. [PMID: 38151023 PMCID: PMC10834150 DOI: 10.1016/j.celrep.2023.113520] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 09/12/2023] [Accepted: 11/14/2023] [Indexed: 12/29/2023] Open
Abstract
Recognizing familiar faces and learning new faces play an important role in social cognition. However, the underlying neural computational mechanisms remain unclear. Here, we record from single neurons in the human amygdala and hippocampus and find a greater neuronal representational distance between pairs of familiar faces than unfamiliar faces, suggesting that neural representations for familiar faces are more distinct. Representational distance increases with exposures to the same identity, suggesting that neural face representations are sharpened with learning and familiarization. Furthermore, representational distance is positively correlated with visual dissimilarity between faces, and exposure to visually similar faces increases representational distance, thus sharpening neural representations. Finally, we construct a computational model that demonstrates an increase in the representational distance of artificial units with training. Together, our results suggest that the neuronal population geometry, quantified by the representational distance, encodes face familiarity, similarity, and learning, forming the basis of face recognition and memory.
Collapse
Affiliation(s)
- Runnan Cao
- Department of Radiology, Washington University in St. Louis, St. Louis, MO 63110, USA; Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA.
| | - Jinge Wang
- Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA
| | - Peter Brunner
- Department of Neurosurgery, Washington University in St. Louis, St. Louis, MO 63110, USA
| | - Jon T Willie
- Department of Neurosurgery, Washington University in St. Louis, St. Louis, MO 63110, USA
| | - Xin Li
- Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA
| | - Ueli Rutishauser
- Departments of Neurosurgery and Neurology, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| | | | - Shuo Wang
- Department of Radiology, Washington University in St. Louis, St. Louis, MO 63110, USA; Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA; Department of Neurosurgery, Washington University in St. Louis, St. Louis, MO 63110, USA.
| |
Collapse
|
7
|
Grootswagers T, Robinson AK, Shatek SM, Carlson TA. Mapping the dynamics of visual feature coding: Insights into perception and integration. PLoS Comput Biol 2024; 20:e1011760. [PMID: 38190390 PMCID: PMC10798643 DOI: 10.1371/journal.pcbi.1011760] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 01/19/2024] [Accepted: 12/13/2023] [Indexed: 01/10/2024] Open
Abstract
The basic computations performed in the human early visual cortex are the foundation for visual perception. While we know a lot about these computations, a key missing piece is how the coding of visual features relates to our perception of the environment. To investigate visual feature coding, interactions, and their relationship to human perception, we investigated neural responses and perceptual similarity judgements to a large set of visual stimuli that varied parametrically along four feature dimensions. We measured neural responses using electroencephalography (N = 16) to 256 grating stimuli that varied in orientation, spatial frequency, contrast, and colour. We then mapped the response profiles of the neural coding of each visual feature and their interactions, and related these to independently obtained behavioural judgements of stimulus similarity. The results confirmed fundamental principles of feature coding in the visual system, such that all four features were processed simultaneously but differed in their dynamics, and there was distinctive conjunction coding for different combinations of features in the neural responses. Importantly, modelling of the behaviour revealed that every stimulus feature contributed to perceptual judgements, despite the untargeted nature of the behavioural task. Further, the relationship between neural coding and behaviour was evident from initial processing stages, signifying that the fundamental features, not just their interactions, contribute to perception. This study highlights the importance of understanding how feature coding progresses through the visual hierarchy and the relationship between different stages of processing and perception.
Collapse
Affiliation(s)
- Tijl Grootswagers
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney, Australia
- School of Computer, Data and Mathematical Sciences, Western Sydney University, Sydney, Australia
| | - Amanda K. Robinson
- Queensland Brain Institute, The University of Queensland, Brisbane, Australia
| | - Sophia M. Shatek
- School of Psychology, The University of Sydney, Sydney, Australia
| | | |
Collapse
|
8
|
Rosanne O, Alves de Oliveira A, Falk TH. EEG Amplitude Modulation Analysis across Mental Tasks: Towards Improved Active BCIs. SENSORS (BASEL, SWITZERLAND) 2023; 23:9352. [PMID: 38067725 PMCID: PMC10708818 DOI: 10.3390/s23239352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 11/15/2023] [Accepted: 11/20/2023] [Indexed: 12/18/2023]
Abstract
Brain-computer interface (BCI) technology has emerged as an influential communication tool with extensive applications across numerous fields, including entertainment, marketing, mental state monitoring, and particularly medical neurorehabilitation. Despite its immense potential, the reliability of BCI systems is challenged by the intricacies of data collection, environmental factors, and noisy interferences, making the interpretation of high-dimensional electroencephalogram (EEG) data a pressing issue. While the current trends in research have leant towards improving classification using deep learning-based models, our study proposes the use of new features based on EEG amplitude modulation (AM) dynamics. Experiments on an active BCI dataset comprised seven mental tasks to show the importance of the proposed features, as well as their complementarity to conventional power spectral features. Through combining the seven mental tasks, 21 binary classification tests were explored. In 17 of these 21 tests, the addition of the proposed features significantly improved classifier performance relative to using power spectral density (PSD) features only. Specifically, the average kappa score for these classifications increased from 0.57 to 0.62 using the combined feature set. An examination of the top-selected features showed the predominance of the AM-based measures, comprising over 77% of the top-ranked features. We conclude this paper with an in-depth analysis of these top-ranked features and discuss their potential for use in neurophysiology.
Collapse
Affiliation(s)
- Olivier Rosanne
- Institut National de la Recherche Scientifique, University of Quebec, Montreal, QC H5A 1K6, Canada;
| | - Alcyr Alves de Oliveira
- Graduate Program in Psychology and Health, Federal University of Health Sciences of Porto Alegre, Porto Alegre 90050-170, Brazil;
| | - Tiago H. Falk
- Institut National de la Recherche Scientifique, University of Quebec, Montreal, QC H5A 1K6, Canada;
| |
Collapse
|
9
|
Corriveau A, Kidder A, Teichmann L, Wardle SG, Baker CI. Sustained neural representations of personally familiar people and places during cued recall. Cortex 2023; 158:71-82. [PMID: 36459788 PMCID: PMC9840701 DOI: 10.1016/j.cortex.2022.08.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 05/28/2022] [Accepted: 08/29/2022] [Indexed: 01/18/2023]
Abstract
The recall and visualization of people and places from memory is an everyday occurrence, yet the neural mechanisms underpinning this phenomenon are not well understood. In particular, the temporal characteristics of the internal representations generated by active recall are unclear. Here, we used magnetoencephalography (MEG) and multivariate pattern analysis to measure the evolving neural representation of familiar places and people across the whole brain when human participants engage in active recall. To isolate self-generated imagined representations, we used a retro-cue paradigm in which participants were first presented with two possible labels before being cued to recall either the first or second item. We collected personalized labels for specific locations and people familiar to each participant. Importantly, no visual stimuli were presented during the recall period, and the retro-cue paradigm allowed the dissociation of responses associated with the labels from those corresponding to the self-generated representations. First, we found that following the retro-cue it took on average ∼1000 ms for distinct neural representations of freely recalled people or places to develop. Second, we found distinct representations of personally familiar concepts throughout the 4 s recall period. Finally, we found that these representations were highly stable and generalizable across time. These results suggest that self-generated visualizations and recall of familiar places and people are subserved by a stable neural mechanism that operates relatively slowly when under conscious control.
Collapse
Affiliation(s)
- Anna Corriveau
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20814, USA; Department of Psychology, The University of Chicago, Chicago, IL 60637, USA.
| | - Alexis Kidder
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20814, USA; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA.
| | - Lina Teichmann
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20814, USA
| | - Susan G Wardle
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20814, USA
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20814, USA
| |
Collapse
|
10
|
Teichmann L, Moerel D, Rich AN, Baker CI. The nature of neural object representations during dynamic occlusion. Cortex 2022; 153:66-86. [PMID: 35597052 PMCID: PMC9247008 DOI: 10.1016/j.cortex.2022.04.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Revised: 03/18/2022] [Accepted: 04/01/2022] [Indexed: 12/01/2022]
Abstract
Objects disappearing briefly from sight due to occlusion is an inevitable occurrence in everyday life. Yet we generally have a strong experience that occluded objects continue to exist, despite the fact that they objectively disappear. This indicates that neural object representations must be maintained during dynamic occlusion. However, it is unclear what the nature of such representation is and in particular whether it is perception-like or more abstract, for example, reflecting limited features such as position or movement direction only. In this study, we address this question by examining how different object features such as object shape, luminance, and position are represented in the brain when a moving object is dynamically occluded. We apply multivariate decoding methods to Magnetoencephalography (MEG) data to track how object representations unfold over time. Our methods allow us to contrast the representations of multiple object features during occlusion and enable us to compare the neural responses evoked by visible and occluded objects. The results show that object position information is represented during occlusion to a limited extent while object identity features are not maintained through the period of occlusion. Together, this suggests that the nature of object representations during dynamic occlusion is different from visual representations during perception.
Collapse
Affiliation(s)
- Lina Teichmann
- Perception in Action Research Centre & School of Psychological Sciences, Macquarie University, 16 University Ave, North Ryde, NSW, 2109, Australia; Laboratory of Brain and Cognition, 10 Center Drive, 10/4C104, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA.
| | - Denise Moerel
- Perception in Action Research Centre & School of Psychological Sciences, Macquarie University, 16 University Ave, North Ryde, NSW, 2109, Australia; School of Psychology, University of Sydney, Sydney, NSW, Australia.
| | - Anina N Rich
- Perception in Action Research Centre & School of Psychological Sciences, Macquarie University, 16 University Ave, North Ryde, NSW, 2109, Australia.
| | - Chris I Baker
- Laboratory of Brain and Cognition, 10 Center Drive, 10/4C104, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA.
| |
Collapse
|
11
|
Moshel ML, Robinson AK, Carlson TA, Grootswagers T. Are you for real? Decoding realistic AI-generated faces from neural activity. Vision Res 2022; 199:108079. [PMID: 35749833 DOI: 10.1016/j.visres.2022.108079] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Revised: 05/30/2022] [Accepted: 06/06/2022] [Indexed: 11/17/2022]
Abstract
Can we trust our eyes? Until recently, we rarely had to question whether what we see is indeed what exists, but this is changing. Artificial neural networks can now generate realistic images that challenge our perception of what is real. This new reality can have significant implications for cybersecurity, counterfeiting, fake news, and border security. We investigated how the human brain encodes and interprets realistic artificially generated images using behaviour and brain imaging. We found that we could reliably decode AI generated faces using people's neural activity. However, while at a group level people performed near chance classifying real and realistic fakes, participants tended to interchange the labels, classifying real faces as realistic fakes and vice versa. Understanding this difference between brain and behavioural responses may be key in determining the 'real' in our new reality. Stimuli, code, and data for this study can be found at https://osf.io/n2z73/.
Collapse
Affiliation(s)
- Michoel L Moshel
- School of Psychology, University of Sydney, NSW, Australia; School of Psychology, Macquarie University, NSW, Australia.
| | - Amanda K Robinson
- School of Psychology, University of Sydney, NSW, Australia; Queensland Brain Institute, The University of Queensland, QLD, Australia
| | | | - Tijl Grootswagers
- School of Psychology, University of Sydney, NSW, Australia; The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, NSW, Australia
| |
Collapse
|
12
|
Giannopulu I, Brotto G, Lee T, Frangos A, To D. Synchronised neural signature of creative mental imagery in reality and augmented reality. Heliyon 2022; 8:e09017. [PMID: 35309391 PMCID: PMC8928117 DOI: 10.1016/j.heliyon.2022.e09017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 11/05/2021] [Accepted: 02/23/2022] [Indexed: 11/23/2022] Open
Abstract
Creativity, transforming imaginative thinking into reality, is a mental imagery simulation in essence. It can be incorporeal, concerns sophisticated and/or substantial thinking, and involves objects. In the present study, a mental imagery task consisting of creating a scene using familiar (FA) or abstract (AB) physical or virtual objects in real (RMI) and augmented reality (VMI) environments, and an execution task involving effectively creating a scene in augmented reality (VE), were utilised. The beta and gamma neural oscillations of healthy participants were recorded via a 32 channel wireless 10/20 international EGG system. In real and augmented environments and for both the mental imagery and execution tasks, the participants displayed a similar cortico-cortical neural signature essentially based on synchronous vs asynchronous beta and gamma oscillatory activities between anterior (i.e. frontal) and posterior (i.e. parietal, occipito-parietal and occipito-temporal) areas bilaterally. The findings revealed a transient synchronised neural architecture that appears to be consistent with the hypothesis according to which, creativity, because of its inherent complexity, cannot be confined to a single brain area but engages various interconnected networks.
Collapse
Affiliation(s)
- I. Giannopulu
- Creative Robotics Lab, UNSW, 2021, Sydney, Australia
- Clinical Research and Technological Innovation, 75016, Paris, France
| | - G. Brotto
- Interdisciplinary Centre for the Artificial Mind (iCAM), Bond University, 4229, Robina, Australia
| | - T.J. Lee
- Interdisciplinary Centre for the Artificial Mind (iCAM), Bond University, 4229, Robina, Australia
| | - A. Frangos
- Interdisciplinary Centre for the Artificial Mind (iCAM), Bond University, 4229, Robina, Australia
| | - D. To
- Interdisciplinary Centre for the Artificial Mind (iCAM), Bond University, 4229, Robina, Australia
| |
Collapse
|
13
|
Zhu L, Cui G, Li Y, Zhang J, Kong W, Cichocki A, Li J. Attention allocation on mobile app interfaces when human interacts with them. Cogn Neurodyn 2021; 16:859-870. [PMID: 35847542 PMCID: PMC9279534 DOI: 10.1007/s11571-021-09760-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 11/09/2021] [Accepted: 11/22/2021] [Indexed: 11/29/2022] Open
Abstract
AbstractWith the popularity of smartphones and the pervasion of mobile apps, people spend more and more time to interact with a diversity of apps on their smartphones, especially for young population. This raises a question: how people allocate attention to interfaces of apps during using them. To address this question, we, in this study, designed an experiment with two sessions (i.e., Session1: browsing original interfaces; Session 2: browsing interfaces after removal of colors and background) integrating with an eyetracking system. Attention fixation durations were recorded by an eye-tracker while participants browsed app interfaces. The whole screen of smartphone was divided into four even regions to explore fixation durations. The results revealed that participants gave significantly longer total fixation duration on the bottom left region compared to other regions in the session (1) Longer total fixation duration on the bottom was preserved, but there is no significant difference between left side and right side in the session2. Similar to the finding of total fixation duration, first fixation duration is also predominantly paid on the bottom area of the interface. Moreover, the skill in the use of mobile phone was quantified by assessing familiarity and accuracy of phone operation and was investigated in the association with the fixation durations. We found that first fixation duration of the bottom left region is significantly negatively correlated with the smartphone operation level in the session 1, but there is no significant correlation between them in the session (2) According to the results of ratio exploration, the ratio of the first fixation duration to the total fixation duration is not significantly different between areas of interest for both sessions. The findings of this study provide insights into the attention allocation during browsing app interfaces and are of implications on the design of app interfaces and advertisements as layout can be optimized according to the attention allocation to maximally deliver information.
Collapse
Affiliation(s)
- Li Zhu
- Computer & Software School, Hangzhou Dianzi University, Hangzhou, Zhejiang Province 310018 China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, 310018 Hangzhou, China
| | - Gaochao Cui
- Department of Information System, Saitama Institute of Technology, Fukaya, Saitama 369-0293 Japan
| | - Yan Li
- School of Art and Textile and Clothing Engineering, Changshu Institute of Techonology, Changshu, 215500 China
- Graduate School of Advanced Imaging Science, Multimedia and Film, Chung-Ang University, Seoul, 06974 Korea
| | - Jianhai Zhang
- Computer & Software School, Hangzhou Dianzi University, Hangzhou, Zhejiang Province 310018 China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, 310018 Hangzhou, China
| | - Wanzeng Kong
- Computer & Software School, Hangzhou Dianzi University, Hangzhou, Zhejiang Province 310018 China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, 310018 Hangzhou, China
| | - Andrzej Cichocki
- Computer & Software School, Hangzhou Dianzi University, Hangzhou, Zhejiang Province 310018 China
- Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia 143026
| | - Junhua Li
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, CO4 3SQ UK
- Laboratory for Brain-Bionic Intelligence and Computational Neuraoscience, Wuyi University, Jiangmen, 529020 China
| |
Collapse
|
14
|
Hudac CM, Naples A, DesChamps TD, Coffman MC, Kresse A, Ward T, Mukerji C, Aaronson B, Faja S, McPartland JC, Bernier R. Modeling temporal dynamics of face processing in youth and adults. Soc Neurosci 2021; 16:345-361. [PMID: 33882266 PMCID: PMC8324546 DOI: 10.1080/17470919.2021.1920050] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
A hierarchical model of temporal dynamics was examined in adults (n = 34) and youth (n = 46) across the stages of face processing during the perception of static and dynamic faces. Three ERP components (P100, N170, N250) and spectral power in the mu range were extracted, corresponding to cognitive stages of face processing: low-level vision processing, structural encoding, higher-order processing, and action understanding. Youth and adults exhibited similar yet distinct patterns of hierarchical temporal dynamics such that earlier cognitive stages predicted later stages, directly and indirectly. However, latent factors indicated unique profiles related to behavioral performance for adults and youth and age as a continuous factor. The application of path analysis to electrophysiological data can yield novel insights into the cortical dynamics of social information processing.
Collapse
Affiliation(s)
- Caitlin M Hudac
- Center for Youth Development and Intervention and Department of Psychology, University of Alabama, Tuscaloosa, AL, USA
| | - Adam Naples
- Yale Child Study Center, Yale University, New Haven, CT, USA
| | - Trent D DesChamps
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Marika C Coffman
- Center for Autism and Brain Development and Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA
| | - Anna Kresse
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Tracey Ward
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA.,The Seattle Clinic, Seattle, WA, USA
| | - Cora Mukerji
- Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | - Benjamin Aaronson
- Department of Pediatrics, University of Washington, Seattle, WA, USA
| | | | | | - Raphael Bernier
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| |
Collapse
|
15
|
Karimi-Rouzbahani H, Ramezani F, Woolgar A, Rich A, Ghodrati M. Perceptual difficulty modulates the direction of information flow in familiar face recognition. Neuroimage 2021; 233:117896. [PMID: 33667671 PMCID: PMC7614447 DOI: 10.1016/j.neuroimage.2021.117896] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Revised: 02/10/2021] [Accepted: 02/17/2021] [Indexed: 02/07/2023] Open
Abstract
Humans are fast and accurate when they recognize familiar faces. Previous neurophysiological studies have shown enhanced representations for the dichotomy of familiar vs. unfamiliar faces. As familiarity is a spectrum, however, any neural correlate should reflect graded representations for more vs. less familiar faces along the spectrum. By systematically varying familiarity across stimuli, we show a neural familiarity spectrum using electroencephalography. We then evaluated the spatiotemporal dynamics of familiar face recognition across the brain. Specifically, we developed a novel informational connectivity method to test whether peri-frontal brain areas contribute to familiar face recognition. Results showed that feed-forward flow dominates for the most familiar faces and top-down flow was only dominant when sensory evidence was insufficient to support face recognition. These results demonstrate that perceptual difficulty and the level of familiarity influence the neural representation of familiar faces and the degree to which peri-frontal neural networks contribute to familiar face recognition.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom; Perception in Action Research Centre and Department of Cognitive Science Macquarie University, Australia.
| | - Farzad Ramezani
- Department of Computer Science, School of Mathematics, Statistics, and Computer Science, University of Tehran, Iran
| | - Alexandra Woolgar
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom; Perception in Action Research Centre and Department of Cognitive Science Macquarie University, Australia
| | - Anina Rich
- Perception in Action Research Centre and Department of Cognitive Science Macquarie University, Australia
| | - Masoud Ghodrati
- Neuroscience Program, Biomedicine Discovery Institute, Monash University, Australia.
| |
Collapse
|
16
|
Blauch NM, Behrmann M, Plaut DC. Computational insights into human perceptual expertise for familiar and unfamiliar face recognition. Cognition 2021; 208:104341. [PMID: 32586632 PMCID: PMC9944378 DOI: 10.1016/j.cognition.2020.104341] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2019] [Revised: 05/22/2020] [Accepted: 05/26/2020] [Indexed: 02/02/2023]
Abstract
Humans are generally thought to be experts at face recognition, and yet identity perception for unfamiliar faces is surprisingly poor compared to that for familiar faces. Prior theoretical work has argued that unfamiliar face identity perception suffers because the majority of identity-invariant visual variability is idiosyncratic to each identity, and thus, each face identity must be learned essentially from scratch. Using a high-performing deep convolutional neural network, we evaluate this claim by examining the effects of visual experience in untrained, object-expert and face-expert networks. We found that only face training led to substantial generalization in an identity verification task of novel unfamiliar identities. Moreover, generalization increased with the number of previously learned identities, highlighting the generality of identity-invariant information in face images. To better understand how familiarity builds upon generic face representations, we simulated familiarization with face identities by fine-tuning the network on images of the previously unfamiliar identities. Familiarization produced a sharp boost in verification, but only approached ceiling performance in the networks that were highly trained on faces. Moreover, in these face-expert networks, the sharp familiarity benefit was seen only at the identity-based output probability layer, and did not depend on changes to perceptual representations; rather, familiarity effects required learning only at the level of identity readout from a fixed expert representation. Our results thus reconcile the existence of a large familiar face advantage with claims that both familiar and unfamiliar face identity processing depend on shared expert perceptual representations.
Collapse
Affiliation(s)
- Nicholas M Blauch
- Program in Neural Computation, Carnegie Mellon University, United States of America; Neuroscience Institute, Carnegie Mellon University, United States of America.
| | - Marlene Behrmann
- Neuroscience Institute, Carnegie Mellon University, United States of America; Department of Psychology, Carnegie Mellon University, United States of America
| | - David C Plaut
- Neuroscience Institute, Carnegie Mellon University, United States of America; Department of Psychology, Carnegie Mellon University, United States of America
| |
Collapse
|
17
|
Wang F, Zhang L, Yue L, Zeng Y, Zhao Q, Gong Q, Zhang J, Liu D, Luo X, Xia X, Wan L, Hu L. A novel method to simultaneously record spinal cord electrophysiology and electroencephalography signals. Neuroimage 2021; 232:117892. [PMID: 33617992 DOI: 10.1016/j.neuroimage.2021.117892] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2020] [Revised: 01/13/2021] [Accepted: 02/16/2021] [Indexed: 11/16/2022] Open
Abstract
The brain and the spinal cord together make up the central nervous system (CNS). The functions of the human brain have been the focus of neuroscience research for a long time. However, the spinal cord is largely ignored, and the functional interaction of these two parts of the CNS is only partly understood. This study developed a novel method to simultaneously record spinal cord electrophysiology (SCE) and electroencephalography (EEG) signals and validated its performance using a classical resting-state study design with two experimental conditions: eyes-closed (EC) and eyes-open (EO). We recruited nine postherpetic neuralgia patients implanted with a spinal cord stimulator, which was modified to record SCE signals simultaneously with EEG signals. For both EEG and SCE, similar differences were found in delta- and alpha-band oscillations between the EC and EO conditions, and the spectral power of these frequency bands was able to predict EC/EO behaviors. Moreover, causal connectivity analysis suggested a top-down regulation in delta-band oscillations from the brain to the spinal cord. Altogether, this study demonstrates the validity of simultaneous SCE-EEG recording and shows that the novel method is a valuable tool to investigate the brain-spinal interaction. With this method, we can better unite knowledge about the brain and the spinal cord for a deeper understanding of the functions of the whole CNS.
Collapse
Affiliation(s)
- Feixue Wang
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China; CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China; Research Center of Brain Cognitive Neuroscience, Liaoning Normal University, Dalian, China
| | - Libo Zhang
- CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Lupeng Yue
- CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Yuxuan Zeng
- CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China
| | - Qing Zhao
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China; CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China
| | - Qingjuan Gong
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China
| | - Jianbo Zhang
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China
| | - Dongyang Liu
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China
| | - Xiuying Luo
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China
| | - Xiaolei Xia
- CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Li Wan
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China.
| | - Li Hu
- Department of Pain Management, The State Key Clinical Specialty in Pain Medicine, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou, China; CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
18
|
Caharel S, Rossion B. The N170 is Sensitive to Long-term (Personal) Familiarity of a Face Identity. Neuroscience 2021; 458:244-255. [PMID: 33460730 DOI: 10.1016/j.neuroscience.2020.12.036] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Revised: 11/30/2020] [Accepted: 12/31/2020] [Indexed: 01/23/2023]
Abstract
The N170 is a large deflection of the human electroencephalogram (EEG), peaking at about 170 milliseconds over the occipito-temporal cortex after the sudden onset of a face stimulus. The N170 reflects perceptual awareness of a face and its onset corresponds to the emergence of reliable face-selectivity in the human brain. However, whether sensitivity to the long-term familiarity of a face identity emerges already at this early time-point remains debated. Here we provide a brief survey of the 45 published studies comparing the N170 response to unfamiliar and familiar (famous, experimentally familiarized, personally familiar and own) faces. Even though effects of familiarity on the N170 are relatively small and inconsistent across studies, this overview indicates that face familiarity significantly increases the N170 amplitude. This effect is especially present for personally familiar faces, learned in natural conditions. In the human brain, effects linked to familiarity with specific facial identities therefore appear to emerge between 150 and 200 ms in occipito-temporal brain regions, i.e., shortly after the onset of face-selectivity but at the same time as the earliest high-level effects of immediate unfamiliar face identity repetition. This observation challenges standard neurocognitive models with a clear-cut distinction between perceptual and memory stages in human face recognition.
Collapse
Affiliation(s)
- Stéphanie Caharel
- Université de Lorraine, Laboratoire Lorrain de Psychologie et Neurosciences (2LPN - EA 7489), Nancy F-54000, France; Institut Universitaire de France, Paris F-75000, France
| | - Bruno Rossion
- Université de Lorraine, CNRS, CRAN, F-54000 Nancy, France; CHRU-Nancy, Service de Neurologie, F-54000, France.
| |
Collapse
|
19
|
Yan X, Rossion B. A robust neural familiar face recognition response in a dynamic (periodic) stream of unfamiliar faces. Cortex 2020; 132:281-295. [DOI: 10.1016/j.cortex.2020.08.016] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2020] [Revised: 07/22/2020] [Accepted: 08/10/2020] [Indexed: 01/23/2023]
|
20
|
Rossion B, Retter TL, Liu‐Shuang J. Understanding human individuation of unfamiliar faces with oddball fast periodic visual stimulation and electroencephalography. Eur J Neurosci 2020; 52:4283-4344. [DOI: 10.1111/ejn.14865] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2020] [Revised: 05/19/2020] [Accepted: 05/30/2020] [Indexed: 01/08/2023]
Affiliation(s)
- Bruno Rossion
- CNRS, CRAN UMR7039 Université de Lorraine F‐54000Nancy France
- Service de Neurologie, CHRU‐Nancy Université de Lorraine F‐54000Nancy France
| | - Talia L. Retter
- Department of Behavioural and Cognitive Sciences Faculty of Language and Literature Humanities, Arts and Education University of Luxembourg Luxembourg Luxembourg
| | - Joan Liu‐Shuang
- Institute of Research in Psychological Science Institute of Neuroscience Université de Louvain Louvain‐la‐Neuve Belgium
| |
Collapse
|
21
|
The Influence of Object-Color Knowledge on Emerging Object Representations in the Brain. J Neurosci 2020; 40:6779-6789. [PMID: 32703903 DOI: 10.1523/jneurosci.0158-20.2020] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2020] [Revised: 07/13/2020] [Accepted: 07/15/2020] [Indexed: 11/21/2022] Open
Abstract
The ability to rapidly and accurately recognize complex objects is a crucial function of the human visual system. To recognize an object, we need to bind incoming visual features, such as color and form, together into cohesive neural representations and integrate these with our preexisting knowledge about the world. For some objects, typical color is a central feature for recognition; for example, a banana is typically yellow. Here, we applied multivariate pattern analysis on time-resolved neuroimaging (MEG) data to examine how object-color knowledge affects emerging object representations over time. Our results from 20 participants (11 female) show that the typicality of object-color combinations influences object representations, although not at the initial stages of object and color processing. We find evidence that color decoding peaks later for atypical object-color combinations compared with typical object-color combinations, illustrating the interplay between processing incoming object features and stored object knowledge. Together, these results provide new insights into the integration of incoming visual information with existing conceptual object knowledge.SIGNIFICANCE STATEMENT To recognize objects, we have to be able to bind object features, such as color and shape, into one coherent representation and compare it with stored object knowledge. The MEG data presented here provide novel insights about the integration of incoming visual information with our knowledge about the world. Using color as a model to understand the interaction between seeing and knowing, we show that there is a unique pattern of brain activity for congruently colored objects (e.g., a yellow banana) relative to incongruently colored objects (e.g., a red banana). This effect of object-color knowledge only occurs after single object features are processed, demonstrating that conceptual knowledge is accessed relatively late in the visual processing hierarchy.
Collapse
|
22
|
Familiarization increases face individuation measured with fast periodic visual stimulation. Biol Psychol 2020; 153:107883. [DOI: 10.1016/j.biopsycho.2020.107883] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2019] [Revised: 03/20/2020] [Accepted: 03/22/2020] [Indexed: 01/01/2023]
|
23
|
Yan X, Zimmermann FGS, Rossion B. An implicit neural familiar face identity recognition response across widely variable natural views in the human brain. Cogn Neurosci 2020; 11:143-156. [DOI: 10.1080/17588928.2020.1712344] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Affiliation(s)
- Xiaoqian Yan
- CNRS, CRAN, Université de Lorraine, Nancy, France
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Belgium
| | - Friederike GS Zimmermann
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Belgium
- BG Klinikum Humburg, Neurologie, Hamburg, Germany
| | - Bruno Rossion
- CNRS, CRAN, Université de Lorraine, Nancy, France
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Belgium
- CHRU-Nancy, Service de Neurologie, France
| |
Collapse
|
24
|
Grootswagers T, Robinson AK, Shatek SM, Carlson TA. Untangling featural and conceptual object representations. Neuroimage 2019; 202:116083. [PMID: 31400529 DOI: 10.1016/j.neuroimage.2019.116083] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2019] [Revised: 07/29/2019] [Accepted: 08/06/2019] [Indexed: 10/26/2022] Open
Abstract
How are visual inputs transformed into conceptual representations by the human visual system? The contents of human perception, such as objects presented on a visual display, can reliably be decoded from voxel activation patterns in fMRI, and in evoked sensor activations in MEG and EEG. A prevailing question is the extent to which brain activation associated with object categories is due to statistical regularities of visual features within object categories. Here, we assessed the contribution of mid-level features to conceptual category decoding using EEG and a novel fast periodic decoding paradigm. Our study used a stimulus set consisting of intact objects from the animate (e.g., fish) and inanimate categories (e.g., chair) and scrambled versions of the same objects that were unrecognizable and preserved their visual features (Long et al., 2018). By presenting the images at different periodic rates, we biased processing to different levels of the visual hierarchy. We found that scrambled objects and their intact counterparts elicited similar patterns of activation, which could be used to decode the conceptual category (animate or inanimate), even for the unrecognizable scrambled objects. Animacy decoding for the scrambled objects, however, was only possible at the slowest periodic presentation rate. Animacy decoding for intact objects was faster, more robust, and could be achieved at faster presentation rates. Our results confirm that the mid-level visual features preserved in the scrambled objects contribute to animacy decoding, but also demonstrate that the dynamics vary markedly for intact versus scrambled objects. Our findings suggest a complex interplay between visual feature coding and categorical representations that is mediated by the visual system's capacity to use image features to resolve a recognisable object.
Collapse
Affiliation(s)
- Tijl Grootswagers
- School of Psychology, University of Sydney, Sydney, NSW, Australia; Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia.
| | - Amanda K Robinson
- School of Psychology, University of Sydney, Sydney, NSW, Australia; Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia
| | - Sophia M Shatek
- School of Psychology, University of Sydney, Sydney, NSW, Australia
| | - Thomas A Carlson
- School of Psychology, University of Sydney, Sydney, NSW, Australia
| |
Collapse
|
25
|
Robinson AK, Grootswagers T, Carlson TA. The influence of image masking on object representations during rapid serial visual presentation. Neuroimage 2019; 197:224-231. [DOI: 10.1016/j.neuroimage.2019.04.050] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2019] [Revised: 03/22/2019] [Accepted: 04/17/2019] [Indexed: 01/20/2023] Open
|
26
|
Zimmermann FGS, Yan X, Rossion B. An objective, sensitive and ecologically valid neural measure of rapid human individual face recognition. ROYAL SOCIETY OPEN SCIENCE 2019; 6:181904. [PMID: 31312474 PMCID: PMC6599768 DOI: 10.1098/rsos.181904] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Accepted: 05/10/2019] [Indexed: 06/10/2023]
Abstract
Humans may be the only species able to rapidly and automatically recognize a familiar face identity in a crowd of unfamiliar faces, an important social skill. Here, by combining electroencephalography (EEG) and fast periodic visual stimulation (FPVS), we introduce an ecologically valid, objective and sensitive neural measure of this human individual face recognition function. Natural images of various unfamiliar faces are presented at a fast rate of 6 Hz, allowing one fixation per face, with variable natural images of a highly familiar face identity, a celebrity, appearing every seven images (0.86 Hz). Following a few minutes of stimulation, a high signal-to-noise ratio neural response reflecting the generalized discrimination of the familiar face identity from unfamiliar faces is observed over the occipito-temporal cortex at 0.86 Hz and harmonics. When face images are presented upside-down, the individual familiar face recognition response is negligible, being reduced by a factor of 5 over occipito-temporal regions. Differences in the magnitude of the individual face recognition response across different familiar face identities suggest that factors such as exposure, within-person variability and distinctiveness mediate this response. Our findings of a biological marker for fast and automatic recognition of individual familiar faces with ecological stimuli open an avenue for understanding this function, its development and neural basis in neurotypical individual brains along with its pathology. This should also have implications for the use of facial recognition measures in forensic science.
Collapse
Affiliation(s)
- Friederike G. S. Zimmermann
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Louvain-la-Neuve, Belgium
- BG Klinikum Hamburg, Bergedorfer Straße 10, 21033 Hamburg, Germany
| | - Xiaoqian Yan
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Louvain-la-Neuve, Belgium
| | - Bruno Rossion
- Institute of Research in Psychological Science, Institute of Neuroscience, Université de Louvain, Louvain-la-Neuve, Belgium
- Université de Lorraine, CNRS, CRAN, 54000 Nancy, France
- CHRU-Nancy, Service de Neurologie, 54000 Nancy, France
| |
Collapse
|
27
|
Dobs K, Isik L, Pantazis D, Kanwisher N. How face perception unfolds over time. Nat Commun 2019; 10:1258. [PMID: 30890707 PMCID: PMC6425020 DOI: 10.1038/s41467-019-09239-1] [Citation(s) in RCA: 111] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Accepted: 02/24/2019] [Indexed: 11/08/2022] Open
Abstract
Within a fraction of a second of viewing a face, we have already determined its gender, age and identity. A full understanding of this remarkable feat will require a characterization of the computational steps it entails, along with the representations extracted at each. Here, we used magnetoencephalography (MEG) to measure the time course of neural responses to faces, thereby addressing two fundamental questions about how face processing unfolds over time. First, using representational similarity analysis, we found that facial gender and age information emerged before identity information, suggesting a coarse-to-fine processing of face dimensions. Second, identity and gender representations of familiar faces were enhanced very early on, suggesting that the behavioral benefit for familiar faces results from tuning of early feed-forward processing mechanisms. These findings start to reveal the time course of face processing in humans, and provide powerful new constraints on computational theories of face perception.
Collapse
Affiliation(s)
- Katharina Dobs
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
- McGovern Institute of Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
- The Center for Brains, Minds, and Machines, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
| | - Leyla Isik
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- McGovern Institute of Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- The Center for Brains, Minds, and Machines, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Dimitrios Pantazis
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- McGovern Institute of Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- McGovern Institute of Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- The Center for Brains, Minds, and Machines, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| |
Collapse
|
28
|
The representational dynamics of visual objects in rapid serial visual processing streams. Neuroimage 2019; 188:668-679. [DOI: 10.1016/j.neuroimage.2018.12.046] [Citation(s) in RCA: 41] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2018] [Revised: 12/17/2018] [Accepted: 12/22/2018] [Indexed: 11/15/2022] Open
|
29
|
Haigh SM, Robinson AK, Grover P, Behrmann M. Differentiation of Types of Visual Agnosia Using EEG. Vision (Basel) 2018; 2:E44. [PMID: 31735907 PMCID: PMC6836011 DOI: 10.3390/vision2040044] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2018] [Revised: 12/05/2018] [Accepted: 12/13/2018] [Indexed: 11/23/2022] Open
Abstract
Visual recognition deficits are the hallmark symptom of visual agnosia, a neuropsychological disorder typically associated with damage to the visual system. Most research into visual agnosia focuses on characterizing the deficits through detailed behavioral testing, and structural and functional brain scans are used to determine the spatial extent of any cortical damage. Although the hierarchical nature of the visual system leads to clear predictions about the temporal dynamics of cortical deficits, there has been little research on the use of neuroimaging methods with high temporal resolution to characterize the temporal profile of agnosia deficits. Here, we employed high-density electroencephalography (EEG) to investigate alterations in the temporal dynamics of the visual system in two individuals with visual agnosia. In the context of a steady state visual evoked potential paradigm (SSVEP), individuals viewed pattern-reversing checkerboards of differing spatial frequency, and we assessed the responses of the visual system in the frequency and temporal domain. JW, a patient with early visual cortex damage, showed impaired SSVEP response relative to a control group and to the second patient (SM) who had right temporal lobe damage. JW also showed lower decoding accuracy for early visual responses (around 100 ms). SM, whose lesion is more anterior in the visual system, showed good decoding accuracy initially but low decoding after 500 ms. Overall, EEG and multivariate decoding methods can yield important insights into the temporal dynamics of visual responses in individuals with visual agnosia.
Collapse
Affiliation(s)
- Sarah M. Haigh
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Department of Psychology and Center for Integrative Neuroscience, University of Nevada, Reno, NV 89557, USA
| | - Amanda K. Robinson
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- ARC Centre of Excellence in Cognition and its Disorders, Department of Cognitive Science, Macquarie University, Sydney 2109, Australia
- School of Psychology, The University of Sydney, Sydney 2006, Australia
| | - Pulkit Grover
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Marlene Behrmann
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| |
Collapse
|