1
|
Karimi-Rouzbahani H. Evidence for Multiscale Multiplexed Representation of Visual Features in EEG. Neural Comput 2024; 36:412-436. [PMID: 38363657 DOI: 10.1162/neco_a_01649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 12/01/2023] [Indexed: 02/18/2024]
Abstract
Distinct neural processes such as sensory and memory processes are often encoded over distinct timescales of neural activations. Animal studies have shown that this multiscale coding strategy is also implemented for individual components of a single process, such as individual features of a multifeature stimulus in sensory coding. However, the generalizability of this encoding strategy to the human brain has remained unclear. We asked if individual features of visual stimuli were encoded over distinct timescales. We applied a multiscale time-resolved decoding method to electroencephalography (EEG) collected from human subjects presented with grating visual stimuli to estimate the timescale of individual stimulus features. We observed that the orientation and color of the stimuli were encoded in shorter timescales, whereas spatial frequency and the contrast of the same stimuli were encoded in longer timescales. The stimulus features appeared in temporally overlapping windows along the trial supporting a multiplexed coding strategy. These results provide evidence for a multiplexed, multiscale coding strategy in the human visual system.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- Neurosciences Centre, Mater Hospital, Brisbane 4101, Australia
- Queensland Brain Institute, University of Queensland, Brisbane 4067, Australia
- Mater Research Institute, University of Queensland, Brisbane 4101, Australia
| |
Collapse
|
2
|
Klink H, Kaiser D, Stecher R, Ambrus GG, Kovács G. Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding. Cereb Cortex 2023; 33:11634-11645. [PMID: 37885126 DOI: 10.1093/cercor/bhad397] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 09/29/2023] [Accepted: 09/30/2023] [Indexed: 10/28/2023] Open
Abstract
Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.
Collapse
Affiliation(s)
- Hannah Klink
- Department of Neurology, Universitätsklinikum, Kastanienstraße1 Jena, D-07747 Jena, Thüringen, Germany
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich Schiller University Jena, Leutragraben 1, D-07743 Jena, Thüringen, Germany
| | - Daniel Kaiser
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus-Liebig-University Gießen, Arndtstraße 2, D-35392 Gießen, Hessen, Germany
- Center for Mind, Brain and Behavior (CMBB), Justus-Liebig-University Gießen and Philipps-University Marburg, Hans-Meerwein-Straße 6 Mehrzweckgeb, 03C022, Marburg, D-35032, Hessen, Germany
| | - Rico Stecher
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus-Liebig-University Gießen, Arndtstraße 2, D-35392 Gießen, Hessen, Germany
| | - Géza G Ambrus
- Department of Psychology, Bournemouth University, Poole House P319, Talbot Campus, Fern Barrow, Poole, Dorset BH12 5BB, United Kingdom
| | - Gyula Kovács
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich Schiller University Jena, Leutragraben 1, D-07743 Jena, Thüringen, Germany
| |
Collapse
|
3
|
von Seth J, Nicholls VI, Tyler LK, Clarke A. Recurrent connectivity supports higher-level visual and semantic object representations in the brain. Commun Biol 2023; 6:1207. [PMID: 38012301 PMCID: PMC10682037 DOI: 10.1038/s42003-023-05565-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 11/09/2023] [Indexed: 11/29/2023] Open
Abstract
Visual object recognition has been traditionally conceptualised as a predominantly feedforward process through the ventral visual pathway. While feedforward artificial neural networks (ANNs) can achieve human-level classification on some image-labelling tasks, it's unclear whether computational models of vision alone can accurately capture the evolving spatiotemporal neural dynamics. Here, we probe these dynamics using a combination of representational similarity and connectivity analyses of fMRI and MEG data recorded during the recognition of familiar, unambiguous objects. Modelling the visual and semantic properties of our stimuli using an artificial neural network as well as a semantic feature model, we find that unique aspects of the neural architecture and connectivity dynamics relate to visual and semantic object properties. Critically, we show that recurrent processing between the anterior and posterior ventral temporal cortex relates to higher-level visual properties prior to semantic object properties, in addition to semantic-related feedback from the frontal lobe to the ventral temporal lobe between 250 and 500 ms after stimulus onset. These results demonstrate the distinct contributions made by semantic object properties in explaining neural activity and connectivity, highlighting it as a core part of object recognition not fully accounted for by current biologically inspired neural networks.
Collapse
Affiliation(s)
- Jacqueline von Seth
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| | | | - Lorraine K Tyler
- Department of Psychology, University of Cambridge, Cambridge, UK
- Cambridge Centre for Ageing and Neuroscience (Cam-CAN), University of Cambridge and MRC Cognition and Brain Sciences Unit, Cambridge, UK
| | - Alex Clarke
- Department of Psychology, University of Cambridge, Cambridge, UK.
| |
Collapse
|
4
|
Kovács G, Li C, Ambrus GG, Burton AM. The neural dynamics of familiarity-dependent face identity representation. Psychophysiology 2023; 60:e14304. [PMID: 37009756 DOI: 10.1111/psyp.14304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 03/20/2023] [Accepted: 03/20/2023] [Indexed: 04/04/2023]
Abstract
Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.
Collapse
Affiliation(s)
- Gyula Kovács
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Jena, Germany
| | - Chenglin Li
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Jena, Germany
- School of Psychology, Zhejiang Normal University, Jinhua, China
| | - Géza Gergely Ambrus
- Department of Psychology, Faculty of Science and Technology, Bournemouth University, Poole, UK
| | - A Mike Burton
- Department of Psychology, University of York, York, UK
- Faculty of Society and Design, Bond University, Gold Coast, Qld, Australia
| |
Collapse
|
5
|
Popova T, Wiese H. Developing familiarity during the first eight months of knowing a person: A longitudinal EEG study on face and identity learning. Cortex 2023; 165:26-37. [PMID: 37245406 DOI: 10.1016/j.cortex.2023.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 02/03/2023] [Accepted: 04/23/2023] [Indexed: 05/30/2023]
Abstract
It is well-established that familiar and unfamiliar faces are processed differently, but surprisingly little is known about how familiarity builds up over time and how novel faces gradually become represented in the brain. Here, we used event-related brain potentials (ERPs) in a pre-registered, longitudinal study to examine the neural processes accompanying face and identity learning during the first eight months of knowing a person. Specifically, we examined how increasing real-life familiarity affects visual recognition (N250 Familiarity Effect) and the integration of person-related knowledge (Sustained Familiarity Effect, SFE). Sixteen first-year undergraduates were tested in three sessions, approximately one, five, and eight months after the start of the academic year, with highly variable "ambient" images of a new friend they had met at university and of an unfamiliar person. We observed clear ERP familiarity effects for the new friend after one month of familiarity. While there was an increase in the N250 effect over the course of the study, no change in the SFE was observed. These results suggest that visual face representations develop faster relative to the integration of identity-specific knowledge.
Collapse
|
6
|
Liu X, Melcher D. The effect of familiarity on behavioral oscillations in face perception. Sci Rep 2023; 13:10145. [PMID: 37349366 PMCID: PMC10287701 DOI: 10.1038/s41598-023-34812-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 05/08/2023] [Indexed: 06/24/2023] Open
Abstract
Studies on behavioral oscillations demonstrate that visual sensitivity fluctuates over time and visual processing varies periodically, mirroring neural oscillations at the same frequencies. Do these behavioral oscillations reflect fixed and relatively automatic sensory sampling, or top-down processes such as attention or predictive coding? To disentangle these theories, the current study used a dual-target rapid serial visual presentation paradigm, where participants indicated the gender of a face target embedded in streams of distractors presented at 30 Hz. On critical trials, two identical targets were presented with varied stimulus onset asynchrony from 200 to 833 ms. The target was either familiar or unfamiliar faces, divided into different blocks. We found a 4.6 Hz phase-coherent fluctuation in gender discrimination performance across both trial types, consistent with previous reports. In addition, however, we found an effect at the alpha frequency, with behavioral oscillations in the familiar blocks characterized by a faster high-alpha peak than for the unfamiliar face blocks. These results are consistent with the combination of both a relatively stable modulation in the theta band and faster modulation of the alpha oscillations. Therefore, the overall pattern of perceptual sampling in visual perception may depend, at least in part, on task demands. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 16/08/2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/A98UF .
Collapse
Affiliation(s)
- Xiaoyi Liu
- New York University Abu Dhabi, Abu Dhabi, UAE
| | | |
Collapse
|
7
|
Mokari-Mahallati M, Ebrahimpour R, Bagheri N, Karimi-Rouzbahani H. Deeper neural network models better reflect how humans cope with contrast variation in object recognition. Neurosci Res 2023:S0168-0102(23)00007-X. [PMID: 36681154 DOI: 10.1016/j.neures.2023.01.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Revised: 11/27/2022] [Accepted: 01/17/2023] [Indexed: 01/20/2023]
Abstract
Visual inputs are far from ideal in everyday situations such as in the fog where the contrasts of input stimuli are low. However, human perception remains relatively robust to contrast variations. To provide insights about the underlying mechanisms of contrast invariance, we addressed two questions. Do contrast effects disappear along the visual hierarchy? Do later stages of the visual hierarchy contribute to contrast invariance? We ran a behavioral experiment where we manipulated the level of stimulus contrast and the involvement of higher-level visual areas through immediate and delayed backward masking of the stimulus. Backward masking led to significant drop in performance in our visual categorization task, supporting the role of higher-level visual areas in contrast invariance. To obtain mechanistic insights, we ran the same categorization task on three state-of the-art computational models of human vision each with a different depth in visual hierarchy. We found contrast effects all along the visual hierarchy, no matter how far into the hierarchy. Moreover, that final layers of deeper hierarchical models, which had been shown to be best models of final stages of the visual system, coped with contrast effects more effectively. These results suggest that, while contrast effects reach the final stages of the hierarchy, those stages play a significant role in compensating for contrast variations in the visual system.
Collapse
Affiliation(s)
- Masoumeh Mokari-Mahallati
- Department of Electrical Engineering, Shahid Rajaee Teacher Training University, Tehran, Islamic Republic of Iran
| | - Reza Ebrahimpour
- Center for Cognitive Science, Institute for Convergence Science and Technology (ICST), Sharif University of Technology, Tehran P.O.Box:11155-1639, Islamic Republic of Iran; Department of Computer Engineering, Shahid Rajaee Teacher Training University, Tehran, Islamic Republic of Iran; School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Tehran, Islamic Republic of Iran.
| | - Nasour Bagheri
- Department of Electrical Engineering, Shahid Rajaee Teacher Training University, Tehran, Islamic Republic of Iran
| | - Hamid Karimi-Rouzbahani
- MRC Cognition & Brain Sciences Unit, University of Cambridge, UK; Mater Research Institute, Faculty of Medicine, University of Queensland, Australia
| |
Collapse
|
8
|
Dalski A, Kovács G, Ambrus GG. No semantic information is necessary to evoke general neural signatures of face familiarity: evidence from cross-experiment classification. Brain Struct Funct 2023; 228:449-462. [PMID: 36244002 PMCID: PMC9944719 DOI: 10.1007/s00429-022-02583-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 10/09/2022] [Indexed: 11/28/2022]
Abstract
Recent theories on the neural correlates of face identification stressed the importance of the available identity-specific semantic and affective information. However, whether such information is essential for the emergence of neural signal of familiarity has not yet been studied in detail. Here, we explored the shared representation of face familiarity between perceptually and personally familiarized identities. We applied a cross-experiment multivariate pattern classification analysis (MVPA), to test if EEG patterns for passive viewing of personally familiar and unfamiliar faces are useful in decoding familiarity in a matching task where familiarity was attained thorough a short perceptual task. Importantly, no additional semantic, contextual, or affective information was provided for the familiarized identities during perceptual familiarization. Although the two datasets originate from different sets of participants who were engaged in two different tasks, familiarity was still decodable in the sorted, same-identity matching trials. This finding indicates that the visual processing of the faces of personally familiar and purely perceptually familiarized identities involve similar mechanisms, leading to cross-classifiable neural patterns.
Collapse
Affiliation(s)
- Alexia Dalski
- Department of Psychology, Philipps-Universität Marburg, 35039 Marburg, Germany ,Center for Mind, Brain and Behavior – CMBB, Philipps-Universität Marburg and Justus-Liebig-Universität Giessen, 35039 Marburg, Germany
| | - Gyula Kovács
- Institute of Psychology, Friedrich Schiller University Jena, 07743 Jena, Germany
| | - Géza Gergely Ambrus
- Institute of Psychology, Friedrich Schiller University Jena, 07743, Jena, Germany. .,Department of Psychology, Bournemouth University, Poole House, Talbot Campus, Fern Barrow, Poole, BH12 5BB, Dorset, UK.
| |
Collapse
|
9
|
Trinh A, Dunn JD, White D. Verifying unfamiliar identities: Effects of processing name and face information in the same identity-matching task. Cogn Res Princ Implic 2022; 7:92. [PMID: 36224440 PMCID: PMC9556678 DOI: 10.1186/s41235-022-00441-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 09/27/2022] [Indexed: 11/10/2022] Open
Abstract
Matching the identity of unfamiliar faces is important in applied identity verification tasks, for example when verifying photo ID at border crossings, in secure access areas, or when issuing identity credentials. In these settings, other biographical details-such as name or date of birth on an identity document-are also often compared to existing records, but the impact of these concurrent checks on decisions has not been examined. Here, we asked participants to sequentially compare name, then face information between an ID card and digital records to detect errors. Across four experiments (combined n = 274), despite being told that mismatches between written name pairs and face image pairs were independent, participants were more likely to say that face images matched when names also matched. Across all experiments, we found that this bias was unaffected by the image quality, suggesting that the source of the bias is somewhat independent of perceptual processes. In a final experiment, we show that this decisional bias was found only for name checks, but not when participants were asked to check ID card expiration dates or unrelated object names. We conclude that the bias arises from processing identity information and propose that it operates at the level of unfamiliar person identity representations. Results are interpreted in the context of theoretical models of face processing, and we discuss applied implications.
Collapse
Affiliation(s)
- Anita Trinh
- grid.1005.40000 0004 4902 0432School of Psychology, UNSW Sydney, Kensington, NSW 2052 Australia
| | - James D. Dunn
- grid.1005.40000 0004 4902 0432School of Psychology, UNSW Sydney, Kensington, NSW 2052 Australia
| | - David White
- grid.1005.40000 0004 4902 0432School of Psychology, UNSW Sydney, Kensington, NSW 2052 Australia
| |
Collapse
|
10
|
Characterizing the shared signals of face familiarity: Long-term acquaintance, voluntary control, and concealed knowledge. Brain Res 2022; 1796:148094. [PMID: 36116487 DOI: 10.1016/j.brainres.2022.148094] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2022] [Revised: 09/10/2022] [Accepted: 09/12/2022] [Indexed: 11/20/2022]
Abstract
In a recent study using cross-experiment multivariate classification of EEG patterns, we found evidence for a shared familiarity signal for faces, patterns of neural activity that successfully separate trials for familiar and unfamiliar faces across participants and modes of familiarization. Here, our aim was to expand upon this research to further characterize the spatio-temporal properties of this signal. By utilizing the information content present for incidental exposure to personally familiar and unfamiliar faces, we tested how the information content in the neural signal unfolds over time under different task demands - giving truthful or deceptive responses to photographs of genuinely familiar and unfamiliar individuals. For this goal, we re-analyzed data from two previously published experiments using within-experiment leave-one-subject-out and cross-experiment classification of face familiarity. We observed that the general face familiarity signal, consistent with its previously described spatio-temporal properties, is present for long-term personally familiar faces under passive viewing, as well as for acknowledged and concealed familiarity responses. Also, central-posterior regions contain information related to deception. We propose that signals in the 200-400 ms window are modulated by top-down task-related anticipation, while the patterns in the 400-600 ms window are influenced by conscious effort to deceive. To our knowledge, this is the first report describing the representational dynamics of concealed knowledge for faces, using time-resolved multivariate classification.
Collapse
|
11
|
Li C, Burton AM, Ambrus GG, Kovács G. A neural measure of the degree of face familiarity. Cortex 2022; 155:1-12. [DOI: 10.1016/j.cortex.2022.06.012] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 06/05/2022] [Accepted: 06/08/2022] [Indexed: 11/03/2022]
|
12
|
Karimi-Rouzbahani H, Woolgar A, Henson R, Nili H. Caveats and Nuances of Model-Based and Model-Free Representational Connectivity Analysis. Front Neurosci 2022; 16:755988. [PMID: 35360178 PMCID: PMC8960982 DOI: 10.3389/fnins.2022.755988] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Accepted: 02/02/2022] [Indexed: 11/30/2022] Open
Abstract
Brain connectivity analyses have conventionally relied on statistical relationship between one-dimensional summaries of activation in different brain areas. However, summarizing activation patterns within each area to a single dimension ignores the potential statistical dependencies between their multi-dimensional activity patterns. Representational Connectivity Analyses (RCA) is a method that quantifies the relationship between multi-dimensional patterns of activity without reducing the dimensionality of the data. We consider two variants of RCA. In model-free RCA, the goal is to quantify the shared information for two brain regions. In model-based RCA, one tests whether two regions have shared information about a specific aspect of the stimuli/task, as defined by a model. However, this is a new approach and the potential caveats of model-free and model-based RCA are still understudied. We first explain how model-based RCA detects connectivity through the lens of models, and then present three scenarios where model-based and model-free RCA give discrepant results. These conflicting results complicate the interpretation of functional connectivity. We highlight the challenges in three scenarios: complex intermediate models, common patterns across regions, and transformation of representational structure across brain regions. The article is accompanied by scripts (https://osf.io/3nxfa/) that reproduce the results. In each case, we suggest potential ways to mitigate the difficulties caused by inconsistent results. The results of this study shed light on some understudied aspects of RCA, and allow researchers to use the method more effectively.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
- Department of Computing, Macquarie University, Sydney, NSW, Australia
| | - Alexandra Woolgar
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
| | - Richard Henson
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
- Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
| | - Hamed Nili
- Department of Excellence for Neural Information Processing, Center for Molecular Neurobiology (ZMNH), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
13
|
Karimi-Rouzbahani H, Woolgar A. When the Whole Is Less Than the Sum of Its Parts: Maximum Object Category Information and Behavioral Prediction in Multiscale Activation Patterns. Front Neurosci 2022; 16:825746. [PMID: 35310090 PMCID: PMC8924472 DOI: 10.3389/fnins.2022.825746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 01/24/2022] [Indexed: 11/19/2022] Open
Abstract
Neural codes are reflected in complex neural activation patterns. Conventional electroencephalography (EEG) decoding analyses summarize activations by averaging/down-sampling signals within the analysis window. This diminishes informative fine-grained patterns. While previous studies have proposed distinct statistical features capable of capturing variability-dependent neural codes, it has been suggested that the brain could use a combination of encoding protocols not reflected in any one mathematical feature alone. To check, we combined 30 features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across three datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed across most of the time points by the multiscale feature of Wavelet coefficients. Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the EEG neural codes better than any combination of protocols. Our findings put new constraints on the models of neural information encoding in EEG.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
- Department of Cognitive Science, Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia
- Department of Computing, Macquarie University, Sydney, NSW, Australia
| | - Alexandra Woolgar
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
- Department of Cognitive Science, Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia
| |
Collapse
|
14
|
Dalski A, Kovács G, Ambrus GG. Evidence for a General Neural Signature of Face Familiarity. Cereb Cortex 2021; 32:2590-2601. [PMID: 34628490 DOI: 10.1093/cercor/bhab366] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Revised: 08/12/2021] [Accepted: 08/13/2021] [Indexed: 11/12/2022] Open
Abstract
We explored the neural signatures of face familiarity using cross-participant and cross-experiment decoding of event-related potentials, evoked by unknown and experimentally familiarized faces from a set of experiments with different participants, stimuli, and familiarization-types. Human participants of both sexes were either familiarized perceptually, via media exposure, or by personal interaction. We observed significant cross-experiment familiarity decoding involving all three experiments, predominantly over posterior and central regions of the right hemisphere in the 270-630 ms time window. This shared face familiarity effect was most prominent across the Media and the Personal, as well as between the Perceptual and Personal experiments. Cross-experiment decodability makes this signal a strong candidate for a general neural indicator of face familiarity, independent of familiarization methods, participants, and stimuli. Furthermore, the sustained pattern of temporal generalization suggests that it reflects a single automatic processing cascade that is maintained over time.
Collapse
Affiliation(s)
- Alexia Dalski
- Institute of Psychology, Friedrich Schiller University Jena, D-07743 Jena, Germany
- Department of Psychology, Philipps-Universität Marburg, D-35039 Marburg, Germany
- Center for Mind, Brain and Behavior - CMBB, Philipps-Universität Marburg and Justus-Liebig-Universität Giessen, D-35039 Marburg, Germany
| | - Gyula Kovács
- Institute of Psychology, Friedrich Schiller University Jena, D-07743 Jena, Germany
| | - Géza Gergely Ambrus
- Institute of Psychology, Friedrich Schiller University Jena, D-07743 Jena, Germany
| |
Collapse
|
15
|
Meyer K, Sommer W, Hildebrandt A. Reflections and New Perspectives on Face Cognition as a Specific Socio-Cognitive Ability. J Intell 2021; 9:jintelligence9020030. [PMID: 34207993 PMCID: PMC8293405 DOI: 10.3390/jintelligence9020030] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 05/11/2021] [Accepted: 06/08/2021] [Indexed: 01/07/2023] Open
Abstract
The study of socio-cognitive abilities emerged from intelligence research, and their specificity remains controversial until today. In recent years, the psychometric structure of face cognition (FC)—a basic facet of socio-cognitive abilities—was extensively studied. In this review, we summarize and discuss the divergent psychometric structures of FC in easy and difficult tasks. While accuracy in difficult tasks was consistently shown to be face-specific, the evidence for easy tasks was inconsistent. The structure of response speed in easy tasks was mostly—but not always—unitary across object categories, including faces. Here, we compare studies to identify characteristics leading to face specificity in easy tasks. The following pattern emerges: in easy tasks, face specificity is found when modeling speed in a single task; however, when modeling speed across multiple, different easy tasks, only a unitary factor structure is reported. In difficult tasks, however, face specificity occurs in both single task approaches and task batteries. This suggests different cognitive mechanisms behind face specificity in easy and difficult tasks. In easy tasks, face specificity relies on isolated cognitive sub-processes such as face identity recognition. In difficult tasks, face-specific and task-independent cognitive processes are employed. We propose a descriptive model and argue for FC to be integrated into common taxonomies of intelligence.
Collapse
Affiliation(s)
- Kristina Meyer
- Charité—Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Psychiatric University Hospital Charité at St. Hedwig Hospital, Große Hamburger Str. 5-11, 10115 Berlin, Germany
- Correspondence:
| | - Werner Sommer
- Institut für Psychologie, Humboldt-Universität zu Berlin and Department of Psychology, Zhejiang Normal University, Jinhua 321004, China;
| | - Andrea Hildebrandt
- Department of Psychology, Carl von Ossietzky Universität Oldenburg and the Research Center Neurosensory Science, Carl von Ossietzky Universität Oldenburg, 26129 Oldenburg, Germany;
| |
Collapse
|
16
|
Jackson JB, Feredoes E, Rich AN, Lindner M, Woolgar A. Concurrent neuroimaging and neurostimulation reveals a causal role for dlPFC in coding of task-relevant information. Commun Biol 2021; 4:588. [PMID: 34002006 PMCID: PMC8128861 DOI: 10.1038/s42003-021-02109-x] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2020] [Accepted: 04/14/2021] [Indexed: 02/03/2023] Open
Abstract
Dorsolateral prefrontal cortex (dlPFC) is proposed to drive brain-wide focus by biasing processing in favour of task-relevant information. A longstanding debate concerns whether this is achieved through enhancing processing of relevant information and/or by inhibiting irrelevant information. To address this, we applied transcranial magnetic stimulation (TMS) during fMRI, and tested for causal changes in information coding. Participants attended to one feature, whilst ignoring another feature, of a visual object. If dlPFC is necessary for facilitation, disruptive TMS should decrease coding of attended features. Conversely, if dlPFC is crucial for inhibition, TMS should increase coding of ignored features. Here, we show that TMS decreases coding of relevant information across frontoparietal cortex, and the impact is significantly stronger than any effect on irrelevant information, which is not statistically detectable. This provides causal evidence for a specific role of dlPFC in enhancing task-relevant representations and demonstrates the cognitive-neural insights possible with concurrent TMS-fMRI-MVPA.
Collapse
Affiliation(s)
- Jade B Jackson
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.
- Perception in Action Research Centre, Department of Cognitive Science, Macquarie University, Sydney, NSW, Australia.
| | - Eva Feredoes
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK
| | - Anina N Rich
- Perception in Action Research Centre, Department of Cognitive Science, Macquarie University, Sydney, NSW, Australia
| | - Michael Lindner
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK
| | - Alexandra Woolgar
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
- Perception in Action Research Centre, Department of Cognitive Science, Macquarie University, Sydney, NSW, Australia
| |
Collapse
|
17
|
Karimi-Rouzbahani H, Woolgar A, Rich AN. Neural signatures of vigilance decrements predict behavioural errors before they occur. eLife 2021; 10:e60563. [PMID: 33830017 PMCID: PMC8060034 DOI: 10.7554/elife.60563] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 04/02/2021] [Indexed: 11/24/2022] Open
Abstract
There are many monitoring environments, such as railway control, in which lapses of attention can have tragic consequences. Problematically, sustained monitoring for rare targets is difficult, with more misses and longer reaction times over time. What changes in the brain underpin these 'vigilance decrements'? We designed a multiple-object monitoring (MOM) paradigm to examine how the neural representation of information varied with target frequency and time performing the task. Behavioural performance decreased over time for the rare target (monitoring) condition, but not for a frequent target (active) condition. This was mirrored in neural decoding using magnetoencephalography: coding of critical information declined more during monitoring versus active conditions along the experiment. We developed new analyses that can predict behavioural errors from the neural data more than a second before they occurred. This facilitates pre-empting behavioural errors due to lapses in attention and provides new insight into the neural correlates of vigilance decrements.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Medical Research Council Cognition and Brain Sciences Unit, University of CambridgeCambridgeUnited Kingdom
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| | - Alexandra Woolgar
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Medical Research Council Cognition and Brain Sciences Unit, University of CambridgeCambridgeUnited Kingdom
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| | - Anina N Rich
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| |
Collapse
|