1
|
Nara S, Kaiser D. Integrative processing in artificial and biological vision predicts the perceived beauty of natural images. SCIENCE ADVANCES 2024; 10:eadi9294. [PMID: 38427730 PMCID: PMC10906925 DOI: 10.1126/sciadv.adi9294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 01/29/2024] [Indexed: 03/03/2024]
Abstract
Previous research shows that the beauty of natural images is already determined during perceptual analysis. However, it is unclear which perceptual computations give rise to the perception of beauty. Here, we tested whether perceived beauty is predicted by spatial integration across an image, a perceptual computation that reduces processing demands by aggregating image parts into more efficient representations of the whole. We quantified integrative processing in an artificial deep neural network model, where the degree of integration was determined by the amount of deviation between activations for the whole image and its constituent parts. This quantification of integration predicted beauty ratings for natural images across four studies with different stimuli and designs. In a complementary functional magnetic resonance imaging study, we show that integrative processing in human visual cortex similarly predicts perceived beauty. Together, our results establish integration as a computational principle that facilitates perceptual analysis and thereby mediates the perception of beauty.
Collapse
Affiliation(s)
- Sanjeev Nara
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus Liebig University Gießen, Gießen Germany
| | - Daniel Kaiser
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus Liebig University Gießen, Gießen Germany
- Center for Mind, Brain and Behavior (CMBB), Philipps-University Marburg and Justus Liebig University Gießen, Marburg, Germany
| |
Collapse
|
2
|
Kaiser D. Spectral brain signatures of aesthetic natural perception in the α and β frequency bands. J Neurophysiol 2022; 128:1501-1505. [PMID: 36259673 DOI: 10.1152/jn.00385.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
During our everyday lives, visual beauty is often conveyed by sustained and dynamic visual stimulation, such as when we walk through an enchanting forest or watch our pets playing. Here, I devised an MEG experiment that mimics such situations: participants viewed 8 s videos of everyday situations and rated their beauty. Using multivariate analysis, I linked aesthetic ratings to 1) sustained MEG broadband responses and 2) spectral MEG responses in the α and β frequency bands. These effects were not accounted for by a set of high- and low-level visual descriptors of the videos, suggesting that they are genuinely related to aesthetic perception. My findings provide the first characterization of spectral brain signatures linked to aesthetic experiences in the real world.NEW & NOTEWORTHY In the real world, aesthetic experiences arise from complex and dynamic inputs. This study shows that such aesthetic experiences are represented in a spectral neural code: cortical α and β activity track our judgments of the aesthetic appearance of natural videos, providing a new starting point for studying the neural correlates of beauty through rhythmic brain activity.
Collapse
Affiliation(s)
- Daniel Kaiser
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus-Liebig-University, Gießen, Germany.,Center for Mind, Brain and Behavior (CMBB), Philipps-University Marburg and Justus-Liebig-University Gießen, Germany
| |
Collapse
|
3
|
Chen L, Cichy RM, Kaiser D. Semantic Scene-Object Consistency Modulates N300/400 EEG Components, but Does Not Automatically Facilitate Object Representations. Cereb Cortex 2022; 32:3553-3567. [PMID: 34891169 DOI: 10.1093/cercor/bhab433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 11/02/2021] [Accepted: 11/03/2021] [Indexed: 11/13/2022] Open
Abstract
During natural vision, objects rarely appear in isolation, but often within a semantically related scene context. Previous studies reported that semantic consistency between objects and scenes facilitates object perception and that scene-object consistency is reflected in changes in the N300 and N400 components in EEG recordings. Here, we investigate whether these N300/400 differences are indicative of changes in the cortical representation of objects. In two experiments, we recorded EEG signals, while participants viewed semantically consistent or inconsistent objects within a scene; in Experiment 1, these objects were task-irrelevant, while in Experiment 2, they were directly relevant for behavior. In both experiments, we found reliable and comparable N300/400 differences between consistent and inconsistent scene-object combinations. To probe the quality of object representations, we performed multivariate classification analyses, in which we decoded the category of the objects contained in the scene. In Experiment 1, in which the objects were not task-relevant, object category could be decoded from ~100 ms after the object presentation, but no difference in decoding performance was found between consistent and inconsistent objects. In contrast, when the objects were task-relevant in Experiment 2, we found enhanced decoding of semantically consistent, compared with semantically inconsistent, objects. These results show that differences in N300/400 components related to scene-object consistency do not index changes in cortical object representations but rather reflect a generic marker of semantic violations. Furthermore, our findings suggest that facilitatory effects between objects and scenes are task-dependent rather than automatic.
Collapse
Affiliation(s)
- Lixiang Chen
- Department of Education and Psychology, Freie Universität Berlin, Berlin 14195, Germany
| | - Radoslaw Martin Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin 14195, Germany
| | - Daniel Kaiser
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus-Liebig-Universität Gießen, Gießen 35392, Germany.,Center for Mind, Brain and Behavior (CMBB), Philipps-Universität Marburg and Justus-Liebig-Universität Gießen, Marburg 35032, Germany
| |
Collapse
|
4
|
Kaiser D. Characterizing Dynamic Neural Representations of Scene Attractiveness. J Cogn Neurosci 2022; 34:1988-1997. [PMID: 35802607 DOI: 10.1162/jocn_a_01891] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Aesthetic experiences during natural vision are varied: They can arise from viewing scenic landscapes, interesting architecture, or attractive people. Recent research in the field of neuroaesthetics has taught us a lot about where in the brain such aesthetic experiences are represented. Much less is known about when such experiences arise during the cortical processing cascade. Particularly, the dynamic neural representation of perceived attractiveness for rich natural scenes is not well understood. Here, I present data from an EEG experiment, in which participants provided attractiveness judgments for a set of diverse natural scenes. Using multivariate pattern analysis, I demonstrate that scene attractiveness is mirrored in early brain signals that arise within 200 msec of vision, suggesting that the aesthetic appeal of scenes is first resolved during perceptual processing. In more detailed analyses, I show that even such early neural correlates of scene attractiveness are partly related to interindividual variation in aesthetic preferences and that they generalize across scene contents. Together, these results characterize the time-resolved neural dynamics that give rise to aesthetic experiences in complex natural environments.
Collapse
|
5
|
Kaiser D, Jacobs AM, Cichy RM. Modelling brain representations of abstract concepts. PLoS Comput Biol 2022; 18:e1009837. [PMID: 35120139 PMCID: PMC8849470 DOI: 10.1371/journal.pcbi.1009837] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 02/16/2022] [Accepted: 01/14/2022] [Indexed: 11/18/2022] Open
Abstract
Abstract conceptual representations are critical for human cognition. Despite their importance, key properties of these representations remain poorly understood. Here, we used computational models of distributional semantics to predict multivariate fMRI activity patterns during the activation and contextualization of abstract concepts. We devised a task in which participants had to embed abstract nouns into a story that they developed around a given background context. We found that representations in inferior parietal cortex were predicted by concept similarities emerging in models of distributional semantics. By constructing different model families, we reveal the models’ learning trajectories and delineate how abstract and concrete training materials contribute to the formation of brain-like representations. These results inform theories about the format and emergence of abstract conceptual representations in the human brain. How do we conceive abstract concepts, like love, peace, or truth? In this study, we investigate how our brains support the activation and contextualization of such abstract concepts. We asked participants to embed abstract nouns into a coherent story while we recorded functional MRI. Using multivariate analysis techniques, we computed how similar different abstract concepts were represented during this task. We then modelled these neural similarities among concepts with computational models of distributional semantics which capture the words’ co-occurance statistics in large natural language corpora. Our results reveal a correspondence between the computational models and brain representations in the inferior parietal cortex. This correspondence held when the computational models were only trained on subsets of the corpora that contained as few as 100,000 sentences and only abstract or concrete words. Our findings establish a neural correlate of abstract concept representation in the inferior parietal cortex, and they provide a first characterization of the format of these representations.
Collapse
Affiliation(s)
- Daniel Kaiser
- Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus-Liebig-Universität Gießen, Gießen, Germany
- Center for Mind, Brain and Behavior (CMBB), Philipps-Universität Marburg and Justus-Liebig-Universität Gießen, Marburg, Germany
- Department of Psychology, University of York, York, United Kingdom
- * E-mail:
| | - Arthur M. Jacobs
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
- Center for Cognitive Neuroscience Berlin, Freie Universität Berlin, Berlin, Germany
| | - Radoslaw M. Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
- Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany
| |
Collapse
|
6
|
Ambrus GG, Eick CM, Kaiser D, Kovács G. Getting to Know You: Emerging Neural Representations during Face Familiarization. J Neurosci 2021; 41:5687-5698. [PMID: 34031162 PMCID: PMC8244976 DOI: 10.1523/jneurosci.2466-20.2021] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Revised: 02/22/2021] [Accepted: 04/05/2021] [Indexed: 11/21/2022] Open
Abstract
The successful recognition of familiar persons is critical for social interactions. Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. In three EEG experiments on human participants of both sexes, we elucidated how representations of face familiarity and identity emerge from different qualities of familiarization: brief perceptual exposure (Experiment 1), extensive media familiarization (Experiment 2), and real-life personal familiarization (Experiment 3). Time-resolved representational similarity analysis revealed that familiarization quality has a profound impact on representations of face familiarity: they were strongly visible after personal familiarization, weaker after media familiarization, and absent after perceptual familiarization. Across all experiments, we found no enhancement of face identity representation, suggesting that familiarity and identity representations emerge independently during face familiarization. Our results emphasize the importance of extensive, real-life familiarization for the emergence of robust face familiarity representations, constraining models of face perception and recognition memory.SIGNIFICANCE STATEMENT Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. To elucidate how face representations change as we get familiar with someone, we conducted three EEG experiments where we used brief perceptual exposure, extensive media familiarization, or real-life personal familiarization. Using multivariate representational similarity analysis, we demonstrate that the method of familiarization has a profound impact on face representations, and emphasize the importance of real-life familiarization. Additionally, familiarization shapes representations of face familiarity and identity differently: as we get to know someone, familiarity signals seem to appear before the formation of identity representations.
Collapse
Affiliation(s)
- Géza Gergely Ambrus
- Institute of Psychology, Friedrich Schiller University Jena, Leutragraben 1, D-07743 Jena, Germany
| | - Charlotta Marina Eick
- Institute of Psychology, Friedrich Schiller University Jena, Leutragraben 1, D-07743 Jena, Germany
| | - Daniel Kaiser
- Department of Psychology, University of York, Heslington, York, YO10 5DD, United Kingdom
| | - Gyula Kovács
- Institute of Psychology, Friedrich Schiller University Jena, Leutragraben 1, D-07743 Jena, Germany
| |
Collapse
|