1
|
Adámek P, Grygarová D, Jajcay L, Bakštein E, Fürstová P, Juríčková V, Jonáš J, Langová V, Neskoroďana I, Kesner L, Horáček J. The Gaze of Schizophrenia Patients Captured by Bottom-up Saliency. SCHIZOPHRENIA (HEIDELBERG, GERMANY) 2024; 10:21. [PMID: 38378724 PMCID: PMC10879495 DOI: 10.1038/s41537-024-00438-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Accepted: 01/19/2024] [Indexed: 02/22/2024]
Abstract
Schizophrenia (SCHZ) notably impacts various human perceptual modalities, including vision. Prior research has identified marked abnormalities in perceptual organization in SCHZ, predominantly attributed to deficits in bottom-up processing. Our study introduces a novel paradigm to differentiate the roles of top-down and bottom-up processes in visual perception in SCHZ. We analysed eye-tracking fixation ground truth maps from 28 SCHZ patients and 25 healthy controls (HC), comparing these with two mathematical models of visual saliency: one bottom-up, based on the physical attributes of images, and the other top-down, incorporating machine learning. While the bottom-up (GBVS) model revealed no significant overall differences between groups (beta = 0.01, p = 0.281, with a marginal increase in SCHZ patients), it did show enhanced performance by SCHZ patients with highly salient images. Conversely, the top-down (EML-Net) model indicated no general group difference (beta = -0.03, p = 0.206, lower in SCHZ patients) but highlighted significantly reduced performance in SCHZ patients for images depicting social interactions (beta = -0.06, p < 0.001). Over time, the disparity between the groups diminished for both models. The previously reported bottom-up bias in SCHZ patients was apparent only during the initial stages of visual exploration and corresponded with progressively shorter fixation durations in this group. Our research proposes an innovative approach to understanding early visual information processing in SCHZ patients, shedding light on the interplay between bottom-up perception and top-down cognition.
Collapse
Affiliation(s)
- Petr Adámek
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic.
- Third Faculty of Medicine, Charles University, Prague, Czech Republic.
| | - Dominika Grygarová
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Third Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Lucia Jajcay
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Institute of Computer Science of the Czech Academy of Sciences, Prague, Czech Republic
- Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic
| | - Eduard Bakštein
- Early Episodes of SMI Research Center, National Institute of Mental Health, Klecany, Czech Republic
- Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University, Prague, Czech Republic
| | - Petra Fürstová
- Early Episodes of SMI Research Center, National Institute of Mental Health, Klecany, Czech Republic
| | - Veronika Juríčková
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- First Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Juraj Jonáš
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Faculty of Humanities, Charles University, Prague, Czech Republic
| | - Veronika Langová
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Third Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Iryna Neskoroďana
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
| | - Ladislav Kesner
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Department of Art History, Masaryk University, Brno, Czech Republic
| | - Jiří Horáček
- Center for Advanced Studies of Brain and Consciousness, National Institute of Mental Health, Klecany, Czech Republic
- Third Faculty of Medicine, Charles University, Prague, Czech Republic
| |
Collapse
|
2
|
Chana K, Mikuni J, Schnebel A, Leder H. Reading in the city: mobile eye-tracking and evaluation of text in an everyday setting. Front Psychol 2023; 14:1205913. [PMID: 37928598 PMCID: PMC10622808 DOI: 10.3389/fpsyg.2023.1205913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 08/21/2023] [Indexed: 11/07/2023] Open
Abstract
Reading is often regarded as a mundane aspect of everyday life. However, little is known about the natural reading experiences in daily activities. To fill this gap, this study presents two field studies (N = 39 and 26, respectively), where we describe how people explore visual environments and divide their attention toward text elements in highly ecological settings, i.e., urban street environments, using mobile eye-tracking glasses. Further, the attention toward the text elements (i.e., shop signs) as well as their memorability, measured via follow-up recognition test, were analysed in relation to their aesthetic quality, which is assumed to be key for attracting visual attention and memorability. Our results revealed that, within these urban streets, text elements were looked at most, and looking behaviour was strongly directed, especially toward shop signs, across both street contexts; however, aesthetic values were not correlated either with the most looked at signs or the viewing time for the signs. Aesthetic ratings did however have an effect on memorability, with signs rated higher being better recognised. The results will be discussed in terms aesthetic reading experiences and implications for future field studies.
Collapse
Affiliation(s)
- Kirren Chana
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
- Department of Foreign Languages and Literatures, University of Verona, Verona, Italy
| | - Jan Mikuni
- Vienna Cognitive Science Hub, University of Vienna, Vienna, Austria
| | - Alina Schnebel
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
| | - Helmut Leder
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
- Vienna Cognitive Science Hub, University of Vienna, Vienna, Austria
| |
Collapse
|
3
|
Peacock CE, Singh P, Hayes TR, Rehrig G, Henderson JM. Searching for meaning: Local scene semantics guide attention during natural visual search in scenes. Q J Exp Psychol (Hove) 2023; 76:632-648. [PMID: 35510885 PMCID: PMC11132926 DOI: 10.1177/17470218221101334] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Models of visual search in scenes include image salience as a source of attentional guidance. However, because scene meaning is correlated with image salience, it could be that the salience predictor in these models is driven by meaning. To test this proposal, we generated meaning maps that represented the spatial distribution of semantic informativeness in scenes, and salience maps which represented the spatial distribution of conspicuous image features and tested their influence on fixation densities from two object search tasks in real-world scenes. The results showed that meaning accounted for significantly greater variance in fixation densities than image salience, both overall and in early attention across both studies. Here, meaning explained 58% and 63% of the theoretical ceiling of variance in attention across both studies, respectively. Furthermore, both studies demonstrated that fast initial saccades were not more likely to be directed to higher salience regions than slower initial saccades, and initial saccades of all latencies were directed to regions containing higher meaning than salience. Together, these results demonstrated that even though meaning was task-neutral, the visual system still selected meaningful over salient scene regions for attention during search.
Collapse
Affiliation(s)
- Candace E Peacock
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
- Department of Psychology, University of California, Davis, Davis, CA, USA
| | - Praveena Singh
- Center for Neuroscience, University of California, Davis, Davis, CA, USA
| | - Taylor R Hayes
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Gwendolyn Rehrig
- Department of Psychology, University of California, Davis, Davis, CA, USA
| | - John M Henderson
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
- Department of Psychology, University of California, Davis, Davis, CA, USA
| |
Collapse
|
4
|
Hayes TR, Henderson JM. Scene inversion reveals distinct patterns of attention to semantically interpreted and uninterpreted features. Cognition 2022; 229:105231. [DOI: 10.1016/j.cognition.2022.105231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Revised: 07/19/2022] [Accepted: 07/20/2022] [Indexed: 11/03/2022]
|
5
|
Modeling eye movement in dynamic interactive tasks for maximizing situation awareness based on Markov decision process. Sci Rep 2022; 12:13298. [PMID: 35918377 PMCID: PMC9346140 DOI: 10.1038/s41598-022-17433-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Accepted: 07/25/2022] [Indexed: 11/09/2022] Open
Abstract
For complex dynamic interactive tasks (such as aviating), operators need to continuously extract information from areas of interest (AOIs) through eye movement to maintain high level of situation awareness (SA), as failures of SA may cause task performance degradation, even system accident. Most of the current eye movement models focus on either static tasks (such as image viewing) or simple dynamic tasks (such as video watching), without considering SA. In this study, an eye movement model with the goal of maximizing SA is proposed based on Markov decision process (MDP), which is designed to describe the dynamic eye movement of experienced operators in dynamic interactive tasks. Two top-down factors, expectancy and value, are introduced into this model to represent the update probability and the importance of information in AOIs, respectively. In particular, the model regards sequence of eye fixations to different AOIs as sequential decisions to maximize the SA-related reward (value) in the context of uncertain information update (expectancy). Further, this model was validated with a flight simulation experiment. Results show that the predicted probabilities of fixation on and shift between AOIs are highly correlated (\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$R = 0.928$$\end{document}R=0.928 and \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$R = 0.951$$\end{document}R=0.951, respectively) with those of the experiment data.
Collapse
|