1
|
Mandolesi S, Gambelli D, Naspetti S, Zanoli R. Exploring Visitors' Visual Behavior Using Eye-Tracking: The Case of the " Studiolo Del Duca". J Imaging 2022; 8:jimaging8010008. [PMID: 35049849 PMCID: PMC8781978 DOI: 10.3390/jimaging8010008] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Revised: 12/21/2021] [Accepted: 12/31/2021] [Indexed: 02/01/2023] Open
Abstract
Although the understanding of cognitive disciplines has progressed, we know relatively little about how the human brain perceives art. Thanks to the growing interest in visual perception, eye-tracking technology has been increasingly used for studying the interaction between individuals and artworks. In this study, eye-tracking was used to provide insights into non-expert visitors’ visual behaviour as they move freely in the historical room of the “Studiolo del Duca” of the Ducal Palace in Urbino, Italy. Visitors looked for an average of almost two minutes. This study revealed which parts of the artefact captured visitors’ attention and also gives interesting information about the main patterns of fruition.
Collapse
Affiliation(s)
- Serena Mandolesi
- Dipartimento di Scienze e Ingegneria della Materia, dell’Ambiente ed Urbanistica (SIMAU), Università Politecnica delle Marche, Via Brecce Bianche, 60131 Ancona, Italy; (S.M.); (S.N.)
| | - Danilo Gambelli
- Dipartimento di Scienze Agrarie, Alimentari e Ambientali (D3A), Università Politecnica delle Marche, Via Brecce Bianche, 60131 Ancona, Italy;
| | - Simona Naspetti
- Dipartimento di Scienze e Ingegneria della Materia, dell’Ambiente ed Urbanistica (SIMAU), Università Politecnica delle Marche, Via Brecce Bianche, 60131 Ancona, Italy; (S.M.); (S.N.)
| | - Raffaele Zanoli
- Dipartimento di Scienze Agrarie, Alimentari e Ambientali (D3A), Università Politecnica delle Marche, Via Brecce Bianche, 60131 Ancona, Italy;
- Correspondence: ; Tel.: +39-071-220-4929
| |
Collapse
|
2
|
Synchronization of acquisition devices in neuroimaging: An application using co-registration of eye movements and electroencephalography. Behav Res Methods 2021; 54:2545-2564. [PMID: 34918232 DOI: 10.3758/s13428-021-01756-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/22/2021] [Indexed: 11/08/2022]
Abstract
Interest in applications for the simultaneous acquisition of data from different devices is growing. In neuroscience for example, co-registration complements and overcomes some of the shortcomings of individual methods. However, precise synchronization of the different data streams involved is required before joint data analysis. Our article presents and evaluates a synchronization method which maximizes the alignment of information across time. Synchronization through common triggers is widely used in all existing methods, because it is very simple and effective. However, this solution has been found to fail in certain practical situations, namely for the spurious detection of triggers and/or when the timestamps of triggers sampled by each acquisition device are not jointly distributed linearly for the entire duration of an experiment. We propose two additional mechanisms, the "Longest Common Subsequence" algorithm and a piecewise linear regression, in order to overcome the limitations of the classical method of synchronizing common triggers. The proposed synchronization method was evaluated using both real and artificial data. Co-registrations of electroencephalographic signals (EEG) and eye movements were used for real data. We compared the effectiveness of our method to another open source method implemented using EYE-EEG toolbox. Overall, we show that our method, implemented in C++ as a DOS application, is very fast, robust and fully automatic.
Collapse
|
3
|
Sciaraffa N, Borghini G, Di Flumeri G, Cincotti F, Babiloni F, Aricò P. Joint Analysis of Eye Blinks and Brain Activity to Investigate Attentional Demand during a Visual Search Task. Brain Sci 2021; 11:brainsci11050562. [PMID: 33925209 PMCID: PMC8146019 DOI: 10.3390/brainsci11050562] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 04/23/2021] [Accepted: 04/27/2021] [Indexed: 12/26/2022] Open
Abstract
In several fields, the need for a joint analysis of brain activity and eye activity to investigate the association between brain mechanisms and manifest behavior has been felt. In this work, two levels of attentional demand, elicited through a conjunction search task, have been modelled in terms of eye blinks, brain activity, and brain network features. Moreover, the association between endogenous neural mechanisms underlying attentional demand and eye blinks, without imposing a time-locked structure to the analysis, has been investigated. The analysis revealed statistically significant spatial and spectral modulations of the recorded brain activity according to the different levels of attentional demand, and a significant reduction in the number of eye blinks when a higher amount of attentional investment was required. Besides, the integration of information coming from high-density electroencephalography (EEG), brain source localization, and connectivity estimation allowed us to merge spectral and causal information between brain areas, characterizing a comprehensive model of neurophysiological processes behind attentional demand. The analysis of the association between eye and brain-related parameters revealed a statistically significant high correlation (R > 0.7) of eye blink rate with anterofrontal brain activity at 8 Hz, centroparietal brain activity at 12 Hz, and a significant moderate correlation with the participation of right Intra Parietal Sulcus in alpha band (R = -0.62). Due to these findings, this work suggests the possibility of using eye blinks measured from one sensor placed on the forehead as an unobtrusive measure correlating with neural mechanisms underpinning attentional demand.
Collapse
Affiliation(s)
- Nicolina Sciaraffa
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- Correspondence:
| | - Gianluca Borghini
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy;
| | - Gianluca Di Flumeri
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
| | - Febo Cincotti
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy;
- Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Via Ariosto 25, 00185 Rome, Italy
| | - Fabio Babiloni
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310005, China
| | - Pietro Aricò
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
| |
Collapse
|
4
|
Anderson DE, Bhatt VR, Schmid K, Lunning M, Holstein SA, Rizzo M. Electrophysiological Measure of Impaired Information Processing in Drivers with Hematological Malignancy. TRANSPORTATION RESEARCH RECORD 2018; 2672:64-73. [PMID: 31031516 PMCID: PMC6482833 DOI: 10.1177/0361198118791666] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
The broad goal of this study is to measure remote effects of cancer on brain physiology and behaviors that underpin instrumental activities of daily living such as automobile driving. Studies of hematological malignancies (HM) have demonstrated impairments in multiple brain functions shown to be critical for safe automobile driving. In the current pilot study, brain physiology during driving simulation was examined in 14 HM patients and 13 healthy comparison drivers. Electroencephalography was used to measure the eye fixation-related potential (EFRP)-a positive amplitude deflection evoked approximately 100 milliseconds after eye movement termination. Previous studies have demonstrated sensitivity of EFRP activity to information-processing capacity. All drivers completed visual search tasks to evaluate the relationship between driving-related changes in performance and EFRP activity. Results showed smaller EFRP amplitudes in drivers who had: (1) greater driving-related changes in visual search performance (p = 0.03, Cohen's d = 0.91); and (2) HM diagnosis (p = 0.18, Cohen's d = 0.54). Extending previous studies, these results provide neural evidence of reduced information-processing capacity associated with cancer diagnosis. Future large-scale studies are needed to confirm these results, given the high level of uncertainty and small sample size. This study provides a novel platform for linking changes in brain physiology and safety-critical driving behaviors.
Collapse
Affiliation(s)
- David E. Anderson
- Department of Neurological Sciences, University of Nebraska Medical Center, Nebraska Medical Center, Omaha, NE
| | - Vijaya R. Bhatt
- Department of Internal Medicine, Division of Oncology & Hematology, Nebraska Medical Center, Omaha, NE
| | - Kendra Schmid
- Department of Biostatistics, University of Nebraska Medical Center, Nebraska Medical Center, Omaha, NE
| | - Matthew Lunning
- Department of Internal Medicine, Division of Oncology & Hematology, Nebraska Medical Center, Omaha, NE
| | - Sarah A. Holstein
- Department of Internal Medicine, Division of Oncology & Hematology, Nebraska Medical Center, Omaha, NE
| | - Matthew Rizzo
- Department of Neurological Sciences, University of Nebraska Medical Center, Nebraska Medical Center, Omaha, NE
| |
Collapse
|
5
|
Fudali-Czyż A, Francuz P, Augustynowicz P. The Effect of Art Expertise on Eye Fixation-Related Potentials During Aesthetic Judgment Task in Focal and Ambient Modes. Front Psychol 2018; 9:1972. [PMID: 30459676 PMCID: PMC6232682 DOI: 10.3389/fpsyg.2018.01972] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2018] [Accepted: 09/26/2018] [Indexed: 02/03/2023] Open
Abstract
This study aimed to determine the effect of expertise on the eye fixation-related potentials (EFRPs) during the aesthetic evaluation of images, independently in focal and ambient modes of visual processing. Focal and ambient modes were identified by averaging EFRP waveforms about the beginning of long eye fixations followed by short saccades and short fixations followed by long saccades, respectively. Thirty experts with formal training in visual arts and thirty-two non-experts freely viewed 150 figurative paintings presented for 20 s, each. After viewing the painting, the participant answered the question: “Is this painting beautiful?” Differences were found between the group of experts and non-experts due to the amplitude of EFRPs but only in focal mode, which is related to top-down, focused attention on the objects. Long fixations of experts had a higher amplitude of the parietal P2 recorded from right site than non-experts. In the group of experts, the frontal P2 was higher for long fixations on not beautiful paintings in comparison to long fixation on beautiful paintings. Moreover, in focal mode, there were higher occipital lambda response and N1-P2 complex for not beautiful than beautiful paintings. These results are discussed in the light of the results of studies on the effect of visual art expertise on event-related potentials (ERPs), ERP studies during aesthetic judgment task, and the knowledge of different modes of visual processing and EFRPs.
Collapse
Affiliation(s)
- Agnieszka Fudali-Czyż
- Department of Experimental Psychology, The John Paul II Catholic University of Lublin, Lublin, Poland
| | - Piotr Francuz
- Department of Experimental Psychology, The John Paul II Catholic University of Lublin, Lublin, Poland
| | - Paweł Augustynowicz
- Department of Experimental Psychology, The John Paul II Catholic University of Lublin, Lublin, Poland
| |
Collapse
|
6
|
Frey A, Lemaire B, Vercueil L, Guérin-Dugué A. An Eye Fixation-Related Potential Study in Two Reading Tasks: Reading to Memorize and Reading to Make a Decision. Brain Topogr 2018; 31:640-660. [DOI: 10.1007/s10548-018-0629-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2017] [Accepted: 01/05/2018] [Indexed: 01/13/2023]
|
7
|
Combining EEG and eye movement recording in free viewing: Pitfalls and possibilities. Brain Cogn 2016; 107:55-83. [DOI: 10.1016/j.bandc.2016.06.004] [Citation(s) in RCA: 71] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2015] [Revised: 06/14/2016] [Accepted: 06/17/2016] [Indexed: 11/19/2022]
|
8
|
Ušćumlić M, Blankertz B. Active visual search in non-stationary scenes: coping with temporal variability and uncertainty. J Neural Eng 2016; 13:016015. [PMID: 26726921 DOI: 10.1088/1741-2560/13/1/016015] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE State-of-the-art experiments for studying neural processes underlying visual cognition often constrain sensory inputs (e.g., static images) and our behavior (e.g., fixed eye-gaze, long eye fixations), isolating or simplifying the interaction of neural processes. Motivated by the non-stationarity of our natural visual environment, we investigated the electroencephalography (EEG) correlates of visual recognition while participants overtly performed visual search in non-stationary scenes. We hypothesized that visual effects (such as those typically used in human-computer interfaces) may increase temporal uncertainty (with reference to fixation onset) of cognition-related EEG activity in an active search task and therefore require novel techniques for single-trial detection. APPROACH We addressed fixation-related EEG activity in an active search task with respect to stimulus-appearance styles and dynamics. Alongside popping-up stimuli, our experimental study embraces two composite appearance styles based on fading-in, enlarging, and motion effects. Additionally, we explored whether the knowledge obtained in the pop-up experimental setting can be exploited to boost the EEG-based intention-decoding performance when facing transitional changes of visual content. MAIN RESULTS The results confirmed our initial hypothesis that the dynamic of visual content can increase temporal uncertainty of the cognition-related EEG activity in active search with respect to fixation onset. This temporal uncertainty challenges the pivotal aim to keep the decoding performance constant irrespective of visual effects. Importantly, the proposed approach for EEG decoding based on knowledge transfer between the different experimental settings gave a promising performance. SIGNIFICANCE Our study demonstrates that the non-stationarity of visual scenes is an important factor in the evolution of cognitive processes, as well as in the dynamic of ocular behavior (i.e., dwell time and fixation duration) in an active search task. In addition, our method to improve single-trial detection performance in this adverse scenario is an important step in making brain-computer interfacing technology available for human-computer interaction applications.
Collapse
Affiliation(s)
- Marija Ušćumlić
- Neurotechnology group, Technische Universität Berlin, Berlin, Germany
| | | |
Collapse
|