1
|
Gordon SM, Dalangin B, Touryan J. Saccade size predicts onset time of object processing during visual search of an open world virtual environment. Neuroimage 2024; 298:120781. [PMID: 39127183 DOI: 10.1016/j.neuroimage.2024.120781] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Revised: 08/02/2024] [Accepted: 08/08/2024] [Indexed: 08/12/2024] Open
Abstract
OBJECTIVE To date the vast majority of research in the visual neurosciences have been forced to adopt a highly constrained perspective of the vision system in which stimuli are processed in an open-loop reactive fashion (i.e., abrupt stimulus presentation followed by an evoked neural response). While such constraints enable high construct validity for neuroscientific investigation, the primary outcomes have been a reductionistic approach to isolate the component processes of visual perception. In electrophysiology, of the many neural processes studied under this rubric, the most well-known is, arguably, the P300 evoked response. There is, however, relatively little known about the real-world corollary of this component in free-viewing paradigms where visual stimuli are connected to neural function in a closed-loop. While growing evidence suggests that neural activity analogous to the P300 does occur in such paradigms, it is an open question when this response occurs and what behavioral or environmental factors could be used to isolate this component. APPROACH The current work uses convolutional networks to decode neural signals during a free-viewing visual search task in a closed-loop paradigm within an open-world virtual environment. From the decoded activity we construct fixation-locked response profiles that enable estimations of the variable latency of any P300 analogue around the moment of fixation. We then use these estimates to investigate which factors best reduce variable latency and, thus, predict the onset time of the response. We consider measurable, search-related factors encompassing top-down (i.e., goal driven) and bottom-up (i.e., stimulus driven) processes, such as fixation duration and salience. We also consider saccade size as an intermediate factor reflecting the integration of these two systems. MAIN RESULTS The results show that of these factors only saccade size reliably determines the onset time of P300 analogous activity for this task. Specifically, we find that for large saccades the variability in response onset is small enough to enable analysis using traditional ensemble averaging methods. SIGNIFICANCE The results show that P300 analogous activity does occur during closed-loop, free-viewing visual search while highlighting distinct differences between the open-loop version of this response and its real-world analogue. The results also further establish saccades, and saccade size, as a key factor in real-world visual processing.
Collapse
Affiliation(s)
| | | | - Jonathan Touryan
- DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD, USA
| |
Collapse
|
2
|
Das Chakladar D, Roy PP. Cognitive workload estimation using physiological measures: a review. Cogn Neurodyn 2024; 18:1445-1465. [PMID: 39104683 PMCID: PMC11297869 DOI: 10.1007/s11571-023-10051-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 11/14/2023] [Accepted: 11/28/2023] [Indexed: 08/07/2024] Open
Abstract
Estimating cognitive workload levels is an emerging research topic in the cognitive neuroscience domain, as participants' performance is highly influenced by cognitive overload or underload results. Different physiological measures such as Electroencephalography (EEG), Functional Magnetic Resonance Imaging, Functional near-infrared spectroscopy, respiratory activity, and eye activity are efficiently used to estimate workload levels with the help of machine learning or deep learning techniques. Some reviews focus only on EEG-based workload estimation using machine learning classifiers or multimodal fusion of different physiological measures for workload estimation. However, a detailed analysis of all physiological measures for estimating cognitive workload levels still needs to be discovered. Thus, this survey highlights the in-depth analysis of all the physiological measures for assessing cognitive workload. This survey emphasizes the basics of cognitive workload, open-access datasets, the experimental paradigm of cognitive tasks, and different measures for estimating workload levels. Lastly, we emphasize the significant findings from this review and identify the open challenges. In addition, we also specify future scopes for researchers to overcome those challenges.
Collapse
Affiliation(s)
- Debashis Das Chakladar
- Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, Roorkee, Uttarakhand India
| | - Partha Pratim Roy
- Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, Roorkee, Uttarakhand India
| |
Collapse
|
3
|
Del Campo VL, Morán JFO, Cagigal VM, Martín JM, Pagador JB, Hornero R. The use of the eye-fixation-related potential to investigate visual perception in professional domains with high attentional demand: a literature review. Neurol Sci 2024; 45:1849-1860. [PMID: 38157102 DOI: 10.1007/s10072-023-07275-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2023] [Accepted: 12/17/2023] [Indexed: 01/03/2024]
Abstract
INTRODUCTION Visual attention is a cognitive skill related to visual perception and neural activity, and also moderated by expertise, in time-constrained professional domains (e.g., aviation, driving, sport, surgery). However, the contribution of both perceptual and neural processes on performance has been studied separately in the literature. DEVELOPMENT We defend an integration of visual and neural signals to offer a more complete picture of the visual attention displayed by professionals of different skill levels when performing free-viewing tasks. Specifically, we propose to zoom the analysis in data related to the quiet eye and P300 component jointly, as a novel signal processing approach to evaluate professionals' visual attention. CONCLUSION This review highlights the advantages of using portable eye trackers and electroencephalogram systems altogether, as a promising technique for a better understanding of early cognitive components related to attentional processes. Altogether, the eye-fixation-related potentials method may provide a better understanding of the cognitive mechanisms employed by the participants in natural settings, revealing what visual information is of interest for participants and distinguishing the neural bases of visual attention between targets and non-targets whenever they perceive a stimulus during free viewing experiments.
Collapse
Affiliation(s)
- Vicente Luis Del Campo
- Laboratorio de Aprendizaje y Control Motor, Facultad de Ciencias del Deporte, Universidad de Extremadura, Avda. de La Universidad, S/N, 10003, Cáceres, Spain.
| | | | - Víctor Martínez Cagigal
- Grupo de Ingeniería Biomédica, Universidad de Valladolid, E.T.S.I. Telecomunicación, Paseo Belén 15, 47011, Valladolid, Spain
- Centro de Investigación Biomédica en Red - Bioingeniería, Biomateriales y Biomedicina (CIBER-BBN), E.T.S.I. Telecomunicación, Paseo Belén 15, 47011, Valladolid, Spain
| | - Jesús Morenas Martín
- Laboratorio de Aprendizaje y Control Motor, Facultad de Ciencias del Deporte, Universidad de Extremadura, Avda. de La Universidad, S/N, 10003, Cáceres, Spain
| | - J Blas Pagador
- Centro de Cirugía de Mínima Invasión Jesús Usón, Ctra. N-521, Km. 41,8, 10071, Cáceres, Spain
| | - Roberto Hornero
- Grupo de Ingeniería Biomédica, Universidad de Valladolid, E.T.S.I. Telecomunicación, Paseo Belén 15, 47011, Valladolid, Spain
- Centro de Investigación Biomédica en Red - Bioingeniería, Biomateriales y Biomedicina (CIBER-BBN), E.T.S.I. Telecomunicación, Paseo Belén 15, 47011, Valladolid, Spain
| |
Collapse
|
4
|
Gordon SM, McDaniel JR, King KW, Lawhern VJ, Touryan J. Decoding neural activity to assess individual latent state in ecologically valid contexts. J Neural Eng 2023; 20:046033. [PMID: 37552980 DOI: 10.1088/1741-2552/acee20] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 08/08/2023] [Indexed: 08/10/2023]
Abstract
Objective.Currently, there exists very few ways to isolate cognitive processes, historically defined via highly controlled laboratory studies, in more ecologically valid contexts. Specifically, it remains unclear as to what extent patterns of neural activity observed under such constraints actually manifest outside the laboratory in a manner that can be used to make accurate inferences about latent states, associated cognitive processes, or proximal behavior. Improving our understanding of when and how specific patterns of neural activity manifest in ecologically valid scenarios would provide validation for laboratory-based approaches that study similar neural phenomena in isolation and meaningful insight into the latent states that occur during complex tasks.Approach.Domain generalization methods, borrowed from the work of the brain-computer interface community, have the potential to capture high-dimensional patterns of neural activity in a way that can be reliably applied across experimental datasets in order to address this specific challenge. We previously used such an approach to decode phasic neural responses associated with visual target discrimination. Here, we extend that work to more tonic phenomena such as internal latent states. We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We apply the trained models to an ecologically valid paradigm in which participants performed multiple, concurrent driving-related tasks while perched atop a six-degrees-of-freedom ride-motion simulator.Main Results.Using the pretrained models, we estimate latent state and the associated patterns of neural activity. As the patterns of neural activity become more similar to those patterns observed in the training data, we find changes in behavior and task performance that are consistent with the observations from the original, laboratory-based paradigms.Significance.These results lend ecological validity to the original, highly controlled, experimental designs and provide a methodology for understanding the relationship between neural activity and behavior during complex tasks.
Collapse
Affiliation(s)
| | | | - Kevin W King
- DCS Corporation, Alexandria, VA, United States of America
| | - Vernon J Lawhern
- DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD, United States of America
| | - Jonathan Touryan
- DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD, United States of America
| |
Collapse
|
5
|
Understanding Clinical Reasoning through Visual Scanpath and Brain Activity Analysis. COMPUTATION 2022. [DOI: 10.3390/computation10080130] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
This paper presents an experimental study that analyzes learners’ visual behaviour and brain activity in clinical reasoning. An acquisition protocol was defined to record eye tracking and EEG data from 15 participants as they interact with a computer-based learning environment called Amnesia, a medical simulation system that assesses the analytical skills of novice medicine students while they solve patient cases. We use gaze data to assess learners’ visual focus and present our methodology to track learners’ reasoning process through scanpath pattern analysis. We also describe our methodology for examining learners’ cognitive states using mental engagement and workload neural indexes. Finally, we discuss the relationship between gaze path information and EEG and how our analyses can lead to new forms of clinical diagnostic reasoning assessment.
Collapse
|
6
|
Suslow T, Günther V, Hensch T, Kersting A, Bodenschatz CM. Alexithymia Is Associated With Deficits in Visual Search for Emotional Faces in Clinical Depression. Front Psychiatry 2021; 12:668019. [PMID: 34267686 PMCID: PMC8275928 DOI: 10.3389/fpsyt.2021.668019] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 06/03/2021] [Indexed: 11/13/2022] Open
Abstract
Background: The concept of alexithymia is characterized by difficulties identifying and describing one's emotions. Alexithymic individuals are impaired in the recognition of others' emotional facial expressions. Alexithymia is quite common in patients suffering from major depressive disorder. The face-in-the-crowd task is a visual search paradigm that assesses processing of multiple facial emotions. In the present eye-tracking study, the relationship between alexithymia and visual processing of facial emotions was examined in clinical depression. Materials and Methods: Gaze behavior and manual response times of 20 alexithymic and 19 non-alexithymic depressed patients were compared in a face-in-the-crowd task. Alexithymia was empirically measured via the 20-item Toronto Alexithymia-Scale. Angry, happy, and neutral facial expressions of different individuals were shown as target and distractor stimuli. Our analyses of gaze behavior focused on latency to the target face, number of distractor faces fixated before fixating the target, number of target fixations, and number of distractor faces fixated after fixating the target. Results: Alexithymic patients exhibited in general slower decision latencies compared to non-alexithymic patients in the face-in-the-crowd task. Patient groups did not differ in latency to target, number of target fixations, and number of distractors fixated prior to target fixation. However, after having looked at the target, alexithymic patients fixated more distractors than non-alexithymic patients, regardless of expression condition. Discussion: According to our results, alexithymia goes along with impairments in visual processing of multiple facial emotions in clinical depression. Alexithymia appears to be associated with delayed manual reaction times and prolonged scanning after the first target fixation in depression, but it might have no impact on the early search phase. The observed deficits could indicate difficulties in target identification and/or decision-making when processing multiple emotional facial expressions. Impairments of alexithymic depressed patients in processing emotions in crowds of faces seem not limited to a specific affective valence. In group situations, alexithymic depressed patients might be slowed in processing interindividual differences in emotional expressions compared with non-alexithymic depressed patients. This could represent a disadvantage in understanding non-verbal communication in groups.
Collapse
Affiliation(s)
- Thomas Suslow
- Department of Psychosomatic Medicine and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| | - Vivien Günther
- Department of Psychosomatic Medicine and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| | - Tilman Hensch
- Department of Psychiatry and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
- Department of Psychology, IU International University of Applied Science, Erfurt, Germany
| | - Anette Kersting
- Department of Psychosomatic Medicine and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| | - Charlott Maria Bodenschatz
- Department of Psychosomatic Medicine and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| |
Collapse
|
7
|
Sciaraffa N, Borghini G, Di Flumeri G, Cincotti F, Babiloni F, Aricò P. Joint Analysis of Eye Blinks and Brain Activity to Investigate Attentional Demand during a Visual Search Task. Brain Sci 2021; 11:brainsci11050562. [PMID: 33925209 PMCID: PMC8146019 DOI: 10.3390/brainsci11050562] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 04/23/2021] [Accepted: 04/27/2021] [Indexed: 12/26/2022] Open
Abstract
In several fields, the need for a joint analysis of brain activity and eye activity to investigate the association between brain mechanisms and manifest behavior has been felt. In this work, two levels of attentional demand, elicited through a conjunction search task, have been modelled in terms of eye blinks, brain activity, and brain network features. Moreover, the association between endogenous neural mechanisms underlying attentional demand and eye blinks, without imposing a time-locked structure to the analysis, has been investigated. The analysis revealed statistically significant spatial and spectral modulations of the recorded brain activity according to the different levels of attentional demand, and a significant reduction in the number of eye blinks when a higher amount of attentional investment was required. Besides, the integration of information coming from high-density electroencephalography (EEG), brain source localization, and connectivity estimation allowed us to merge spectral and causal information between brain areas, characterizing a comprehensive model of neurophysiological processes behind attentional demand. The analysis of the association between eye and brain-related parameters revealed a statistically significant high correlation (R > 0.7) of eye blink rate with anterofrontal brain activity at 8 Hz, centroparietal brain activity at 12 Hz, and a significant moderate correlation with the participation of right Intra Parietal Sulcus in alpha band (R = -0.62). Due to these findings, this work suggests the possibility of using eye blinks measured from one sensor placed on the forehead as an unobtrusive measure correlating with neural mechanisms underpinning attentional demand.
Collapse
Affiliation(s)
- Nicolina Sciaraffa
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- Correspondence:
| | - Gianluca Borghini
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy;
| | - Gianluca Di Flumeri
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
| | - Febo Cincotti
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy;
- Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Via Ariosto 25, 00185 Rome, Italy
| | - Fabio Babiloni
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310005, China
| | - Pietro Aricò
- Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy; (G.B.); (G.D.F.); (F.B.); (P.A.)
- BrainSigns srl, Lungotevere Michelangelo 9, 00192 Rome, Italy
| |
Collapse
|
8
|
Brouwer AM. Challenges and Opportunities in Consumer Neuroergonomics. FRONTIERS IN NEUROERGONOMICS 2021; 2:606646. [PMID: 38235238 PMCID: PMC10790888 DOI: 10.3389/fnrgo.2021.606646] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 02/08/2021] [Indexed: 01/19/2024]
Affiliation(s)
- Anne-Marie Brouwer
- TNO The Netherlands Organisation for Applied Scientific Research, Soesterberg, Netherlands
| |
Collapse
|
9
|
Debie E, Fernandez Rojas R, Fidock J, Barlow M, Kasmarik K, Anavatti S, Garratt M, Abbass HA. Multimodal Fusion for Objective Assessment of Cognitive Workload: A Review. IEEE TRANSACTIONS ON CYBERNETICS 2021; 51:1542-1555. [PMID: 31545761 DOI: 10.1109/tcyb.2019.2939399] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
Considerable progress has been made in improving the estimation accuracy of cognitive workload using various sensor technologies. However, the overall performance of different algorithms and methods remain suboptimal in real-world applications. Some studies in the literature demonstrate that a single modality is sufficient to estimate cognitive workload. These studies are limited to controlled settings, a scenario that is significantly different from the real world where data gets corrupted, interrupted, and delayed. In such situations, the use of multiple modalities is needed. Multimodal fusion approaches have been successful in other domains, such as wireless-sensor networks, in addressing single-sensor weaknesses and improving information quality/accuracy. These approaches are inherently more reliable when a data source is lost. In the cognitive workload literature, sensors, such as electroencephalography (EEG), electrocardiography (ECG), and eye tracking, have shown success in estimating the aspects of cognitive workload. Multimodal approaches that combine data from several sensors together can be more robust for real-time measurement of cognitive workload. In this article, we review the published studies related to multimodal data fusion to estimate the cognitive workload and synthesize their main findings. We identify the opportunities for designing better multimodal fusion systems for cognitive workload modeling.
Collapse
|
10
|
Eye Fixation-Related Potentials during Visual Search on Acquaintance and Newly-Learned Faces. Brain Sci 2021; 11:brainsci11020218. [PMID: 33579017 PMCID: PMC7916779 DOI: 10.3390/brainsci11020218] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Revised: 02/03/2021] [Accepted: 02/07/2021] [Indexed: 11/24/2022] Open
Abstract
Searching familiar faces in the crowd may involve stimulus-driven attention by emotional significance, together with goal-directed attention due to task-relevant needs. The present study investigated the effect of familiarity on attentional processes by exploring eye fixation-related potentials (EFRPs) and eye gazes when humans searched for, among other distracting faces, either an acquaintance’s face or a newly-learned face. Task performance and gaze behavior were indistinguishable for identifying either faces. However, from the EFRP analysis, after a P300 component for successful search of target faces, we found greater deflections of right parietal late positive potentials in response to newly-learned faces than acquaintance’s faces, indicating more involvement of goal-directed attention in processing newly-learned faces. In addition, we found greater occipital negativity elicited by acquaintance’s faces, reflecting emotional responses to significant stimuli. These results may suggest that finding a familiar face in the crowd would involve lower goal-directed attention and elicit more emotional responses.
Collapse
|
11
|
Ischebeck A, Hiebel H, Miller J, Höfler M, Gilchrist ID, Körner C. Target processing in overt serial visual search involves the dorsal attention network: A fixation-based event-related fMRI study. Neuropsychologia 2021; 153:107763. [PMID: 33493526 DOI: 10.1016/j.neuropsychologia.2021.107763] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Revised: 01/15/2021] [Accepted: 01/17/2021] [Indexed: 11/18/2022]
Abstract
In serial visual search we shift attention successively from location to location in search for the target. Although such search has been investigated using fMRI, overt attention (i.e., eye movements) was usually neglected or discouraged. As a result, it is unclear what happens in the instant when our gaze falls upon a target as compared to a distractor. In the present experiment, we used a multiple target search task that required eye movements and employed an analysis based on fixations as events of interest to investigate differences between target and distractor processing. Twenty young healthy adults indicated the number of targets (0-3) among distractors in a 20-item display. Compared to distractor fixations, we found that target fixations gave rise to wide-spread activation in the dorsal attention system, as well as in the visual cortex. Targets that were found later during the search activated the left inferior frontal gyrus and the left supramarginal gyrus more strongly than those that were found earlier. Finally, areas associated with visual and verbal working memory showed increased activation with a larger number of targets in the display.
Collapse
Affiliation(s)
- Anja Ischebeck
- Institute of Psychology, University of Graz, Austria; BioTechMed-Graz, Austria.
| | - Hannah Hiebel
- Institute of Psychology, University of Graz, Austria
| | - Joe Miller
- Institute of Psychology, University of Graz, Austria
| | - Margit Höfler
- Institute of Psychology, University of Graz, Austria; Department for Clinical Neurosciences and Preventive Medicine, Danube University Krems, Austria
| | | | - Christof Körner
- Institute of Psychology, University of Graz, Austria; BioTechMed-Graz, Austria
| |
Collapse
|
12
|
Wang S, Mamelak AN, Adolphs R, Rutishauser U. Abstract goal representation in visual search by neurons in the human pre-supplementary motor area. Brain 2019; 142:3530-3549. [PMID: 31549164 PMCID: PMC6821249 DOI: 10.1093/brain/awz279] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2019] [Revised: 06/14/2019] [Accepted: 07/11/2019] [Indexed: 01/18/2023] Open
Abstract
The medial frontal cortex is important for goal-directed behaviours such as visual search. The pre-supplementary motor area (pre-SMA) plays a critical role in linking higher-level goals to actions, but little is known about the responses of individual cells in this area in humans. Pre-SMA dysfunction is thought to be a critical factor in the cognitive deficits that are observed in diseases such as Parkinson's disease and schizophrenia, making it important to develop a better mechanistic understanding of the pre-SMA's role in cognition. We simultaneously recorded single neurons in the human pre-SMA and eye movements while subjects performed goal-directed visual search tasks. We characterized two groups of neurons in the pre-SMA. First, 40% of neurons changed their firing rate whenever a fixation landed on the search target. These neurons responded to targets in an abstract manner across several conditions and tasks. Responses were invariant to motor output (i.e. button press or not), and to different ways of defining the search target (by instruction or pop-out). Second, ∼50% of neurons changed their response as a function of fixation order. Together, our results show that human pre-SMA neurons carry abstract signals during visual search that indicate whether a goal was reached in an action- and cue-independent manner. This suggests that the pre-SMA contributes to goal-directed behaviour by flexibly signalling goal detection and time elapsed since start of the search, and this process occurs regardless of task. These observations provide insights into how pre-SMA dysfunction might impact cognitive function.
Collapse
Affiliation(s)
- Shuo Wang
- Department of Chemical and Biomedical Engineering, and Rockefeller Neuroscience Institute, West Virginia University, Morgantown, WV, USA
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ralph Adolphs
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Ueli Rutishauser
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| |
Collapse
|
13
|
Cornish L, Hill A, Horswill MS, Becker SI, Watson MO. Eye-tracking reveals how observation chart design features affect the detection of patient deterioration: An experimental study. APPLIED ERGONOMICS 2019; 75:230-242. [PMID: 30509531 DOI: 10.1016/j.apergo.2018.10.005] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/25/2017] [Revised: 10/09/2018] [Accepted: 10/14/2018] [Indexed: 06/09/2023]
Abstract
Particular design features intended to improve usability - including graphically displayed observations and integrated colour-based scoring-systems - have been shown to increase the speed and accuracy with which users of hospital observation charts detect abnormal patient observations. We used eye-tracking to evaluate two potential cognitive mechanisms underlying these effects. Novice chart-users completed a series of experimental trials in which they viewed patient data presented on one of three observation chart designs (varied within-subjects), and indicated which observation was abnormal (or that none were). A chart that incorporated both graphically displayed observations and an integrated colour-based scoring-system yielded faster, more accurate responses and fewer, shorter fixations than a graphical chart without a colour-based scoring-system. The latter, in turn, yielded the same advantages over a tabular chart (which incorporated neither design feature). These results suggest that both colour-based scoring-systems and graphically displayed observations improve search efficiency and reduce the cognitive resources required to process vital sign data.
Collapse
Affiliation(s)
- Lillian Cornish
- School of Psychology, The University of Queensland, St Lucia, Brisbane, Queensland, 4072, Australia
| | - Andrew Hill
- School of Psychology, The University of Queensland, St Lucia, Brisbane, Queensland, 4072, Australia; Clinical Skills Development Service, Metro North Hospital and Health Service, Herston, Brisbane, Queensland, 4006, Australia.
| | - Mark S Horswill
- School of Psychology, The University of Queensland, St Lucia, Brisbane, Queensland, 4072, Australia
| | - Stefanie I Becker
- School of Psychology, The University of Queensland, St Lucia, Brisbane, Queensland, 4072, Australia
| | - Marcus O Watson
- School of Psychology, The University of Queensland, St Lucia, Brisbane, Queensland, 4072, Australia; Clinical Skills Development Service, Metro North Hospital and Health Service, Herston, Brisbane, Queensland, 4006, Australia; School of Medicine, The University of Queensland, Herston, Brisbane, Queensland, 4006, Australia
| |
Collapse
|
14
|
Target probability modulates fixation-related potentials in visual search. Biol Psychol 2018; 138:199-210. [PMID: 30253233 DOI: 10.1016/j.biopsycho.2018.09.007] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2018] [Revised: 09/12/2018] [Accepted: 09/17/2018] [Indexed: 01/06/2023]
Abstract
This study investigated the influence of target probability on the neural response to target detection in free viewing visual search. Participants were asked to indicate the number of targets (one or two) among distractors in a visual search task while EEG and eye movements were co-registered. Target probability was manipulated by varying the set size of the displays between 10, 22, and 30 items. Fixation-related potentials time-locked to first target fixations revealed a pronounced P300 at the centro-parietal cortex with larger amplitudes for set sizes 22 and 30 than for set size 10. With increasing set size, more distractor fixations preceded the detection of the target, resulting in a decreased target probability and, consequently, a larger P300. For distractors, no increase of P300 amplitude with set size was observed. The findings suggest that set size specifically affects target but not distractor processing in overt serial visual search.
Collapse
|