1
|
Chen S, Reichle ED, Liu Y. Direct lexical control of eye movements in Chinese reading: Evidence from the co-registration of EEG and eye tracking. Cogn Psychol 2024; 153:101683. [PMID: 39217858 DOI: 10.1016/j.cogpsych.2024.101683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Revised: 07/10/2024] [Accepted: 08/21/2024] [Indexed: 09/04/2024]
Abstract
The direct-lexical-control hypothesis stipulates that some aspect of a word's processing determines the duration of the fixation on that word and/or the next. Although the direct lexical control is incorporated into most current models of eye-movement control in reading, the precise implementation varies and the assumptions of the hypothesis may not be feasible given that lexical processing must occur rapidly enough to influence fixation durations. Conclusive empirical evidence supporting this hypothesis is therefore lacking. In this article, we report the results of an eye-tracking experiment using the boundary paradigm in which native speakers of Chinese read sentences in which target words were either high- or low-frequency and preceded by a valid or invalid preview. Eye movements were co-registered with electroencephalography, allowing standard analyses of eye-movement measures, divergence point analyses of fixation-duration distributions, and fixated-related potentials on the target words. These analyses collectively provide strong behavioral and neural evidence of early lexical processing and thus strong support for the direct-lexical-control hypothesis. We discuss the implications of the findings for our understanding of how the hypothesis might be implemented, the neural systems that support skilled reading, and the nature of eye-movement control in the reading of Chinese versus alphabetic scripts.
Collapse
Affiliation(s)
- Shuyuan Chen
- Department of Psychology, Sun Yat-sen University, China
| | - Erik D Reichle
- School of Psychological Sciences, Macquarie University, Australia
| | - Yanping Liu
- Department of Psychology, Sun Yat-sen University, China.
| |
Collapse
|
2
|
Levari T, Snedeker J. Understanding words in context: A naturalistic EEG study of children's lexical processing. JOURNAL OF MEMORY AND LANGUAGE 2024; 137:104512. [PMID: 38855737 PMCID: PMC11160963 DOI: 10.1016/j.jml.2024.104512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2024]
Abstract
When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.
Collapse
Affiliation(s)
- Tatyana Levari
- Department of Psychology, Harvard University, United States
| | - Jesse Snedeker
- Department of Psychology, Harvard University, United States
| |
Collapse
|
3
|
Mei X, Chen S, Xia X, Yang B, Liu Y. Neural correlates for word-frequency effect in Chinese natural reading. Atten Percept Psychophys 2024:10.3758/s13414-024-02894-7. [PMID: 38995494 DOI: 10.3758/s13414-024-02894-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/10/2024] [Indexed: 07/13/2024]
Abstract
Word frequency effect has always been of interest for reading research because of its critical role in exploring mental processing underlying reading behaviors. Access to word frequency information has long been considered an indicator of the beginning of lexical processing and the most sensitive marker for studying when the brain begins to extract semantic information Sereno & Rayner, Brain and Cognition, 42, 78-81, (2000), Trends in Cognitive Sciences, 7, 489-493, (2003). While the word frequency effect has been extensively studied in numerous eye-tracking and traditional EEG research using the RSVP paradigm, there is a lack of corresponding evidence in studies of natural reading. To find the neural correlates of the word frequency effect, we conducted a study of Chinese natural reading using EEG and eye-tracking coregistration to examine the time course of lexical processing. Our results reliably showed that the word frequency effect first appeared in the N200 time window and the bilateral occipitotemporal regions. Additionally, the word frequency effect was reflected in the N400 time window, spreading from the occipital region to the central parietal and frontal regions. Our current study provides the first neural correlates for word-frequency effect in natural Chinese reading so far, shedding new light on understanding lexical processing in natural reading and could serve as an important basis for further reading study when considering neural correlates in a realistic manner.
Collapse
Affiliation(s)
- Xiaolin Mei
- Department of Psychology, Sun Yat-sen University, No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China
- School of Optometry, The Hong Kong Polytechnic University, Hong Kong, China
| | - Shuyuan Chen
- Department of Psychology, Sun Yat-sen University, No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China
| | - Xinyi Xia
- Department of Psychology, Sun Yat-sen University, No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China
| | - Bo Yang
- Department of Psychology, Sun Yat-sen University, No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China
| | - Yanping Liu
- Department of Psychology, Sun Yat-sen University, No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China.
| |
Collapse
|
4
|
Melcher D, Alaberkyan A, Anastasaki C, Liu X, Deodato M, Marsicano G, Almeida D. An early effect of the parafoveal preview on post-saccadic processing of English words. Atten Percept Psychophys 2024:10.3758/s13414-024-02916-4. [PMID: 38956003 DOI: 10.3758/s13414-024-02916-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/05/2024] [Indexed: 07/04/2024]
Abstract
A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.
Collapse
Affiliation(s)
- David Melcher
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates.
- Center for Brain and Health, NYUAD Research Institute, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates.
| | - Ani Alaberkyan
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
| | - Chrysi Anastasaki
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
| | - Xiaoyi Liu
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
- Department of Psychology, Princeton University, Washington Rd, Princeton, NJ, 08540, USA
| | - Michele Deodato
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
- Center for Brain and Health, NYUAD Research Institute, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
| | - Gianluca Marsicano
- Department of Psychology, University of Bologna, Viale Berti Pichat 5, 40121, Bologna, Italy
- Centre for Studies and Research in Cognitive Neuroscience, University of Bologna, Via Rasi e Spinelli 176, 47023, Cesena, Italy
| | - Diogo Almeida
- Psychology Program, Division of Science, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, United Arab Emirates
| |
Collapse
|
5
|
Huang X, Wong BWL, Ng HTY, Sommer W, Dimigen O, Maurer U. Neural mechanism underlying preview effects and masked priming effects in visual word processing. Atten Percept Psychophys 2024:10.3758/s13414-024-02904-8. [PMID: 38956004 DOI: 10.3758/s13414-024-02904-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/07/2024] [Indexed: 07/04/2024]
Abstract
Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.
Collapse
Affiliation(s)
- Xin Huang
- Department of Psychology, The Chinese University of Hong Kong, Sino Building 3/F, Shatin, New Territories, Hong Kong, SAR, China
| | - Brian W L Wong
- Department of Psychology, The Chinese University of Hong Kong, Sino Building 3/F, Shatin, New Territories, Hong Kong, SAR, China
- BCBL, Basque Center on Brain, Language and Cognition, Donostia-San Sebastián, Spain
| | - Hezul Tin-Yan Ng
- Wofoo Joseph Lee Consulting and Counselling Psychology Research Centre, Lingnan University, Hong Kong, China
| | - Werner Sommer
- Institut für Psychologie, Humboldt-Universität zu Berlin, Berlin, Germany
- Department of Psychology, Zhejiang Normal University, Jin Hua, China
- Department of Physics and Life Science Imaging Centre, Hong Kong Baptist University, Hong Kong, China
| | - Olaf Dimigen
- Institut für Psychologie, Humboldt-Universität zu Berlin, Berlin, Germany.
- Department of Psychology, University of Groningen, Grote Kruisstraat 2-1, 9712 TS, Groningen, The Netherlands.
- The Research School of Behavioural and Cognitive Neurosciences, University of Groningen, Groningen, the Netherlands.
| | - Urs Maurer
- Department of Psychology, The Chinese University of Hong Kong, Sino Building 3/F, Shatin, New Territories, Hong Kong, SAR, China.
- Centre for Developmental Psychology, The Chinese University of Hong Kong, Hong Kong, China.
- Brain and Mind Institute, The Chinese University of Hong Kong, Hong Kong, China.
| |
Collapse
|
6
|
Li N, Wang S, Kornrumpf F, Sommer W, Dimigen O. Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials. Psychophysiology 2024; 61:e14524. [PMID: 38297818 DOI: 10.1111/psyp.14524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 12/20/2023] [Accepted: 12/26/2023] [Indexed: 02/02/2024]
Abstract
The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.
Collapse
Affiliation(s)
- Nan Li
- School of Foreign Studies, South China Normal University, Guangzhou, China
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, China
- Center for Language Cognition and Assessment, Guangzhou, China
| | - Suiping Wang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, China
| | - Florian Kornrumpf
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Werner Sommer
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- Department of Psychology, Zhejiang Normal University, Jin Hua, China
- Department of Physics and Life Sciences Imaging Center, Hong Kong Baptist University, Hong Kong, China
| | - Olaf Dimigen
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- Max-Planck-Institute for Human Development, Berlin, Germany
- University of Groningen, Groningen, The Netherlands
| |
Collapse
|
7
|
Huber-Huber C, Melcher D. Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study. Atten Percept Psychophys 2023:10.3758/s13414-023-02802-5. [PMID: 37917292 DOI: 10.3758/s13414-023-02802-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/27/2023] [Indexed: 11/04/2023]
Abstract
Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.
Collapse
Affiliation(s)
- Christoph Huber-Huber
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Corso Bettini 31, 38068, Rovereto, Italy.
| | - David Melcher
- Center for Brain & Health, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
- Psychology Program, Division of Science, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| |
Collapse
|
8
|
Harris AM, Eayrs JO, Lavie N. Establishing gaze markers of perceptual load during multi-target visual search. Cogn Res Princ Implic 2023; 8:56. [PMID: 37648839 PMCID: PMC10468466 DOI: 10.1186/s41235-023-00498-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 06/22/2023] [Indexed: 09/01/2023] Open
Abstract
Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.
Collapse
Affiliation(s)
- Anthony M Harris
- Institute of Cognitive Neuroscience, University College London, London, UK.
- Queensland Brain Institute, The University of Queensland, Brisbane, Australia.
| | - Joshua O Eayrs
- Institute of Cognitive Neuroscience, University College London, London, UK
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Nilli Lavie
- Institute of Cognitive Neuroscience, University College London, London, UK
| |
Collapse
|
9
|
Zhang W, Dong J, Li Y, Zhen A, Yan H. General cognitive processing for orthographic discrepancy engages foveal attention during sentence comprehension. Atten Percept Psychophys 2023; 85:2056-2064. [PMID: 37442894 DOI: 10.3758/s13414-023-02759-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/26/2023] [Indexed: 07/15/2023]
Abstract
Words are processed in the parafovea and fovea in succession during natural reading, but the classic rapid serial visual presentation (RSVP) paradigm presents words only in the fovea. Unlike the RSVP paradigm, the RSVP with flanker (RSVP-flanker) paradigm is similar to natural text reading. Previous studies using the RSVP-flanker paradigm have suggested that high-level semantic/syntactic integration engages foveal fixation after parafoveal semantic access. However, it is less clear how general cognitive processing, such as discrepancy monitoring and error correction, unfolds across the parafoveal and foveal visual fields. In the current study, Chinese sentences were presented with the RSVP-flanker paradigm. Critical words were manipulated so that they were expected (EXP), semantically violated (VIO), or orthographically similar (ORT). Because of the nuanced differences between the EXP and ORT conditions, the processing of ORT critical words required more general cognitive processing, such as discrepancy monitoring and error correction. In contrast, the processing of VIO words may require more semantic integration. The ERP results showed more positive foveal late positive component (LPC) in ORT versus VIO. Moreover, there was no parafoveal LPC effect, but a robust foveal LPC effect, in the contrasts of VIO/ORT versus EXP, replicating previous results. Together, the results provide substantial evidence that general cognitive processing for orthographic discrepancy occurs at the foveal perception.
Collapse
Affiliation(s)
- Wenjia Zhang
- Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, Xi'an, China
| | - Jie Dong
- Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, Xi'an, China
| | - Yongbin Li
- Department of Radiology, The First Hospital of Xi'an, Xi'an, 710002, Shaanxi, China
| | - Anna Zhen
- Department of Psychology, St. John's University, 8000 Utopia Parkway, Queens, NY, 11439, USA
| | - Hao Yan
- Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, Xi'an, China.
| |
Collapse
|
10
|
Snell J, Yeaton J, Mirault J, Grainger J. Parallel word reading revealed by fixation-related brain potentials. Cortex 2023; 162:1-11. [PMID: 36948090 DOI: 10.1016/j.cortex.2023.02.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Revised: 02/05/2022] [Accepted: 02/13/2023] [Indexed: 02/27/2023]
Abstract
During reading, the brain is confronted with many relevant objects at once. But does lexical processing occur for multiple words simultaneously? Cognitive science has yet to answer this prominent question. Recently it has been argued that the issue warrants supplementing the field's traditional toolbox (response times, eye-tracking) with neuroscientific techniques (EEG, fMRI). Indeed, according to the OB1-reader model, upcoming words need not impact oculomotor behavior per se, but parallel processing of these words must nonetheless be reflected in neural activity. Here we combined eye-tracking with EEG, time-locking the neural window of interest to the fixation on target words in sentence reading. During these fixations, we manipulated the identity of the subsequent word so that it posed either a syntactically legal or illegal continuation of the sentence. In line with previous research, oculomotor measures were unaffected. Yet, syntax impacted brain potentials as early as 100 ms after the target fixation onset. Given the EEG literature on syntax processing, the presently observed timings suggest parallel word reading. We reckon that parallel word processing typifies reading, and that OB1-reader offers a good platform for theorizing about the reading brain.
Collapse
Affiliation(s)
| | | | - Jonathan Mirault
- Aix-Marseille University & Centre Nationale de Recherche Scientifique, France
| | - Jonathan Grainger
- Aix-Marseille University & Centre Nationale de Recherche Scientifique, France; Institute of Language Communication and the Brain, Aix-Marseille University, France
| |
Collapse
|
11
|
Schubert E, Bode S. Positive emotions and their upregulation increase willingness to consume healthy foods. Appetite 2023; 181:106420. [PMID: 36513297 DOI: 10.1016/j.appet.2022.106420] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 12/07/2022] [Accepted: 12/08/2022] [Indexed: 12/14/2022]
Abstract
While highly relevant for everyday life, it is unclear whether experiencing incidental positive or negative emotional states, and active emotion regulation, influence the weighting of perceived taste and health in food choices. In Experiment 1, we examined two emotion regulation strategies, reappraisal and distraction, used to decrease negative emotions. Participants were cued to experience or decrease their emotional response for either neutral or negative incidental emotion-inducing images. They subsequently rated their willingness to consume foods, which varied in their taste and health attributes. Mixed-effects model analysis showed that compared to neutral, negative emotions decreased willingness to consume, regardless of perceived taste and health, but neither emotion regulation strategy had a significant effect. Experiment 2 used images inducing incidental positive emotions in combination with three emotion regulation strategies: reappraisal, distraction, and increasing positive emotions. Experiencing positive emotions generally increased willingness to consume, with stronger effects for tasty and healthy foods. Decreasing positive emotions via reappraisal decreased willingness to consume, particularly for healthy foods. Increasing positive emotion intensity further increased willingness to consume, with stronger effects for healthy foods. The results suggest that experiencing positive emotions increases desire particularly strongly for healthy foods, which can additionally be modulated via emotion regulation. This has important implications for designing health-related interventions targeting mood improvement.
Collapse
Affiliation(s)
- Elektra Schubert
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia
| | - Stefan Bode
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia.
| |
Collapse
|
12
|
Hollenstein N, Tröndle M, Plomecka M, Kiegeland S, Özyurt Y, Jäger LA, Langer N. The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data. Front Psychol 2023; 13:1028824. [PMID: 36710838 PMCID: PMC9878684 DOI: 10.3389/fpsyg.2022.1028824] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Accepted: 12/20/2022] [Indexed: 01/13/2023] Open
Abstract
We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.
Collapse
Affiliation(s)
- Nora Hollenstein
- Center for Language Technology, University of Copenhagen, Copenhagen, Denmark
| | - Marius Tröndle
- Department of Psychology, University of Zurich, Zurich, Switzerland
| | - Martyna Plomecka
- Department of Psychology, University of Zurich, Zurich, Switzerland
| | | | | | - Lena A. Jäger
- Department of Computational Linguistics, University of Zurich, Zurich, Switzerland
- Department of Computer Science, University of Potsdam, Potsdam, Germany
| | - Nicolas Langer
- Department of Psychology, University of Zurich, Zurich, Switzerland
| |
Collapse
|
13
|
Luo C, Qiao S, Zhuang X, Ma G. Dynamic attentional bias for pictorial and textual food cues in the visual search paradigm. Appetite 2023; 180:106318. [PMID: 36206971 DOI: 10.1016/j.appet.2022.106318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 08/24/2022] [Accepted: 09/22/2022] [Indexed: 11/07/2022]
Abstract
Previous studies have found that individuals have an attentional bias for food cues, which may be related to the energy level or the type of stimulus (e.g., pictorial or textual food cues) of the food cues. However, the available evidence is inconsistent, and there is no consensus about how the type of stimulus and food energy modulate food-related attentional bias. Searching for food is one of the most important daily behaviors. In this study, a modified visual search paradigm was used to explore the attentional bias for food cues, and eye movements were recorded. Food cues consisted of both food words and food pictures with different energy levels (i.e., high- and low-calorie foods). The results showed that there was an attentional avoidance in the early stage but a later-stage attentional approach for all food cues in the pictorial condition. This was especially true for high-calorie food pictures. Participants showed a later-stage conflicting attentional bias for foods with different energy levels in the textual condition. They showed an attentional approach to high-calorie food words but an attentional avoidance of low-calorie food words. These data show that food-related attentional bias varied along with different time courses, which was also modulated by the type of stimulus and food energy. These findings regarding dynamic attentional bias could be explained using the Goal Conflict Model of eating behavior.
Collapse
Affiliation(s)
- Changlin Luo
- Shaanxi Key Laboratory of Behavior and Cognitive Neuroscience, School of Psychology, Shaanxi Normal University, Xi'an, China
| | - Siyuan Qiao
- Shaanxi Key Laboratory of Behavior and Cognitive Neuroscience, School of Psychology, Shaanxi Normal University, Xi'an, China
| | - Xiangling Zhuang
- Shaanxi Key Laboratory of Behavior and Cognitive Neuroscience, School of Psychology, Shaanxi Normal University, Xi'an, China
| | - Guojie Ma
- Shaanxi Key Laboratory of Behavior and Cognitive Neuroscience, School of Psychology, Shaanxi Normal University, Xi'an, China.
| |
Collapse
|
14
|
Pan Y, Popov T, Frisson S, Jensen O. Saccades are locked to the phase of alpha oscillations during natural reading. PLoS Biol 2023; 21:e3001968. [PMID: 36649331 PMCID: PMC9882905 DOI: 10.1371/journal.pbio.3001968] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 01/27/2023] [Accepted: 12/16/2022] [Indexed: 01/18/2023] Open
Abstract
We saccade 3 to 5 times per second when reading. However, little is known about the neuronal mechanisms coordinating the oculomotor and visual systems during such rapid processing. Here, we ask if brain oscillations play a role in the temporal coordination of the visuomotor integration. We simultaneously acquired MEG and eye-tracking data while participants read sentences silently. Every sentence was embedded with a target word of either high or low lexical frequency. Our key finding demonstrated that saccade onsets were locked to the phase of alpha oscillations (8 to 13 Hz), and in particular, for saccades towards low frequency words. Source modelling demonstrated that the alpha oscillations to which the saccades were locked, were generated in the right-visual motor cortex (BA 7). Our findings suggest that the alpha oscillations serve to time the processing between the oculomotor and visual systems during natural reading, and that this coordination becomes more pronounced for demanding words.
Collapse
Affiliation(s)
- Yali Pan
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
| | - Tzvetan Popov
- Methods of Plasticity Research, Department of Psychology, University of Zurich, Zurich, Switzerland
| | - Steven Frisson
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
| | - Ole Jensen
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
| |
Collapse
|
15
|
Wu JY, Ching CTS, Wang HMD, Liao LD. Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications. BIOSENSORS 2022; 12:1097. [PMID: 36551064 PMCID: PMC9776100 DOI: 10.3390/bios12121097] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 11/15/2022] [Indexed: 06/17/2023]
Abstract
Wearable devices are being developed faster and applied more widely. Wearables have been used to monitor movement-related physiological indices, including heartbeat, movement, and other exercise metrics, for health purposes. People are also paying more attention to mental health issues, such as stress management. Wearable devices can be used to monitor emotional status and provide preliminary diagnoses and guided training functions. The nervous system responds to stress, which directly affects eye movements and sweat secretion. Therefore, the changes in brain potential, eye potential, and cortisol content in sweat could be used to interpret emotional changes, fatigue levels, and physiological and psychological stress. To better assess users, stress-sensing devices can be integrated with applications to improve cognitive function, attention, sports performance, learning ability, and stress release. These application-related wearables can be used in medical diagnosis and treatment, such as for attention-deficit hyperactivity disorder (ADHD), traumatic stress syndrome, and insomnia, thus facilitating precision medicine. However, many factors contribute to data errors and incorrect assessments, including the various wearable devices, sensor types, data reception methods, data processing accuracy and algorithms, application reliability and validity, and actual user actions. Therefore, in the future, medical platforms for wearable devices and applications should be developed, and product implementations should be evaluated clinically to confirm product accuracy and perform reliable research.
Collapse
Affiliation(s)
- Ju-Yu Wu
- Institute of Biomedical Engineering and Nanomedicine, National Health Research Institutes, Zhunan Township, Miaoli County 35053, Taiwan
- Program in Tissue Engineering and Regenerative Medicine, National Chung Hsing University, South District, Taichung City 402, Taiwan
| | - Congo Tak-Shing Ching
- Graduate Institute of Biomedical Engineering, National Chung Hsing University, South District, Taichung City 402, Taiwan
- Department of Electrical Engineering, National Chi Nan University, No. 1 University Road, Puli Township, Nantou County 545301, Taiwan
| | - Hui-Min David Wang
- Program in Tissue Engineering and Regenerative Medicine, National Chung Hsing University, South District, Taichung City 402, Taiwan
- Graduate Institute of Biomedical Engineering, National Chung Hsing University, South District, Taichung City 402, Taiwan
| | - Lun-De Liao
- Institute of Biomedical Engineering and Nanomedicine, National Health Research Institutes, Zhunan Township, Miaoli County 35053, Taiwan
- Program in Tissue Engineering and Regenerative Medicine, National Chung Hsing University, South District, Taichung City 402, Taiwan
| |
Collapse
|
16
|
Cui X, Richlan F, Zhou W. Fixation-related fMRI analysis reveals the neural basis of parafoveal processing in self-paced reading of Chinese words. Brain Struct Funct 2022; 227:2609-2621. [PMID: 35997831 DOI: 10.1007/s00429-022-02552-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2022] [Accepted: 08/10/2022] [Indexed: 11/27/2022]
Abstract
While parafoveal word processing plays an important role in natural reading, the underlying neural mechanism remains unclear. The present study investigated the neural basis of parafoveal processing during Chinese word reading with the co-registration of eye-tracking and functional magnetic resonance imaging (fMRI) using fixation-related fMRI analysis. In the gaze-contingent boundary paradigm, preview conditions (words that are identical, orthographically similar, and unrelated to target words), pre-target word frequency and target word frequency were manipulated. When fixating the pre-target word, the identical preview condition elicited lower brain activation in the left fusiform gyrus relative to unrelated and orthographically similar preview conditions and there were significant interactions of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus, left fusiform gyrus and supplementary motor area. When fixating the target word, there was a significant main effect of preview condition on brain activation of the right fusiform gyrus and a significant interaction of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus. These results suggest that fixation-related brain activation provides immediate measures and new perspectives to understand the mechanism of parafoveal processing in self-paced reading.
Collapse
Affiliation(s)
- Xiaohui Cui
- Beijing Key Lab of Learning and Cognition, School of Psychology, Capital Normal University, Beijing, China.,Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, China.,Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Fabio Richlan
- Centre for Cognitive Neuroscience and Department of Psychology, Paris-Lodron-University of Salzburg, Salzburg, Austria
| | - Wei Zhou
- Beijing Key Lab of Learning and Cognition, School of Psychology, Capital Normal University, Beijing, China.
| |
Collapse
|
17
|
Li N, Dimigen O, Sommer W, Wang S. Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task. Psychophysiology 2022; 59:e14053. [PMID: 35512086 DOI: 10.1111/psyp.14053] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Revised: 02/27/2022] [Accepted: 03/02/2022] [Indexed: 11/29/2022]
Abstract
During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.
Collapse
Affiliation(s)
- Nan Li
- School of Foreign Studies, South China Normal University, Guangzhou, China.,Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, China.,Center for Language Cognition and Assessment, Guangzhou, China
| | - Olaf Dimigen
- Department of Psychology, Humboldt University at Berlin, Berlin, Germany
| | - Werner Sommer
- Department of Psychology, Humboldt University at Berlin, Berlin, Germany
| | - Suiping Wang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, China
| |
Collapse
|
18
|
Parafoveal processing of repeated words during reading. Psychon Bull Rev 2022; 29:1451-1460. [PMID: 35217984 PMCID: PMC9436839 DOI: 10.3758/s13423-021-02054-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/29/2021] [Indexed: 12/04/2022]
Abstract
In an eye-tracking experiment during reading, we examined the repetition effect, whereby words that are repeated in the same paragraph receive shorter fixation durations. Target words that were either high-frequency or low-frequency words and of which the parafoveal preview was either correct or with all letters replaced were embedded three times in the same paragraph. Shorter fixation times and higher skipping rates were observed for high-frequency compared to low-frequency words, words for which the parafoveal preview was correct versus incorrect, and as the word was being repeated more often. An interaction between frequency and repetition indicated that the reduction in fixation times due to repetition was more pronounced for low-frequency words. We also observed influences of word repetition on parafoveal processing, as repeated words were skipped more often. An interaction between parafoveal preview and repetition indicated an absent repetition effect when the preview was incorrect, but this effect was short lived, as it was restricted to the first fixation duration on the target word.
Collapse
|
19
|
Antúnez M, Milligan S, Hernández-Cabrera JA, Barber HA, Schotter ER. Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements. Psychophysiology 2021; 59:e13986. [PMID: 34942021 DOI: 10.1111/psyp.13986] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 10/02/2021] [Accepted: 11/30/2021] [Indexed: 11/29/2022]
Abstract
Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.
Collapse
Affiliation(s)
- Martín Antúnez
- Cognitive Psychology Department, Universidad de La Laguna, Santa Cruz de Tenerife, Spain
| | - Sara Milligan
- Department of Psychology, University of South Florida, Tampa, Florida, USA
| | | | - Horacio A Barber
- Cognitive Psychology Department, Universidad de La Laguna, Santa Cruz de Tenerife, Spain.,Institute for Biomedical Technologies (ITB), Universidad de La Laguna, Santa Cruz de Tenerife, Spain.,Basque Center on Cognition, Brain and Language (BCBL), San Sebastián, Spain
| | | |
Collapse
|
20
|
Lörch L. The association of eye movements and performance accuracy in a novel sight-reading task. J Eye Mov Res 2021; 14. [PMID: 34754405 PMCID: PMC8573852 DOI: 10.16910/jemr.14.4.5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
The present study investigated how eye movements were associated with performance
accuracy during sight-reading. Participants performed a complex span task in which
sequences of single quarter note symbols that either enabled chunking or did not enable
chunking were presented for subsequent serial recall. In between the presentation of each
note, participants sight-read a notated melody on an electric piano in the tempo of 70 bpm.
All melodies were unique but contained four types of note pairs: eighth-eighth, eighthquarter,
quarter-eighth, quarter-quarter. Analyses revealed that reading with fewer
fixations was associated with a more accurate note onset. Fewer fixations might be
advantageous for sight-reading as fewer saccades have to be planned and less information
has to be integrated. Moreover, the quarter-quarter note pair was read with a larger number
of fixations and the eighth-quarter note pair was read with a longer gaze duration. This
suggests that when rhythm is processed, additional beats might trigger re-fixations and
unconventional rhythmical patterns might trigger longer gazes. Neither recall accuracy nor
chunking processes were found to explain additional variance in the eye movement data.
Collapse
|
21
|
Pan Y, Frisson S, Jensen O. Neural evidence for lexical parafoveal processing. Nat Commun 2021; 12:5234. [PMID: 34475391 PMCID: PMC8413448 DOI: 10.1038/s41467-021-25571-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 08/17/2021] [Indexed: 11/08/2022] Open
Abstract
In spite of the reduced visual acuity, parafoveal information plays an important role in natural reading. However, competing models on reading disagree on whether words are previewed parafoveally at the lexical level. We find neural evidence for lexical parafoveal processing by combining a rapid invisible frequency tagging (RIFT) approach with magnetoencephalography (MEG) and eye-tracking. In a silent reading task, target words are tagged (flickered) subliminally at 60 Hz. The tagging responses measured when fixating on the pre-target word reflect parafoveal processing of the target word. We observe stronger tagging responses during pre-target fixations when followed by low compared with high lexical frequency targets. Moreover, this lexical parafoveal processing is associated with individual reading speed. Our findings suggest that reading unfolds in the fovea and parafovea simultaneously to support fluent reading.
Collapse
Affiliation(s)
- Yali Pan
- Centre for Human Brain Health, University of Birmingham, Birmingham, UK.
- School of Psychology, University of Birmingham, Birmingham, UK.
| | - Steven Frisson
- Centre for Human Brain Health, University of Birmingham, Birmingham, UK
- School of Psychology, University of Birmingham, Birmingham, UK
| | - Ole Jensen
- Centre for Human Brain Health, University of Birmingham, Birmingham, UK
- School of Psychology, University of Birmingham, Birmingham, UK
| |
Collapse
|
22
|
Huber-Huber C, Buonocore A, Melcher D. The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception. J Vis 2021; 21:12. [PMID: 34283203 PMCID: PMC8300052 DOI: 10.1167/jov.21.7.12] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Accepted: 05/18/2021] [Indexed: 01/02/2023] Open
Abstract
A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.
Collapse
Affiliation(s)
- Christoph Huber-Huber
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, The Netherlands
- CIMeC, University of Trento, Italy
| | - Antimo Buonocore
- Werner Reichardt Centre for Integrative Neuroscience, Tübingen University, Tübingen, BW, Germany
- Hertie Institute for Clinical Brain Research, Tübingen University, Tübingen, BW, Germany
| | - David Melcher
- CIMeC, University of Trento, Italy
- Division of Science, New York University Abu Dhabi, UAE
| |
Collapse
|
23
|
Franzen L, Stark Z, Johnson AP. Individuals with dyslexia use a different visual sampling strategy to read text. Sci Rep 2021; 11:6449. [PMID: 33742007 PMCID: PMC7979812 DOI: 10.1038/s41598-021-84945-9] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2020] [Accepted: 02/18/2021] [Indexed: 11/09/2022] Open
Abstract
Individuals with dyslexia present with reading-related deficits including inaccurate and/or less fluent word recognition and poor decoding abilities. Slow reading speed and worse text comprehension can occur as secondary consequences of these deficits. Reports of visual symptoms such as atypical eye movements during reading gave rise to a search for these deficits' underlying mechanisms. This study sought to replicate established behavioral deficits in reading and cognitive processing speed while investigating their underlying mechanisms in more detail by developing a comprehensive profile of eye movements specific to reading in adult dyslexia. Using a validated standardized reading assessment, our findings confirm a reading speed deficit among adults with dyslexia. We observed different eye movements in readers with dyslexia across numerous eye movement metrics including the duration of a stop (i.e., fixation), the length of jumps (i.e., saccades), and the number of times a reader's eyes expressed a jump atypical for reading. We conclude that individuals with dyslexia visually sample written information in a laborious and more effortful manner that is fundamentally different from those without dyslexia. Our findings suggest a mix of aberrant cognitive linguistic and oculomotor processes being present in adults with dyslexia.
Collapse
Affiliation(s)
- Léon Franzen
- Department of Psychology, Concordia University, Montréal, Canada.
- Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, UK.
| | - Zoey Stark
- Department of Psychology, Concordia University, Montréal, Canada
| | - Aaron P Johnson
- Department of Psychology, Concordia University, Montréal, Canada
- CRIR/Lethbridge-Layton-Mackay Centre de Réadaptation du CIUSSS du Centre-Ouest-de-l'Île-de-Montréal, Montréal, Canada
- Réseau de Recherche en Santé de La Vision, Montréal, Canada
| |
Collapse
|
24
|
Dimigen O, Ehinger BV. Regression-based analysis of combined EEG and eye-tracking data: Theory and applications. J Vis 2021; 21:3. [PMID: 33410892 PMCID: PMC7804566 DOI: 10.1167/jov.21.1.3] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2019] [Accepted: 08/14/2020] [Indexed: 12/27/2022] Open
Abstract
Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.
Collapse
Affiliation(s)
- Olaf Dimigen
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Benedikt V Ehinger
- Institute of Cognitive Science, Universität Osnabrück, Osnabrück, Germany
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
25
|
Tan G, Xu K, Liu J, Liu H. A Trend on Autism Spectrum Disorder Research: Eye Tracking-EEG Correlative Analytics. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3102646] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
26
|
Vaughan-Evans A, Liversedge SP, Fitzsimmons G, Jones MW. Syntactic co-activation in natural reading. VISUAL COGNITION 2020. [DOI: 10.1080/13506285.2020.1841866] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
| | - Simon P. Liversedge
- School of Psychology and Computing, University of Central Lancashire, Preston, England, UK
| | | | - Manon W. Jones
- School of Psychology, Bangor University, Bangor, Wales, UK
| |
Collapse
|
27
|
Roelke A, Vorstius C, Radach R, Hofmann MJ. Fixation-related NIRS indexes retinotopic occipital processing of parafoveal preview during natural reading. Neuroimage 2020; 215:116823. [PMID: 32289457 DOI: 10.1016/j.neuroimage.2020.116823] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 03/23/2020] [Accepted: 03/28/2020] [Indexed: 11/19/2022] Open
Abstract
While word frequency and predictability effects have been examined extensively, any evidence on interactive effects as well as parafoveal influences during whole sentence reading remains inconsistent and elusive. Novel neuroimaging methods utilize eye movement data to account for the hemodynamic responses of very short events such as fixations during natural reading. In this study, we used the rapid sampling frequency of near-infrared spectroscopy (NIRS) to investigate neural responses in the occipital and orbitofrontal cortex to word frequency and predictability. We observed increased activation in the right ventral occipital cortex when the fixated word N was of low frequency, which we attribute to an enhanced cost during saccade planning. Importantly, unpredictable (in contrast to predictable) low frequency words increased the activity in the left dorsal occipital cortex at the fixation of the preceding word N-1, presumably due to an upcoming breach of top-down modulated expectation. Opposite to studies that utilized a serial presentation of words (e.g. Hofmann et al., 2014), we did not find such an interaction in the orbitofrontal cortex, implying that top-down timing of cognitive subprocesses is not required during natural reading. We discuss the implications of an interactive parafoveal-on-foveal effect for current models of eye movements.
Collapse
Affiliation(s)
- Andre Roelke
- General and Biological Psychology, University of Wuppertal, Max-Horkheimer-Str. 20, D-42119, Wuppertal, Germany.
| | - Christian Vorstius
- General and Biological Psychology, University of Wuppertal, Max-Horkheimer-Str. 20, D-42119, Wuppertal, Germany
| | - Ralph Radach
- General and Biological Psychology, University of Wuppertal, Max-Horkheimer-Str. 20, D-42119, Wuppertal, Germany
| | - Markus J Hofmann
- General and Biological Psychology, University of Wuppertal, Max-Horkheimer-Str. 20, D-42119, Wuppertal, Germany
| |
Collapse
|
28
|
Carter BT, Luke SG. Best practices in eye tracking research. Int J Psychophysiol 2020; 155:49-62. [PMID: 32504653 DOI: 10.1016/j.ijpsycho.2020.05.010] [Citation(s) in RCA: 81] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2019] [Revised: 05/26/2020] [Accepted: 05/27/2020] [Indexed: 12/14/2022]
Abstract
This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.
Collapse
|
29
|
Mirault J, Yeaton J, Broqua F, Dufau S, Holcomb PJ, Grainger J. Parafoveal-on-foveal repetition effects in sentence reading: A co-registered eye-tracking and electroencephalogram study. Psychophysiology 2020; 57:e13553. [PMID: 32091627 PMCID: PMC7507185 DOI: 10.1111/psyp.13553] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2019] [Revised: 01/27/2020] [Accepted: 02/01/2020] [Indexed: 11/29/2022]
Abstract
When reading, can the next word in the sentence (word n + 1) influence how you read the word you are currently looking at (word n)? Serial models of sentence reading state that this generally should not be the case, whereas parallel models predict that this should be the case. Here we focus on perhaps the simplest and the strongest Parafoveal‐on‐Foveal (PoF) manipulation: word n + 1 is either the same as word n or a different word. Participants read sentences for comprehension and when their eyes left word n, the repeated or unrelated word at position n + 1 was swapped for a word that provided a syntactically correct continuation of the sentence. We recorded electroencephalogram and eye‐movements, and time‐locked the analysis of fixation‐related potentials (FRPs) to fixation of word n. We found robust PoF repetition effects on gaze durations on word n, and also on the initial landing position on word n. Most important is that we also observed significant effects in FRPs, reaching significance at 260 ms post‐fixation of word n. Repetition of the target word n at position n + 1 caused a widely distributed reduced negativity in the FRPs. Given the timing of this effect, we argue that it is driven by orthographic processing of word n + 1, while readers were still looking at word n, plus the spatial integration of orthographic information extracted from these two words in parallel. A hotly debated issue in reading research concerns the serial versus parallel nature of word identification processes. Evidence in favor of parallel processing has been criticized because it has often been obtained with artificial reading paradigms. Here we show that fixation‐related potentials elicited by a target word embedded in a normal sentence are impacted by the nature of the word immediately to the right of the fixated target word (n), and that this impact became significant 260 ms after fixation of word n. This is in line with parallel orthographic processing across adjacent words during sentence reading.
Collapse
Affiliation(s)
- Jonathan Mirault
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France
| | - Jeremy Yeaton
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France
| | - Fanny Broqua
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France
| | - Stéphane Dufau
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France.,Institute for Language, Aix-Marseille University, Aix-en-Provence, France
| | - Phillip J Holcomb
- Department of Psychology, San Diego State University, San Diego, CA, USA
| | - Jonathan Grainger
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France.,Institute for Language, Aix-Marseille University, Aix-en-Provence, France
| |
Collapse
|
30
|
Degno F, Liversedge SP. Eye Movements and Fixation-Related Potentials in Reading: A Review. Vision (Basel) 2020; 4:E11. [PMID: 32028566 PMCID: PMC7157570 DOI: 10.3390/vision4010011] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2019] [Revised: 01/27/2020] [Accepted: 01/28/2020] [Indexed: 11/19/2022] Open
Abstract
The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.
Collapse
Affiliation(s)
- Federica Degno
- School of Psychology, University of Central Lancashire, Marsh Ln, Preston PR1 2HE, UK;
| | | |
Collapse
|
31
|
A co-registration investigation of inter-word spacing and parafoveal preview: Eye movements and fixation-related potentials. PLoS One 2019; 14:e0225819. [PMID: 31851679 PMCID: PMC6919607 DOI: 10.1371/journal.pone.0225819] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2019] [Accepted: 11/13/2019] [Indexed: 11/20/2022] Open
Abstract
Participants' eye movements (EMs) and EEG signal were simultaneously recorded to examine foveal and parafoveal processing during sentence reading. All the words in the sentence were manipulated for inter-word spacing (intact spaces vs. spaces replaced by a random letter) and parafoveal preview (identical preview vs. random letter string preview). We observed disruption for unspaced text and invalid preview conditions in both EMs and fixation-related potentials (FRPs). Unspaced and invalid preview conditions received longer reading times than spaced and valid preview conditions. In addition, the FRP data showed that unspaced previews disrupted reading in earlier time windows of analysis, compared to string preview conditions. Moreover, the effect of parafoveal preview was greater for spaced relative to unspaced conditions, in both EMs and FRPs. These findings replicate well-established preview effects, provide novel insight into the neural correlates of reading with and without inter-word spacing and suggest that spatial selection precedes lexical processing.
Collapse
|
32
|
de Lissa P, McArthur G, Hawelka S, Palermo R, Mahajan Y, Degno F, Hutzler F. Peripheral preview abolishes N170 face-sensitivity at fixation: Using fixation-related potentials to investigate dynamic face processing. VISUAL COGNITION 2019. [DOI: 10.1080/13506285.2019.1676855] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Peter de Lissa
- iBMLab, Department of Psychology, University of Fribourg, Fribourg, Switzerland
- Department of Cognitive Science, Macquarie University, Sydney, Australia
| | - Genevieve McArthur
- Department of Cognitive Science, Macquarie University, Sydney, Australia
| | - Stefan Hawelka
- Centre for Cognitive Neuroscience, Salzburg University, Salzburg, Austria
| | - Romina Palermo
- School of Psychological Science, University of Western Australia, Perth, Australia
| | - Yatin Mahajan
- The MARCS Institute, University of Western Sydney, Australia
| | - Federica Degno
- School of Psychology, University of Central Lancashire, Preston, UK
| | - Florian Hutzler
- Centre for Cognitive Neuroscience, Salzburg University, Salzburg, Austria
| |
Collapse
|
33
|
Alday PM, Kretzschmar F. Speed-Accuracy Tradeoffs in Brain and Behavior: Testing the Independence of P300 and N400 Related Processes in Behavioral Responses to Sentence Categorization. Front Hum Neurosci 2019; 13:285. [PMID: 31507392 PMCID: PMC6718734 DOI: 10.3389/fnhum.2019.00285] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Accepted: 08/05/2019] [Indexed: 01/23/2023] Open
Abstract
Although the N400 was originally discovered in a paradigm designed to elicit a P300 (Kutas and Hillyard, 1980), its relationship with the P300 and how both overlapping event-related potentials (ERPs) determine behavioral profiles is still elusive. Here we conducted an ERP (N = 20) and a multiple-response speed-accuracy tradeoff (SAT) experiment (N = 16) on distinct participant samples using an antonym paradigm (The opposite of black is white/nice/yellow with acceptability judgment). We hypothesized that SAT profiles incorporate processes of task-related decision-making (P300) and stimulus-related expectation violation (N400). We replicated previous ERP results (Roehm et al., 2007): in the correct condition (white), the expected target elicits a P300, while both expectation violations engender an N400 [reduced for related (yellow) vs. unrelated targets (nice)]. Using multivariate Bayesian mixed-effects models, we modeled the P300 and N400 responses simultaneously and found that correlation between residuals and subject-level random effects of each response window was minimal, suggesting that the components are largely independent. For the SAT data, we found that antonyms and unrelated targets had a similar slope (rate of increase in accuracy over time) and an asymptote at ceiling, while related targets showed both a lower slope and a lower asymptote, reaching only approximately 80% accuracy. Using a GLMM-based approach (Davidson and Martin, 2013), we modeled these dynamics using response time and condition as predictors. Replacing the predictor for condition with the averaged P300 and N400 amplitudes from the ERP experiment, we achieved identical model performance. We then examined the piecewise contribution of the P300 and N400 amplitudes with partial effects (see Hohenstein and Kliegl, 2015). Unsurprisingly, the P300 amplitude was the strongest contributor to the SAT-curve in the antonym condition and the N400 was the strongest contributor in the unrelated condition. In brief, this is the first demonstration of how overlapping ERP responses in one sample of participants predict behavioral SAT profiles of another sample. The P300 and N400 reflect two independent but interacting processes and the competition between these processes is reflected differently in behavioral parameters of speed and accuracy.
Collapse
Affiliation(s)
- Phillip M. Alday
- Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands
| | - Franziska Kretzschmar
- CRC 1252 “Prominence in Language”, University of Cologne, Cologne, Germany
- Institute of German Language and Literature I, University of Cologne, Cologne, Germany
| |
Collapse
|
34
|
Himmelstoss NA, Schuster S, Hutzler F, Moran R, Hawelka S. Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading. LANGUAGE, COGNITION AND NEUROSCIENCE 2019; 35:595-612. [PMID: 32656295 PMCID: PMC7324136 DOI: 10.1080/23273798.2019.1616102] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/11/2018] [Accepted: 04/26/2019] [Indexed: 06/11/2023]
Abstract
Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.
Collapse
Affiliation(s)
| | - Sarah Schuster
- Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria
| | - Florian Hutzler
- Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria
| | - Rosalyn Moran
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
| | - Stefan Hawelka
- Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria
| |
Collapse
|
35
|
An electrophysiological investigation of orthographic spatial integration in reading. Neuropsychologia 2019; 129:276-283. [PMID: 31002854 PMCID: PMC6582630 DOI: 10.1016/j.neuropsychologia.2019.04.009] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2018] [Revised: 04/14/2019] [Accepted: 04/15/2019] [Indexed: 11/29/2022]
Abstract
During reading, word recognition speed is influenced by the amount of orthographic overlap with surrounding words. The nature of this phenomenon is not understood: some theories attribute it to low-level visual operations (i.e., parafoveal feature detectors influencing foveal letter detectors), whereas other theories assume that orthographic processing (i.e., letter position coding and word activation) occurs across multiple words in parallel. To arbitrate between these theories, we used electroencephalography to reveal the time course of orthographic spatial integration in a lexical decision task. Foveal target words were flanked on each side by parafoveal words, manipulated across three conditions: repetition flankers (e.g. rock rock rock), unrelated flankers (step rock step) and a no-flanker condition. Linear mixed-effect models were constructed to analyze EEG data on a trial-by-trial basis. Word recognition was worse in the unrelated flanker condition than in the repetition and no-flanker conditions. This behavioral pattern was accompanied by increased negativity in the N250 and N400 windows, associated with the activation of sub-lexical and lexico-semantic representations, respectively. Crucially, the absence of effects prior to 200 ms post-stimulus onset provides evidence against the involvement of low-level visual processes. We conclude that orthographic spatial integration is driven by parallel processing of multiple words, which leads to the activation of a larger set of sub-lexical nodes and more difficult processing at the lexical level when those words are orthographically unrelated. Words are recognized faster when they share many letters with surrounding words. EEG measurements suggest that this effect unfolds 200 ms after stimulus onset. The locus of effects contradicts the involvement of low-level visual processing. Instead, the phenomenon appears to be driven by parallel orthographic processing.
Collapse
|