1
|
Dufau S, Yeaton J, Badier JM, Chen S, Holcomb PJ, Grainger J. Sentence superiority in the reading brain. Neuropsychologia 2024; 198:108885. [PMID: 38604495 DOI: 10.1016/j.neuropsychologia.2024.108885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Revised: 02/06/2024] [Accepted: 04/07/2024] [Indexed: 04/13/2024]
Abstract
When a sequence of written words is briefly presented and participants are asked to identify just one word at a post-cued location, then word identification accuracy is higher when the word is presented in a grammatically correct sequence compared with an ungrammatical sequence. This sentence superiority effect has been reported in several behavioral studies and two EEG investigations. Taken together, the results of these studies support the hypothesis that the sentence superiority effect is primarily driven by rapid access to a sentence-level representation via partial word identification processes that operate in parallel over several words. Here we used MEG to examine the neural structures involved in this early stage of written sentence processing, and to further specify the timing of the different processes involved. Source activities over time showed grammatical vs. ungrammatical differences first in the left inferior frontal gyrus (IFG: 321-406 ms), then the left anterior temporal lobe (ATL: 466-531 ms), and finally in both left IFG (549-602 ms) and left posterior superior temporal gyrus (pSTG: 553-622 ms). We interpret the early IFG activity as reflecting the rapid bottom-up activation of sentence-level representations, including syntax, enabled by partly parallel word processing. Subsequent activity in ATL and pSTG is thought to reflect the constraints imposed by such sentence-level representations on on-going word-based semantic activation (ATL), and the subsequent development of a more detailed sentence-level representation (pSTG). These results provide further support for a cascaded interactive-activation account of sentence reading.
Collapse
Affiliation(s)
- Stéphane Dufau
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France; Institute for Language, Communication, and the Brain, Aix-Marseille University, Aix-en-Provence, France
| | - Jeremy Yeaton
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France; Department of Language Science, University of California, Irvine, CA, USA
| | - Jean-Michel Badier
- Institute for Language, Communication, and the Brain, Aix-Marseille University, Aix-en-Provence, France; Institut de Neurosciences des Systèmes (INS), INSERM, Aix-Marseille University, Marseille, France
| | - Sophie Chen
- Institute for Language, Communication, and the Brain, Aix-Marseille University, Aix-en-Provence, France; Institut de Neurosciences des Systèmes (INS), INSERM, Aix-Marseille University, Marseille, France
| | - Phillip J Holcomb
- Department of Psychology, San Diego State University, San Diego, CA, USA
| | - Jonathan Grainger
- Laboratoire de Psychologie Cognitive, Centre National de la Recherche Scientifique, Aix-Marseille University, Marseille, France; Institute for Language, Communication, and the Brain, Aix-Marseille University, Aix-en-Provence, France.
| |
Collapse
|
2
|
Weissman B, Cohn N, Tanner D. The electrophysiology of lexical prediction of emoji and text. Neuropsychologia 2024; 198:108881. [PMID: 38579906 DOI: 10.1016/j.neuropsychologia.2024.108881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 02/18/2024] [Accepted: 03/27/2024] [Indexed: 04/07/2024]
Abstract
As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two well-established ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
Collapse
Affiliation(s)
- Benjamin Weissman
- Department of Cognitive Science Rensselaer Polytechnic Institute 110 8th Street, Troy, NY, 12180, USA; Department of Linguistics University of Illinois at Urbana-Champaign 707 S Mathews Ave, Urbana, IL, 61801, USA.
| | - Neil Cohn
- Department of Communication and Cognition Tilburg University PO Box 90153, 5000, LE Tilburg, the Netherlands
| | - Darren Tanner
- Department of Linguistics University of Illinois at Urbana-Champaign 707 S Mathews Ave, Urbana, IL, 61801, USA; AI For Good Lab Microsoft 1 Microsoft Way, Redmond, WA, USA
| |
Collapse
|
3
|
Karimi H, Weber P, Zinn J. Information entropy facilitates (not impedes) lexical processing during language comprehension. Psychon Bull Rev 2024:10.3758/s13423-024-02463-x. [PMID: 38361106 DOI: 10.3758/s13423-024-02463-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/15/2024] [Indexed: 02/17/2024]
Abstract
It is well known that contextual predictability facilitates word identification, but it is less clear whether the uncertainty associated with the current context (i.e., its lexical entropy) influences sentence processing. On the one hand, high entropy contexts may lead to interference due to greater number of lexical competitors. On the other hand, predicting multiple lexical competitors may facilitate processing through the preactivation of shared semantic features. In this study, we examined whether entropy measured at the trial level (i.e., for each participant, for each item) corresponds to facilitatory or inhibitory effects. Trial-level entropy captures each individual's knowledge about specific contexts and is therefore a more valid and sensitive measure of entropy (relative to the commonly employed item-level entropy). Participants (N = 112) completed two experimental sessions (with counterbalanced orders) that were separated by a 3- to 14-day interval. In one session, they produced up to 10 completions for sentence fragments (N = 647). In another session, they read the same sentences including a target word (whose entropy value was calculated based on the produced completions) while reading times were measured. We observed a facilitatory (not inhibitory) effect of trial-level entropy on lexical processing over and above item-level measures of lexical predictability (including cloze probability, surprisal, and semantic constraint). Extra analyses revealed that greater semantic overlap between the target and the produced responses facilitated target processing. Thus, the results lend support to theories of lexical prediction maintaining that prediction involves broad activation of semantic features rather than activation of full lexical forms.
Collapse
Affiliation(s)
- Hossein Karimi
- Department of Psychology, Mississippi State University, 215 Magruder Hall, Mississippi State, MS, USA.
| | - Pete Weber
- Department of Psychology, Mississippi State University, 215 Magruder Hall, Mississippi State, MS, USA
| | - Jaden Zinn
- Department of Psychology, Mississippi State University, 215 Magruder Hall, Mississippi State, MS, USA
| |
Collapse
|
4
|
Zhou W, Wang S, Yan M. Fixation-related fMRI analysis reveals the neural basis of natural reading of unspaced and spaced Chinese sentences. Cereb Cortex 2023; 33:10401-10410. [PMID: 37566912 DOI: 10.1093/cercor/bhad290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 07/15/2023] [Accepted: 07/16/2023] [Indexed: 08/13/2023] Open
Abstract
Although there are many eye-movement studies focusing on natural sentence reading and functional magnetic resonance imaging research on reading with serial visual presentation paradigms, there is a scarcity of investigations into the neural mechanism of natural sentence reading. The present study recruited 33 adults to read unspaced and spaced Chinese sentences with the eye tracking and functional magnetic resonance imaging data recorded simultaneously. By using fixation-related functional magnetic resonance imaging analysis, this study showed that natural reading of Chinese sentences produced activations in ventral visual, dorsal attention, and semantic brain regions, which were modulated by the properties of words such as word length and word frequency. The multivoxel pattern analysis showed that the activity pattern in the left middle temporal gyrus could significantly predict the visual layout categories (i.e. unspaced vs. spaced conditions). Dynamic causal modeling analysis showed that there were bidirectional brain connections between the left middle temporal gyrus and the left inferior occipital cortex in the unspaced Chinese sentence reading but not in the spaced reading. These results provide a neural mechanism for the natural reading of Chinese sentences from the perspective of word segmentation.
Collapse
Affiliation(s)
- Wei Zhou
- Beijing Key Lab of Learning and Cognition, School of Psychology, Capital Normal University, Beijing 100048, China
- Chinese Institute for Brain Research, Beijing 102206, China
| | - Sile Wang
- Beijing Key Lab of Learning and Cognition, School of Psychology, Capital Normal University, Beijing 100048, China
- School of Psychology, South China Normal University, Guangzhou 510631, China
| | - Ming Yan
- Department of Psychology and Center for Cognitive and Brain Sciences, University of Macau, Taipa, Macau
| |
Collapse
|
5
|
Herrera C, Whittle N, Leek MR, Brodbeck C, Lee G, Barcenas C, Barnes S, Holshouser B, Yi A, Venezia JH. Cortical networks for recognition of speech with simultaneous talkers. Hear Res 2023; 437:108856. [PMID: 37531847 DOI: 10.1016/j.heares.2023.108856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 07/05/2023] [Accepted: 07/21/2023] [Indexed: 08/04/2023]
Abstract
The relative contributions of superior temporal vs. inferior frontal and parietal networks to recognition of speech in a background of competing speech remain unclear, although the contributions themselves are well established. Here, we use fMRI with spectrotemporal modulation transfer function (ST-MTF) modeling to examine the speech information represented in temporal vs. frontoparietal networks for two speech recognition tasks with and without a competing talker. Specifically, 31 listeners completed two versions of a three-alternative forced choice competing speech task: "Unison" and "Competing", in which a female (target) and a male (competing) talker uttered identical or different phrases, respectively. Spectrotemporal modulation filtering (i.e., acoustic distortion) was applied to the two-talker mixtures and ST-MTF models were generated to predict brain activation from differences in spectrotemporal-modulation distortion on each trial. Three cortical networks were identified based on differential patterns of ST-MTF predictions and the resultant ST-MTF weights across conditions (Unison, Competing): a bilateral superior temporal (S-T) network, a frontoparietal (F-P) network, and a network distributed across cortical midline regions and the angular gyrus (M-AG). The S-T network and the M-AG network responded primarily to spectrotemporal cues associated with speech intelligibility, regardless of condition, but the S-T network responded to a greater range of temporal modulations suggesting a more acoustically driven response. The F-P network responded to the absence of intelligibility-related cues in both conditions, but also to the absence (presence) of target-talker (competing-talker) vocal pitch in the Competing condition, suggesting a generalized response to signal degradation. Task performance was best predicted by activation in the S-T and F-P networks, but in opposite directions (S-T: more activation = better performance; F-P: vice versa). Moreover, S-T network predictions were entirely ST-MTF mediated while F-P network predictions were ST-MTF mediated only in the Unison condition, suggesting an influence from non-acoustic sources (e.g., informational masking) in the Competing condition. Activation in the M-AG network was weakly positively correlated with performance and this relation was entirely superseded by those in the S-T and F-P networks. Regarding contributions to speech recognition, we conclude: (a) superior temporal regions play a bottom-up, perceptual role that is not qualitatively dependent on the presence of competing speech; (b) frontoparietal regions play a top-down role that is modulated by competing speech and scales with listening effort; and (c) performance ultimately relies on dynamic interactions between these networks, with ancillary contributions from networks not involved in speech processing per se (e.g., the M-AG network).
Collapse
Affiliation(s)
| | - Nicole Whittle
- VA Loma Linda Healthcare System, Loma Linda, CA, United States
| | - Marjorie R Leek
- VA Loma Linda Healthcare System, Loma Linda, CA, United States; Loma Linda University, Loma Linda, CA, United States
| | | | - Grace Lee
- Loma Linda University, Loma Linda, CA, United States
| | | | - Samuel Barnes
- Loma Linda University, Loma Linda, CA, United States
| | | | - Alex Yi
- VA Loma Linda Healthcare System, Loma Linda, CA, United States; Loma Linda University, Loma Linda, CA, United States
| | - Jonathan H Venezia
- VA Loma Linda Healthcare System, Loma Linda, CA, United States; Loma Linda University, Loma Linda, CA, United States.
| |
Collapse
|
6
|
Chang M, Zhang K, Sun Y, Li S, Wang J. The graded predictive pre-activation in Chinese sentence reading: evidence from eye movements. Front Psychol 2023; 14:1136488. [PMID: 37457059 PMCID: PMC10342199 DOI: 10.3389/fpsyg.2023.1136488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 06/12/2023] [Indexed: 07/18/2023] Open
Abstract
Previous research has revealed that graded pre-activation rather than specific lexical prediction is more likely to be the mechanism for the word predictability effect in English. However, whether graded pre-activation underlies the predictability effect in Chinese reading is unknown. Accordingly, the present study tested the generality of the graded pre-activation account in Chinese reading. We manipulated the contextual constraint of sentences and the predictability of target words as independent variables. Readers' eye movement behaviors were recorded via an eye tracker. We examined whether processing an unpredictable word in a solid constraining context incurs a prediction error cost when this unpredictable word has a predictable alternative. The results showed no cues of prediction error cost on the early eye movement measures, supported by the Bayes Factor analyses. The current research indicates that graded predictive pre-activation underlies the predictability effect in Chinese reading.
Collapse
Affiliation(s)
- Min Chang
- School of Education Science, Nantong University, Nantong, China
| | - Kuo Zhang
- Department of Social Psychology, Nankai University, Tianjin, China
| | - Yue Sun
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
| | - Sha Li
- School of Psychology, Fujian Normal University, Fuzhou, China
| | - Jingxin Wang
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
| |
Collapse
|
7
|
Ma X, Liu Y, Clariana R, Gu C, Li P. From eye movements to scanpath networks: A method for studying individual differences in expository text reading. Behav Res Methods 2023; 55:730-750. [PMID: 35445941 PMCID: PMC10027820 DOI: 10.3758/s13428-022-01842-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/17/2022] [Indexed: 11/08/2022]
Abstract
Eye movements have been examined as an index of attention and comprehension during reading in the literature for over 30 years. Although eye-movement measurements are acknowledged as reliable indicators of readers' comprehension skill, few studies have analyzed eye-movement patterns using network science. In this study, we offer a new approach to analyze eye-movement data. Specifically, we recorded visual scanpaths when participants were reading expository science text, and used these to construct scanpath networks that reflect readers' processing of the text. Results showed that low ability and high ability readers' scanpath networks exhibited distinctive properties, which are reflected in different network metrics including density, centrality, small-worldness, transitivity, and global efficiency. Such patterns provide a new way to show how skilled readers, as compared with less skilled readers, process information more efficiently. Implications of our analyses are discussed in light of current theories of reading comprehension.
Collapse
Affiliation(s)
- Xiaochuan Ma
- Department of Psychology, The Pennsylvania State University, Moore Building, University Park, PA, 16802, USA
| | - Yikang Liu
- Department of Biomedical Engineering, The Pennsylvania State University, Millennium Science Complex, University Park, PA, 16802, USA
| | - Roy Clariana
- Department of Learning and Performance Systems, Keller Building, The Pennsylvania State University, University Park, PA, 16802, USA
| | - Chanyuan Gu
- Department of Chinese and Bilingual Studies, Faculty of Humanities, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
| | - Ping Li
- Department of Chinese and Bilingual Studies, Faculty of Humanities, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong.
| |
Collapse
|
8
|
Russo AG, De Martino M, Elia A, Di Salle F, Esposito F. Negative correlation between word-level surprisal and intersubject neural synchronization during narrative listening. Cortex 2022; 155:132-149. [DOI: 10.1016/j.cortex.2022.07.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 02/10/2022] [Accepted: 07/06/2022] [Indexed: 11/30/2022]
|
9
|
Brothers T, Kuperberg GR. Word predictability effects are linear, not logarithmic: Implications for probabilistic models of sentence comprehension. JOURNAL OF MEMORY AND LANGUAGE 2021; 116:104174. [PMID: 33100508 PMCID: PMC7584137 DOI: 10.1016/j.jml.2020.104174] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
During language comprehension, we routinely use information from the prior context to help identify the meaning of individual words. While measures of online processing difficulty, such as reading times, are strongly influenced by contextual predictability, there is disagreement about the mechanisms underlying this lexical predictability effect, with different models predicting different linking functions - linear (Reichle, Rayner & Pollatsek, 2003) or logarithmic (Levy, 2008). To help resolve this debate, we conducted two highly-powered experiments (self-paced reading, N = 216; cross-modal picture naming, N = 36), and a meta-analysis of prior eye-tracking while reading studies (total N = 218). We observed a robust linear relationship between lexical predictability and word processing times across all three studies. Beyond their methodological implications, these findings also place important constraints on predictive processing models of language comprehension. In particular, these results directly contradict the empirical predictions of surprisal theory, while supporting a proportional pre-activation account of lexical prediction effects in comprehension.
Collapse
Affiliation(s)
- Trevor Brothers
- Department of Psychology, Tufts University, Medford, MA
USA
- Department of Psychiatry and the Athinoula A. Martinos
Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical
School, Boston, MA USA
| | - Gina R. Kuperberg
- Department of Psychology, Tufts University, Medford, MA
USA
- Department of Psychiatry and the Athinoula A. Martinos
Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical
School, Boston, MA USA
| |
Collapse
|
10
|
Ischebeck A, Hiebel H, Miller J, Höfler M, Gilchrist ID, Körner C. Target processing in overt serial visual search involves the dorsal attention network: A fixation-based event-related fMRI study. Neuropsychologia 2021; 153:107763. [PMID: 33493526 DOI: 10.1016/j.neuropsychologia.2021.107763] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Revised: 01/15/2021] [Accepted: 01/17/2021] [Indexed: 11/18/2022]
Abstract
In serial visual search we shift attention successively from location to location in search for the target. Although such search has been investigated using fMRI, overt attention (i.e., eye movements) was usually neglected or discouraged. As a result, it is unclear what happens in the instant when our gaze falls upon a target as compared to a distractor. In the present experiment, we used a multiple target search task that required eye movements and employed an analysis based on fixations as events of interest to investigate differences between target and distractor processing. Twenty young healthy adults indicated the number of targets (0-3) among distractors in a 20-item display. Compared to distractor fixations, we found that target fixations gave rise to wide-spread activation in the dorsal attention system, as well as in the visual cortex. Targets that were found later during the search activated the left inferior frontal gyrus and the left supramarginal gyrus more strongly than those that were found earlier. Finally, areas associated with visual and verbal working memory showed increased activation with a larger number of targets in the display.
Collapse
Affiliation(s)
- Anja Ischebeck
- Institute of Psychology, University of Graz, Austria; BioTechMed-Graz, Austria.
| | - Hannah Hiebel
- Institute of Psychology, University of Graz, Austria
| | - Joe Miller
- Institute of Psychology, University of Graz, Austria
| | - Margit Höfler
- Institute of Psychology, University of Graz, Austria; Department for Clinical Neurosciences and Preventive Medicine, Danube University Krems, Austria
| | | | - Christof Körner
- Institute of Psychology, University of Graz, Austria; BioTechMed-Graz, Austria
| |
Collapse
|
11
|
Schuster S, Himmelstoss NA, Hutzler F, Richlan F, Kronbichler M, Hawelka S. Cloze enough? Hemodynamic effects of predictive processing during natural reading. Neuroimage 2020; 228:117687. [PMID: 33385553 DOI: 10.1016/j.neuroimage.2020.117687] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2020] [Revised: 11/25/2020] [Accepted: 12/23/2020] [Indexed: 10/22/2022] Open
Abstract
Evidence accrues that readers form multiple hypotheses about upcoming words. The present study investigated the hemodynamic effects of predictive processing during natural reading by means of combining fMRI and eye movement recordings. In particular, we investigated the neural and behavioral correlates of precision-weighted prediction errors, which are thought to be indicative of subsequent belief updating. Participants silently read sentences in which we manipulated the cloze probability and the semantic congruency of the final word that served as an index for precision and prediction error respectively. With respect to the neural correlates, our findings indicate an enhanced activation within the left inferior frontal and middle temporal gyrus suggesting an effect of precision on prediction update in higher (lexico-)semantic levels. Despite being evident at the neural level, we did not observe any evidence that this mechanism resulted in disproportionate reading times on participants' eye movements. The results speak against discrete predictions, but favor the notion that multiple words are activated in parallel during reading.
Collapse
Affiliation(s)
- Sarah Schuster
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria
| | - Nicole Alexandra Himmelstoss
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria
| | - Florian Hutzler
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria.
| | - Fabio Richlan
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria
| | - Martin Kronbichler
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria; Neuroscience Institute and Department of Neurology, Christian Doppler Clinic, Paracelsus Private Medical University, Ignaz-Harrer-Str. 79, 5020 Salzburg, Austria
| | - Stefan Hawelka
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Hellbrunnerstr. 34, 5020 Salzburg, Austria
| |
Collapse
|
12
|
Henderson JM, Goold JE, Choi W, Hayes TR. Neural Correlates of Fixated Low- and High-level Scene Properties during Active Scene Viewing. J Cogn Neurosci 2020; 32:2013-2023. [PMID: 32573384 PMCID: PMC11164273 DOI: 10.1162/jocn_a_01599] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
During real-world scene perception, viewers actively direct their attention through a scene in a controlled sequence of eye fixations. During each fixation, local scene properties are attended, analyzed, and interpreted. What is the relationship between fixated scene properties and neural activity in the visual cortex? Participants inspected photographs of real-world scenes in an MRI scanner while their eye movements were recorded. Fixation-related fMRI was used to measure activation as a function of lower- and higher-level scene properties at fixation, operationalized as edge density and meaning maps, respectively. We found that edge density at fixation was most associated with activation in early visual areas, whereas semantic content at fixation was most associated with activation along the ventral visual stream including core object and scene-selective areas (lateral occipital complex, parahippocampal place area, occipital place area, and retrosplenial cortex). The observed activation from semantic content was not accounted for by differences in edge density. The results are consistent with active vision models in which fixation gates detailed visual analysis for fixated scene regions, and this gating influences both lower and higher levels of scene analysis.
Collapse
Affiliation(s)
| | | | - Wonil Choi
- Gwangju Institute of Science and Technology
| | | |
Collapse
|
13
|
Gambi C, Jindal P, Sharpe S, Pickering MJ, Rabagliati H. The Relation Between Preschoolers' Vocabulary Development and Their Ability to Predict and Recognize Words. Child Dev 2020; 92:1048-1066. [PMID: 32865231 DOI: 10.1111/cdev.13465] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
By age 2, children are developing foundational language processing skills, such as quickly recognizing words and predicting words before they occur. How do these skills relate to children's structural knowledge of vocabulary? Multiple aspects of language processing were simultaneously measured in a sample of 2-to-5-year-olds (N = 215): While older children were more fluent at recognizing words, at predicting words in a graded fashion, and at revising incorrect predictions, only revision was associated with concurrent vocabulary knowledge once age was accounted for. However, an exploratory longitudinal follow-up (N = 55) then found that word recognition and prediction skills were associated with rate of subsequent vocabulary development, but revision skills were not. We argue that prediction skills may facilitate language learning through enhancing processing speed.
Collapse
|
14
|
Russo AG, De Martino M, Mancuso A, Iaconetta G, Manara R, Elia A, Laudanna A, Di Salle F, Esposito F. Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening. Neuroimage 2020; 222:117281. [PMID: 32828929 DOI: 10.1016/j.neuroimage.2020.117281] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2020] [Revised: 06/22/2020] [Accepted: 08/13/2020] [Indexed: 11/16/2022] Open
Abstract
Probabilistic language models are increasingly used to provide neural representations of linguistic features under naturalistic settings. Word surprisal models can be applied to continuous fMRI recordings during task-free listening of narratives, to detect regions linked to language prediction and comprehension. Here, to this purpose, a novel semantics-weighted lexical surprisal is applied to naturalistic fMRI data. FMRI was performed at 3 Tesla in 31 subjects during task-free listening to a 12-minute audiobook played in both original and word-reversed (control) version. Lexical-only and semantics-weighted lexical surprisal models were estimated for the original and control word series. The two series were alternatively chosen to build the predictor of interest in the first-level general linear model and were compared in the second-level (group) analysis. The addition of the surprisal predictor to the stimulus-related predictors significantly improved the fitting of the neural signal. In average, the semantics-weighted model yielded lower surprisal values and, in some areas, better fitting of the fMRI data compared to the lexical-only model. The two models produced both overlapping and distinct activations: while lexical-only surprisal activated secondary auditory areas in the superior temporal gyri and the cerebellum, semantics-weighted surprisal additionally activated the left inferior frontal gyrus. These results confirm the usefulness of surprisal models in the naturalistic fMRI analysis of linguistic processes and suggest that the use of semantic information may increase the sensitivity of a probabilistic language model in higher-order language-related areas, with possible implications for future naturalistic fMRI studies of language under normal and (clinically or pharmacologically) modified conditions.
Collapse
Affiliation(s)
- Andrea G Russo
- Department of Political and Communication Sciences, University of Salerno, Fisciano, Salerno, Italy; Department of Medicine, Surgery and Dentistry, "Scuola Medica Salernitana", University of Salerno, Baronissi, Salerno, Italy.
| | - Maria De Martino
- Department of Political and Communication Sciences, University of Salerno, Fisciano, Salerno, Italy
| | - Azzurra Mancuso
- Department of Political and Communication Sciences, University of Salerno, Fisciano, Salerno, Italy
| | - Giorgio Iaconetta
- Department of Medicine, Surgery and Dentistry, "Scuola Medica Salernitana", University of Salerno, Baronissi, Salerno, Italy; Department of Diagnostic Imaging, University Hospital "San Giovanni di Dio e Ruggi D'Aragona", Salerno, Italy
| | - Renzo Manara
- Department of Medicine, Surgery and Dentistry, "Scuola Medica Salernitana", University of Salerno, Baronissi, Salerno, Italy; Department of Diagnostic Imaging, University Hospital "San Giovanni di Dio e Ruggi D'Aragona", Salerno, Italy
| | - Annibale Elia
- Department of Political and Communication Sciences, University of Salerno, Fisciano, Salerno, Italy
| | - Alessandro Laudanna
- Department of Political and Communication Sciences, University of Salerno, Fisciano, Salerno, Italy
| | - Francesco Di Salle
- Department of Medicine, Surgery and Dentistry, "Scuola Medica Salernitana", University of Salerno, Baronissi, Salerno, Italy; Department of Diagnostic Imaging, University Hospital "San Giovanni di Dio e Ruggi D'Aragona", Salerno, Italy
| | - Fabrizio Esposito
- Department of Medicine, Surgery and Dentistry, "Scuola Medica Salernitana", University of Salerno, Baronissi, Salerno, Italy; Department of Diagnostic Imaging, University Hospital "San Giovanni di Dio e Ruggi D'Aragona", Salerno, Italy
| |
Collapse
|
15
|
Carter BT, Luke SG. Best practices in eye tracking research. Int J Psychophysiol 2020; 155:49-62. [PMID: 32504653 DOI: 10.1016/j.ijpsycho.2020.05.010] [Citation(s) in RCA: 61] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2019] [Revised: 05/26/2020] [Accepted: 05/27/2020] [Indexed: 12/14/2022]
Abstract
This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.
Collapse
|
16
|
Carter BT, Luke SG. The effect of convolving word length, word frequency, function word predictability and first pass reading time in the analysis of a fixation-related fMRI dataset. Data Brief 2019; 25:104171. [PMID: 31463340 PMCID: PMC6706769 DOI: 10.1016/j.dib.2019.104171] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2019] [Revised: 05/06/2019] [Accepted: 06/14/2019] [Indexed: 01/10/2023] Open
Abstract
The data presented in this document was created to explore the effect of including or excluding word length, word frequency, the lexical predictability of function words and first pass reading time (or the duration of the first fixation on a word) as either baseline regressors or duration modulators on the final analysis for a fixation-related fMRI investigation of linguistic processing. The effect of these regressors was a central question raised during the review of Linguistic networks associated with lexical, semantic and syntactic predictability in reading: A fixation-related fMRI study [1]. Three datasets were created and compared to the original dataset to determine their effect. The first examines the effect of adding word length and word frequency as baseline regressors. The second examines the effect of removing first pass reading time as a duration modulator. The third examines the inclusion of function word predictability into the baseline hemodynamic response function. Statistical maps were created for each dataset and compared to the primary dataset (published in [1]) across the linguistic conditions of the initial dataset (lexical predictability, semantic predictability or syntax predictability).
Collapse
|