1
|
Hintz F, Voeten CC, Dobó D, Lukics KS, Lukács Á. The role of general cognitive skills in integrating visual and linguistic information during sentence comprehension: individual differences across the lifespan. Sci Rep 2024; 14:17797. [PMID: 39090337 PMCID: PMC11294566 DOI: 10.1038/s41598-024-68674-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Accepted: 07/26/2024] [Indexed: 08/04/2024] Open
Abstract
Individuals exhibit massive variability in general cognitive skills that affect language processing. This variability is partly developmental. Here, we recruited a large sample of participants (N = 487), ranging from 9 to 90 years of age, and examined the involvement of nonverbal processing speed (assessed using visual and auditory reaction time tasks) and working memory (assessed using forward and backward Digit Span tasks) in a visual world task. Participants saw two objects on the screen and heard a sentence that referred to one of them. In half of the sentences, the target object could be predicted based on verb-selectional restrictions. We observed evidence for anticipatory processing on predictable compared to non-predictable trials. Visual and auditory processing speed had main effects on sentence comprehension and facilitated predictive processing, as evidenced by an interaction. We observed only weak evidence for the involvement of working memory in predictive sentence comprehension. Age had a nonlinear main effect (younger adults responded faster than children and older adults), but it did not differentially modulate predictive and non-predictive processing, nor did it modulate the involvement of processing speed and working memory. Our results contribute to delineating the cognitive skills that are involved in language-vision interactions.
Collapse
Affiliation(s)
- Florian Hintz
- Deutscher Sprachatlas, Philipps University of Marburg, Pilgrimstein 16, 35032, Marburg, Germany.
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands.
- Center for Mind, Brain and Behavior, Philipps University Marburg & Justus Liebig University Giessen, Marburg & Giessen, Germany.
| | - Cesko C Voeten
- Department of Linguistics, University of Pennsylvania, Philadelphia, PA, USA
- Department of Biology, University of Pennsylvania, Philadelphia, PA, USA
- University of Amsterdam, Amsterdam, The Netherlands
- Fryske Akademy, Leeuwarden, The Netherlands
| | - Dorottya Dobó
- Department of Cognitive Science, Faculty of Natural Sciences, Budapest University of Technology and Economics, Budapest, Hungary
- MTA-BME Momentum Language Acquisition Research Group, Eötvös Loránd Research Network (ELKH), Budapest, Hungary
| | - Krisztina Sára Lukics
- Department of Cognitive Science, Faculty of Natural Sciences, Budapest University of Technology and Economics, Budapest, Hungary
- MTA-BME Momentum Language Acquisition Research Group, Eötvös Loránd Research Network (ELKH), Budapest, Hungary
| | - Ágnes Lukács
- Department of Cognitive Science, Faculty of Natural Sciences, Budapest University of Technology and Economics, Budapest, Hungary
- MTA-BME Momentum Language Acquisition Research Group, Eötvös Loránd Research Network (ELKH), Budapest, Hungary
| |
Collapse
|
2
|
Truman A, Kutas M. Flexible Conceptual Representations. Cogn Sci 2024; 48:e13475. [PMID: 38923016 DOI: 10.1111/cogs.13475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 05/14/2024] [Accepted: 06/05/2024] [Indexed: 06/28/2024]
Abstract
A view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context-dependent, and task-dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age-old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all-or-none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.
Collapse
Affiliation(s)
- Alyssa Truman
- Department of Cognitive Science, University of California, San Diego
| | - Marta Kutas
- Department of Cognitive Science, University of California, San Diego
| |
Collapse
|
3
|
James AN, Minnihan CJ, Watson DG. Language Experience Predicts Eye Movements During Online Auditory Comprehension. J Cogn 2023; 6:30. [PMID: 37397351 PMCID: PMC10312251 DOI: 10.5334/joc.285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 06/03/2023] [Indexed: 07/04/2023] Open
Abstract
Experience-based theories of language processing suggest that listeners use the properties of their previous linguistic input to constrain comprehension in real time (e.g. MacDonald & Christiansen, 2002; Smith & Levy, 2013; Stanovich & West, 1989; Mishra, Pandey, Singh, & Huettig, 2012). This project investigates the prediction that individual differences in experience will predict differences in sentence comprehension. Participants completed a visual world eye-tracking task following Altmann and Kamide (1999) which manipulates whether the verb licenses the anticipation of a specific referent in the scene (e.g. The boy will eat/move the cake). Within this paradigm, we ask (1) are there reliable individual differences in language-mediated eye movements during this task? If so, (2) do individual differences in language experience correlate with these differences, and (3) can this relationship be explained by other, more general cognitive abilities? Study 1 finds evidence that language experience predicts an overall facilitation in fixating the target, and Study 2 replicates this effect and finds that it remains when controlling for working memory, inhibitory control, phonological ability, and perceptual speed.
Collapse
Affiliation(s)
| | | | - Duane G. Watson
- Department of Psychology and Human Development, Peabody College, Vanderbilt University, US
| |
Collapse
|
4
|
Garrido Rodriguez G, Norcliffe E, Brown P, Huettig F, Levinson SC. Anticipatory Processing in a Verb-Initial Mayan Language: Eye-Tracking Evidence During Sentence Comprehension in Tseltal. Cogn Sci 2023; 47:e13292. [PMID: 36652288 DOI: 10.1111/cogs.13219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 04/13/2022] [Accepted: 09/11/2022] [Indexed: 01/19/2023]
Abstract
We present a visual world eye-tracking study on Tseltal (a Mayan language) and investigate whether verbal information can be used to anticipate an upcoming referent. Basic word order in transitive sentences in Tseltal is Verb-Object-Subject (VOS). The verb is usually encountered first, making argument structure and syntactic information available at the outset, which should facilitate anticipation of the post-verbal arguments. Tseltal speakers listened to verb-initial sentences with either an object-predictive verb (e.g., "eat") or a general verb (e.g., "look for") (e.g., "Ya slo'/sle ta stukel on te kereme," Is eating/is looking (for) by himself the avocado the boy/ "The boy is eating/is looking (for) an avocado by himself") while seeing a visual display showing one potential referent (e.g., avocado) and three distractors (e.g., bag, toy car, coffee grinder). We manipulated verb type (predictive vs. general) and recorded participants' eye movements while they listened and inspected the visual scene. Participants' fixations to the target referent were analyzed using multilevel logistic regression models. Shortly after hearing the predictive verb, participants fixated the target object before it was mentioned. In contrast, when the verb was general, fixations to the target only started to increase once the object was heard. Our results suggest that Tseltal hearers pre-activate semantic features of the grammatical object prior to its linguistic expression. This provides evidence from a verb-initial language for online incremental semantic interpretation and anticipatory processing during language comprehension. These processes are comparable to the ones identified in subject-initial languages, which is consistent with the notion that different languages follow similar universal processing principles.
Collapse
Affiliation(s)
- Gabriela Garrido Rodriguez
- Language and Cognition Department, Max Planck Institute for Psycholinguistics.,Language Development Department, Max Planck Institute for Psycholinguistics.,School of Languages and Linguistics, The University of Melbourne.,ARC Centre of Excellence for the Dynamics of Language, The University of Melbourne
| | | | - Penelope Brown
- Language Development Department, Max Planck Institute for Psycholinguistics
| | - Falk Huettig
- Psychology of Language Department, Max Planck Institute for Psycholinguistics.,Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen.,Centre for Language Studies, Radboud University Nijmegen
| | - Stephen C Levinson
- Language and Cognition Department, Max Planck Institute for Psycholinguistics.,Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen
| |
Collapse
|
5
|
Angulo-Chavira AQ, Castellón-Flores AM, Barrón-Martínez JB, Arias-Trejo N. Word prediction using closely and moderately related verbs in Down syndrome. Front Psychol 2022; 13:934826. [PMID: 36262448 PMCID: PMC9574260 DOI: 10.3389/fpsyg.2022.934826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 09/13/2022] [Indexed: 12/02/2022] Open
Abstract
People with Down syndrome (DS) have several difficulties in language learning, and one of the areas most affected is language production. Theoretical frameworks argue that prediction depends on the production system. Yet, people with DS can predict upcoming nouns using semantically related verbs. Possibly, prediction skills in people with DS are driven by their associative mechanism rather than by the prediction mechanism based on the production system. This study explores prediction mechanisms in people with DS and their relationship with production skills. Three groups were evaluated in a preferential-looking task: young adults, children with DS, and a typically developing control group paired by sex and mental age. Participants saw two images, a target and a distractor. They also heard a sentence in one of the three conditions: with a verb that was closely related to the object (e.g., "The woman read the book"), with a verb that was moderately related to the object (e.g., "My uncle waited for the bus"), or with a verb that was unrelated to the object (e.g., "My sister threw a broom"). Their productive vocabulary was then measured. In the young adult and typically developing groups, the results showed prediction in sentences with highly and moderately related verbs. Participants with DS, however, showed prediction skills only in the highly related context. There was no influence of chronological age, mental age, or production on prediction skills. These results indicate that people with DS base prediction mainly on associative mechanisms and they have difficulty in generating top-down predictions.
Collapse
Affiliation(s)
- Armando Q. Angulo-Chavira
- Laboratorio de Psicolingüística, Facultad de Psicología, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Alejandra M. Castellón-Flores
- Laboratorio de Psicolingüística, Facultad de Psicología, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Julia B. Barrón-Martínez
- Facultad de Estudios Superiores Zaragoza, Universidad Nacional Autónoma de México, Mexico City, Mexico
| | - Natalia Arias-Trejo
- Laboratorio de Psicolingüística, Facultad de Psicología, Universidad Nacional Autónoma de México, Mexico City, Mexico
| |
Collapse
|
6
|
Developmental changes in how children generalize from their experience to support predictive linguistic processing. J Exp Child Psychol 2022; 219:105349. [DOI: 10.1016/j.jecp.2021.105349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 11/20/2021] [Accepted: 12/06/2021] [Indexed: 11/22/2022]
|
7
|
Ito A, Sakai H. Everyday Language Exposure Shapes Prediction of Specific Words in Listening Comprehension: A Visual World Eye-Tracking Study. Front Psychol 2021; 12:607474. [PMID: 33633638 PMCID: PMC7899992 DOI: 10.3389/fpsyg.2021.607474] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Accepted: 01/18/2021] [Indexed: 11/21/2022] Open
Abstract
We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g., /sakana/; fish), an orthographic competitor (e.g., /tuno/; horn), a phonological competitor (e.g., /sakura/; cherry blossom), or an unrelated word (e.g., /hon/; book). The three other objects were distractors. The Tokyo group fixated the target and the orthographic competitor over the unrelated objects before the target word was mentioned, suggesting that they pre-activated the orthographic form of the target word. The Berlin group showed a weaker bias toward the target than the Tokyo group, and they showed a tendency to fixate the orthographic competitor only when the orthographic similarity was very high. Thus, prediction effects were weaker in the Berlin group than in the Tokyo group. We found no evidence for the prediction of phonological information. The obtained group differences support probabilistic models of prediction, which regard the built-up language experience as a basis of prediction.
Collapse
Affiliation(s)
- Aine Ito
- Department of German Studies and Linguistics, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Hiromu Sakai
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan
| |
Collapse
|
8
|
Literacy Advantages Beyond Reading: Prediction of Spoken Language. Trends Cogn Sci 2020; 23:464-475. [PMID: 31097411 DOI: 10.1016/j.tics.2019.03.008] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Revised: 03/28/2019] [Accepted: 03/28/2019] [Indexed: 01/28/2023]
Abstract
Literacy has many obvious benefits: it exposes the reader to a wealth of new information and enhances syntactic knowledge. However, we argue that literacy has an additional, often overlooked, benefit: it enhances people's ability to predict spoken language thereby aiding comprehension. Readers are under pressure to process information more quickly than listeners and reading provides excellent conditions - in particular, a stable environment - for training the predictive system. It also leads to increased awareness of words as linguistic units and to more fine-grained phonological and additional orthographic representations, which sharpen lexical representations and facilitate the retrieval of predicted representations. Thus, reading trains core processes and representations involved in language prediction that are common to both reading and listening.
Collapse
|
9
|
Rommers J, Dell GS, Benjamin AS. Word predictability blurs the lines between production and comprehension: Evidence from the production effect in memory. Cognition 2020; 198:104206. [DOI: 10.1016/j.cognition.2020.104206] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 12/02/2019] [Accepted: 01/26/2020] [Indexed: 11/16/2022]
|
10
|
Evaluating information-theoretic measures of word prediction in naturalistic sentence reading. Neuropsychologia 2019; 134:107198. [DOI: 10.1016/j.neuropsychologia.2019.107198] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 08/09/2019] [Accepted: 09/19/2019] [Indexed: 11/18/2022]
|
11
|
James AN, Fraundorf SH, Lee EK, Watson DG. Individual differences in syntactic processing: Is there evidence for reader-text interactions? JOURNAL OF MEMORY AND LANGUAGE 2018; 102:155-181. [PMID: 30713367 PMCID: PMC6350810 DOI: 10.1016/j.jml.2018.05.006] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
There remains little consensus about whether there exist meaningful individual differences in syntactic processing and, if so, what explains them. We argue that this partially reflects the fact that few psycholinguistic studies of individual differences include multiple constructs, multiple measures per construct, or tests for reliable measures. Here, we replicated three major syntactic phenomena in the psycholinguistic literature: use of verb distributional statistics, difficulty of object-versus subject-extracted relative clauses, and resolution of relative clause attachment ambiguities. We examine whether any individual differences in these phenomena could be predicted by language experience or general cognitive abilities (phonological ability, verbal working memory capacity, inhibitory control, perceptual speed). We find correlations between individual differences and offline, but not online, syntactic phenomena. Condition effects on reading time were not consistent within individuals, limiting their ability to correlate with other measures. We suggest that this might explain controversy over individual differences in language processing.
Collapse
Affiliation(s)
- Ariel N. James
- University of Illinois at Urbana-Champaign, Department of Psychology, 603 East Daniel Street, Champaign, IL 61820
| | - Scott H. Fraundorf
- University of Pittsburgh, Department of Psychology and Learning Research and Development Center, 608 Learning Research and Development Center, 3939 O’Hara Street, Pittsburgh, PA 15260
| | - Eun-Kyung Lee
- Yonsei University, Department of English Language and Literature, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea
| | - Duane G. Watson
- Vanderbilt University, Department of Psychology and Human Development, Peabody College #552, 230 Appleton Place, Nashville, TN 37203-5721
| |
Collapse
|
12
|
Saryazdi R, Chambers CG. Mapping language to visual referents: Does the degree of image realism matter? Acta Psychol (Amst) 2018; 182:91-99. [PMID: 29154035 DOI: 10.1016/j.actpsy.2017.11.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2017] [Revised: 10/10/2017] [Accepted: 11/04/2017] [Indexed: 11/28/2022] Open
Abstract
Studies of real-time spoken language comprehension have shown that listeners rapidly map unfolding speech to available referents in the immediate visual environment. This has been explored using various kinds of 2-dimensional (2D) stimuli, with convenience or availability typically motivating the choice of a particular image type. However, work in other areas has suggested that certain cognitive processes are sensitive to the level of realism in 2D representations. The present study examined the process of mapping language to depictions of objects that are more or less realistic, namely photographs versus clipart images. A custom stimulus set was first created by generating clipart images directly from photographs of real objects. Two visual world experiments were then conducted, varying whether referent identification was driven by noun or verb information. A modest benefit for clipart stimuli was observed during real-time processing, but only for noun-driving mappings. The results are discussed in terms of their implications for studies of visually situated language processing.
Collapse
|
13
|
Lieberman AM, Borovsky A, Mayberry RI. Prediction in a visual language: real-time sentence processing in American Sign Language across development. LANGUAGE, COGNITION AND NEUROSCIENCE 2017; 33:387-401. [PMID: 29687014 PMCID: PMC5909983 DOI: 10.1080/23273798.2017.1411961] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/20/2017] [Accepted: 11/17/2017] [Indexed: 06/08/2023]
Abstract
Prediction during sign language comprehension may enable signers to integrate linguistic and non-linguistic information within the visual modality. In two eyetracking experiments, we investigated American Sign language (ASL) semantic prediction in deaf adults and children (aged 4-8 years). Participants viewed ASL sentences in a visual world paradigm in which the sentence-initial verb was either neutral or constrained relative to the sentence-final target noun. Adults and children made anticipatory looks to the target picture before the onset of the target noun in the constrained condition only, showing evidence for semantic prediction. Crucially, signers alternated gaze between the stimulus sign and the target picture only when the sentential object could be predicted from the verb. Signers therefore engage in prediction by optimizing visual attention between divided linguistic and referential signals. These patterns suggest that prediction is a modality-independent process, and theoretical implications are discussed.
Collapse
Affiliation(s)
- Amy M Lieberman
- Boston University, School of Education, 2 Silber Way, Boston, MA 02215, ,
| | - Arielle Borovsky
- Purdue University, Speech, Language, and Hearing Sciences, 715 Clinic Dr, West Lafayette, IN 47907, ,
| | - Rachel I Mayberry
- University of California, San Diego, Department of Linguistics, 9500 Gilman Drive, #0108, La Jolla, CA 92093-0108, ,
| |
Collapse
|
14
|
de Groot F, Huettig F, Olivers CNL. Revisiting the looking at nothing phenomenon: Visual and semantic biases in memory search. VISUAL COGNITION 2016. [DOI: 10.1080/13506285.2016.1221013] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
15
|
Mani N, Daum MM, Huettig F. “Proactive” in many ways: Developmental evidence for a dynamic pluralistic approach to prediction. Q J Exp Psychol (Hove) 2016; 69:2189-201. [DOI: 10.1080/17470218.2015.1111395] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
The anticipation of the forthcoming behaviour of social interaction partners is a useful ability supporting interaction and communication between social partners. Associations and prediction based on the production system (in line with views that listeners use the production system covertly to anticipate what the other person might be likely to say) are two potential factors, which have been proposed to be involved in anticipatory language processing. We examined the influence of both factors on the degree to which listeners predict upcoming linguistic input. Are listeners more likely to predict book as an appropriate continuation of the sentence “The boy reads a”, based on the strength of the association between the words read and book (strong association) and read and letter (weak association)? Do more proficient producers predict more? What is the interplay of these two influences on prediction? The results suggest that associations influence language-mediated anticipatory eye gaze in two-year-olds and adults only when two thematically appropriate target objects compete for overt attention but not when these objects are presented separately. Furthermore, children's prediction abilities are strongly related to their language production skills when appropriate target objects are presented separately but not when presented together. Both influences on prediction in language processing thus appear to be context dependent. We conclude that multiple factors simultaneously influence listeners’ anticipation of upcoming linguistic input and that only such a dynamic approach to prediction can capture listeners’ prowess at predictive language processing.
Collapse
Affiliation(s)
- Nivedita Mani
- Psychology of Language Research Group, University of Goettingen, Goettingen, Germany
| | - Moritz M Daum
- Department of Psychology, University of Zurich, Zurich, Switzerland
| | - Falk Huettig
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands
- Department of Psychology of Language, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| |
Collapse
|
16
|
Rommers J, Dickson DS, Norton JJS, Wlotko EW, Federmeier KD. Alpha and theta band dynamics related to sentential constraint and word expectancy. LANGUAGE, COGNITION AND NEUROSCIENCE 2016; 32:576-589. [PMID: 28761896 PMCID: PMC5533299 DOI: 10.1080/23273798.2016.1183799] [Citation(s) in RCA: 65] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2015] [Accepted: 04/15/2016] [Indexed: 05/25/2023]
Abstract
Despite strong evidence for prediction during language comprehension, the underlying mechanisms, and the extent to which they are specific to language, remain unclear. Re-analyzing an ERP study, we examined responses in the time-frequency domain to expected and unexpected (but plausible) words in strongly and weakly constraining sentences, and found results similar to those reported in nonverbal domains. Relative to expected words, unexpected words elicited an increase in the theta band (4-7 Hz) in strongly constraining contexts, suggesting the involvement of control processes to deal with the consequences of having a prediction disconfirmed. Prior to critical word onset, strongly constraining sentences exhibited a decrease in the alpha band (8-12 Hz) relative to weakly constraining sentences, suggesting that comprehenders can take advantage of predictive sentence contexts to prepare for the input. The results suggest that the brain recruits domain-general preparation and control mechanisms when making and assessing predictions during sentence comprehension.
Collapse
Affiliation(s)
- Joost Rommers
- Department of Psychology, University of Illinois, Urbana-Champaign, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana-Champaign, USA
| | | | - James J. S. Norton
- Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana-Champaign, USA
- Program in Neuroscience, University of Illinois, Urbana-Champaign, USA
| | - Edward W. Wlotko
- Sackler School of Graduate Biomedical Sciences, Tufts University, Boston, USA
| | - Kara D. Federmeier
- Department of Psychology, University of Illinois, Urbana-Champaign, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana-Champaign, USA
- Program in Neuroscience, University of Illinois, Urbana-Champaign, USA
| |
Collapse
|
17
|
Farmer TA, Fine AB, Misyak JB, Christiansen MH. Reading span task performance, linguistic experience, and the processing of unexpected syntactic events. Q J Exp Psychol (Hove) 2016; 70:413-433. [PMID: 26652283 DOI: 10.1080/17470218.2015.1131310] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Accounts of individual differences in online language processing ability often focus on the explanatory utility of verbal working memory, as measured by reading span tasks. Although variability in reading span task performance likely reflects individual differences in multiple underlying traits, skills, and processes, accumulating evidence suggests that reading span scores also reflect variability in the linguistic experiences of an individual. Here, through an individual differences approach, we first demonstrate that reading span scores correlate significantly with measures of the amount of experience an individual has had with written language (gauged by measures that provide "proxy estimates" of print exposure). We then explore the relationship between reading span scores and online language processing ability. Individuals with higher reading spans demonstrated greater sensitivity to violations of statistical regularities found in natural language-as evinced by higher reading times (RTs) on the disambiguating region of garden-path sentences-relative to their lower span counterparts. This result held after statistically controlling for individual differences in a non-linguistic operation span task. Taken together, these results suggest that accounts of individual differences in sentence processing can benefit from a stronger focus on experiential factors, especially when considered in relation to variability in perceptual and learning abilities that influence the amount of benefit gleaned from such experience.
Collapse
Affiliation(s)
- Thomas A Farmer
- a Department of Psychological and Brain Sciences , University of Iowa , Iowa City , IA , USA.,b Department of Linguistics , University of Iowa , Iowa City , IA , USA
| | - Alex B Fine
- c Department of Psychology , The Hebrew University of Jerusalem , Jerusalem , Israel
| | - Jennifer B Misyak
- d Behavioural Science Group, Warwick Business School , University of Warwick , Coventry , UK
| | | |
Collapse
|
18
|
Bobb SC, Huettig F, Mani N. Predicting visual information during sentence processing: Toddlers activate an object's shape before it is mentioned. J Exp Child Psychol 2015; 151:51-64. [PMID: 26687440 DOI: 10.1016/j.jecp.2015.11.002] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2015] [Revised: 10/30/2015] [Accepted: 11/03/2015] [Indexed: 11/29/2022]
Abstract
We examined the contents of language-mediated prediction in toddlers by investigating the extent to which toddlers are sensitive to visual shape representations of upcoming words. Previous studies with adults suggest limits to the degree to which information about the visual form of a referent is predicted during language comprehension in low constraint sentences. Toddlers (30-month-olds) heard either contextually constraining sentences or contextually neutral sentences as they viewed images that were either identical or shape-related to the heard target label. We observed that toddlers activate shape information of upcoming linguistic input in contextually constraining semantic contexts; hearing a sentence context that was predictive of the target word activated perceptual information that subsequently influenced visual attention toward shape-related targets. Our findings suggest that visual shape is central to predictive language processing in toddlers.
Collapse
Affiliation(s)
- Susan C Bobb
- Department of Psychology, Gordon College, Wenham, MA 01984, USA.
| | - Falk Huettig
- Max Planck Institute for Psycholinguistics, 6525 XD Nijmegen, The Netherlands; Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen, 6525 EZ Nijmegen, The Netherlands
| | - Nivedita Mani
- "Psychology of Language" Research Group, University of Göttingen, D-37073 Göttingen, Germany
| |
Collapse
|
19
|
de Groot F, Koelewijn T, Huettig F, Olivers CN. A stimulus set of words and pictures matched for visual and semantic similarity. JOURNAL OF COGNITIVE PSYCHOLOGY 2015. [DOI: 10.1080/20445911.2015.1101119] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
20
|
Huettig F. Four central questions about prediction in language processing. Brain Res 2015; 1626:118-35. [PMID: 25708148 DOI: 10.1016/j.brainres.2015.02.014] [Citation(s) in RCA: 143] [Impact Index Per Article: 15.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2014] [Revised: 02/05/2015] [Accepted: 02/07/2015] [Indexed: 11/19/2022]
Abstract
The notion that prediction is a fundamental principle of human information processing has been en vogue over recent years. The investigation of language processing may be particularly illuminating for testing this claim. Linguists traditionally have argued prediction plays only a minor role during language understanding because of the vast possibilities available to the language user as each word is encountered. In the present review I consider four central questions of anticipatory language processing: Why (i.e. what is the function of prediction in language processing)? What (i.e. what are the cues used to predict up-coming linguistic information and what type of representations are predicted)? How (what mechanisms are involved in predictive language processing and what is the role of possible mediating factors such as working memory)? When (i.e. do individuals always predict up-coming input during language processing)? I propose that prediction occurs via a set of diverse PACS (production-, association-, combinatorial-, and simulation-based prediction) mechanisms which are minimally required for a comprehensive account of predictive language processing. Models of anticipatory language processing must be revised to take multiple mechanisms, mediating factors, and situational context into account. Finally, I conjecture that the evidence considered here is consistent with the notion that prediction is an important aspect but not a fundamental principle of language processing. This article is part of a Special Issue entitled SI: Prediction and Attention.
Collapse
Affiliation(s)
- Falk Huettig
- Max Planck Institute for Psycholinguistics, P.O. Box 310, 6500 AH Nijmegen, The Netherlands; Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, The Netherlands.
| |
Collapse
|