Neural bases of event knowledge and syntax integration in comprehension of complex sentences

Psycholinguistic mechanisms of classifier processing in sign language

Nonsigners viewing sign language are sometimes able to guess the meaning of signs by relying on the overt connection between form and meaning, or iconicity (cf. Ortega, Özyürek, & Peeters, 2020; Strickland et al., 2015). One word class in sign languages that appears to be highly iconic is classifiers: verb-like signs that can refer to location change or handling. Classifier use and meaning are governed by linguistic rules, yet in comparison with lexical verb signs, classifiers are highly variable in their morpho-phonology (variety of potential handshapes and motion direction within the sign). These open-class linguistic items in sign languages prompt a question about the mechanisms of their processing: Are they part of a gestural-semiotic system (processed like the gestures of nonsigners), or are they processed as linguistic verbs? To examine the psychological mechanisms of classifier comprehension, we recorded the electroencephalogram (EEG) activity of signers who watched videos of signed sentences with classifiers. We manipulated the sentence word order of the stimuli (subject-object-verb [SOV] vs. object-subject-verb [OSV]), contrasting the two conditions, which, according to different processing hypotheses, should incur increased processing costs for OSV orders. As previously reported for lexical signs, we observed an N400 effect for OSV compared with SOV, reflecting increased cognitive load for linguistic processing. These findings support the hypothesis that classifiers are a linguistic part of speech in sign language, extending the current understanding of processing mechanisms at the interface of linguistic form and meaning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Linguistic Information in Auditory Dynamic Events Contributes to the Detection of Fine, Not Coarse Event Boundaries

Human observers (comprehenders) segment dynamic information into discrete events. That is, although there is continuous sensory information, comprehenders perceive boundaries between two meaningful units of information. In narrative comprehension, comprehenders use linguistic, non-linguistic , and physical cues for this event boundary perception. Yet, it is an open question - both from a theoretical and an empirical perspective - how linguistic and non-linguistic cues contribute to this process. The current study explores how linguistic cues contribute to the participants' ability to segment continuous auditory information into discrete, hierarchically structured events. Native speakers of German and non-native speakers, who neither spoke nor understood German, segmented a German audio drama into coarse and fine events. Whereas native participants could make use of linguistic, non-linguistic, and physical cues for segmentation, non-native participants could only use non-linguistic and physical cues. We analyzed segmentation behavior in terms of the ability to identify coarse and fine event boundaries and the resulting hierarchical structure. Non-native listeners identified almost identical coarse event boundaries as native listeners, but missed some of the fine event boundaries identified by the native listeners. Interestingly, hierarchical event perception (as measured by hierarchical alignment and enclosure) was comparable for native and non-native participants. In summary, linguistic cues contributed particularly to the identification of certain fine event boundaries. The results are discussed with regard to the current theories of event cognition.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.

Subject preference emerges as cross-modal strategy for linguistic processing

Research on spoken languages has identified a "subject preference" processing strategy for tackling input that is syntactically ambiguous as to whether a sentence-initial NP is a subject or object. The present study documents that the "subject preference" strategy is also seen in the processing of a sign language, supporting the hypothesis that the "subject"-first strategy is universal and not dependent on the language modality (spoken vs. signed). Deaf signers of Austrian Sign Language (ÖGS) were shown videos of locally ambiguous signed sentences in SOV and OSV word orders. Electroencephalogram (EEG) data indicated higher cognitive load in response to OSV stimuli (i.e. a negativity for OSV compared to SOV), indicative of syntactic reanalysis cost. A finding that is specific to the visual modality is that the ERP (event-related potential) effect reflecting linguistic reanalysis occurred earlier than might have been expected, that is, before the time point when the path movement of the disambiguating sign was visible. We suggest that in the visual modality, transitional movement of the articulators prior to the disambiguating verb position or co-occurring non-manual (face/body) markings were used in resolving the local ambiguity in ÖGS. Thus, whereas the processing strategy of "subject preference" is cross-modal at the linguistic level, the cues that enable the processor to apply that strategy differ in signing as compared to speech.

Neural bases of syntax-semantics interface processing

The binding problem-question of how information between the modules of the linguistic system is integrated during language processing-is as yet unresolved. The remarkable speed of language processing and comprehension (Pulvermüller et al. 2009) suggests that at least coarse semantic information (e.g. noun animacy) and syntactically-relevant information (e.g. verbal template) are integrated rapidly to allow for coarse comprehension. This EEG study investigated syntax-semantics interface processing during word-by-word sentence reading. As alpha-band neural activity serves as an inhibition mechanism for local networks, we used topographical distribution of alpha power to help identify the timecourse of the binding process. We manipulated the syntactic parameter of verbal event structure, and semantic parameter of noun animacy in reduced relative clauses (RRCs, e.g. "The witness/mansion seized/protected by the agent was in danger"), to investigate the neural bases of interaction between syntactic and semantic networks during sentence processing. The word-by-word stimulus presentation method in the present experiment required manipulation of both syntactic structure and semantic features in the working memory. The results demonstrated a gradient distribution of early components (biphasic posterior P1-N2 and anterior N1-P2) over function words "by" and "the", and the verb, corresponding to facilitation or conflict resulting from the syntactic (telicity) and semantic (animacy) cues in the preceding portion of the sentence. This was followed by assimilation of power distribution in the α band at the second noun. The flattened distribution of α power during the mental manipulation with high demand on working memory-thematic role re-assignment-demonstrates a state of α equilibrium with strong functional coupling between posterior and anterior regions. These results demonstrate that the processing of semantic and syntactic features during sentence comprehension proceeds in highly integrated fashion using gating of attentional resources to facilitate rapid comprehension, with attentional suppression of global alpha power to facilitate interaction of local networks.