Predictive processing of syntactic structure: Sluicing and ellipsis in real-time sentence processing

Incremental structure building in the processing of ellipsis

This article presents the results of two experiments conducted to examine how ellipsis sites are processed during the processing of backward sluicing, which is superficially similar to non-sluicing wh-filler-gap dependencies. Previous studies on long-distance wh-filler-gap dependencies established that the processing of these dependencies is sensitive to the syntactic structure of materials within the dependency: CP vs. NP. Results from two maze experiments show that backward sluicing processing is sensitive to the same structural factors, confirming that the same processing mechanism underlies both constructions. We suggest that an active search mechanism is operating at the core for these structures and with the interaction of the ellipsis-specific mechanism, e.g., a word-by-word copying mechanism, the parser builds antecedent structure within the ellipsis site incrementally during the processing of backward sluicing.

Online revision process in clause-boundary garden-path sentences

A long-standing question in sentence processing research concerns the online parsing process in clause-boundary garden-path sentences, such as After Mary dressed John bathed. In this sentence, "John" must be parsed as the matrix subject DP but can be locally analysed as the object of the embedded verb. There is considerable evidence that the parser misanalyses these garden-path sentences. However, the controversy lies in whether the parser revises them during the online parsing process. The present study investigated this revision process through two self-paced reading experiments utilising grammatical constraints on reflexives and subject or object relative clauses embedded within the locally ambiguous DP. The results provided evidence of revision when a subject relative clause was embedded but not when an object relative clause was embedded. These findings suggest that the parser assigns grammatical structures that correspond to input strings during the revision of clause-boundary ambiguities but that object relative clauses affect the online revision process.

Predictive structure building in language comprehension: a large sample study on incremental licensing and parallelism

In online language comprehension, the parser incrementally builds hierarchical syntactic structures. The predictive nature of this structure-building process has been the subject of extensive debate. A previous study observed that when a wh-phrase indicates parallelism between the upcoming wh-clause and a preceding clause (e.g., John told some stories, but we couldn't remember which stories…), the parser predictively constructs the wh-clause. This observation demonstrates predictive structure building. However, the study also suggests that the parser does not make a prediction when the wh-phrase indicates that parallelism does not hold (e.g., John told some stories … with which stories…), a potential limit to the prediction of syntactic structures. Crucially, these findings are controversial because the study did not observe processing difficulty when disambiguating input indicated that the predicted continuation was inconsistent with the globally grammatical structure (garden-path effects). The controversial results may be due to a lack of statistical power. Therefore, the present study conducted a large-scale replication study (324 participants and 24 sets of materials). The results revealed that the parser predicts the clausal structure, irrespective of the type of wh-phrase. There was also evidence of garden-path effects, supporting the finding that the parser makes a prediction. These observations suggest that the prediction algorithm inherent in the human parser is more powerful than assumed by the previous study and that the parser attempts to construct globally grammatical structures during revision.

Predictive Processing in Sign Languages: A Systematic Review

The objective of this article was to review existing research to assess the evidence for predictive processing (PP) in sign language, the conditions under which it occurs, and the effects of language mastery (sign language as a first language, sign language as a second language, bimodal bilingualism) on the neural bases of PP. This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework. We searched peer-reviewed electronic databases (SCOPUS, Web of Science, PubMed, ScienceDirect, and EBSCO host) and gray literature (dissertations in ProQuest). We also searched the reference lists of records selected for the review and forward citations to identify all relevant publications. We searched for records based on five criteria (original work, peer-reviewed, published in English, research topic related to PP or neural entrainment, and human sign language processing). To reduce the risk of bias, the remaining two authors with expertise in sign language processing and a variety of research methods reviewed the results. Disagreements were resolved through extensive discussion. In the final review, 7 records were included, of which 5 were published articles and 2 were dissertations. The reviewed records provide evidence for PP in signing populations, although the underlying mechanism in the visual modality is not clear. The reviewed studies addressed the motor simulation proposals, neural basis of PP, as well as the development of PP. All studies used dynamic sign stimuli. Most of the studies focused on semantic prediction. The question of the mechanism for the interaction between one’s sign language competence (L1 vs. L2 vs. bimodal bilingual) and PP in the manual-visual modality remains unclear, primarily due to the scarcity of participants with varying degrees of language dominance. There is a paucity of evidence for PP in sign languages, especially for frequency-based, phonetic (articulatory), and syntactic prediction. However, studies published to date indicate that Deaf native/native-like L1 signers predict linguistic information during sign language processing, suggesting that PP is an amodal property of language processing.

Processing gapping: Parallelism and grammatical constraints

This study aims to test two hypotheses about the online processing of Gapping: whether the parser inserts an ellipsis site in an incremental fashion in certain coordinated structures (the Incremental Ellipsis Hypothesis), or whether ellipsis is a late and dispreferred option (the Ellipsis as a Last Resort Hypothesis). We employ two offline acceptability rating experiments and a sentence fragment completion experiment to investigate to what extent the distribution of Gapping is controlled by grammatical and extra-grammatical constraints. Furthermore, an eye-tracking while reading experiment demonstrated that the parser inserts an ellipsis site incrementally but only when grammatical and extra-grammatical constraints allow for the insertion of the ellipsis site. This study shows that incremental building of the Gapping structure follows from the parser's general preference to keep the structure of the two conjuncts maximally parallel in a coordination structure as well as from grammatical restrictions on the distribution of Gapping such as the Coordination Constraint.

Looking for a Location: Dissociated Effects of Event-Related Plausibility and Verb-Argument Information on Predictive Processing in Aphasia

PURPOSE

This study examined the influence of verb-argument information and event-related plausibility on prediction of upcoming event locations in people with aphasia, as well as older and younger, neurotypical adults. It investigated how these types of information interact during anticipatory processing and how the ability to take advantage of the different types of information is affected by aphasia.

METHOD

This study used a modified visual-world task to examine eye movements and offline photo selection. Twelve adults with aphasia (aged 54-82 years) as well as 44 young adults (aged 18-31 years) and 18 older adults (aged 50-71 years) participated.

RESULTS

Neurotypical adults used verb argument status and plausibility information to guide both eye gaze (a measure of anticipatory processing) and image selection (a measure of ultimate interpretation). Argument status did not affect the behavior of people with aphasia in either measure. There was only limited evidence of interaction between these 2 factors in eye gaze data.

CONCLUSIONS

Both event-related plausibility and verb-based argument status contributed to anticipatory processing of upcoming event locations among younger and older neurotypical adults. However, event-related likelihood had a much larger role in the performance of people with aphasia than did verb-based knowledge regarding argument structure.

The Electrophysiology of Basic Phrase Building

A defining trait of linguistic competence is the ability to combine elements into increasingly complex structures to denote, and to comprehend, a potentially infinite number of meanings. Recent magnetoencephalography (MEG) work has investigated these processes by comparing the response to nouns in combinatorial (blue car) and non-combinatorial (rnsh car) contexts. In the current study we extended this paradigm using electroencephalography (EEG) to dissociate the role of semantic content from phonological well-formedness (yerl car). We used event-related potential (ERP) recordings in order to better relate the observed neurophysiological correlates of basic combinatorial operations to prior ERP work on comprehension. We found that nouns in combinatorial contexts (blue car) elicited a greater centro-parietal negativity between 180-400ms, independent of the phonological well-formedness of the context word. We discuss the potential relationship between this ‘combinatorial’ effect and classic N400 effects. We also report preliminary evidence for an early anterior negative deflection immediately preceding the critical noun in combinatorial contexts, which we tentatively interpret as an electrophysiological reflex of syntactic structure initialization.

Predicting complex syntactic structure in real time: Processing of negative sentences in Russian

In Russian negative sentences the verb's direct object may appear either in the accusative case, which is licensed by the verb (as is common cross-linguistically), or in the genitive case, which is licensed by the negation (Russian-specific "genitive-of-negation" phenomenon). Such sentences were used to investigate whether case marking is employed for anticipating syntactic structure, and whether lexical heads other than the verb can be predicted on the basis of a case-marked noun phrase. Experiment 1, a completion task, confirmed that genitive-of-negation is part of Russian speakers' active grammatical repertoire. In Experiments 2 and 3, the genitive/accusative case manipulation on the preverbal object led to shorter reading times at the negation and verb in the genitive versus accusative condition. Furthermore, Experiment 3 manipulated linear order of the direct object and the negated verb in order to distinguish whether the abovementioned facilitatory effect was predictive or integrative in nature, and concluded that the parser actively predicts a verb and (otherwise optional) negation on the basis of a preceding genitive-marked object. Similarly to a head-final language, case-marking information on preverbal noun phrases (NPs) is used by the parser to enable incremental structure building in a free-word-order language such as Russian.

Structural Prediction in Aphasia: Evidence from either

Young neurotypical adults engage in prediction during language comprehension (e.g., Altmann & Kamide, 1999; Staub & Clifton, 2006; Yoshida, Dickey & Sturt, 2013). The role of prediction in aphasic comprehension is less clear. Some evidence suggests that lexical prediction may be spared in aphasia (Dickey et al., 2014; Love & Webb, 1977; cf. Mack et al, 2013), and there is even indication that structural prediction may be spared in some people with aphasia (PWA; e.g. Hanne, Burchert, De Bleser, & Vashishth, 2015). The current self-paced reading experiment manipulated the presence of either to examine structural prediction among PWA and a set of similar-aged neurotypical control participants. Consistent with intact structural prediction for both groups of participants, when either preceded a disjunction, reading times were faster on the or and second disjunct (cf. Staub & Clifton, 2006). For neurotypical controls, this effect of the presence vs. absence of either shrank reliably as more experimental items were encountered, whereas for PWA there was a non-significant trend for it to grow as more experimental items were encountered. These findings indicate that PWA and older neurotypical individuals can use a lexical cue to predict the structural form of upcoming material during comprehension, but that on-line adaptation to patterns in the local context may be different for the two groups.

Abstract linguistic structure correlates with temporal activity during naturalistic comprehension

Neurolinguistic accounts of sentence comprehension identify a network of relevant brain regions, but do not detail the information flowing through them. We investigate syntactic information. Does brain activity implicate a computation over hierarchical grammars or does it simply reflect linear order, as in a Markov chain? To address this question, we quantify the cognitive states implied by alternative parsing models. We compare processing-complexity predictions from these states against fMRI timecourses from regions that have been implicated in sentence comprehension. We find that hierarchical grammars independently predict timecourses from left anterior and posterior temporal lobe. Markov models are predictive in these regions and across a broader network that includes the inferior frontal gyrus. These results suggest that while linear effects are wide-spread across the language network, certain areas in the left temporal lobe deal with abstract, hierarchical syntactic representations.

Filling the Silence: Reactivation, not Reconstruction

In a self-paced reading experiment, we investigated the processing of sluicing constructions (“sluices”) whose antecedent contained a known garden-path structure in German. Results showed decreased processing times for sluices with garden-path antecedents as well as a disadvantage for antecedents with non-canonical word order downstream from the ellipsis site. A post-hoc analysis showed the garden-path advantage also to be present in the region right before the ellipsis site. While no existing account of ellipsis processing explicitly predicted the results, we argue that they are best captured by combining a local antecedent mismatch effect with memory trace reactivation through reanalysis.

Processing implicit control: evidence from reading times

Sentences such as “The ship was sunk to collect the insurance” exhibit an unusual form of anaphora, implicit control, where neither anaphor nor antecedent is audible. The non-finite reason clause has an understood subject, PRO, that is anaphoric; here it may be understood as naming the agent of the event of the host clause. Yet since the host is a short passive, this agent is realized by no audible dependent. The putative antecedent to PRO is therefore implicit, which it normally cannot be. What sorts of representations subserve the comprehension of this dependency? Here we present four self-paced reading time studies directed at this question. Previous work showed no processing cost for implicit vs. explicit control, and took this to support the view that PRO is linked syntactically to a silent argument in the passive. We challenge this conclusion by reporting that we also find no processing cost for remote implicit control, as in: “The ship was sunk. The reason was to collect the insurance.” Here the dependency crosses two independent sentences, and so cannot, we argue, be mediated by syntax. Our Experiments 1–4 examined the processing of both implicit (short passive) and explicit (active or long passive) control in both local and remote configurations. Experiments 3 and 4 added either “3 days ago” or “just in order” to the local conditions, to control for the distance between the passive and infinitival verbs, and for the predictability of the reason clause, respectively. We replicate the finding that implicit control does not impose an additional processing cost. But critically we show that remote control does not impose a processing cost either. Reading times at the reason clause were never slower when control was remote. In fact they were always faster. Thus, efficient processing of local implicit control cannot show that implicit control is mediated by syntax; nor, in turn, that there is a silent but grammatically active argument in passives.

Verb gapping: an action-gap compatibility study

This study addresses the processing of verb-gapping sentences, e.g., John closes a juice bottle and Jim [ ] a lemonade bottle. The goal was to explore if there would be an interaction between language comprehension and motor action not only for overt action verbs but also for gapped verbs. Participants read gapping sentences that either described clockwise or counter-clockwise manual rotations (e.g., closes vs. opens a juice bottle). Adopting a paradigm developed by Zwaan and Taylor (2006), sentence presentation was frame-by-frame. Participants proceeded from frame to frame by turning a knob either clockwise or counter clockwise. Analyses of the frame reading-times yielded a significant effect of compatibility between the linguistically conveyed action and the knob turning for the overt-verb (e.g., closes/opens a juice bottle) as well as for the gapped-verb frame (e.g., a lemonade bottle) - with longer reading times in the match condition than in the mismatch condition - but not for any of the other frames (e.g., and Jim). The results are promising in providing novel evidence for the real-time reactivation of gapped verbs and in suggesting that action simulation is not bound to the processing of overt verbs.

An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal.