Using the Visual World Paradigm to Study Retrieval Interference in Spoken Language Comprehension

It's about time! Time as a parameter for lexical and syntactic processing: an eye-tracking-while-listening investigation

We examined the time-course of lexical activation, deactivation, and the syntactic operation of dependency linking during the online processing of object-relative sentence constructions using eye-tracking-while-listening. We explored how manipulating temporal aspects of the language input affects the tight lexical and syntactic temporal constraints found in sentence processing. The three temporal manipulations were (1) increasing the duration of the direct object noun, (2) adding the disfluency uh after the noun, and (3) replacing the disfluency with a silent pause. The findings from this experiment revealed that the disfluent and silence temporal manipulations enhanced the processing of subject and object noun phrases by modulating activation and deactivation. The manipulations also changed the time-course of dependency linking (increased reactivation of the direct object). The modulated activation dynamics of these lexical items are thought to play a role in mitigating interference and suggest that deactivation plays a beneficial role in complex sentence processing.

Pronouns Are as Sensitive to Structural Constraints as Reflexives in Early Processing: Evidence From Visual World Paradigm Eye-Tracking

A number of studies in the extant literature report findings that suggest asymmetry in the way reflexive and pronoun anaphors are interpreted in the early stages of processing: that pronouns are less sensitive to structural constraints, as formulated by Binding Theory, than reflexives, in the initial antecedent retrieval process. However, in previous visual world paradigm eye-tracking studies, these conclusions were based on sentences that placed the critical anaphors within picture noun phrases or prepositional phrases, which have independently been shown not to neatly conform to the Binding Theory principles. We present results from a visual world paradigm eye-tracking experiment that show that when critical anaphors are placed in the indirect object position immediately following a verb as a recipient argument, pronoun and reflexive processing are equally sensitive to structural constraints.

Language-driven anticipatory eye movements in virtual reality

Predictive language processing is often studied by measuring eye movements as participants look at objects on a computer screen while they listen to spoken sentences. This variant of the visual-world paradigm has revealed that information encountered by a listener at a spoken verb can give rise to anticipatory eye movements to a target object, which is taken to indicate that people predict upcoming words. The ecological validity of such findings remains questionable, however, because these computer experiments used two-dimensional stimuli that were mere abstractions of real-world objects. Here we present a visual-world paradigm study in a three-dimensional (3-D) immersive virtual reality environment. Despite significant changes in the stimulus materials and the different mode of stimulus presentation, language-mediated anticipatory eye movements were still observed. These findings thus indicate that people do predict upcoming words during language comprehension in a more naturalistic setting where natural depth cues are preserved. Moreover, the results confirm the feasibility of using eyetracking in rich and multimodal 3-D virtual environments.

Vocabulary size and Native Speaker self-identification influence flexibility in linguistic prediction among adult bilinguals

When language users predict upcoming speech, they generate pluralistic expectations, weighted by likelihood (Kuperberg & Jaeger, 2016). Many variables influence the prediction of highly-likely sentential outcomes, but less is known regarding variables affecting the prediction of less-likely outcomes. Here we explore how English vocabulary size and self-identification as a Native Speaker (NS) of English modulate adult bi-/multilinguals' pre-activation of less-likely sentential outcomes in two visual-world experiments. Participants heard transitive sentences containing an agent, action and theme (The pirate chases the ship) while viewing four referents varying in expectancy by relation to the agent and action. In experiment 1 (N=70), spoken themes referred to highly-expected items (e.g., ship). Results indicate lower-skill (smaller vocabulary size) and less confident (not identifying as NS) bi-/multilinguals activate less-likely action-related referents more than their higher-skill/confidence peers. In experiment 2 (N=65), themes were one of two less-likely items (The pirate chases the bone/cat). Results approaching significance indicate an opposite but similar size effect: higher-skill/confidence listeners activate less-likely action-related (e.g., bone) referents slightly more than lower-skill/confidence listeners. Results across experiments suggest higher-skill/confidence participants more flexibly modulate their linguistic predictions per the demands of the task, with similar but not identical patterns emerging when bi-/multilinguals are grouped by self-ascribed NS-status versus vocabulary size.

Retrieval and Encoding Interference: Cross-Linguistic Evidence from Anaphor Processing

The main goal of this paper was to disentangle encoding and retrieval interference effects in anaphor processing and thus to evaluate the hypothesis predicting that structurally inaccessible nouns (distractors) are not considered to be potential anaphor antecedents during language processing (Nicol and Swinney, 1989). Three self-paced reading experiments were conducted: one in German, comparing gender-unmarked reflexives and gender-marked pronouns, and two in Russian, comparing gender-marked and -unmarked reflexives. In the German experiment, no interference effects were found. In the first experiment in Russian, an unexpected reading times pattern emerged: in the condition where the distractor matched the gender of the reflexive's antecedent, reading of the gender-unmarked, but not the gender-marked reflexives was slowed down. The same reading times pattern was replicated in a second experiment in Russian where the order of the reflexive and the main verb was inverted. We conclude that the results of the two experiments in Russian are inconsistent with the retrieval interference account, but can be explained by encoding interference and additional semantic processing efforts associated with the processing of gender-marked reflexives. In sum, we found no evidence that would allow us to reject the syntax as an early filer account (Nicol and Swinney, 1989).