The effect of linguistic and visual salience in visual world studies

When It's Harder to Ignorar than to Ignore: Evidence of Greater Attentional Capture from a Non-Dominant Language

AIMS AND OBJECTIVES

Imagine you're driving and you become so distracted by the radio that you miss your turn. Which is more likely to have caught your attention, a broadcast in your native tongue or one in your second language? The present study explores the effect of language proficiency on our ability to inhibit irrelevant phonological information.

METHODOLOGY

Participants were asked to identify which of two drawings changed color while ignoring irrelevant words in either their native language, English, or a less proficient language, Spanish. The drawings appeared on screen for either 200 or 2000ms prior to word-onset, which was followed 200ms later by a color-change. On critical trials, the irrelevant word shared phonological features with the label of the non-target drawing. Trials were blocked by preview time and language.

DATA AND ANALYSIS

Reaction time data from 19 bilinguals were analyzed utilizing generalized linear mixed-effects models, with fixed effects of Competition (competitor vs. control), and Language (English vs. Spanish) and random effects for Subject and Item within each preview window.

FINDINGS/CONCLUSIONS

No interference was observed when participants heard their native tongue in either preview condition. However, participants in the long-preview condition were significantly slower to respond when there was phonological competition in their less proficient language, despite the fact that the task required no language processing.

ORIGINALITY

Past work has indicated that languages are processed more automatically and cause greater interference as proficiency increases. We propose that though higher-proficiency languages may receive greater activation overall, lower-proficiency languages may be more likely to exogenously capture attention due to both relatively greater salience, and relatively less control.

SIGNIFICANCE

The present findings have implications for how we understand the dynamic relationship between language proficiency, activation, and inhibition, suggesting that the salience of the less familiar influences our ability to ignore irrelevant information.

Modulation of scene consistency and task demand on language-driven eye movements for audio-visual integration

Previous psycholinguistic studies have demonstrated that people tend to direct fixations toward the visual object to which spoken input refers during language comprehension. However, it is still unclear how the visual scene, especially the semantic consistency between object and background, affects the word-object mapping process during comprehension. Two visual world paradigm experiments were conducted to investigate how the scene consistency dynamically influenced the language-driven eye movements in a speech comprehension and a scene comprehension task. In each trial, participants listened to a spoken sentence while viewing a picture with two critical objects: one is the mentioned target object (e.g., tiger), which was embedded in either a consistent (e.g., field), inconsistent (e.g., sky) or blank background; the other is an unmentioned non-target object (e.g., eagle), which was always consistent with its background. The results showed that the fixation proportion of the inconsistent target was higher than the consistent target, and the task demand can affect the strength and the direction of the inconsistency effect before and after the target had been mentioned. In summary, the spoken language, scene-based knowledge and task demand were intertwined to determine eye movements during audio-visual integration for comprehension.