Not all errors are the same: ERP sensitivity to error typicality in foreign accented speech perception

Neural correlates of listening to nonnative-accented speech in multi-talker background noise

We examined the neural correlates underlying the semantic processing of native- and nonnative-accented sentences, presented in quiet or embedded in multi-talker noise. Implementing a semantic violation paradigm, 36 English monolingual young adults listened to American-accented (native) and Chinese-accented (nonnative) English sentences with or without semantic anomalies, presented in quiet or embedded in multi-talker noise, while EEG was recorded. After hearing each sentence, participants verbally repeated the sentence, which was coded and scored as an offline comprehension accuracy measure. In line with earlier behavioral studies, the negative impact of background noise on sentence repetition accuracy was higher for nonnative-accented than for native-accented sentences. At the neural level, the N400 effect for semantic anomaly was larger for native-accented than for nonnative-accented sentences, and was also larger for sentences presented in quiet than in noise, indicating impaired lexical-semantic access when listening to nonnative-accented speech or sentences embedded in noise. No semantic N400 effect was observed for nonnative-accented sentences presented in noise. Furthermore, the frequency of neural oscillations in the alpha frequency band (an index of online cognitive listening effort) was higher when listening to sentences in noise versus in quiet, but no difference was observed across the accent conditions. Semantic anomalies presented in background noise also elicited higher theta activity, whereas processing nonnative-accented anomalies was associated with decreased theta activity. Taken together, we found that listening to nonnative accents or background noise is associated with processing challenges during online semantic access, leading to decreased comprehension accuracy. However, the underlying cognitive mechanism (e.g., associated listening efforts) might manifest differently across accented speech processing and speech in noise processing.

Would you respect a norm if it sounds foreign? Foreign-accented speech affects decision-making processes

Does listening to a foreign-accented speaker bias native speakers' behavior? We investigated whether the accent, i.e., a foreign accent versus a native accent, in which a social norm is presented affects native speakers' decision to respect the norm (Experiments 1 and 2) and the judgement for not respecting it (Experiment 2). In Experiment 1, we presented 128 native Spanish speakers with new social norms, adapted from the measures imposed by the Spanish Government to fight the Covid-19 pandemic (e.g., 'To avoid the spread of the Covid-19 virus, keep your distance'), whereas in Experiment 2, we presented 240 native Spanish speakers with everyday social norms learned from childhood (e.g., 'Not littering on the street or in public places'), that have an intrinsic cultural and linguistic link. In Experiment 1, the norms were uttered either in a native accent, or in a foreign accent unfamiliar to our participants to avoid stereotypes. In Experiment 2, we added an accent negatively perceived in Spain to assess the role of language attitudes on decision making. Overall, accent did not directly impact participants' final decisions, but it influenced the decision-making process. The factors that seem to underlie this effect are emotionality and language attitudes. These findings add up to the recent Foreign Accent effect observed on moral judgements and further highlight the role of the speaker's identity in decision making.

When A Nonnative Accent Lets You Spot All the Errors: Examining the Syntactic Interlanguage Benefit

In our continuously globalizing world, cross-cultural and cross-linguistic communications are far from exceptional. A wealth of research has indicated that the processing of nonnative-accented speech can be challenging for native listeners, both at the level of phonology (e.g., Munro & Derwing, 1995) and syntax (Caffarra & Martin, 2019). However, few online studies have examined the underpinnings of accented speech recognition from the perspective of the "nonnative listener," even though behavioral studies indicate that accented input may be easier to process for such individuals (i.e., the interlanguage speech intelligibility benefit; Bent & Bradlow, 2003). The current EEG study first examined the phonological and syntactic analysis of nonnative-accented speech among nonnative listeners. As such, 30 English learners of Spanish listened to syntactically correct and incorrect Spanish sentences produced in native and nonnative-accented Spanish. The violation in the incorrect sentences was caused by errors that are typical (i.e., gender errors; *la color) or atypical for English learners of Spanish (i.e., number errors; *los color). Results indicated that nonnative listeners elicit a phonological mismatch negativity (PMN) when attending to speech produced by a native Spanish speaker. Furthermore, the nonnative listeners showed a P600 for all grammatical violations, indicating that they repair all errors regardless of their typicality or the accent in which they are produced. Follow-up analyses compared our novel data to the data of native listeners from the methodologically identical precursor study (Caffarra & Martin, 2019). These analyses showed that native and nonnative listeners exhibit directionally opposite PMN effects; whereas natives exhibited a larger PMN for English-accented Spanish, nonnatives displayed a larger PMN in response to native Spanish utterances (a classic interlanguage speech intelligibility benefit). An additional difference was observed at the syntactic level: Whereas natives repaired only atypical number errors when they were English-accented, nonnative participants exhibited a P600 in response to all English-accented syntactic errors, regardless of their typicality (a syntactic interlanguage speech intelligibility benefit). Altogether, these results suggest that accented speech is not inherently difficult to process; in fact, nonnatives may benefit from the presence of a nonnative accent. Thus, our data provide some of the first electrophysiological evidence supporting the existence of the classic interlanguage speech intelligibility benefit and its novel syntactic counterpart.

Children Treat Grammatical Errors Differently for Native and Non-Native Speakers

Both children and adults demonstrate biases against non-native speakers. However, in some situations, adults act more generously towards non-native speakers than towards native speakers. In particular, adults judge errors from non-native speakers less harshly, presumably because they expect such errors. In the present study, we asked whether 5-6-year-old children place less weight on errors from speakers with a foreign accent. In Experiment 1, 5- and 6-year-old children (N = 80) listened to pairs of either native or foreign-accented speakers (between-subjects) label objects. For native speaker pairings, children preferred information provided by grammatical speakers over information from speakers who made subject-verb agreement errors. In contrast, children chose between foreign-accented speakers at chance. In Experiment 2 (N = 40), children preferred information from grammatical foreign-accented speakers over information from foreign-accented speakers who produced word-order violations. These findings constitute the first demonstration that children treat speech errors differently based on a speaker’s language background.

Interlocutor modelling in comprehending speech from interleaved interlocutors of different dialectic backgrounds

It has been shown that, in language comprehension, listeners model certain attributes of their interlocutor (e.g., dialectic background, age, gender) and interpret speech against that model; for example, they understand cross-dialectally ambiguous words such as flat and gas for their American English (AE) meanings more often when listening to an AE interlocutor than a British English (BE) interlocutor. This study further investigated whether listeners construct concurrent interlocutor models when communicating with interleaved interlocutors of different dialectic backgrounds, and, if they do, how they choose between concurrent models to interpret words. In two experiments, participants heard a word (e.g., flat) spoken by a BE or AE interlocutor and provided a word associate (indicating which meaning of the word was accessed). When different interlocutors were encountered in separate blocks, participants accessed more AE meanings when listening to an AE rather than a BE interlocutor, and the accent effect was not larger for words pronounced more differently in BE and AE (e.g., fall sounds more distinctly British vs. American than flat does). These results suggest that participants constructed an interlocutor model (e.g., of a BE or an AE speaker) and used it (instead of accent details in a word) to guide word meaning access. When interlocutors were interleaved in the same block, we observed a comparable accent effect, which increased as a function of between-accent differences in pronunciation. These results suggest that participants constructed concurrent interlocutor models and used accent details in a word to select the appropriate interlocutor model. We also observed that the accent effect was comparable for two interleaved interlocutors of the same gender (e.g., a female BE interlocutor and a female AE interlocutor) and for two interleaved interlocutors of different genders (e.g., a female BE interlocutor and a male AE interlocutor). These results suggest that participants did not use gender-related voice details for model selection when accent details were sufficient for interlocutor model selection.

Faces with foreign accents: An event-related potential study of accented sentence comprehension

Mounting Event-Related Potential (ERP) research testing the neurocognitive processes of foreign- and native-accented speech comprehension shows that listeners process foreign-accented speech in qualitatively and quantitatively different ways from native-accented speech. Previous ERP research has presented listeners with foreign- and native-accented sentences only, devoid of any cues to the speaker's identity. Additional cues about the foreign-accented speaker identity, such as faces, may be useful in aiding listeners' processing, potentially reducing differences between neurocognitive processes associated with foreign-accented and native-accented speech comprehension. This study tested whether providing listeners with extra-linguistic face cues to the speakers' identity affects the ERP correlates of grammatical and semantic processing of foreign-accented and native-accented sentences. Specifically, we presented listeners with face cues to speaker identity followed by native- and foreign-accented sentences that were identical to Grey and Van Hell (2017), who did not provide face cues. To elucidate the effects of face cues, we compared the present study's Face Cue data with the Grey and Van Hell (2017) No Face Cue data. For grammar processing, the results showed a biphasic Nref-P600 for native-accented speech and a P600 for foreign-accented speech in the Face Cue group. In the No Face Cue group, results for grammar processing showed an Nref for native-accented speech and no significant ERP effects for foreign-accented speech. This result indicates that face cues were effective in aiding processing of foreign-accented sentences. For semantic processing, there were robust N400 effects for native-accented speech and delayed N400s for foreign-accented speech in the Face Cue and No Face Cue groups. This pattern indicates that the extra-linguistic face cue to foreign-accented speaker identity did not affect semantic processing. Overall, the results provide neurocognitive evidence that the integration of speaker and face information aids listeners during the processing of foreign-accented speech, particularly for grammatical processing.

Processing Code-Switches in the Presence of Others: An ERP Study

Code-switching is highly socially constrained. For instance, code-switching is only felicitous when those present are fluent in both languages. This means that bilinguals need to dynamically adjust their language control and expectation of code-switching to the current social situation or context. The aim of the present EEG study was to investigate how and when language control in the comprehension of code-switches is affected by the assumed language knowledge of others in the context. Spanish-English bilinguals read sentences with and without code-switches together with another Spanish-English bilingual or with an English monolingual. Switches elicited an early fronto-central positivity. This effect was smaller overall when a bilingual was present at the start of the study. In addition, the late positive complex found for switches was smaller when a bilingual was present rather than a monolingual, but only for those participants who were sensitive to the other's language knowledge in their off-line judgments. These findings suggest that the bilinguals in our study expected and activated both languages when initially paired with a bilingual and that they more easily accommodated code-switches, in the presence of a bilingual than in the presence of a monolingual. Our findings support the view that language control can be modulated by the perceived language knowledge of others present, and are compatible with a dynamic control model of bilingual language comprehension.

Who are you talking to? The role of addressee identity in utterance comprehension

Experimental evidence suggests that speaker and addressee quickly adapt to each other from the earliest moments of sentence processing, and that interlocutor-related information is rapidly integrated with other sources of nonpragmatic information (e.g., semantic, morphosyntactic, etc.). These findings have been taken as support for one-step models of sentence comprehension. The results from the present event-related potential study challenge this theoretical framework providing a case where discourse level information is integrated only at a late stage of processing, when morphosyntactic analysis has been already initiated. We considered the case of Basque allocutive agreement, where information about addressee gender is encoded in verbal inflection. Two different types of Basque grammatical violations were presented together with the corresponding control conditions: one could be detected based on a morphosyntactic mismatch (person agreement violation), while the other could be detected only if the addressee's gender was considered (allocutive violation). Morphosyntactic violations elicited greater N400 effects followed by P600 effects, while allocutive violations elicited only P600 effects. These results provide new constraints to one-step accounts as they represent a case where speakers do not immediately adjust to the addressee's perspective. We propose that the relevance of discourse-level information might be a crucial variable to reconcile the dichotomy between one- and two-step models.

The impact of foreign accent on irony interpretation

In modern multi-cultural societies, conversations between foreign speakers and native listeners have become very common. These exchanges often include the use of figurative language. The present study examines, for the first time, whether native listeners’ non-literal interpretation of discourse is influenced by indexical cues such as speaker accent. Native listeners were presented with ironic and literal Spanish stories uttered in a native or foreign accent (Spanish and British English accents, respectively). Two types of irony were considered: ironic criticism (frequently used) and ironic praise (less frequently used). Participants were asked to rate stories on their level of irony. Results showed an impact of foreign accent on natives’ non-literal interpretation. The effect was evident in the less frequent ironic constructions (ironic praise), with foreign accented utterances considered less ironic than native accented utterances. These findings revealed that native listeners’ figurative interpretation of ironic praise can change depending on indexical cues, with a reduction of pragmatic inferences in the case of foreign accent.