Effects of native language and training on lexical tone perception: an event-related potential study

Native language advantage in electrical brain responses to speech sound changes in passive and active listening condition

It is not clear whether the brain can detect changes in native and non-native speech sounds in both unattended and attended conditions, but this information would be important to understand the nature of potential native language advantage in speech perception. We recorded event-related potentials (ERPs) for changes in duration and in Chinese lexical tone in a repeated vowel /a/ in native speakers of Finnish and Chinese in passive and active listening conditions. ERP amplitudes reflecting deviance detection (mismatch negativity; MMN and N2b) and attentional shifts towards changes in speech sounds (P3a and P3b) were investigated. In the passive listening condition, duration changes elicited increased amplitude in the MMN latency window for both standard and deviant sounds in the Finnish speakers compared to the Chinese speakers, but no group differences were observed for P3a. In passive listening to lexical tones, P3a was increased in amplitude for both standard and deviant stimuli in Chinese speakers compared to Finnish speakers, but the groups did not differ in MMN. In active listening, both tone and duration changes elicited N2b and P3b, but the groups differed only in pattern of results for the deviant type. The results thus suggest an overall increased sensitivity to native speech sounds, especially in passive listening, while the mechanisms of change detection and attentional shifting seem to work well for both native and non-native speech sounds in the attentive mode.

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

The dynamics of the mind are complex. Mental processes unfold continuously in time and may be sensitive to a myriad of interacting variables, especially in naturalistic settings. But statistical models used to analyze data from cognitive experiments often assume simplistic dynamics. Recent advances in deep learning have yielded startling improvements to simulations of dynamical cognitive processes, including speech comprehension, visual perception, and goal-directed behavior. But due to poor interpretability, deep learning is generally not used for scientific analysis. Here, we bridge this gap by showing that deep learning can be used, not just to imitate, but to analyze complex processes, providing flexible function approximation while preserving interpretability. To do so, we define and implement a nonlinear regression model in which the probability distribution over the response variable is parameterized by convolving the history of predictors over time using an artificial neural network, thereby allowing the shape and continuous temporal extent of effects to be inferred directly from time series data. Our approach relaxes standard simplifying assumptions (e.g., linearity, stationarity, and homoscedasticity) that are implausible for many cognitive processes and may critically affect the interpretation of data. We demonstrate substantial improvements on behavioral and neuroimaging data from the language processing domain, and we show that our model enables discovery of novel patterns in exploratory analyses, controls for diverse confounds in confirmatory analyses, and opens up research questions in cognitive (neuro)science that are otherwise hard to study.

Intonation processing of interrogative words in Mandarin: an event-related potential study

Intonation is the variation in pitch used in speech, which forms the premise of tonal and non-tonal languages. Interrogative words are words that introduce questions. Previous research lacks clarity regarding the specific cues used in the processing of word intonation. To address this gap, this study used the event-related potential electroencephalogram (EEG) research method to explore the intonation processing of tone two (mid-rising) interrogative words in Mandarin. For this, the word "shui," meaning "who," was selected as the experimental material. To avoid the influence of the environment, gender, and semantics, the Hum version, corresponding to the stimulus material, was also adopted for the experiment. This study used a passive oddball paradigm to examine the clues of intonation information processing in automatic cognitive processing through amplitude, latency, time window, and evoked location potential mismatch negativity. The standard stimulus was the declarative intonation with a high probability of occurrence (90%), and the deviant stimulus was the interrogative intonation with a low probability of occurrence (10%). In the time window of 370-450 ms, the mismatch negativity was found at the F3, F4, C3, Cz, and C4 channels. The findings show that, in the passive oddball paradigm, lexical semantics are essential for intonation processing at the pre-attentive level, which is dominated by the frontal and central areas of the brain. The results support the functional and comprehensive hypotheses that the processing of intonation is based on the function of language and that bilateral regions are involved in this processing. This study makes an important contribution by providing event-related potential evidence that lexical semantics plays a key role in the pre-attentive processing of intonation, as shown by the significant differences between semantic and non-semantic conditions.

Observing gesture at learning enhances subsequent phonological and semantic processing of L2 words: An N400 study

This study employed the N400 event-related potential (ERP) to investigate how observing different types of gestures at learning affects the subsequent processing of L2 Mandarin words differing in lexical tone by L1 English speakers. The effects of pitch gestures conveying lexical tones (e.g., upwards diagonal movements for rising tone), semantic gestures conveying word meanings (e.g., waving goodbye for to wave), and no gesture were compared. In a lexical tone discrimination task, larger N400s for Mandarin target words mismatching vs. matching Mandarin prime words in lexical tone were observed for words learned with pitch gesture. In a meaning discrimination task, larger N400s for English target words mismatching vs. matching Mandarin prime words in meaning were observed for words learned with pitch and semantic gesture. These findings provide the first neural evidence that observing gestures during L2 word learning enhances subsequent phonological and semantic processing of learned L2 words.

Are Serbian and English listeners insensitive to lexical pitch accents in Serbian?

The paper investigated possible perceptual insensitivity effects in the perception of lexical pitch accents by native and non-native listeners, that is, by Serbian and English listeners, respectively. The objective of the study was to explore which word-prosodic categories listeners used when they were required to contrast and recall sequences of lexical pitch accents. To that effect, Serbian and English listeners performed a Sequence Recall Task (SRT) in which they contrasted pairs of non-words with different Serbian lexical pitch accent types, and recalled the sequences of these non-words under different memory load conditions. Listeners' answers were coded correct and incorrect and the accuracy scores between the groups were compared and analyzed. Both groups had almost identical levels of accuracy and they performed well above chance level on each contrast. Neither group exhibited any effects of perceptual insensitivity to lexical pitch accents. English (non-native) listeners did not differ in their performance from native Serbian listeners, which, contrary to what previous research suggested, implied that one's native language word-prosodic category inventory did not preclude the encoding of non-native word-prosodic categories. Instead, non-native listeners were capable of deploying different prosodic resources such as post-lexical pitch accents to recall the sequences.

The tone atlas of perceptual discriminability and perceptual distance: Four tone languages and five language groups

Some prior investigations suggest that tone perception is flexible, reasonably independent of native phonology, whereas others suggest it is constrained by native phonology. We address this issue in a systematic and comprehensive investigation of adult tone perception. Sampling from diverse tone and non-tone speaking communities, we tested discrimination of the three major tone systems (Cantonese, Thai, Mandarin) that dominate the tone perception literature, in relation to native language and language experience as well as stimulus variation (tone properties, presentation order, pitch cues) using linear mixed effect modelling and multidimensional scaling. There was an overall discrimination advantage for tone language speakers and for native tones. However, language- and tone-specific effects, and presentation order effects also emerged. Thus, over and above native phonology, stimulus variation exerts a powerful influence on tone discrimination. This study provides a tone atlas, a reference guide to inform empirical studies of tone sensitivity, both retrospectively and prospectively.

Neural responses in novice learners' perceptual learning and generalization of lexical tones: The effect of training variability

The acoustics of lexical tones are highly variable across talkers, and require second-language (L2) learners' flexibility in accommodating talker-specific tonal variations for successful learning. This study investigated how tone training with high vs. low talker-variability modulated novice learners' neural responses to non-native tones. A passive oddball paradigm tested Mandarin-speaking participants' neural responses to Cantonese low-high and low-mid tonal contrasts in the pretest and posttest. Participants were trained using a tone identification task with feedback, either with high or low talker-variability. The results of mismatch negativity (MMN) showed no group difference in the pretest whereas the high-variability group demonstrated greater neural sensitivity to the low-high tonal contrast produced by a novel talker and a trained talker in the posttest. The finding provides (tentative) novel evidence that training variability may benefit perceptual learning of the relatively easy tone pair and facilitate the formation of talker-independent representations of non-native tones by novice learners.

Electrophysiological Signatures of Perceiving Alternated Tone in Mandarin Chinese: Mismatch Negativity to Underlying Tone Conflict

Although phonological alternation is prevalent in languages, the process of perceiving phonologically alternated sounds is poorly understood, especially at the neurolinguistic level. We examined the process of perceiving Mandarin 3rd tone sandhi (T3 + T3 → T2 + T3) with a mismatch negativity (MMN) experiment. Our design has two independent variables (whether the deviant undergoes tone sandhi; whether the standard and the deviant have matched underlying tone). These two independent variables modulated ERP responses in both the first and the second syllables. Notably, despite the apparent segmental conflict between the standard and the deviant in all conditions, MMN is only observed when neither the standard nor the deviant undergoes tone sandhi, suggesting that discovering the underlying representation of an alternated sound could interfere with the generation of MMN. A tentative model with three hypothesized underlying processing mechanisms is proposed to explain the observed latency and amplitude differences across conditions. The results are also discussed in light of the potential electrophysiological signatures involved in the process of perceiving alternated sounds.

Auditory processing deficit in individuals with dyslexia: A meta-analysis of mismatch negativity

Several previous studies have used mismatch negativity (MMN) to examine the auditory processing deficit in individuals with dyslexia. However, researchers have not clearly determined whether the deficit is general or specific and how it potentially changes with age. Meta-analysis was adopted to quantitatively identify the auditory processing deficit in individuals with dyslexia. By analysing 81 results within 25 publications that employed passive oddball paradigms to explore auditory processing in individuals with dyslexia, we identified that MMN impairment in auditory processing of speech was observed in children (Cohen's d = 0.296) and adults with dyslexia (Cohen's d = 0.486). Besides, adults with dyslexia showed atypical auditory processing of non-speech (Cohen's d = 0.409), which appeared to be related to the types of stimuli. Based on these findings, for individuals with dyslexia, the auditory processing deficit in speech will persist into adulthood, and the auditory processing deficit is general in adults with dyslexia. Because few studies used appropriate non-speech stimuli to examine the auditory processing in children with dyslexia, future studies should focus more on this area.

Effects of native language experience on Mandarin lexical tone processing in proficient second language learners

Learning the acoustic and phonological information in lexical tones is significant for learners of tonal languages. Although there is a wealth of knowledge from studies of second language (L2) tone learning, it remains unclear how L2 learners process acoustic versus phonological information differently depending on whether their first language (L1) is a tonal language. In the present study, we first examined proficient L2 learners of Mandarin with tonal and nontonal L1 in a behavioral experiment (identifying a Mandarin tonal continuum) to construct tonal contrasts that could differentiate the phonological from the acoustic information in Mandarin lexical tones for the L2 learners. We then conducted an ERP experiment to investigate these learners' automatic processing of acoustic and phonological information in Mandarin lexical tones by mismatch negativity (MMN). Although both groups of L2 learners showed similar behavioral identification features for the Mandarin tonal continuum as native speakers, L2 learners with nontonal L1, as compared with both native speakers and L2 learners with tonal L1, showed longer reaction time to the tokens of the Mandarin tonal continuum. More importantly, the MMN data further revealed distinct roles of acoustic and phonological information on the automatic processing of L2 lexical tones between the two groups of L2 learners. Taken together, the results indicate that the processing of acoustic and phonological information in L2 lexical tones may be modulated by L1 experience with a tonal language. The theoretical implications of the current study are discussed in light of models of L2 speech learning.

Revisiting Stress "Deafness" in European Portuguese - A Behavioral and ERP Study

European Portuguese (EP) is a language with variable stress, and the main cues for stress are duration and vowel reduction. A previous behavioral study has reported a stress “deafness” effect in EP when vowel quality cues are unavailable. The present study recorded both event-related potentials (ERPs) and behavioral data to examine the stress processing by native EP speakers in the absence of the vowel quality cues. Our behavioral result was consistent with previous research, showing that when vowel reduction is absent EP speakers demonstrated a stress “deafness” effect similar to that found in speakers of languages with fixed stress or without any lexical stress marking. In the ERP task, both the trochaic and iambic conditions yielded mismatch negativity (MMN) and late negativity, suggesting that EP speakers are able to discriminate the two stress patterns without vowel reduction at the pre-attentive stage. Moreover, the ERP and behavioral data revealed compatible results regarding the pattern of stress bias in EP. In the EPR task, the MMN and late negativity components were more negative and span over a larger temporal window in the iambic condition than in the trochaic condition, indicating a higher sensitivity for the iambic stress pattern. In the behavioral task, EP speakers responded more accurately and more quickly to the iambic stress. These results match recent developmental findings in the acquisition of stress, but speak against the dominant view in EP phonological literature which assumes penultimate stress to be the regular stress pattern. In addition, both the ERP and the behavioral data showed that EP speakers’ stress processing was influenced by their working memory (WM) capacity. The participants with high WM capacity outperformed the participants with limited WM capacity in the iambic condition. In sum, our results broaden the current knowledge on stress processing by EP speakers at both the pre-attentive and attentive levels.

Brain Processing (Auditory Event-Related Potential) of Stressed Versus Unstressed Words in Thai Speech

This study investigated the effect of the stressed word in Thai language on auditory event-related potential (aERP) in unattended conditions. We presented 30 healthy participants with monosyllabic Thai words consisting of either stressed or unstressed words. We instructed them not to attend to the sound stimuli, but rather to watch and memorize the contents of a silent natural documentary without subtitles. The two listening conditions consisted of 20% deviant stimuli (70 stressed and 70 unstressed words, respectively) and 80% standard stimuli (other 280 unstressed words) presented pseudorandomly and binaurally via a pair of earphones. Participants' aERPs from the two conditions were evaluated by the mismatch negativity (MMN) component of aERP. The mismatch negativity amplitudes in the stressed word condition were significantly higher than those in the unstressed word condition, especially in frontal and left fronto-central brain areas. Therefore, these data show the role of the frontal and left fronto-central brain regions in auditory preattentive processing of stressed word perception among native Thai speakers. This is the first study demonstration that stressed meaningful monosyllable words in tonal language facilitate word perception in this preattentive stage. This result has implications for developing clinical tests evaluating preattentive speech perception.

The Duration of Auditory Sensory Memory for Vowel Processing: Neurophysiological and Behavioral Measures

Speech perception behavioral research suggests that rates of sensory memory decay are dependent on stimulus properties at more than one level (e.g., acoustic level, phonemic level). The neurophysiology of sensory memory decay rate has rarely been examined in the context of speech processing. In a lexical tone study, we showed that long-term memory representation of lexical tone slows the decay rate of sensory memory for these tones. Here, we tested the hypothesis that long-term memory representation of vowels slows the rate of auditory sensory memory decay in a similar way to that of lexical tone. Event-related potential (ERP) responses were recorded to Mandarin non-words contrasting the vowels /i/ vs. /u/ and /y/ vs. /u/ from first-language (L1) Mandarin and L1 American English participants under short and long interstimulus interval (ISI) conditions (short ISI: an average of 575 ms, long ISI: an average of 2675 ms). Results revealed poorer discrimination of the vowel contrasts for English listeners than Mandarin listeners, but with different patterns for behavioral perception and neural discrimination. As predicted, English listeners showed the poorest discrimination and identification for the vowel contrast /y/ vs. /u/, and poorer performance in the long ISI condition. In contrast to Yu et al. (2017), however, we found no effect of ISI reflected in the neural responses, specifically the mismatch negativity (MMN), P3a and late negativity ERP amplitudes. We did see a language group effect, with Mandarin listeners generally showing larger MMN and English listeners showing larger P3a. The behavioral results revealed that native language experience plays a role in echoic sensory memory trace maintenance, but the failure to find an effect of ISI on the ERP results suggests that vowel and lexical tone memory traces decay at different rates.

Highlights:

We examined the interaction between auditory sensory memory decay and language experience.

We compared MMN, P3a, LN and behavioral responses in short vs. long interstimulus intervals.

We found that different from lexical tone contrast, MMN, P3a, and LN changes to vowel contrasts are not influenced by lengthening the ISI to 2.6 s.

We also found that the English listeners discriminated the non-native vowel contrast with lower accuracy under the long ISI condition.

Monolingual and Bilingual Infants' Ability to Use Non-native Tone for Word Learning Deteriorates by the Second Year After Birth

Previous studies reported a non-native word learning advantage for bilingual infants at around 18 months. We investigated developmental changes in infant interpretation of sounds that aid in object mapping. Dutch monolingual and bilingual (exposed to Dutch and a second non-tone-language) infants' word learning ability was examined on two novel label-object pairings using syllables differing in Mandarin tones as labels (flat vs. falling). Infants aged 14-15 months, regardless of language backgrounds, were sensitive to violations in the label-objects pairings when lexical tones were switched compared to when they were the same as habituated. Conversely at 17-18 months, neither monolingual nor bilingual infants demonstrated learning. Linking with existing literature, infants' ability to associate non-native tones with meanings may be related to tonal acoustic properties and/or perceptual assimilation to native prosodic categories. These findings provide new insights into the relation between infant tone perception, learning, and interpretative narrowing from a developmental perspective.

What Can Lexical Tone Training Studies in Adults Tell Us about Tone Processing in Children?

A growing number of studies on the acquisition of lexical tone by adult learners have revealed that factors such as language background, musical experience, cognitive abilities, and neuroanatomy all play a role in determining tone learning success. On the basis of these findings, it has been argued that the effectiveness of tone learning in adulthood depends on individual differences in these factors. However, it is not clear whether similar individual differences play an analogous role in tone learning in childhood. Indeed, relatively few studies have made comparisons between how adults and children learn lexical tones. Here, we review recent developments for tone learning in both adults and children. The review covers tone training in a range of contexts, including in naive listeners, in native speakers of other tone languages, in listeners with varying levels of musical experience, and in individuals with speech and hearing disorders. Finally, we discuss the parallels between adult and child tone learning, and provide recommendations concerning how findings in adult tone training can provide insights into tone learning for children by accommodating the needs of individual learners.

Differential sensitivity to changes in pitch acceleration in the auditory brainstem and cortex

The cortical pitch-specific response (CPR) is differentially sensitive to pitch contours varying in rate of acceleration-time-variant Mandarin Tone2 (T2) versus constant, linear rising ramp (Linear)-as a function of language experience (Krishnan, Gandour, & Suresh, 2014). CPR and brainstem frequency following response (FFR) data were recorded concurrently from native Mandarin listeners using the same stimuli. Results showed that T2 elicited larger responses than Linear at both cortical and brainstem levels (CPR: Na-Pb, Pb-Nb; FFR). However, Pb-Nb exhibited a larger difference in magnitude between T2 and Linear than either Na-Pb or FFR. This finding highlights differential weighting of brain responses elicited by a specific temporal attribute of pitch. Consistent with the notion of a distributed, integrated hierarchical pitch processing network, temporal attributes of pitch are differentially weighted by subcortical and cortical level processing.

Neurophysiological and Behavioral Responses of Mandarin Lexical Tone Processing

Language experience enhances discrimination of speech contrasts at a behavioral- perceptual level, as well as at a pre-attentive level, as indexed by event-related potential (ERP) mismatch negativity (MMN) responses. The enhanced sensitivity could be the result of changes in acoustic resolution and/or long-term memory representations of the relevant information in the auditory cortex. To examine these possibilities, we used a short (ca. 600 ms) vs. long (ca. 2,600 ms) interstimulus interval (ISI) in a passive, oddball discrimination task while obtaining ERPs. These ISI differences were used to test whether cross-linguistic differences in processing Mandarin lexical tone are a function of differences in acoustic resolution and/or differences in long-term memory representations. Bisyllabic nonword tokens that differed in lexical tone categories were presented using a passive listening multiple oddball paradigm. Behavioral discrimination and identification data were also collected. The ERP results revealed robust MMNs to both easy and difficult lexical tone differences for both groups at short ISIs. At long ISIs, there was either no change or an enhanced MMN amplitude for the Mandarin group, but reduced MMN amplitude for the English group. In addition, the Mandarin listeners showed a larger late negativity (LN) discriminative response than the English listeners for lexical tone contrasts in the long ISI condition. Mandarin speakers outperformed English speakers in the behavioral tasks, especially under the long ISI conditions with the more similar lexical tone pair. These results suggest that the acoustic correlates of lexical tone are fairly robust and easily discriminated at short ISIs, when the auditory sensory memory trace is strong. At longer ISIs beyond 2.5 s language-specific experience is necessary for robust discrimination.

Normalization of lexical tones and nonlinguistic pitch contours: Implications for speech-specific processing mechanism

Context is indispensable for accurate tone perception, especially when the target tone system is as complex as that of Cantonese. However, not all contexts are equally beneficial. Speech contexts are usually more effective in improving lexical tone identification than nonspeech contexts matched in pitch information. Some potential factors which may contribute to these unequal effects have been proposed but, thus far, their plausibility remains unclear. To shed light on this issue, the present study compares the perception of lexical tones and their nonlinguistic counterparts under specific contextual (speech, nonspeech) and attentional (with/without focal attention) conditions. The results reveal a prominent congruency effect-target sounds tend to be identified more accurately when embedded in contexts of the same nature (speech/nonspeech). This finding suggests that speech and nonspeech sounds are partly processed by domain-specific mechanisms and that information from the same domain can be integrated more effectively than that from different domains. Therefore, domain-specific processing of speech could be the most likely cause of the unequal context effect. Moreover, focal attention is not a prerequisite for extracting contextual cues from speech and nonspeech during perceptual normalization. This finding implies that context encoding is highly automatic for native listeners.

Categorical perception of lexical tones by English learners of Mandarin Chinese

Whether native speakers of non-tonal languages can acquire categorical representations of lexical tones remains controversial. This study investigates the acquisition of lexical tone categories by native English speakers learning Mandarin Chinese as a foreign language by comparing the categorical perception (CP) of lexical tones between three groups of listeners: (1) native English speakers who had taken advanced Mandarin courses in colleges; (2) native English speakers with no knowledge of Mandarin Chinese; and (3) native Mandarin speakers. Two tonal continua derived from natural speech within carrier phrases were created through interpolation within two tonal contrasts (tone 1/tone 4, T1/T4; tone 2/tone 3, T2/T3). Results showed categorical-like perception of tones by native Mandarin speakers. The inexperienced English speakers performed near chance on discrimination tasks and showed significantly broader identification boundaries than the other two groups. The learners of Mandarin showed similar pattern of CP to native Mandarin speakers, but with higher overall discrimination scores. Findings suggest that CP of lexical tone may be available to advanced second language learners.

Individual aptitude in Mandarin lexical tone perception predicts effectiveness of high-variability training

Although the high-variability training method can enhance learning of non-native speech categories, this can depend on individuals’ aptitude. The current study asked how general the effects of perceptual aptitude are by testing whether they occur with training materials spoken by native speakers and whether they depend on the nature of the to-be-learned material. Forty-five native Dutch listeners took part in a 5-day training procedure in which they identified bisyllabic Mandarin pseudowords (e.g., asa) pronounced with different lexical tone combinations. The training materials were presented to different groups of listeners at three levels of variability: low (many repetitions of a limited set of words recorded by a single speaker), medium (fewer repetitions of a more variable set of words recorded by three speakers), and high (similar to medium but with five speakers). Overall, variability did not influence learning performance, but this was due to an interaction with individuals’ perceptual aptitude: increasing variability hindered improvements in performance for low-aptitude perceivers while it helped improvements in performance for high-aptitude perceivers. These results show that the previously observed interaction between individuals’ aptitude and effects of degree of variability extends to natural tokens of Mandarin speech. This interaction was not found, however, in a closely matched study in which native Dutch listeners were trained on the Japanese geminate/singleton consonant contrast. This may indicate that the effectiveness of high-variability training depends not only on individuals’ aptitude in speech perception but also on the nature of the categories being acquired.

Temporal relation between top-down and bottom-up processing in lexical tone perception

Speech perception entails both top-down processing that relies primarily on language experience and bottom-up processing that depends mainly on instant auditory input. Previous models of speech perception often claim that bottom-up processing occurs in an early time window, whereas top-down processing takes place in a late time window after stimulus onset. In this paper, we evaluated the temporal relation of both types of processing in lexical tone perception. We conducted a series of event-related potential (ERP) experiments that recruited Mandarin participants and adopted three experimental paradigms, namely dichotic listening, lexical decision with phonological priming, and semantic violation. By systematically analyzing the lateralization patterns of the early and late ERP components that are observed in these experiments, we discovered that: auditory processing of pitch variations in tones, as a bottom-up effect, elicited greater right hemisphere activation; in contrast, linguistic processing of lexical tones, as a top-down effect, elicited greater left hemisphere activation. We also found that both types of processing co-occurred in both the early (around 200 ms) and late (around 300–500 ms) time windows, which supported a parallel model of lexical tone perception. Unlike the previous view that language processing is special and performed by dedicated neural circuitry, our study have elucidated that language processing can be decomposed into general cognitive functions (e.g., sensory and memory) and share neural resources with these functions.

Temporal order perception of auditory stimuli is selectively modified by tonal and non-tonal language environments

The close relationship between temporal perception and speech processing is well established. The present study focused on the specific question whether the speech environment could influence temporal order perception in subjects whose language backgrounds are distinctively different, i.e., Chinese (tonal language) vs. Polish (non-tonal language). Temporal order thresholds were measured for both monaurally presented clicks and binaurally presented tone pairs. Whereas the click experiment showed similar order thresholds for the two language groups, the experiment with tone pairs resulted in different observations: while Chinese demonstrated better performance in discriminating the temporal order of two "close frequency" tone pairs (600 Hz and 1200 Hz), Polish subjects showed a reversed pattern, i.e., better performance for "distant frequency" tone pairs (400 Hz and 3000 Hz). These results indicate on the one hand a common temporal mechanism for perceiving the order of two monaurally presented stimuli, and on the other hand neuronal plasticity for perceiving the order of frequency-related auditory stimuli. We conclude that the auditory brain is modified with respect to temporal processing by long-term exposure to a tonal or a non-tonal language. As a consequence of such an exposure different cognitive modes of operation (analytic vs. holistic) are selected: the analytic mode is adopted for "distant frequency" tone pairs in Chinese and for "close frequency" tone pairs in Polish subjects, whereas the holistic mode is selected for "close frequency" tone pairs in Chinese and for "distant frequency" tone pairs in Polish subjects, reflecting a double dissociation of function.

Changes in oscillatory brain networks after lexical tone training

Learning foreign speech contrasts involves creating new representations of sound categories in memory. This formation of new memory representations is likely to involve changes in neural networks as reflected by oscillatory brain activity. To explore this, we conducted time-frequency analyses of electro-encephalography (EEG) data recorded in a passive auditory oddball paradigm using Thai language tones. We compared native speakers of English (a non-tone language) and native speakers of Mandarin Chinese (a tone language), before and after a two-day laboratory training. Native English speakers showed a larger gamma-band power and stronger alpha-band synchrony across EEG channels than the native Chinese speakers, especially after training. This is compatible with the view that forming new speech categories on the basis of unfamiliar perceptual dimensions involves stronger gamma activity and more coherent activity in alpha-band networks than forming new categories on the basis of familiar dimensions.

Reading acquisition reorganizes the phonological awareness network only in alphabetic writing systems

It is unknown how experience with different types of orthographies influences the neural basis of oral language processing. In order to determine the effects of alphabetic and nonalphabetic writing systems, the current study examined the influence of learning to read on oral language in English and Chinese speakers. Children (8-12 years olds) and adults made rhyming judgments to pairs of spoken words during functional magnetic resonance imaging (fMRI). Developmental increases were seen only for English speakers in the left hemisphere phonological network (superior temporal gyrus (STG), inferior parietal lobule, and inferior frontal gyrus). The increase in the STG was more pronounced for words with conflicting orthography (e.g. pint-mint; jazz-has) even though access to orthography was irrelevant to the task. Moreover, higher reading skill was correlated with greater activation in the STG only for English speaking children. The effects suggest that learning to read reorganizes the phonological awareness network only for alphabetic and not logographic writing systems because of differences in the principles for mapping between orthographic and phonological representations. The reorganization of the auditory cortex may result in better phonological awareness skills in alphabetic readers.

Native experience with a tone language enhances pitch discrimination and the timing of neural responses to pitch change

Native tone language experience has been linked with alterations in the production and perception of pitch in language, as well as with the brain response to linguistic and non-linguistic tones. Here we use two experiments to address whether these changes apply to the discrimination of simple pitch changes and pitch intervals. Event related potentials (ERPs) were recorded from native Mandarin speakers and a control group during a same/different task with pairs of pure tones differing only in pitch height, and with pure tone pairs differing only in interval distance. Behaviorally, Mandarin speakers were more accurate than controls at detecting both pitch and interval changes, showing a sensitivity to small pitch changes and interval distances that was absent in the control group. Converging evidence from ERPs obtained during the same tasks revealed an earlier response to change relative to no-change trials in Mandarin speakers, as well as earlier differentiation of trials by change direction relative to controls. These findings illustrate the cross-domain influence of language experience on the perception of pitch, suggesting that the native use of tonal pitch contours in language leads to a general enhancement in the acuity of pitch representations.

Involuntary switching of attention mediates differences in event-related responses to complex tones between early and late Spanish-English bilinguals

Most research with bilinguals has used speech stimuli to demonstrate differences in auditory processing abilities. Two main factors have been identified as modulators of such differences: proficiency and age of acquisition of the second language (L2). However, whether the bilingual brain differs from the monolingual in the efficient processing of non-verbal auditory events (known to be critical to the acoustic analysis of the speech stream) remains unclear. In this EEG/ERP study, using the mismatch negativity (MMN), P3a, and late negativity (LN), we examined differences in discrimination, involuntary switching of attention and reorienting of attention between monolinguals and bilinguals as they processed complex tones. Further, we examined the role that age of acquisition plays in modulating such responses. A group of English monolinguals and a group of proficient Spanish-English bilinguals were presented with a multiple-deviant oddball paradigm with four deviant conditions (duration, frequency, silent gap, and frequency modulation). Late bilinguals, who learned English after age 10, exhibited larger MMN and P3a responses than early bilinguals, across all deviant conditions. Significant associations were found between amplitude of the responses and both age of L2 acquisition and years of L2 experience. Individuals who acquired English at later ages and had fewer years of L2 experience had larger MMN, P3a, and LN responses than those who learned it earlier. These findings demonstrate that age of L2 acquisition is an important modulator of auditory responses in bilinguals even when processing non-speech signals. Involuntary attention switching is suggested as the main factor driving these differences.

Categorical perception of lexical tones in Chinese revealed by mismatch negativity

The present study investigated the neurophysiological correlates of categorical perception of Chinese lexical tones in Mandarin Chinese. Relative to standard stimuli, both within- and across-category deviants elicited mismatch negativity (MMN) in bilateral frontal-central recording sites. The MMN elicited in the right sites was marginally larger than in the left sites, which reflects the role of the right hemisphere in acoustic processing. At the same time, relative to within-category deviants, the across-category deviants elicited larger MMN in the left recording sites, reflecting the long-term phonemic traces of lexical tones. These results provide strong neurophysiological evidence in support of categorical perception of lexical tones in Chinese. More important, they demonstrate that acoustic and phonological information is processed in parallel within the MMN time window for the perception of lexical tones. Finally, homologous nonspeech stimuli elicited similar MMN patterns, indicating that lexical tone knowledge influences the perception of nonspeech signals.

Individual variability in cue-weighting and lexical tone learning

Speech sound patterns can be discerned using multiple acoustic cues. The relative weighting of these cues is known to be language-specific. Speech-sound training in adults induces changes in cue-weighting such that relevant acoustic cues are emphasized. In the current study, the extent to which individual variability in cue weighting contributes to differential success in learning to use foreign sound patterns was examined. Sixteen English-speaking adult participants underwent a sound-to-meaning training paradigm, during which they learned to incorporate Mandarin linguistic pitch contours into words. In addition to cognitive tests, measures of pitch pattern discrimination and identification were collected from all participants. Reaction time data from the discrimination task was subjected to 3-way multidimensional scaling to extract dimensions underlying tone perception. Two dimensions relating to pitch height and pitch direction were found to underlie non-native tone space. Good learners attended more to pitch direction relative to poor learners, before and after training. Training increased the ability to identify and label pitch direction. The results demonstrate that variability in the ability to successfully learn to use pitch in lexical contexts can be explained by pre-training differences in cue-weighting.

The role of the auditory brainstem in processing linguistically-relevant pitch patterns

Historically, the brainstem has been neglected as a part of the brain involved in language processing. We review recent evidence of language-dependent effects in pitch processing based on comparisons of native vs. nonnative speakers of a tonal language from electrophysiological recordings in the auditory brainstem. We argue that there is enhancing of linguistically-relevant pitch dimensions or features well before the auditory signal reaches the cerebral cortex. We propose that long-term experience with a tone language sharpens the tuning characteristics of neurons along the pitch axis with enhanced sensitivity to linguistically-relevant, rapidly changing sections of pitch contours. Though not specific to a speech context, experience-dependent brainstem mechanisms for pitch representation are clearly sensitive to particular aspects of pitch contours that native speakers of a tone language have been exposed to. Such experience-dependent effects on lower-level sensory processing are compatible with more integrated, hierarchically organized pathways to language and the brain.

Communication disorders in speakers of tone languages: etiological bases and clinical considerations

Lexical tones are a phonetic contrast necessary for conveying meaning in a majority of the world's languages. Various hearing, speech, and language disorders affect the ability to perceive or produce lexical tones, thereby seriously impairing individuals' communicative abilities. The number of tone language speakers is increasing, even in otherwise English-speaking nations, yet insufficient emphasis has been placed on clinical assessment and rehabilitation of lexical tone disorders. The similarities and dissimilarities between lexical tones and other speech sounds make a richer scientific understanding of their physiological bases paramount to more effective remediation of speech and language disorders in general. Here we discuss the cognitive and biological bases of lexical tones, emphasizing the neural structures and networks that support their acquisition, perception, and cognitive representation. We present emerging research on lexical tone learning in the context of the clinical disorders of hearing, speech, and language that this body of research will help to address.

Thai lexical tone perception in native speakers of Thai, English and Mandarin Chinese: an event-related potentials training study

BACKGROUND

Tone languages such as Thai and Mandarin Chinese use differences in fundamental frequency (F0, pitch) to distinguish lexical meaning. Previous behavioral studies have shown that native speakers of a non-tone language have difficulty discriminating among tone contrasts and are sensitive to different F0 dimensions than speakers of a tone language. The aim of the present ERP study was to investigate the effect of language background and training on the non-attentive processing of lexical tones. EEG was recorded from 12 adult native speakers of Mandarin Chinese, 12 native speakers of American English, and 11 Thai speakers while they were watching a movie and were presented with multiple tokens of low-falling, mid-level and high-rising Thai lexical tones. High-rising or low-falling tokens were presented as deviants among mid-level standard tokens, and vice versa. EEG data and data from a behavioral discrimination task were collected before and after a two-day perceptual categorization training task.

RESULTS

Behavioral discrimination improved after training in both the Chinese and the English groups. Low-falling tone deviants versus standards elicited a mismatch negativity (MMN) in all language groups. Before, but not after training, the English speakers showed a larger MMN compared to the Chinese, even though English speakers performed worst in the behavioral tasks. The MMN was followed by a late negativity, which became smaller with improved discrimination. The High-rising deviants versus standards elicited a late negativity, which was left-lateralized only in the English and Chinese groups.

CONCLUSION

Results showed that native speakers of English, Chinese and Thai recruited largely similar mechanisms when non-attentively processing Thai lexical tones. However, native Thai speakers differed from the Chinese and English speakers with respect to the processing of late F0 contour differences (high-rising versus mid-level tones). In addition, native speakers of a non-tone language (English) were initially more sensitive to F0 onset differences (low-falling versus mid-level contrast), which was suppressed as a result of training. This result converges with results from previous behavioral studies and supports the view that attentive as well as non-attentive processing of F0 contrasts is affected by language background, but is malleable even in adult learners.