The perception of speech modulation cues in lexical tones is guided by early language-specific experience

The development of tone discrimination in infancy: Evidence from a cross-linguistic, multi-lab report

We report the findings of a multi-language and multi-lab investigation of young infants' ability to discriminate lexical tones as a function of their native language, age and language experience, as well as of tone properties. Given the high prevalence of lexical tones across human languages, understanding lexical tone acquisition is fundamental for comprehensive theories of language learning. While there are some similarities between the developmental course of lexical tone perception and that of vowels and consonants, findings for lexical tones tend to vary greatly across different laboratories. To reconcile these differences and to assess the developmental trajectory of native and non-native perception of tone contrasts, this study employed a single experimental paradigm with the same two pairs of Cantonese tone contrasts (perceptually similar vs. distinct) across 13 laboratories in Asia-Pacific, Europe and North-America testing 5-, 10- and 17-month-old monolingual (tone, pitch-accent, non-tone) and bilingual (tone/non-tone, non-tone/non-tone) infants. Across the age range and language backgrounds, infants who were not exposed to Cantonese showed robust discrimination of the two non-native lexical tone contrasts. Contrary to this overall finding, the statistical model assessing native discrimination by Cantonese-learning infants failed to yield significant effects. These findings indicate that lexical tone sensitivity is maintained from 5 to 17 months in infants acquiring tone and non-tone languages, challenging the generalisability of the existing theoretical accounts of perceptual narrowing in the first months of life. RESEARCH HIGHLIGHTS: This is a multi-language and multi-lab investigation of young infants' ability to discriminate lexical tones. This study included data from 13 laboratories testing 5-, 10-, and 17-month-old monolingual (tone, pitch-accent, non-tone) and bilingual (tone/non-tone, non-tone/non-tone) infants. Overall, infants discriminated a perceptually similar and a distinct non-native tone contrast, although there was no evidence of a native tone-language advantage in discrimination. These results demonstrate maintenance of tone discrimination throughout development.

An auditory perspective on phonological development in infancy

Introduction

The auditory system encodes the phonetic features of languages by processing spectro-temporal modulations in speech, which can be described at two time scales: relatively slow amplitude variations over time (AM, further distinguished into the slowest <8-16 Hz and faster components 16-500 Hz), and frequency modulations (FM, oscillating at higher rates about 600-10 kHz). While adults require only the slowest AM cues to identify and discriminate speech sounds, infants have been shown to also require faster AM cues (>8-16 Hz) for similar tasks.

Methods

Using an observer-based psychophysical method, this study measured the ability of typical-hearing 6-month-olds, 10-month-olds, and adults to detect a change in the vowel or consonant features of consonant-vowel syllables when temporal modulations are selectively degraded. Two acoustically degraded conditions were designed, replacing FM cues with pure tones in 32 frequency bands, and then extracting AM cues in each frequency band with two different low-pass cut- off frequencies: (1) half the bandwidth (Fast AM condition), (2) <8 Hz (Slow AM condition).

Results

In the Fast AM condition, results show that with reduced FM cues, 85% of 6-month-olds, 72.5% of 10-month-olds, and 100% of adults successfully categorize phonemes. Among participants who passed the Fast AM condition, 67% of 6-month-olds, 75% of 10-month-olds, and 95% of adults passed the Slow AM condition. Furthermore, across the three age groups, the proportion of participants able to detect phonetic category change did not differ between the vowel and consonant conditions. However, age-related differences were observed for vowel categorization: while the 6- and 10-month-old groups did not differ from one another, they both independently differed from adults. Moreover, for consonant categorization, 10-month-olds were more impacted by acoustic temporal degradation compared to 6-month-olds, and showed a greater decline in detection success rates between the Fast AM and Slow AM conditions.

Discussion

The degradation of FM and faster AM cues (>8 Hz) appears to strongly affect consonant processing at 10 months of age. These findings suggest that between 6 and 10 months, infants show different developmental trajectories in the perceptual weight of speech temporal acoustic cues for vowel and consonant processing, possibly linked to phonological attunement.

Auditory Processing of Non-speech Stimuli by Children in Dual-Language Immersion Programs

Perception of low-level auditory cues such as frequency modulation (FM) and rise time (RT) is crucial for development of phonemic representations, segmentation of word boundaries, and attunement to prosodic patterns in language. While learning an additional language, children may develop an increased sensitivity to these cues to extract relevant information from multiple types of linguistic input. Performance on these auditory processing tasks such as FM and RT by children learning another language is, however, unknown. Here we examine 92 English-speaking 7-8-year-olds in the U.S. and their performance in FM and RT perceptual tasks at the end of their second year in Cantonese or Spanish dual-language immersion compared to children in general English education programs. Results demonstrate that children in immersion programs have greater sensitivity to FM, but not RT, controlling for various factors. The immersion program students were also observed to have better phonological awareness performance. However, individual differences in FM sensitivity were not associated with phonological awareness, a pattern typically observed in monolinguals. These preliminary findings suggest a possible impact of formal language immersion on low-level auditory processing. Additional research is warranted to understand causal relationships and ultimate impact on language skills in multilinguals.

The Interaction Between Timescale and Pitch Contour at Pre-attentive Processing of Frequency-Modulated Sweeps

Speech comprehension across languages depends on encoding the pitch variations in frequency-modulated (FM) sweeps at different timescales and frequency ranges. While timescale and spectral contour of FM sweeps play important roles in differentiating acoustic speech units, relatively little work has been done to understand the interaction between the two acoustic dimensions at early cortical processing. An auditory oddball paradigm was employed to examine the interaction of timescale and pitch contour at pre-attentive processing of FM sweeps. Event-related potentials to frequency sweeps that vary in linguistically relevant pitch contour (fundamental frequency F0 vs. first formant frequency F1) and timescale (local vs. global) in Mandarin Chinese were recorded. Mismatch negativities (MMNs) were elicited by all types of sweep deviants. For local timescale, FM sweeps with F0 contours yielded larger MMN amplitudes than F1 contours. A reversed MMN amplitude pattern was obtained with respect to F0/F1 contours for global timescale stimuli. An interhemispheric asymmetry of MMN topography was observed corresponding to local and global-timescale contours. Falling but not rising frequency difference waveforms sweep contours elicited right hemispheric dominance. Results showed that timescale and pitch contour interacts with each other in pre-attentive auditory processing of FM sweeps. Findings suggest that FM sweeps, a type of non-speech signal, is processed at an early stage with reference to its linguistic function. That the dynamic interaction between timescale and spectral pattern is processed during early cortical processing of non-speech frequency sweep signal may be critical to facilitate speech encoding at a later stage.

Discrimination and identification of lexical tones and consonants in Mandarin-speaking children using cochlear implants

Mandarin-speaking adults using cochlear implants (CI) experience more difficulties in perceiving lexical tones than consonants. This problem may result from the fact that CIs provide relatively sufficient temporal envelope information for consonant perception in quiet environments, but do not convey the fine spectro-temporal information considered to be necessary for accurate pitch perception. Another possibility is that Mandarin speakers with post-lingual hearing loss have developed language-specific use of these acoustic cues, impeding lexical tone processing under CI conditions. To investigate this latter hypothesis, syllable discrimination and word identification abilities for Mandarin consonants (place and manner) and lexical-tone contrasts (tones 1 vs 3 and 1 vs 2) were measured in 15 Mandarin-speaking children using CIs and age-matched children with normal hearing (NH). In the discrimination task, only children using CIs exhibited significantly lower scores for consonant place contrasts compared to other contrasts, including lexical tones. In the word identification task, children using CIs showed lower performance for all contrasts compared to children with NH, but they both showed specific difficulties with tone 1 vs 2 contrasts. This study suggests that Mandarin-speaking children using CIs are able to discriminate and identify lexical tones and, perhaps more surprisingly, have more difficulties when discriminating consonants.

Auditory identification of frequency-modulated sweeps and reading difficulties in Chinese

BACKGROUND

In Chinese Mandarin, lexical tones play an important role of providing contrasts in word meaning. They are pitch patterns expressed by frequency-modulated (FM) signals. Yet, few studies have looked at the relationship between low-level auditory processing of frequency signals and Chinese reading skills.

AIMS

The study aims to identify the role of auditory frequency processing in Chinese lexical tone awareness as well as character recognition in Chinese-speaking children.

METHODS

Children with (N = 28) and without (N = 27) developmental dyslexia (DD) were recruited. All participants completed two linguistic tasks, Chinese character recognition and lexical tone awareness, and two auditory frequency processing tasks, frequency discrimination and FM sweep direction identification.

RESULTS

The results revealed that Chinese-speaking children with DD were significantly poorer at all tasks. Particularly, Chinese character recognition was significantly related to FM sweep identification. Lexical tone awareness was significantly associated with both auditory frequency processing tasks. Regression analyses suggested the influence of FM sweep identification on Chinese character recognition contributed through lexical tone awareness.

CONCLUSIONS AND IMPLICATION

This study suggests that poor auditory frequency processing may associate with Chinese developmental dyslexia with phonological deficits. In support of the phonological deficit hypothesis, what underlies phonological deficit is likely to be auditory-basis. A potential clinical implication is to reinforce auditory perception and sensitivity through intervention for phonological processing.

Adult Learning of Novel Words in a Non-native Language: Consonants, Vowels, and Tones

While words are distinguished primarily by consonants and vowels in many languages, tones are also used in the majority of the world's languages to cue lexical contrasts. However, studies on novel word learning have largely concentrated on consonants and vowels. To shed more light on the use of tonal information in novel word learning and its relationship with the development of phonological categories, the present study explored how adults' ability to learn minimal pair pseudowords in a tone language is modulated by their native phonological knowledge. Twenty-four adult speakers of three languages were tested: Cantonese, Mandarin, and French. Eye-tracking was used to record eye movements of these learners, while they were watching animated cartoons in Cantonese. On each trial, adults had to learn two new label-object associations, while the labels differed minimally by a consonant, a vowel, or a tone. Learning would therefore attest to participants' ability to use phonological information to distinguish the paired words. Results first revealed that adult learners in each language group performed better than chance in all conditions. Moreover, compared to native Cantonese adults, both Mandarin- and French-speaking adults performed worse on all three contrasts. In addition, French adults were worse on tones when compared to Mandarin adults. Lastly, no advantage for consonantal information in native lexical processing was found for Cantonese-speaking adults as predicted by the “division of labor” proposal, thus confirming crosslinguistic differences in consonant/vowel weight between speakers of tonal vs. non-tonal languages. These findings establish rapid novel word learning in a non-native language (long-term learning will have to be further assessed), modulated by native phonological knowledge. The implications of the findings of this adult study for further infant word learning studies are discussed.

Infants' and Adults' Use of Temporal Cues in Consonant Discrimination

OBJECTIVES

Adults can use slow temporal envelope cues, or amplitude modulation (AM), to identify speech sounds in quiet. Faster AM cues and the temporal fine structure, or frequency modulation (FM), play a more important role in noise. This study assessed whether fast and slow temporal modulation cues play a similar role in infants' speech perception by comparing the ability of normal-hearing 3-month-olds and adults to use slow temporal envelope cues in discriminating consonants contrasts.

DESIGN

English consonant-vowel syllables differing in voicing or place of articulation were processed by 2 tone-excited vocoders to replace the original FM cues with pure tones in 32 frequency bands. AM cues were extracted in each frequency band with 2 different cutoff frequencies, 256 or 8 Hz. Discrimination was assessed for infants and adults using an observer-based testing method, in quiet or in a speech-shaped noise.

RESULTS

For infants, the effect of eliminating fast AM cues was the same in quiet and in noise: a high proportion of infants discriminated when both fast and slow AM cues were available, but less than half of the infants also discriminated when only slow AM cues were preserved. For adults, the effect of eliminating fast AM cues was greater in noise than in quiet: All adults discriminated in quiet whether or not fast AM cues were available, but in noise eliminating fast AM cues reduced the percentage of adults reaching criterion from 71 to 21%.

CONCLUSIONS

In quiet, infants seem to depend on fast AM cues more than adults do. In noise, adults seem to depend on FM cues to a greater extent than infants do. However, infants and adults are similarly affected by a loss of fast AM cues in noise. Experience with the native language seems to change the relative importance of different acoustic cues for speech perception.

The Development of Mismatch Responses to Mandarin Lexical Tone in 12- to 24-Month-Old Infants

This study explores the development of mismatch responses (MMRs) to Mandarin lexical tone changes in infants at 12, 18, and 24 months of age using the multi-deviant oddball paradigm with the low dipping Tone 3 (T3) as the standard, the high level Tone 1 (T1) as the large, and the high rising Tone 2 (T2) as the small deviant. The results show that the large acoustic change between T1/T3 elicited mismatch negativity (MMN) in all three age groups. The small acoustic change between T2/T3 elicited a positive mismatch response (P-MMR) at 12 and 18 months of age, but no MMR was found to the T2/T3 change at 24 months. The coexistence of MMN and P-MMR in the same age group implies that different mechanisms were used for discriminating large and small deviants. Infants were able to detect the T1/T3 change automatically and showed adult-like MMN as early as 6 months of age. However, the detection of the T2/T3 change remains effortful in infants under 24 months of age. These findings support the notion that MMN and P-MMR may be used to index the maturation of speech perception.

One Way or Another: Evidence for Perceptual Asymmetry in Pre-attentive Learning of Non-native Contrasts

Research investigating listeners' neural sensitivity to speech sounds has largely focused on segmental features. We examined Australian English listeners' perception and learning of a supra-segmental feature, pitch direction in a non-native tonal contrast, using a passive oddball paradigm and electroencephalography. The stimuli were two contours generated from naturally produced high-level and high-falling tones in Mandarin Chinese, differing only in pitch direction (Liu and Kager, 2014). While both contours had similar pitch onsets, the pitch offset of the falling contour was lower than that of the level one. The contrast was presented in two orientations (standard and deviant reversed) and tested in two blocks with the order of block presentation counterbalanced. Mismatch negativity (MMN) responses showed that listeners discriminated the non-native tonal contrast only in the second block, reflecting indications of learning through exposure during the first block. In addition, listeners showed a later MMN peak for their second block of test relative to listeners who did the same block first, suggesting linguistic (as opposed to acoustic) processing or a misapplication of perceptual strategies from the first to the second block. The results also showed a perceptual asymmetry for change in pitch direction: listeners who encountered a falling tone deviant in the first block had larger frontal MMN amplitudes than listeners who encountered a level tone deviant in the first block. The implications of our findings for second language speech and the developmental trajectory for tone perception are discussed.

Monolingual and Bilingual Infants' Ability to Use Non-native Tone for Word Learning Deteriorates by the Second Year After Birth

Previous studies reported a non-native word learning advantage for bilingual infants at around 18 months. We investigated developmental changes in infant interpretation of sounds that aid in object mapping. Dutch monolingual and bilingual (exposed to Dutch and a second non-tone-language) infants' word learning ability was examined on two novel label-object pairings using syllables differing in Mandarin tones as labels (flat vs. falling). Infants aged 14-15 months, regardless of language backgrounds, were sensitive to violations in the label-objects pairings when lexical tones were switched compared to when they were the same as habituated. Conversely at 17-18 months, neither monolingual nor bilingual infants demonstrated learning. Linking with existing literature, infants' ability to associate non-native tones with meanings may be related to tonal acoustic properties and/or perceptual assimilation to native prosodic categories. These findings provide new insights into the relation between infant tone perception, learning, and interpretative narrowing from a developmental perspective.

Perceptual Improvement of Lexical Tones in Infants: Effects of Tone Language Experience

To learn words in a tonal language, tone-language learners should not only develop better abilities for perceiving consonants and vowels, but also for lexical tones. The divergent trend of enhancing sensitivity to native phonetic contrasts and reduced sensitivity to non-native phonetic contrast is theoretically essential to evaluate effects of listening to an ambient language on speech perception development. The loss of sensitivity in discriminating lexical tones among non-tonal language-learning infants was apparent between 6 and 12 months of age, but only few studies examined trends of differentiating native lexical tones in infancy. The sensitivity in discriminating lexical tones among 6-8 and 10-12 month-old Mandarin-learning infants (n = 120) was tested in Experiment 1 using three lexical tone contrasts of Mandarin. Facilitation of linguistic experience was shown in the tonal contrast (Tone 1 vs. 3), but both age groups performed similar in the other two tonal contrasts (Tone 2 vs. 4; Tone 2 vs. 3). In Experiment 2, 6-8 and 10-12 month-old Mandarin-learning infants (n = 90) were tested with tonal contrasts that have pitch contours either similar to or inverse from lexical tones in Mandarin, and perceptual improvement was shown only in a tonal contrast with familiar pitch contours (i.e., Tone 1 vs. 3). In Experiment 3, 6-8 and 10-12 month-old English-learning infants (n = 40) were tested with Tone 1 vs. 3 contrast of Mandarin and showed an improvement in the perception of non-native lexical tones. This study reveals that tone-language learning infants develop more accurate representations of lexical tones around their first birthday, and the results of both tone and non-tone language-learning infants imply that the rate of development depends on listening experience and the acoustical salience of specific tone contrasts.

Loudness Contour Can Influence Mandarin Tone Recognition: Vocoder Simulation and Cochlear Implants

Lexical tone recognition with current cochlear implants (CI) remains unsatisfactory due to significantly degraded pitch-related acoustic cues, which dominate the tone recognition by normal-hearing (NH) listeners. Several secondary cues (e.g., amplitude contour, duration, and spectral envelope) that influence tone recognition in NH listeners and CI users have been studied. This work proposes a loudness contour manipulation algorithm, namely Loudness-Tone (L-Tone), to investigate the effects of loudness contour on Mandarin tone recognition and the effectiveness of using loudness cue to enhance tone recognition for CI users. With L-Tone, the intensity of sound samples is multiplied by gain values determined by instantaneous fundamental frequencies (F0s) and pre-defined gain- F0 mapping functions. Perceptual experiments were conducted with a four-channel noise-band vocoder simulation in NH listeners and with CI users. The results suggested that 1) loudness contour is a useful secondary cue for Mandarin tone recognition, especially when pitch cues are significantly degraded; 2) L-Tone can be used to improve Mandarin tone recognition in both simulated and actual CI-hearing without significant negative effect on vowel and consonant recognition. L-Tone is a promising algorithm for incorporation into real-time CI processing and off-line CI rehabilitation training software.