Integral perception, but separate processing: The perceptual normalization of lexical tones and vowels

The neural mechanisms underlying the processing of consonant, vowel and tone during Chinese typing: an fNIRS study

Many studies have explored the role of consonant, vowel, and tone in Chinese word identification or sentence comprehension. However, few studies have explored their roles and neural basis during Chinese word production, especially when involving neural basis. The present fNIRS study investigated the neural mechanisms of consonant, vowel, and tone processing during Chinese typing. Participants were asked to name the Chinese characters displayed on a computer screen by typing on a keyboard while hearing a simultaneously presented auditory stimulus. The auditory stimulus was either consistent with the characters' pronunciation (consistent condition) or mismatched in the consonant, vowel, or tone of the character pronunciation. The fNIRS results showed that compared with the consistent condition (as baseline), the consonant mismatch condition evoked lower levels of oxygenated hemoglobin (HbO) activation in the left inferior frontal gyrus Broca's triangle and left superior temporal gyrus. Vowel mismatch condition evoked a higher level of HbO activation in the top of the left inferior frontal gyrus and left middle frontal gyrus. The regions and patterns of brain activation evoked by tone mismatch were the same as those of vowel mismatch. The study indicated that consonant, vowel and tone all play a role in Chinese character production. The sensitive brain areas were all in the left hemisphere. However, the neural mechanism of consonant processing differed from vowel processing in both brain regions and patterns, while tone and vowel processing shared the same regions.

Production of Tone 2 in disyllabic words in Mandarin Chinese speaking children aged 3-5 with a cochlear implant and a contralateral hearing aid

To investigate Mandarin Tone 2 production of disyllabic words of prelingually deafened children with a cochlear implant (CI) and a contralateral hearing aid (HA) and to evaluate the relationship between their demographic variables and tone-production ability. Thirty prelingually Mandarin-speaking preschoolers with CI+HA and 30 age-matched normal-hearing (NH) children participated in the study. Fourteen disyllabic words were recorded from each child. A total of 840 tokens (14 × 60) were then used in tone-perception tests in which four speech therapists participated. The production of T2-related disyllabic words of the bimodal group was significantly worse than that of the NH group, as reflected in the overall accuracy (88.57% ± 16.31% vs 99.29% ± 21.79%, p < 0.05), the accuracy of T1+T2 (93.33% vs 100%), the accuracy of T2+T1 (66.67 ± 37.91% vs 98.33 ± 9.13%), and the accuracy of T2+T4 (78.33 ± 33.95% vs 100%). In addition, the bimodal group showed significantly inferior production accuracy of T2+T1 than T2+T2 and T3+T2, p < 0.05. Both bimodal age and implantation age were significantly negatively correlated with the overall production accuracy, p < 0.05. For the error patterns, bimodal participants experienced more errors when T2 was in the first position of the tone combination, and T2 was most likely to be mispronounced as T1 and T3. Bimodal patients aged 3-5 have T2-related disyllabic lexical tone production defects, and their performances are related to tone combination, implantation age, and bimodal age.

Development of Achieving Constancy in Lexical Tone Identification With Contextual Cues

PURPOSE

The aim of this study was to explore when and how Mandarin-speaking children use contextual cues to normalize speech variability in perceiving lexical tones. Two different cognitive mechanisms underlying speech normalization (lower level acoustic normalization and higher level acoustic-phonemic normalization) were investigated through the lexical tone identification task in nonspeech contexts and speech contexts, respectively. Besides, another aim of this study was to reveal how domain-general cognitive abilities contribute to the development of the speech normalization process.

METHOD

In this study, 94 five- to eight-year-old Mandarin-speaking children (50 boys, 44 girls) and 24 young adults (14 men, 10 women) were asked to identify ambiguous Mandarin high-level and mid-rising tones in either speech or nonspeech contexts. Furthermore, in this study, we tested participants' pitch sensitivity through a nonlinguistic pitch discrimination task and their working memory using the digit span task.

RESULTS

Higher level acoustic-phonemic normalization of lexical tones emerged at the age of 6 years and was relatively stable thereafter. However, lower level acoustic normalization was less stable across different ages. Neither pitch sensitivity nor working memory affected children's lexical tone normalization.

CONCLUSIONS

Mandarin-speaking children above 6 years of age successfully achieved constancy in lexical tone normalization based on speech contextual cues. The perceptual normalization of lexical tones was not affected by pitch sensitivity and working memory capacity.

Decoding lexical tones and vowels in imagined tonal monosyllables using fNIRS signals

Objective.Speech is a common way of communication. Decoding verbal intent could provide a naturalistic communication way for people with severe motor disabilities. Active brain computer interaction (BCI) speller is one of the most commonly used speech BCIs. To reduce the spelling time of Chinese words, identifying vowels and tones that are embedded in imagined Chinese words is essential. Functional near-infrared spectroscopy (fNIRS) has been widely used in BCI because it is portable, non-invasive, safe, low cost, and has a relatively high spatial resolution.Approach.In this study, an active BCI speller based on fNIRS is presented by covertly rehearsing tonal monosyllables with vowels (i.e. /a/, /i/, /o/, and /u/) and four lexical tones in Mandarin Chinese (i.e. tones 1, 2, 3, and 4) for 10 s.Main results.fNIRS results showed significant differences in the right superior temporal gyrus between imagined vowels with tone 2/3/4 and those with tone 1 (i.e. more activations and stronger connections to other brain regions for imagined vowels with tones 2/3/4 than for those with tone 1). Speech-related areas for tone imagery (i.e. the right hemisphere) provided majority of information for identifying tones, while the left hemisphere had advantages in vowel identification. Having decoded both vowels and tones during the post-stimulus 15 s period, the average classification accuracies exceeded 40% and 70% in multiclass (i.e. four classes) and binary settings, respectively. To spell words more quickly, the time window size for decoding was reduced from 15 s to 2.5 s while the classification accuracies were not significantly reduced.Significance.For the first time, this work demonstrated the possibility of discriminating lexical tones and vowels in imagined tonal syllables simultaneously. In addition, the reduced time window for decoding indicated that the spelling time of Chinese words could be significantly reduced in the fNIRS-based BCIs.

Phonetic Encoding Contributes to the Processing of Linguistic Prosody at the Word Level: Cross-Linguistic Evidence From Event-Related Potentials

PURPOSE

This study aimed to examine whether abstract knowledge of word-level linguistic prosody is independent of or integrated with phonetic knowledge.

METHOD

Event-related potential (ERP) responses were measured from 18 adult listeners while they listened to native and nonnative word-level prosody in speech and in nonspeech. The prosodic phonology (speech) conditions included disyllabic pseudowords spoken in Chinese and in English matched for syllabic structure, duration, and intensity. The prosodic acoustic (nonspeech) conditions were hummed versions of the speech stimuli, which eliminated the phonetic content while preserving the acoustic prosodic features.

RESULTS

We observed language-specific effects on the ERP that native stimuli elicited larger late negative response (LNR) amplitude than nonnative stimuli in the prosodic phonology conditions. However, no such effect was observed in the phoneme-free prosodic acoustic control conditions.

CONCLUSIONS

The results support the integration view that word-level linguistic prosody likely relies on the phonetic content where the acoustic cues embedded in. It remains to be examined whether the LNR may serve as a neural signature for language-specific processing of prosodic phonology beyond auditory processing of the critical acoustic cues at the suprasyllabic level.

The time course of normalizing speech variability in vowels

To achieve perceptual constancy, listeners utilize contextual cues to normalize speech variabilities in speakers. The present study tested the time course of this cognitive process with an event-related potential (ERP) experiment. The first neurophysiological evidence of speech normalization is observed in P2 (130-250 ms), which is functionally related to phonetic and phonological processes. Furthermore, the normalization process was found to ease lexical retrieval, as indexed by smaller N400 (350-470 ms) after larger P2. A cross-language vowel perception task was carried out to further specify whether normalization was processed in the phonetic and/or phonological stage(s). It was found that both phonetic and phonological cues in the speech context contributed to vowel normalization. The results suggest that vowel normalization in the speech context can be observed in the P2 time window and largely overlaps with phonetic and phonological processes.

Music Does Not Facilitate Lexical Tone Normalization: A Speech-Specific Perceptual Process

Listeners utilize the immediate contexts to efficiently normalize variable vocal streams into standard phonology units. However, researchers debated whether non-speech contexts can also serve as valid clues for speech normalization. Supporters of the two sides proposed a general-auditory hypothesis and a speech-specific hypothesis to explain the underlying mechanisms. A possible confounding factor of this inconsistency is the listeners' perceptual familiarity of the contexts, as the non-speech contexts were perceptually unfamiliar to listeners. In this study, we examined this confounding factor by recruiting a group of native Cantonese speakers with sufficient musical training experience and a control group with minimal musical training. Participants performed lexical tone judgment tasks in three contextual conditions, i.e., speech, non-speech, and music context conditions. Both groups were familiar with the speech context and not familiar with the non-speech context. The musician group was more familiar with the music context than the non-musician group. The results evidenced the lexical tone normalization process in speech context but not non-speech nor music contexts. More importantly, musicians did not outperform non-musicians on any contextual conditions even if the musicians were experienced at pitch perception, indicating that there is no noticeable transfer in pitch perception from the music domain to the linguistic domain for tonal language speakers. The findings showed that even high familiarity with a non-linguistic context cannot elicit an effective lexical tone normalization process, supporting the speech-specific basis of the perceptual normalization process.