Enhanced production and perception of musical pitch in tone language speakers

Voice actors show enhanced neural tracking of pitch, prosody perception, and music perception

Experiences with sound that make strong demands on the precision of perception, such as musical training and experience speaking a tone language, can enhance auditory neural encoding. Are high demands on the precision of perception necessary for training to drive auditory neural plasticity? Voice actors are an ideal subject population for answering this question. Voice acting requires exaggerating prosodic cues to convey emotion, character, and linguistic structure, drawing upon attention to sound, memory for sound features, and accurate sound production, but not fine perceptual precision. Here we assessed neural encoding of pitch using the frequency-following response (FFR), as well as prosody, music, and sound perception, in voice actors and a matched group of non-actors. We find that the consistency of neural sound encoding, prosody perception, and musical phrase perception are all enhanced in voice actors, suggesting that a range of neural and behavioural auditory processing enhancements can result from training which lacks fine perceptual precision. However, fine discrimination was not enhanced in voice actors but was linked to degree of musical experience, suggesting that low-level auditory processing can only be enhanced by demanding perceptual training. These findings suggest that training which taxes attention, memory, and production but is not perceptually taxing may be a way to boost neural encoding of sound and auditory pattern detection in individuals with poor auditory skills.

Perceptual Sensitivity to Tonal Alignment in Nuer

This paper examines the perceptual threshold in patterns of tonal timing (alignment) of Falling versus Low tones. The results indicate a remarkable sensitivity among the listeners. In a perception experiment with 30 participants, we tested how native speakers of the West Nilotic language Nuer responded to stimuli in which the timing of the F0 fall that distinguishes Low versus Fall following a High target is manipulated. We measured the threshold for the responses to shift tone perception from 25% to 75%. The results show that listeners only needed an average of 19 ms to differentiate between the melodic shapes and as little as 13 ms for one item. Perceptual sensitivity this fine-grained is not expected based on what is known about the Just Noticeable Difference (JND) from previous studies. Results from non-tonal languages report a sensitivity threshold for tonal timing of at least 50 ms at category boundaries. This difference suggests that whether or not subjects speak a tone language may be a determining factor in their JND.

Articulation posture influences pitch during singing imagery

Facial muscle activity contributes to singing and to articulation: in articulation, mouth shape can alter vowel identity; and in singing, facial movement correlates with pitch changes. Here, we examine whether mouth posture causally influences pitch during singing imagery. Based on perception-action theories and embodied cognition theories, we predict that mouth posture influences pitch judgments even when no overt utterances are produced. In two experiments (total N = 160), mouth posture was manipulated to resemble the articulation of either /i/ (as in English meet; retracted lips) or /o/ (as in French rose; protruded lips). Holding this mouth posture, participants were instructed to mentally "sing" given songs (which were all positive in valence) while listening with their inner ear and, afterwards, to assess the pitch of their mental chant. As predicted, compared to the o-posture, the i-posture led to higher pitch in mental singing. Thus, bodily states can shape experiential qualities, such as pitch, during imagery. This extends embodied music cognition and demonstrates a new link between language and music.

The Relationship Between Autism and Pitch Perception is Modulated by Cognitive Abilities

Previous studies reported mixed findings on autistic individuals' pitch perception relative to neurotypical (NT) individuals. We investigated whether this may be partly due to individual differences in cognitive abilities by comparing their performance on various pitch perception tasks on a large sample (n = 164) of autistic and NT children and adults. Our findings revealed that: (i) autistic individuals either showed similar or worse performance than NT individuals on the pitch tasks; (ii) cognitive abilities were associated with some pitch task performance; and (iii) cognitive abilities modulated the relationship between autism diagnosis and pitch perception on some tasks. Our findings highlight the importance of taking an individual differences approach to understand the strengths and weaknesses of pitch processing in autism.

Language experience predicts music processing in a half-million speakers of fifty-four languages

Tonal languages differ from other languages in their use of pitch (tones) to distinguish words. Lifelong experience speaking and hearing tonal languages has been argued to shape auditory processing in ways that generalize beyond the perception of linguistic pitch to the perception of pitch in other domains like music. We conducted a meta-analysis of prior studies testing this idea, finding moderate evidence supporting it. But prior studies were limited by mostly small sample sizes representing a small number of languages and countries, making it challenging to disentangle the effects of linguistic experience from variability in music training, cultural differences, and other potential confounds. To address these issues, we used web-based citizen science to assess music perception skill on a global scale in 34,034 native speakers of 19 tonal languages (e.g., Mandarin, Yoruba). We compared their performance to 459,066 native speakers of other languages, including 6 pitch-accented (e.g., Japanese) and 29 non-tonal languages (e.g., Hungarian). Whether or not participants had taken music lessons, native speakers of all 19 tonal languages had an improved ability to discriminate musical melodies on average, relative to speakers of non-tonal languages. But this improvement came with a trade-off: tonal language speakers were also worse at processing the musical beat. The results, which held across native speakers of many diverse languages and were robust to geographic and demographic variation, demonstrate that linguistic experience shapes music perception, with implications for relations between music, language, and culture in the human mind.

Auditory Pitch Perception in Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

PURPOSE

Pitch plays an important role in auditory perception of music and language. This study provides a systematic review with meta-analysis to investigate whether individuals with autism spectrum disorder (ASD) have enhanced pitch processing ability and to identify the potential factors associated with processing differences between ASD and neurotypicals.

METHOD

We conducted a systematic search through six major electronic databases focusing on the studies that used nonspeech stimuli to provide a qualitative and quantitative assessment across existing studies on pitch perception in autism. We identified potential participant- and methodology-related moderators and conducted metaregression analyses using mixed-effects models.

RESULTS

On the basis of 22 studies with a total of 464 participants with ASD, we obtained a small-to-medium positive effect size (g = 0.26) in support of enhanced pitch perception in ASD. Moreover, the mean age and nonverbal IQ of participants were found to significantly moderate the between-studies heterogeneity.

CONCLUSIONS

Our study provides the first meta-analysis on auditory pitch perception in ASD and demonstrates the existence of different developmental trajectories between autistic individuals and neurotypicals. In addition to age, nonverbal ability is found to be a significant contributor to the lower level/local processing bias in ASD. We highlight the need for further investigation of pitch perception in ASD under challenging listening conditions. Future neurophysiological and brain imaging studies with a longitudinal design are also needed to better understand the underlying neural mechanisms of atypical pitch processing in ASD and to help guide auditory-based interventions for improving language and social functioning.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21614271.

How experience with tone in the native language affects the L2 acquisition of pitch accents

This paper tested the ability of Mandarin learners of German, whose native language has lexical tone, to imitate pitch accent contrasts in German, an intonation language. In intonation languages, pitch accents do not convey lexical information; also, pitch accents are sparser than lexical tones as they only associate with prominent words in the utterance. We compared two kinds of German pitch-accent contrasts: (1) a “non-merger” contrast, which Mandarin listeners perceive as different and (2) a “merger” contrast, which sounds more similar to Mandarin listeners. Speakers of a tone language are generally very sensitive to pitch. Hypothesis 1 (H1) therefore stated that Mandarin learners produce the two kinds of contrasts similarly to native German speakers. However, the documented sensitivity to tonal contrasts, at the expense of processing phrase-level intonational contrasts, may generally hinder target-like production of intonational pitch accents in the L2 (Hypothesis 2, H2). Finally, cross-linguistic influence (CLI) predicts a difference in the realization of these two contrasts as well as improvement with higher proficiency (Hypothesis 3, H3). We used a delayed imitation paradigm, which is well-suited for assessing L2-phonetics and -phonology because it does not necessitate access to intonational meaning. We investigated the imitation of three kinds of accents, which were associated with the sentence-final noun in short wh-questions (e.g., Wer malt denn Mandalas, lit: “Who draws PRT mandalas?” “Who likes drawing mandalas?”). In Experiment 1, 28 native speakers of Mandarin participated (14 low- and 14 high-proficient). The learners’ productions of the two kinds of contrasts were analyzed using General Additive Mixed Models to evaluate differences in pitch accent contrasts over time, in comparison to the productions of native German participants from an earlier study in our lab. Results showed a more pronounced realization of the non-merger contrast compared to German natives and a less distinct realization of the merger contrast, with beneficial effects of proficiency, lending support to H3. Experiment 2 tested low-proficient Italian learners of German (whose L1 is an intonation language) to contextualize the Mandarin data and further investigate CLI. Italian learners realized the non-merger contrast more target-like than Mandarin learners, lending additional support to CLI (H3).

Pitch chroma information is processed in addition to pitch height information with more than two pitch-range categories

Octave equivalence describes the perception that notes separated by a doubling in frequency sound similar. While the octave is used cross-culturally as a basis of pitch perception, experimental demonstration of the phenomenon has proved to be difficult. In past work, members of our group developed a three-range generalization paradigm that reliably demonstrated octave equivalence. In this study we replicate and expand on this previous work trying to answer three questions that help us understand the origins and potential cross-cultural significance of octave equivalence: (1) whether training with three ranges is strictly necessary or whether an easier-to-learn two-range task would be sufficient, (2) whether the task could demonstrate octave equivalence beyond neighbouring octaves, and (3) whether language skills and musical education impact the use of octave equivalence in this task. We conducted a large-sample study using variations of the original paradigm to answer these questions. Results found here suggest that the three-range discrimination task is indeed vital to demonstrating octave equivalence. In a two-range task, pitch height appears to be dominant over octave equivalence. Octave equivalence has an effect only when pitch height alone is not sufficient. Results also suggest that effects of octave equivalence are strongest between neighbouring octaves, and that tonal language and musical training have a positive effect on learning of discriminations but not on perception of octave equivalence during testing. We discuss these results considering their relevance to future research and to ongoing debates about the basis of octave equivalence perception.

Musicianship Influences Language Effect on Musical Pitch Perception

Given its practical implications, the effect of musicianship on language learning has been vastly researched. Interestingly, growing evidence also suggests that language experience can facilitate music perception. However, the precise nature of this facilitation is not fully understood. To address this research gap, I investigated the interactive effect of language and musicianship on musical pitch and rhythmic perception. Cantonese and English listeners, each divided into musician and non-musician groups, completed the Musical Ear Test and the Raven’s 2 Progressive Matrices. Essentially, an interactive effect of language and musicianship was found on musical pitch but not rhythmic perception. Consistent with previous studies, Cantonese language experience appeared to facilitate musical pitch perception. However, this facilitatory effect was only present among the non-musicians. Among the musicians, Cantonese language experience did not offer any perceptual advantage. The above findings reflect that musicianship influences the effect of language on musical pitch perception. Together with the previous findings, the new findings offer two theoretical implications for the OPERA hypothesis—bi-directionality and mechanisms through which language experience and musicianship interact in different domains.

Context-dependent Plasticity and Strength of Subcortical Encoding of Musical Sounds Independently Underlie Pitch Discrimination for Music Melodies

Subcortical auditory nuclei contribute to pitch perception, but how subcortical sound encoding is related to pitch processing for music perception remains unclear. Conventionally, enhanced subcortical sound encoding is considered underlying superior pitch discrimination. However, associations between superior auditory perception and the context-dependent plasticity of subcortical sound encoding are also documented. Here, we explored the subcortical neural correlates to music pitch perception by analyzing frequency-following responses (FFRs) to musical sounds presented in a predictable context and a random context. We found that the FFR inter-trial phase-locking (ITPL) was negatively correlated with behavioral performances of discrimination of pitches in music melodies. It was also negatively correlated with the plasticity indices measuring the variability of FFRs to physically identical sounds between the two contexts. The plasticity indices were consistently positively correlated with pitch discrimination performances, suggesting the subcortical context-dependent plasticity underlying music pitch perception. Moreover, the raw FFR spectral strength was not significantly correlated with pitch discrimination performances. However, it was positively correlated with behavioral performances when the FFR ITPL was controlled by partial correlations, suggesting that the strength of subcortical sound encoding underlies music pitch perception. When the spectral strength was controlled by partial correlations, the negative ITPL-behavioral correlations were maintained. Furthermore, the FFR ITPL, the plasticity indices, and the FFR spectral strength were more correlated with pitch than with rhythm discrimination performances. These findings suggest that the context-dependent plasticity and the strength of subcortical encoding of musical sounds are independently and perhaps specifically associated with pitch perception for music melodies.

Effects of perceptual abilities and lexical knowledge on the phonetic categorization of second language speech

This study examined how phonetic categorization in a second language (L2) is jointly affected by perceptual abilities and lexical knowledge. Adult L1 Mandarin Chinese and L1 English-L2 Mandarin learners performed a phonetic categorization task. The stimuli varied the F0 contour along a continuum resulting in four different tonal word/nonword end point combinations. Both L1 and L2 listeners categorized more ambiguous tokens as words than nonwords, thus demonstrating a lexical bias in their behavior, i.e., the Ganong effect. Non-phonetic, linguistic information can thus modify L2 phonetic categorization of lexical tones. This effect, however, can be constrained by the listener's pitch perception abilities.

Emotion processing in congenital amusia: the deficits do not generalize to written emotion words

Congenital amusia is a lifelong impairment in musical ability. Individuals with amusia are found to show reduced sensitivity to emotion recognition in speech prosody and silent facial expressions, implying a possible cross-modal emotion-processing deficit. However, it is not clear whether the observed deficits are primarily confined to socio-emotional contexts, where visual cues (facial expression) often co-occur with auditory cues (emotion prosody) to express intended emotions, or extend to linguistic emotion processing. In order to better understand the underlying deficiency mechanism of emotion processing in individuals with amusia, we examined whether reduced sensitivity to emotional processing extends to the recognition of emotion category and valence of written words in individuals with amusia. Twenty Cantonese speakers with amusia and 17 controls were tested in three experiments: (1) emotion prosody rating, in which participants rated how much each spoken sentence was expressed in each of the four emotions on 7-point rating scales; (2) written word emotion recognition, in which participants recognized the emotion of written emotion words; and (3) written word valence judgment, in which participants judged the valence of written words. Results showed that participants with amusia preformed significantly less accurately than controls in emotion prosody recognition; in contrast, the two groups showed no significant difference in accuracy rates in both written word tasks (emotion recognition and valence judgment). The results indicate that the impairment of individuals with amusia in emotion processing may not generalize to linguistic emotion processing in written words, implying that the emotion deficit is likely to be restricted to socio-emotional contexts in individuals with amusia.

Human Auditory-Frequency Tuning Is Sensitive to Tonal Language Experience

Purpose The aim of this study is to investigate whether human auditory frequency tuning can be influenced by tonal language experience. Method Perceptual tuning measured via psychophysical tuning curves and cochlear tuning derived via stimulus-frequency otoacoustic emission suppression tuning curves in 14 native speakers of a tonal language (Mandarin) were compared to those of 14 native speakers of a nontonal language (English) at 1 and 4 kHz. Results Group comparisons of both psychophysical tuning curves (p = .046) and stimulus-frequency otoacoustic emission suppression tuning curves (p = .007) in the 4-kHz region indicated sharper frequency tuning in the Mandarin-speaking group relative to the English-speaking group. The auditory tuning was better at the higher (4 kHz) than the lower (1 kHz) probe frequencies (p < .001). Conclusions The sharper auditory tuning in the 4-kHz cochlear region is associated with long-term tonal language (i.e., Mandarin) experience. Experience-dependent plasticity of tonal language may occur before the sound signal reaches central neural stages, as peripheral as the cochlea.

Frequency selectivity of tonal language native speakers probed by suppression tuning curves of spontaneous otoacoustic emissions

Native acquisition of a tonal language (TL) is related to enhanced abilities of pitch perception and production, compared to non-tonal language (NTL) native speakers. Moreover, differences in brain responses to both linguistically relevant and non-relevant pitch changes have been described in TL native speakers. It is so far unclear to which extent differences are present at the peripheral processing level of the cochlea. To determine possible differences in cochlear frequency selectivity between Asian TL speakers and Caucasian NTL speakers, suppression tuning curves (STCs) of spontaneous otoacoustic emissions (SOAEs) were examined in both groups. By presenting pure tones, SOAE levels were suppressed and STCs were derived. SOAEs with center frequencies higher than 4.5 kHz were recorded only in female TL native speakers, which correlated with better high-frequency tone detection thresholds. The suppression thresholds at the tip of the STC and filter quality coefficient Q_10dB did not differ significantly between both language groups. Thus, the characteristics of the STCs of SOAEs do not support the presence of differences in peripheral auditory processing between TL and NTL native speakers.

Reduced Sensitivity to Between-Category Information but Preserved Categorical Perception of Lexical Tones in Tone Language Speakers With Congenital Amusia

Previous studies have shown that for congenital amusics, long-term tone language experience cannot compensate for lexical tone processing difficulties. However, it is still unknown whether such difficulties are merely caused by domain-transferred insensitivity in lower-level acoustic processing and/or by higher-level phonological processing of linguistic pitch as well. The current P300 study links and extends previous studies by uncovering the neurophysiological mechanisms underpinning lexical tone perception difficulties in Mandarin-speaking amusics. Both the behavioral index (d′) and P300 amplitude showed reduced within-category as well as between-category sensitivity among the Mandarin-speaking amusics regardless of the linguistic status of the signal. The results suggest that acoustic pitch processing difficulties in amusics are manifested profoundly and further persist into the higher-level phonological processing that involves the neural processing of different lexical tone categories. Our findings indicate that long-term tone language experience may not compensate for the reduced acoustic pitch processing in tone language speakers with amusia but rather may extend to the neural processing of the phonological information of lexical tones during the attentive stage. However, from both the behavioral and neural evidence, the peakedness scores of the d′ and P300 amplitude were comparable between amusics and controls. It seems that the basic categorical perception (CP) pattern of native lexical tones is preserved in Mandarin-speaking amusics, indicating that they may have normal or near normal long-term categorical memory.

Effects of language experience on domain-general perceptual strategies

Speech and music are highly redundant communication systems, with multiple acoustic cues signaling the existence of perceptual categories. This redundancy makes these systems robust to the influence of noise, but necessitates the development of perceptual strategies: listeners need to decide how much importance to place on each source of information. Prior empirical work and modeling has suggested that cue weights primarily reflect within-task statistical learning, as listeners assess the reliability with which different acoustic dimensions signal a category and modify their weights accordingly. Here we present evidence that perceptual experience can lead to changes in cue weighting that extend across tasks and across domains, suggesting that perceptual strategies reflect both global biases and local (i.e. task-specific) learning. In two experiments, native speakers of Mandarin (N = 45)-where pitch is a crucial cue to word identity-placed more importance on pitch and less importance on other dimensions compared to native speakers of non-tonal languages English (N = 45) and Spanish (N = 27), during the perception of both English speech and musical beats. In a third experiment, we further show that Mandarin speakers are better able to attend to pitch and ignore irrelevant variation in other dimensions in speech compared to English and Spanish speakers, and even struggle to ignore pitch when asked to attend to other dimensions. Thus, an individual's idiosyncratic auditory perceptual strategy reflects a complex mixture of congenital predispositions, task-specific learning, and biases instilled by extensive experience in making use of important dimensions in their native language.

The Influence of Tonal and Atonal Bilingualism on Children's Lexical and Non-Lexical Tone Perception

This study examined how bilingualism in an atonal language, in addition to a tonal language, influences lexical and non-lexical tone perception and word learning during childhood. Forty children aged 5;3-7;2, bilingual either in English and Mandarin or English and another atonal language, were tested on Mandarin lexical tone discrimination, level-pitch sine-wave tone discrimination, and learning of novel words differing minimally in Mandarin lexical tone. Mandarin-English bilingual children discriminated between and learned novel words differing minimally in Mandarin lexical tone more accurately than their atonal-English bilingual peers. However, Mandarin-English and atonal-English bilingual children discriminated between level-pitch sine-wave tones with similar accuracy. Moreover, atonal-English bilingual children showed a tendency to perceive differing Mandarin lexical and level-pitch sine-wave tones as identical, whereas their Mandarin-English peers showed no such tendency. These results indicate that bilingualism in a tonal language in addition to an atonal language-but not bilingualism in two atonal languages-allows for continued sensitivity to lexical tone beyond infancy. Moreover, they suggest that although tonal-atonal bilingualism does not enhance sensitivity to differences in pitch between sine-wave tones beyond infancy any more effectively than atonal-atonal bilingualism, it protects against the development of biases to perceive differing lexical and non-lexical tones as identical. Together, the results indicate that, beyond infancy, tonal-atonal bilinguals process lexical tones using different cognitive mechanisms than atonal-atonal bilinguals, but that both groups process level-pitch non-lexical tone using the same cognitive mechanisms.

A Minimum Temporal Window for Direction Detection of Frequency-Modulated Sweeps: A Magnetoencephalography Study

The ability to rapidly encode the direction of frequency contour contained in frequency-modulated (FM) sweeps is essential for speech processing, music appreciation, and conspecific communications. Psychophysical evidence points to a common temporal window threshold for human listeners in processing rapid changes in frequency glides. No neural evidence has been provided for the existence of a cortical temporal window threshold underlying the encoding of rapid transitions in frequency glides. The present magnetoencephalography study used the cortical mismatch negativity activity (MMNm) to investigate the minimum temporal window required for detecting different magnitudes of directional changes in frequency-modulated sweeps. A deviant oddball paradigm was used in which directional upward or downward frequency sweep serves as the standard and the same type of sweep with the opposite direction serves as its deviant. Stimuli consisted of unidirectional linear frequency-sweep complexes that swept across speech-relevant frequency bands in durations of 10, 20, 40, 80, 160, and 320 ms (with corresponding rates of 50, 25, 12.5, 6.2, 3.1, 1.5 oct/s). The data revealed significant magnetic mismatch field responses across all sweep durations, with slower-rate sweeps eliciting larger MMNm responses. A greater temporally related enhancement in MMNm response was obtained for rising but not falling frequency sweep contours. A hemispheric asymmetry in the MMNm response pattern was observed corresponding to the directionality of frequency sweeps. Contrary to psychophysical findings, we report a temporal window as short as 10 ms sufficient to elicit a robust MMNm response to a directional change in speech-relevant frequency contours. The results suggest that auditory cortex requires extremely brief temporal window to implicitly differentiate a dynamic change in frequency of linguistically relevant pitch contours. That the brain is extremely sensitive to fine spectral changes contained in speech-relevant glides provides cortical evidence for the ecological importance of FM sweeps in speech processing.

Dong speakers outperform Mandarin speakers in behavioral pitch discrimination

Tonal language speakers outperform non-tonal language speakers in behavioral pitch discrimination. Because the tone system differs in complexity across different tonal languages, it is unknown whether pitch discrimination differs across individuals speaking different tonal languages. There are nine tones in Dong but only four in Mandarin. This study investigates whether behavioral pitch discrimination is superior in Dong speakers compared to Mandarin speakers. Behavioral pitch discrimination was indexed by difference limens measured using pure tones and harmonic tones. The results indicate that Dong speakers outperformed Mandarin speakers in pitch discrimination tasks.

The effects of musicality and language background on cue integration in pitch perception

Pitch perception involves the processing of multidimensional acoustic cues, and listeners can exhibit different cue integration strategies in interpreting pitch. This study aims to examine whether musicality and language experience have effects on listeners' pitch perception strategies. Both Mandarin and English listeners were recruited to participate in two experiments: (1) a pitch classification experiment that tested their relative reliance on f0 and spectral cues, and (2) the Montreal Battery of Evaluation of Musical Abilities that objectively quantified their musical aptitude as continuous musicality scores. Overall, the results show a strong musicality effect: Listeners with higher musicality scores relied more on f0 in pitch perception, while listeners with lower musicality scores were more likely to attend to spectral cues. However, there were no effects of language experience on musicality scores or cue integration strategies in pitch perception. These results suggest that less musical or even amusic subjects may not suffer impairment in linguistic pitch processing due to the multidimensional nature of pitch cues.

Pitch-specific contributions of auditory imagery and auditory memory in vocal pitch imitation

Vocal imitation guides both music and language development. Despite the developmental significance of this behavior, a sizable minority of individuals are inaccurate at vocal pitch imitation. Although previous research suggested that inaccurate pitch imitation results from deficient sensorimotor associations between pitch perception and vocal motor planning, the cognitive processes involved in sensorimotor translation are not clearly defined. In the present research, we investigated the roles of basic cognitive processes in the vocal imitation of pitch, as well as the degree to which these processes rely on pitch-specific resources. In the present study, participants completed a battery of pitch and verbal tasks to measure pitch perception, pitch and verbal auditory imagery, pitch and verbal auditory short-term memory, and pitch imitation ability. Information on participants' music background was collected, as well. Pitch imagery, pitch short-term memory, pitch discrimination ability, and musical experience were unique predictors of pitch imitation ability. Furthermore, pitch imagery was a partial mediator of the relationship between pitch short-term memory and pitch imitation ability. These results indicate that vocal imitation recruits cognitive processes that rely on at least partially separate neural resources for pitch and verbal representations.

Verbal and Musical Short-Term Memory: Evidence for Shared Serial Order Processes?

This study explored the validity of an integrative framework for verbal and musical short-term memory (STM). Following this framework, access to domain-specific long-term knowledge bases supports the processing of musical and verbal item information in STM, while domain-general ordering processes support the representation of serial order information in the two domains. We exposed participants to verbal and musical STM tasks assessing either item information, order information, or both item and order information. Using an interindividual differences approach, we observed that performance in item-based STM tasks was not strongly associated between musical and verbal domains. In contrast, strong between-domain associations were observed for STM tasks assessing processing of verbal order and musical rhythm information. These preliminary results are overall in agreement with an integrative approach of verbal and musical STM. At the same time, the results highlight the difficulty of measuring serial order processing in the musical STM domain in a direct and specific manner.

A tonal-language benefit for pitch in normally-hearing and cochlear-implanted children

In tonal languages, voice pitch inflections change the meaning of words, such that the brain processes pitch not merely as an acoustic characterization of sound but as semantic information. In normally-hearing (NH) adults, this linguistic pressure on pitch appears to sharpen its neural encoding and can lead to perceptual benefits, depending on the task relevance, potentially generalizing outside of the speech domain. In children, however, linguistic systems are still malleable, meaning that their encoding of voice pitch information might not receive as much neural specialization but might generalize more easily to ecologically irrelevant pitch contours. This would seem particularly true for early-deafened children wearing a cochlear implant (CI), who must exhibit great adaptability to unfamiliar sounds as their sense of pitch is severely degraded. Here, we provide the first demonstration of a tonal language benefit in dynamic pitch sensitivity among NH children (using both a sweep discrimination and labelling task) which extends partially to children with CI (i.e., in the labelling task only). Strong age effects suggest that sensitivity to pitch contours reaches adult-like levels early in tonal language speakers (possibly before 6 years of age) but continues to develop in non-tonal language speakers well into the teenage years. Overall, we conclude that language-dependent neuroplasticity can enhance behavioral sensitivity to dynamic pitch, even in extreme cases of auditory degradation, but it is most easily observable early in life.

Tone language experience modulates the effect of long-term musical training on musical pitch perception

Long-term musical training is widely reported to enhance music pitch perception. However, it remains unclear whether tone language experience influences the effect of long-term musical training on musical pitch perception. The present study addressed this question by testing 30 Cantonese and 30 non-tonal language speakers, each divided equally into musician and non-musician groups, on pitch height and pitch interval discrimination. Musicians outperformed non-musicians among non-tonal language speakers, but not among Cantonese speakers on the pitch height discrimination task. However, musicians outperformed non-musicians among Cantonese speakers, but not among non-tonal language speakers on the pitch interval discrimination task. These results suggest that the effect of long-term musical training on musical pitch perception is shaped by tone language experience and varies across different pitch perception tasks.

Incongruent pitch cues are associated with increased activation and functional connectivity in the frontal areas

Pitch plays a crucial role in music and speech perception. Pitch perception is characterized by multiple perceptual dimensions, such as pitch height and chroma. Information provided by auditory signals that are related to these perceptual dimensions can be either congruent or incongruent. To create conflicting cues for pitch perception, we modified Shepard tones by varying the pitch height and pitch chroma dimensions in either the same or opposite directions. Our behavioral data showed that most listeners judged pitch changes based on pitch chroma, instead of pitch height, when incongruent information was provided. The reliance on pitch chroma resulted in a stable percept of upward or downward pitch shift, rather than alternating between two different percepts. Across the incongruent and congruent conditions, consistent activation was found in the bilateral superior temporal and inferior frontal areas. In addition, significantly stronger activation was observed in the inferior frontal areas during the incongruent compared to congruent conditions. Enhanced functional connectivity was found between the left temporal and bilateral frontal areas in the incongruent than congruent conditions. Increased intra-hemispheric and inter-hemispheric connectivity was also observed in the frontal areas. Our results suggest the involvement of the frontal lobe in top-down and bottom-up processes to generate a stable percept of pitch change with conflicting perceptual cues.

The effects of ethnicity, musicianship, and tone language experience on pitch perception

Language and music are intertwined: music training can facilitate language abilities, and language experiences can also help with some music tasks. Possible language–music transfer effects are explored in two experiments in this study. In Experiment 1, we tested native Mandarin, Korean, and English speakers on a pitch discrimination task with two types of sounds: speech sounds and fundamental frequency (F0) patterns derived from speech sounds. To control for factors that might influence participants’ performance, we included cognitive ability tasks testing memory and intelligence. In addition, two music skill tasks were used to examine general transfer effects from language to music. Prior studies showing that tone language speakers have an advantage on pitch tasks have been taken as support for three alternative hypotheses: specific transfer effects, general transfer effects, and an ethnicity effect. In Experiment 1, musicians outperformed non-musicians on both speech and F0 sounds, suggesting a music-to-language transfer effect. Korean and Mandarin speakers performed similarly, and they both outperformed English speakers, providing some evidence for an ethnicity effect. Alternatively, this could be due to population selection bias. In Experiment 2, we recruited Chinese Americans approximating the native English speakers’ language background to further test the ethnicity effect. Chinese Americans, regardless of their tone language experiences, performed similarly to their non–Asian American counterparts in all tasks. Therefore, although this study provides additional evidence of transfer effects across music and language, it casts doubt on the contribution of ethnicity to differences observed in pitch perception and general music abilities.

Auditory Perceptual Abilities Are Associated with Specific Auditory Experience

The extent to which auditory experience can shape general auditory perceptual abilities is still under constant debate. Some studies show that specific auditory expertise may have a general effect on auditory perceptual abilities, while others show a more limited influence, exhibited only in a relatively narrow range associated with the area of expertise. The current study addresses this issue by examining experience-dependent enhancement in perceptual abilities in the auditory domain. Three experiments were performed. In the first experiment, 12 pop and rock musicians and 15 non-musicians were tested in frequency discrimination (DLF), intensity discrimination, spectrum discrimination (DLS), and time discrimination (DLT). Results showed significant superiority of the musician group only for the DLF and DLT tasks, illuminating enhanced perceptual skills in the key features of pop music, in which miniscule changes in amplitude and spectrum are not critical to performance. The next two experiments attempted to differentiate between generalization and specificity in the influence of auditory experience, by comparing subgroups of specialists. First, seven guitar players and eight percussionists were tested in the DLF and DLT tasks that were found superior for musicians. Results showed superior abilities on the DLF task for guitar players, though no difference between the groups in DLT, demonstrating some dependency of auditory learning on the specific area of expertise. Subsequently, a third experiment was conducted, testing a possible influence of vowel density in native language on auditory perceptual abilities. Ten native speakers of German (a language characterized by a dense vowel system of 14 vowels), and 10 native speakers of Hebrew (characterized by a sparse vowel system of five vowels), were tested in a formant discrimination task. This is the linguistic equivalent of a DLS task. Results showed that German speakers had superior formant discrimination, demonstrating highly specific effects for auditory linguistic experience as well. Overall, results suggest that auditory superiority is associated with the specific auditory exposure.

Factors affecting pitch discrimination performance in a cohort of extensively phenotyped healthy volunteers

Despite efforts to characterize the different aspects of musical abilities in humans, many elements of this complex area remain unknown. Musical abilities are known to be associated with factors like intelligence, training, and sex, but a comprehensive evaluation of the simultaneous impact of multiple factors has not yet been performed. Here, we assessed 918 healthy volunteers for pitch discrimination abilities—their ability to tell two tones close in pitch apart. We identified the minimal threshold that the participants could detect, and we found that better performance was associated with higher intelligence, East Asian ancestry, male sex, younger age, formal music training–especially before age 6–and English as the native language. All these factors remained significant when controlling for the others, with general intelligence, musical training, and male sex having the biggest impacts. We also performed a small GWAS and gene-based collapsing analysis, identifying no significant associations. Future genetic studies of musical abilities should involve large sample sizes and an unbiased genome-wide approach, with the factors highlighted here included as important covariates.

Effect of Linguistic and Musical Experience on Distributional Learning of Nonnative Lexical Tones

PURPOSE

Evidence suggests that extensive experience with lexical tones or musical training provides an advantage in perceiving nonnative lexical tones. This investigation concerns whether such an advantage is evident in learning nonnative lexical tones based on the distributional structure of the input.

METHOD

Using an established protocol, distributional learning of lexical tones was investigated with tone language (Mandarin) listeners with no musical training (Experiment 1) and nontone language (Australian English) listeners with musical training (Experiment 2). Within each experiment, participants were trained on a bimodal (2-peak) or a unimodal (single peak) distribution along a continuum spanning a Thai lexical tone minimal pair. Discrimination performance on the target minimal pair was assessed before and after training.

RESULTS

Mandarin nonmusicians exhibited clear distributional learning (listeners in the bimodal, but not those in the unimodal condition, improved significantly as a function of training), whereas Australian English musicians did not (listeners in both the bimodal and unimodal conditions improved as a function of training).

CONCLUSIONS

Our findings suggest that veridical perception of lexical tones is not sufficient for distributional learning of nonnative lexical tones to occur. Rather, distributional learning appears to be modulated by domain-specific pitch experience and is constrained possibly by top-down interference.

Pitch Perception in Tone Language-Speaking Adults With and Without Autism Spectrum Disorders

Enhanced low-level pitch perception has been universally reported in autism spectrum disorders (ASD). This study examined whether tone language speakers with ASD exhibit this advantage. The pitch perception skill of 20 Cantonese-speaking adults with ASD was compared with that of 20 neurotypical individuals. Participants discriminated pairs of real syllable, pseudo-syllable (syllables that do not conform the phonotactic rules or are accidental gaps), and non-speech (syllables with attenuated high-frequency segmental content) stimuli contrasting pitch levels. The results revealed significantly higher discrimination ability in both groups for the non-speech stimuli than for the pseudo-syllables with one semitone difference. No significant group differences were noted. Different from previous findings, post hoc analysis found that enhanced pitch perception was observed in a subgroup of participants with ASD showing no history of delayed speech onset. The tone language experience may have modulated the pitch processing mechanism in the speakers in both ASD and non-ASD groups.

Neural bases of congenital amusia in tonal language speakers

Congenital amusia is a lifelong neurodevelopmental disorder of fine-grained pitch processing. In this fMRI study, we examined the neural bases of congenial amusia in speakers of a tonal language - Cantonese. Previous studies on non-tonal language speakers suggest that the neural deficits of congenital amusia lie in the music-selective neural circuitry in the right inferior frontal gyrus (IFG). However, it is unclear whether this finding can generalize to congenital amusics in tonal languages. Tonal language experience has been reported to shape the neural processing of pitch, which raises the question of how tonal language experience affects the neural bases of congenital amusia. To investigate this question, we examined the neural circuitries sub-serving the processing of relative pitch interval in pitch-matched Cantonese level tone and musical stimuli in 11 Cantonese-speaking amusics and 11 musically intact controls. Cantonese-speaking amusics exhibited abnormal brain activities in a widely distributed neural network during the processing of lexical tone and musical stimuli. Whereas the controls exhibited significant activation in the right superior temporal gyrus (STG) in the lexical tone condition and in the cerebellum regardless of the lexical tone and music conditions, no activation was found in the amusics in those regions, which likely reflects a dysfunctional neural mechanism of relative pitch processing in the amusics. Furthermore, the amusics showed abnormally strong activation of the right middle frontal gyrus and precuneus when the pitch stimuli were repeated, which presumably reflect deficits of attending to repeated pitch stimuli or encoding them into working memory. No significant group difference was found in the right IFG in either the whole-brain analysis or region-of-interest analysis. These findings imply that the neural deficits in tonal language speakers might differ from those in non-tonal language speakers, and overlap partly with the neural circuitries of lexical tone processing (e.g. right STG).

Mandarin-English Bilinguals Process Lexical Tones in Newly Learned Words in Accordance with the Language Context

Previous research has mainly considered the impact of tone-language experience on ability to discriminate linguistic pitch, but proficient bilingual listening requires differential processing of sound variation in each language context. Here, we ask whether Mandarin-English bilinguals, for whom pitch indicates word distinctions in one language but not the other, can process pitch differently in a Mandarin context vs. an English context. Across three eye-tracked word-learning experiments, results indicated that tone-intonation bilinguals process tone in accordance with the language context. In Experiment 1, 51 Mandarin-English bilinguals and 26 English speakers without tone experience were taught Mandarin-compatible novel words with tones. Mandarin-English bilinguals out-performed English speakers, and, for bilinguals, overall accuracy was correlated with Mandarin dominance. Experiment 2 taught 24 Mandarin-English bilinguals and 25 English speakers novel words with Mandarin-like tones, but English-like phonemes and phonotactics. The Mandarin-dominance advantages observed in Experiment 1 disappeared when words were English-like. Experiment 3 contrasted Mandarin-like vs. English-like words in a within-subjects design, providing even stronger evidence that bilinguals can process tone language-specifically. Bilinguals (N = 58), regardless of language dominance, attended more to tone than English speakers without Mandarin experience (N = 28), but only when words were Mandarin-like-not when they were English-like. Mandarin-English bilinguals thus tailor tone processing to the within-word language context.

Audiovisual Interval Size Estimation Is Associated with Early Musical Training

Although pitch is a fundamental attribute of auditory perception, substantial individual differences exist in our ability to perceive differences in pitch. Little is known about how these individual differences in the auditory modality might affect crossmodal processes such as audiovisual perception. In this study, we asked whether individual differences in pitch perception might affect audiovisual perception, as it relates to age of onset and number of years of musical training. Fifty-seven subjects made subjective ratings of interval size when given point-light displays of audio, visual, and audiovisual stimuli of sung intervals. Audiovisual stimuli were divided into congruent and incongruent (audiovisual-mismatched) stimuli. Participants’ ratings correlated strongly with interval size in audio-only, visual-only, and audiovisual-congruent conditions. In the audiovisual-incongruent condition, ratings correlated more with audio than with visual stimuli, particularly for subjects who had better pitch perception abilities and higher nonverbal IQ scores. To further investigate the effects of age of onset and length of musical training, subjects were divided into musically trained and untrained groups. Results showed that among subjects with musical training, the degree to which participants’ ratings correlated with auditory interval size during incongruent audiovisual perception was correlated with both nonverbal IQ and age of onset of musical training. After partialing out nonverbal IQ, pitch discrimination thresholds were no longer associated with incongruent audio scores, whereas age of onset of musical training remained associated with incongruent audio scores. These findings invite future research on the developmental effects of musical training, particularly those relating to the process of audiovisual perception.

Naïve Learners Show Cross-Domain Transfer after Distributional Learning: The Case of Lexical and Musical Pitch

Experienced listeners of a particular acoustic cue in either speech or music appear to have an advantage when perceiving a similar cue in the other domain (i.e., they exhibit cross-domain transfer). One explanation for cross-domain transfer relates to the acquisition of the foundations of speech and music: if acquiring pitch-based elements in speech or music results in heightened attention to pitch in general, then cross-domain transfer of pitch may be observed, which may explain the cross-domain phenomenon seen among listeners of a tone language and listeners with musical training. Here, we investigate this possibility in naïve adult learners, who were trained to acquire pitch-based elements using a distributional learning paradigm, to provide a proof-of-concept for the explanation. Learners were exposed to a stimulus distribution spanning either a Thai lexical tone minimal pair or a novel musical chord minimal pair. Within each domain, the distribution highlights pitch to facilitate learning of two different sounds (Bimodal distribution) or the distribution minimizes pitch so that the input is inferred to be from a single sound (Unimodal distribution). Learning was assessed before and after exposure to the distribution using discrimination tasks with both Thai tone and musical chord minimal pairs. We hypothesize: (i) distributional learning for learners in both the tone and the chord distributions, that is, pre-to-post improvement in discrimination after exposure to the Bimodal but not the Unimodal distribution; and (ii) for both the tone and chord conditions, learners in the Bimodal conditions but not those in the Unimodal conditions will show cross-domain transfer, as indexed by improvement in discrimination of test items in the domain other than what they were trained on. The results support both hypotheses, suggesting that distributional learning is not only used to acquire the foundations of speech and music, but may also play a role in cross-domain transfer: as a result of learning primitives based on a particular cue, learners show heightened attention to that cue in any auditory signal.

Pitch perception and production in congenital amusia: Evidence from Cantonese speakers

This study investigated pitch perception and production in speech and music in individuals with congenital amusia (a disorder of musical pitch processing) who are native speakers of Cantonese, a tone language with a highly complex tonal system. Sixteen Cantonese-speaking congenital amusics and 16 controls performed a set of lexical tone perception, production, singing, and psychophysical pitch threshold tasks. Their tone production accuracy and singing proficiency were subsequently judged by independent listeners, and subjected to acoustic analyses. Relative to controls, amusics showed impaired discrimination of lexical tones in both speech and non-speech conditions. They also received lower ratings for singing proficiency, producing larger pitch interval deviations and making more pitch interval errors compared to controls. Demonstrating higher pitch direction identification thresholds than controls for both speech syllables and piano tones, amusics nevertheless produced native lexical tones with comparable pitch trajectories and intelligibility as controls. Significant correlations were found between pitch threshold and lexical tone perception, music perception and production, but not between lexical tone perception and production for amusics. These findings provide further evidence that congenital amusia is a domain-general language-independent pitch-processing deficit that is associated with severely impaired music perception and production, mildly impaired speech perception, and largely intact speech production.

Top-Down Modulation on the Perception and Categorization of Identical Pitch Contours in Speech and Music

Whether pitch in language and music is governed by domain-specific or domain-general cognitive mechanisms is contentiously debated. The aim of the present study was to investigate whether mechanisms governing pitch contour perception operate differently when pitch information is interpreted as either speech or music. By modulating listening mode, this study aspired to demonstrate that pitch contour perception relies on domain-specific cognitive mechanisms, which are regulated by top-down influences from language and music. Three groups of participants (Mandarin speakers, Dutch speaking non-musicians, and Dutch musicians) were exposed to identical pitch contours, and tested on their ability to identify these contours in a language and musical context. Stimuli consisted of disyllabic words spoken in Mandarin, and melodic tonal analogs, embedded in a linguistic and melodic carrier phrase, respectively. Participants classified identical pitch contours as significantly different depending on listening mode. Top-down influences from language appeared to alter the perception of pitch contour in speakers of Mandarin. This was not the case for non-musician speakers of Dutch. Moreover, this effect was lacking in Dutch speaking musicians. The classification patterns of pitch contours in language and music seem to suggest that domain-specific categorization is modulated by top-down influences from language and music.

Effects of musical and linguistic experience on categorization of lexical and melodic tones

This study investigated the categorization of Mandarin lexical tones and music melodic tones by listeners differing in linguistic and musical experience (English musicians, English non-musicians, and Mandarin non-musicians). Linguistic tonal continua were created from the Mandarin rising to level, and falling to level tones. Melodic continua were created by varying the note D under the context of C and E. The tasks involved tone discrimination and identification. Results revealed that musical training facilitated Mandarin tone categorization, with English musicians' tone identification approximating native Mandarin patterns, being more categorical than English non-musicians'. However, English musicians showed higher discrimination accuracy than Mandarin listeners but not English non-musicians. This suggests that musical experience was not advantageous in discriminating linguistic tonal variations, which requires listeners to ignore subtle physical differences in order to make categorical judgments. Similarly, Mandarin tone experience affected melodic tone identification, with Mandarin non-musicians approximating English musicians, showing more categorical patterns than English non-musicians. In contrast, Mandarin non-musicians' melodic discrimination was the poorest among the three groups, indicating that their experience with linguistic tone categorization may have decreased their sensitivity to fine-grained pitch variations. These results demonstrate bi-directional transfer of pitch proficiency between speech and music as a function of experience.

Perception of Melodic Contour and Intonation in Autism Spectrum Disorder: Evidence From Mandarin Speakers

Tone language experience benefits pitch processing in music and speech for typically developing individuals. No known studies have examined pitch processing in individuals with autism who speak a tone language. This study investigated discrimination and identification of melodic contour and speech intonation in a group of Mandarin-speaking individuals with high-functioning autism. Individuals with autism showed superior melodic contour identification but comparable contour discrimination relative to controls. In contrast, these individuals performed worse than controls on both discrimination and identification of speech intonation. These findings provide the first evidence for differential pitch processing in music and speech in tone language speakers with autism, suggesting that tone language experience may not compensate for speech intonation perception deficits in individuals with autism.

Effects of Early Bilingual Experience with a Tone and a Non-Tone Language on Speech-Music Integration

We investigated music and language processing in a group of early bilinguals who spoke a tone language and a non-tone language (Cantonese and Dutch). We assessed online speech-music processing interactions, that is, interactions that occur when speech and music are processed simultaneously in songs, with a speeded classification task. In this task, participants judged sung pseudowords either musically (based on the direction of the musical interval) or phonologically (based on the identity of the sung vowel). We also assessed longer-term effects of linguistic experience on musical ability, that is, the influence of extensive prior experience with language when processing music. These effects were assessed with a task in which participants had to learn to identify musical intervals and with four pitch-perception tasks. Our hypothesis was that due to their experience in two different languages using lexical versus intonational tone, the early Cantonese-Dutch bilinguals would outperform the Dutch control participants. In online processing, the Cantonese-Dutch bilinguals processed speech and music more holistically than controls. This effect seems to be driven by experience with a tone language, in which integration of segmental and pitch information is fundamental. Regarding longer-term effects of linguistic experience, we found no evidence for a bilingual advantage in either the music-interval learning task or the pitch-perception tasks. Together, these results suggest that being a Cantonese-Dutch bilingual does not have any measurable longer-term effects on pitch and music processing, but does have consequences for how speech and music are processed jointly.

Effects of tonal language background on tests of temporal sequencing in children

OBJECTIVE

The aims of the present study were to determine the effects of language background on the performance of the pitch pattern sequence test (PPST) and duration pattern sequence test (DPST). As temporal order sequencing may be affected by age and working memory, these factors were also studied.

DESIGN

Performance of tonal and non-tonal language speakers on PPST and DPST were compared.

STUDY SAMPLE

Twenty-eight native Mandarin (tonal language) speakers and twenty-nine native Malay (non-tonal language) speakers between seven to nine years old participated in this study.

RESULTS

The results revealed that relative to native Malay speakers, native Mandarin speakers demonstrated better scores on the PPST in both humming and verbal labeling responses. However, a similar language effect was not apparent in the DPST. An age effect was only significant in the PPST (verbal labeling). Finally, no significant effect of working memory was found on the PPST and the DPST.

CONCLUSION

These findings suggest that the PPST is affected by tonal language background, and highlight the importance of developing different normative values for tonal and non-tonal language speakers.

On drawing a line through the spectrogram: how do we understand deficits of vocal pitch imitation?

In recent years there has been a remarkable increase in research focusing on deficits of pitch production in singing. A critical concern has been the identification of “poor pitch singers,” which we refer to more generally as individuals having a “vocal pitch imitation deficit.” The present paper includes a critical assessment of the assumption that vocal pitch imitation abilities can be treated as a dichotomy. Though this practice may be useful for data analysis and may be necessary within educational practice, we argue that this approach is complicated by a series of problems. Moreover, we argue that a more informative (and less problematic) approach comes from analyzing vocal pitch imitation abilities on a continuum, referred to as effect magnitude regression, and offer examples concerning how researchers may analyze data using this approach. We also argue that the understanding of this deficit may be better served by focusing on the effects of experimental manipulations on different individuals, rather than attempt to treat values of individual measures, and isolated tasks, as absolute measures of ability.