The Linked Dual Representation model of vocal perception and production

Speech-to-song transformation in perception and production

The speech-to-song transformation is an illusion in which certain spoken phrases are perceived as more song-like after being repeated several times. The present study addresses whether this perceptual transformation leads to a corresponding change in how accurately participants imitate pitch/time patterns in speech. We used illusion-inducing (illusion stimuli) and non-inducing (control stimuli) spoken phrases as stimuli. In each trial, one stimulus was presented eight times in succession. Participants were asked to reproduce the phrase and rate how music-like the phrase sounded after the first and final (eighth) repetitions. The ratings of illusion stimuli reflected more song-like perception after the final repetition than the first repetition, but the ratings of control stimuli did not change over repetitions. The results from imitative production mirrored the perceptual effects: pitch matching of illusion stimuli improved from the first to the final repetition, but pitch matching of control stimuli did not improve. These findings suggest a consistent pattern of speech-to-song transformation in both perception and production, suggesting that distinctions between music and language may be more malleable than originally thought both in perception and production.

The Relationship Between Pitch Discrimination and Acoustic Voice Measures in a Cohort of Female Speakers

BACKGROUND

Evidence across a range of musically trained, hearing disordered and voice disordered populations present conflicting results regarding the relationship between pitch discrimination (PD) and voice quality. PD characteristics of female speakers with and without a musical training background and no self-reported voice disorder, and the relationship between PD and voice quality in this particular population, have not been investigated.

AIMS

To evaluate PD characteristics in a cohort of female participants without a self-reported voice disorder and the relationship between PD and acoustic voice measures.

METHOD

One hundred fourteen female participants were studied, all of whom self-reported as being non-voice disordered. All completed the Newcastle Assessment of Pitch Discrimination which involved a two-tone PD task. Their voices were recorded producing standardized vocal tasks. Voice samples were acoustically analyzed for frequency-domain measures (fundamental frequency and its standard deviation, and harmonics-to-noise ratio) and spectral-domain measures (cepstral peak prominence and the Cepstral/Spectral Index of Dysphonia). Data were analyzed for the whole cohort and for musical and non-musical training backgrounds.

RESULTS

In the whole cohort, there were no significant correlations between PD and acoustic voice measures. PD accuracy in musically trained speakers was better than in non-trained speakers and correlated with fundamental frequency standard deviation in prolonged vowel tasks. Vocalists demonstrated superior PD accuracy and fundamental frequency standard deviation in prolonged vowels compared to instrumentalists but did not show significant correlations between PD and acoustic measures. The Newcastle Assessment of Pitch Discrimination was a reliable tool, showing moderate-good prediction value in differentiating musical background.

CONCLUSIONS

There was little evidence of a relationship between PD and acoustic measures of voice quality, regardless of musical training background and superior PD accuracy among the musically trained. These data do not support ideas concerning the co-development of perception and action among individuals identified as having voice quality measures within normal ranges. Numerous measures of voice quality, including measures sensitive to pitch, did not distinguish across musically and non-musically trained individuals, despite individual differences in pitch discrimination.

Voice actors show enhanced neural tracking of pitch, prosody perception, and music perception

Experiences with sound that make strong demands on the precision of perception, such as musical training and experience speaking a tone language, can enhance auditory neural encoding. Are high demands on the precision of perception necessary for training to drive auditory neural plasticity? Voice actors are an ideal subject population for answering this question. Voice acting requires exaggerating prosodic cues to convey emotion, character, and linguistic structure, drawing upon attention to sound, memory for sound features, and accurate sound production, but not fine perceptual precision. Here we assessed neural encoding of pitch using the frequency-following response (FFR), as well as prosody, music, and sound perception, in voice actors and a matched group of non-actors. We find that the consistency of neural sound encoding, prosody perception, and musical phrase perception are all enhanced in voice actors, suggesting that a range of neural and behavioural auditory processing enhancements can result from training which lacks fine perceptual precision. However, fine discrimination was not enhanced in voice actors but was linked to degree of musical experience, suggesting that low-level auditory processing can only be enhanced by demanding perceptual training. These findings suggest that training which taxes attention, memory, and production but is not perceptually taxing may be a way to boost neural encoding of sound and auditory pattern detection in individuals with poor auditory skills.

Comparative Study on the Acoustic Analysis of Voice in Auditory Brainstem Implantees, Cochlear Implantees, and Normal Hearing Children

The aim of the study was to compare the acoustic characteristics of voice between Auditory Brainstem Implantees, Cochlear Implantees and normal hearing children. Voice parameters such as fundamental frequency, formant frequencies, perturbation measures, and harmonic to noise ratio were measured in a total of 30 children out of which 10 were Auditory Brainstem Implantees, 10 were Cochlear Implantees and 10 were normal hearing children. Parametric and nonparametric statistics were done to establish the nature of significance between the three groups. Overall deviancies were seen in the implanted group for all acoustic parameters. However abnormal deviations were seen in individuals with Auditory Brainstem Implants indicating the deficit in the feedback loop impacting the voice characteristics. The deviancy in feedback could attribute to the poor performance in ABI and CI. The CI performed comparatively better when compared to the ABI group indicating a slight feedback loop due to the type of Implant. However, there needs to be additional evidence supporting this and there is a need to carry out the same study using a larger sample size and a longitudinal design.

The Relationship Between Pitch Discrimination and Fundamental Frequency Variation: Effects of Singing Status and Vocal Hyperfunction

PURPOSE

The purpose of this study was to investigate the relationship between pitch discrimination and fundamental frequency (fo) variation in running speech, with consideration of factors such as singing status and vocal hyperfunction (VH).

METHOD

Female speakers (18-69 years) with typical voices (26 non-singers; 27 singers) and speakers with VH (22 non-singers; 30 singers) completed a pitch discrimination task and read the Rainbow Passage. The pitch discrimination task was a two-alternative forced choice procedure, in which participants determined whether tokens were the same or different. Tokens were a prerecorded sustained /ɑ/ of the participant's own voice and a pitch-shifted version of their sustained /ɑ/, such that the difference in fo was adaptively modified. Pitch discrimination and Rainbow Passage fo variation were calculated for each participant and compared via Pearson's correlations for each group.

RESULTS

A significant strong correlation was found between pitch discrimination and fo variation for non-singers with typical voices. No significant correlations were found for the other three groups, with notable restrictions in the ranges of discrimination for both singer-groups and in the range of fo variation values for non-singers with VH.

CONCLUSIONS

Speakers with worse pitch discrimination may increase their fo variation to produce self-salient intonational changes, which is in contrast to previous findings from articulatory investigations. The erosion of this relationship in groups with singing training and/or with VH may be explained by the known influence of musical training on pitch discrimination or the biomechanical changes associated with VH restricting speakers' abilities to change their fo.

Individual Differences in Singing Behavior during Childhood Predicts Language Performance during Adulthood

Research on singing and language abilities has gained considerable interest in the past decade. While several studies about singing ability and language capacity have been published, investigations on individual differences in singing behavior during childhood and its relationship to language capacity in adulthood have largely been neglected. We wanted to focus our study on whether individuals who had sung more often during childhood than their peers were also better in language and music capacity during adulthood. We used questionnaires to assess singing behavior of adults during childhood and tested them for their singing ability, their music perception skills, and their ability to perceive and pronounce unfamiliar languages. The results have revealed that the more often individuals had sung during childhood, the better their singing ability and language pronunciation skills were, while the amount of childhood singing was less predictive on music and language perception skills. We suggest that the amount of singing during childhood seems to influence the ability to sing and the ability to acquire foreign language pronunciation later in adulthood.

Musical Performance in Adolescents with ADHD, ADD and Dyslexia—Behavioral and Neurophysiological Aspects

Research has shown that dyslexia and attention deficit (hyperactivity) disorder (AD(H)D) are characterized by specific neuroanatomical and neurofunctional differences in the auditory cortex. These neurofunctional characteristics in children with ADHD, ADD and dyslexia are linked to distinct differences in music perception. Group-specific differences in the musical performance of patients with ADHD, ADD and dyslexia have not been investigated in detail so far. We investigated the musical performance and neurophysiological correlates of 21 adolescents with dyslexia, 19 with ADHD, 28 with ADD and 28 age-matched, unaffected controls using a music performance assessment scale and magnetoencephalography (MEG). Musical experts independently assessed pitch and rhythmic accuracy, intonation, improvisation skills and musical expression. Compared to dyslexic adolescents, controls as well as adolescents with ADHD and ADD performed better in rhythmic reproduction, rhythmic improvisation and musical expression. Controls were significantly better in rhythmic reproduction than adolescents with ADD and scored higher in rhythmic and pitch improvisation than adolescents with ADHD. Adolescents with ADD and controls scored better in pitch reproduction than dyslexic adolescents. In pitch improvisation, the ADD group performed better than the ADHD group, and controls scored better than dyslexic adolescents. Discriminant analysis revealed that rhythmic improvisation and musical expression discriminate the dyslexic group from controls and adolescents with ADHD and ADD. A second discriminant analysis based on MEG variables showed that absolute P1 latency asynchrony |R-L| distinguishes the control group from the disorder groups best, while P1 and N1 latencies averaged across hemispheres separate the control, ADD and ADHD groups from the dyslexic group. Furthermore, rhythmic improvisation was negatively correlated with auditory-evoked P1 and N1 latencies, pointing in the following direction: the earlier the P1 and N1 latencies (mean), the better the rhythmic improvisation. These findings provide novel insight into the differences between music processing and performance in adolescents with and without neurodevelopmental disorders. A better understanding of these differences may help to develop tailored preventions or therapeutic interventions.

What do less accurate singers remember? Pitch-matching ability and long-term memory for music

We have only a partial understanding of how people remember nonverbal information such as melodies. Although once learned, melodies can be retained well over long periods of time, remembering newly presented melodies is on average quite difficult. People vary considerably, however, in their level of success in both memory situations. Here, we examine a skill we anticipated would be correlated with memory for melodies: the ability to accurately reproduce pitches. Such a correlation would constitute evidence that melodic memory involves at least covert sensorimotor codes. Experiment 1 looked at episodic memory for new melodies among nonmusicians, both overall and with respect to the Vocal Memory Advantage (VMA): the superiority in remembering melodies presented as sung on a syllable compared to rendered on an instrument. Although we replicated the VMA, our prediction that better pitch matchers would have a larger VMA was not supported, although there was a modest correlation with memory for melodies presented in a piano timbre. Experiment 2 examined long-term memory for the starting pitch of familiar recorded music. Participants selected the starting note of familiar songs on a keyboard, without singing. Nevertheless, we found that better pitch-matchers were more accurate in reproducing the correct starting note. We conclude that sensorimotor coding may be used in storing and retrieving exact melodic information, but is not so useful during early encounters with melodies, as initial coding seems to involve more derived properties such as pitch contour and tonality.

Individuals with autism spectrum disorder are impaired in absolute but not relative pitch and duration matching in speech and song imitation

Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation. However, few studies have identified clear quantitative characteristics of vocal imitation in ASD. This study investigated imitation of speech and song in English-speaking individuals with and without ASD and its modulation by age. Participants consisted of 25 autistic children and 19 autistic adults, who were compared to 25 children and 19 adults with typical development matched on age, gender, musical training, and cognitive abilities. The task required participants to imitate speech and song stimuli with varying pitch and duration patterns. Acoustic analyses of the imitation performance suggested that individuals with ASD were worse than controls on absolute pitch and duration matching for both speech and song imitation, although they performed as well as controls on relative pitch and duration matching. Furthermore, the two groups produced similar numbers of pitch contour, pitch interval-, and time errors. Across both groups, sung pitch was imitated more accurately than spoken pitch, whereas spoken duration was imitated more accurately than sung duration. Children imitated spoken pitch more accurately than adults when it came to speech stimuli, whereas age showed no significant relationship to song imitation. These results reveal a vocal imitation deficit across speech and music domains in ASD that is specific to absolute pitch and duration matching. This finding provides evidence for shared mechanisms between speech and song imitation, which involves independent implementation of relative versus absolute features. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation of actions and gestures. Characteristics of vocal imitation in ASD remain unclear. By comparing speech and song imitation, this study shows that individuals with ASD have a vocal imitative deficit that is specific to absolute pitch and duration matching, while performing as well as controls on relative pitch and duration matching, across speech and music domains.

The effects of musicality and language background on cue integration in pitch perception

Pitch perception involves the processing of multidimensional acoustic cues, and listeners can exhibit different cue integration strategies in interpreting pitch. This study aims to examine whether musicality and language experience have effects on listeners' pitch perception strategies. Both Mandarin and English listeners were recruited to participate in two experiments: (1) a pitch classification experiment that tested their relative reliance on f0 and spectral cues, and (2) the Montreal Battery of Evaluation of Musical Abilities that objectively quantified their musical aptitude as continuous musicality scores. Overall, the results show a strong musicality effect: Listeners with higher musicality scores relied more on f0 in pitch perception, while listeners with lower musicality scores were more likely to attend to spectral cues. However, there were no effects of language experience on musicality scores or cue integration strategies in pitch perception. These results suggest that less musical or even amusic subjects may not suffer impairment in linguistic pitch processing due to the multidimensional nature of pitch cues.

The Role of Auditory Feedback at Vocalization Onset and Mid-Utterance

Auditory feedback plays an important role in monitoring and correcting for errors during speech production. Previous research suggests that at vocalization onset, auditory feedback is compared to a sensory prediction generated by the motor system to ensure the desired fundamental frequency (F0) is produced. After vocalization onset, auditory feedback is compared to the most recently perceived F0 in order to stabilize the vocalization. This study aimed to further investigate whether after vocalization onset, auditory feedback is used strictly to stabilize speakers’ F0, or if it is also influenced by the sensory prediction generated by the motor system. Event-related potentials (ERP) were recorded while participants produced vocalizations and heard the F0 of their auditory feedback perturbed suddenly mid-utterance by half a semitone. For half of the vocalizations, at vocalization onset, participants’ F0 was also raised by half a semitone. Thus, half of the perturbations occurred while participants heard their unaltered auditory feedback, and the other half occurred in auditory feedback that had also been perturbed 50 cents at vocalization onset. If after vocalization onset auditory feedback is strictly used to stabilize speakers’ F0, then similarly sized vocal and ERP responses would be expected across all trials, regardless of whether the perturbation occurred while listening to altered or unaltered auditory feedback. Results indicate that the perturbations to the participants’ unaltered auditory feedback resulted in larger vocal and N1 and P2 ERP responses than perturbations to their altered auditory feedback. These results suggest that after vocalization onset auditory feedback is not strictly used to stabilize speakers’ F0, but is also used to ensure the desired F0 is produced.

Lexical tone recognition in noise in normal-hearing children and prelingually deafened children with cochlear implants

OBJECTIVE

The purpose of the present study was to investigate Mandarin tone recognition in background noise in children with cochlear implants (CIs), and to examine the potential factors contributing to their performance.

DESIGN

Tone recognition was tested using a two-alternative forced-choice paradigm in various signal-to-noise ratio (SNR) conditions (i.e. quiet, +12, +6, 0, and -6 dB). Linear correlation analysis was performed to examine possible relationships between the tone-recognition performance of the CI children and the demographic factors.

STUDY SAMPLE

Sixty-six prelingually deafened children with CIs and 52 normal-hearing (NH) children as controls participated in the study.

RESULTS

Children with CIs showed an overall poorer tone-recognition performance and were more susceptible to noise than their NH peers. Tone confusions between Mandarin tone 2 and tone 3 were most prominent in both CI and NH children except for in the poorest SNR conditions. Age at implantation was significantly correlated with tone-recognition performance of the CI children in noise.

CONCLUSIONS

There is a marked deficit in tone recognition in prelingually deafened children with CIs, particularly in noise listening conditions. While factors that contribute to the large individual differences are still elusive, early implantation could be beneficial to tone development in pediatric CI users.

Pitch perception and production in congenital amusia: Evidence from Cantonese speakers

This study investigated pitch perception and production in speech and music in individuals with congenital amusia (a disorder of musical pitch processing) who are native speakers of Cantonese, a tone language with a highly complex tonal system. Sixteen Cantonese-speaking congenital amusics and 16 controls performed a set of lexical tone perception, production, singing, and psychophysical pitch threshold tasks. Their tone production accuracy and singing proficiency were subsequently judged by independent listeners, and subjected to acoustic analyses. Relative to controls, amusics showed impaired discrimination of lexical tones in both speech and non-speech conditions. They also received lower ratings for singing proficiency, producing larger pitch interval deviations and making more pitch interval errors compared to controls. Demonstrating higher pitch direction identification thresholds than controls for both speech syllables and piano tones, amusics nevertheless produced native lexical tones with comparable pitch trajectories and intelligibility as controls. Significant correlations were found between pitch threshold and lexical tone perception, music perception and production, but not between lexical tone perception and production for amusics. These findings provide further evidence that congenital amusia is a domain-general language-independent pitch-processing deficit that is associated with severely impaired music perception and production, mildly impaired speech perception, and largely intact speech production.

The Neural Basis of Vocal Pitch Imitation in Humans

Vocal imitation is a phenotype that is unique to humans among all primate species, and so an understanding of its neural basis is critical in explaining the emergence of both speech and song in human evolution. Two principal neural models of vocal imitation have emerged from a consideration of nonhuman animals. One hypothesis suggests that putative mirror neurons in the inferior frontal gyrus pars opercularis of Broca's area may be important for imitation. An alternative hypothesis derived from the study of songbirds suggests that the corticostriate motor pathway performs sensorimotor processes that are specific to vocal imitation. Using fMRI with a sparse event-related sampling design, we investigated the neural basis of vocal imitation in humans by comparing imitative vocal production of pitch sequences with both nonimitative vocal production and pitch discrimination. The strongest difference between these tasks was found in the putamen bilaterally, providing a striking parallel to the role of the analogous region in songbirds. Other areas preferentially activated during imitation included the orofacial motor cortex, Rolandic operculum, and SMA, which together outline the corticostriate motor loop. No differences were seen in the inferior frontal gyrus. The corticostriate system thus appears to be the central pathway for vocal imitation in humans, as predicted from an analogy with songbirds.

Syllable Structure Universals and Native Language Interference in Second Language Perception and Production: Positional Asymmetry and Perceptual Links to Accentedness

The present study investigated how syllable structure differences between the first Language (L1) and the second language (L2) affect L2 consonant perception and production at syllable-initial and syllable-final positions. The participants were Mandarin-speaking college students who studied English as a second language. Monosyllabic English words were used in the perception test. Production was recorded from each Chinese subject and rated for accentedness by two native speakers of English. Consistent with previous studies, significant positional asymmetry effects were found across speech sound categories in terms of voicing, place of articulation, and manner of articulation. Furthermore, significant correlations between perception and accentedness ratings were found at the syllable onset position but not for the coda. Many exceptions were also found, which could not be solely accounted for by differences in L1-L2 syllabic structures. The results show a strong effect of language experience at the syllable level, which joins force with acoustic, phonetic, and phonemic properties of individual consonants in influencing positional asymmetry in both domains of L2 segmental perception and production. The complexities and exceptions call for further systematic studies on the interactions between syllable structure universals and native language interference with refined theoretical models to specify the links between perception and production in second language acquisition.

On drawing a line through the spectrogram: how do we understand deficits of vocal pitch imitation?

In recent years there has been a remarkable increase in research focusing on deficits of pitch production in singing. A critical concern has been the identification of “poor pitch singers,” which we refer to more generally as individuals having a “vocal pitch imitation deficit.” The present paper includes a critical assessment of the assumption that vocal pitch imitation abilities can be treated as a dichotomy. Though this practice may be useful for data analysis and may be necessary within educational practice, we argue that this approach is complicated by a series of problems. Moreover, we argue that a more informative (and less problematic) approach comes from analyzing vocal pitch imitation abilities on a continuum, referred to as effect magnitude regression, and offer examples concerning how researchers may analyze data using this approach. We also argue that the understanding of this deficit may be better served by focusing on the effects of experimental manipulations on different individuals, rather than attempt to treat values of individual measures, and isolated tasks, as absolute measures of ability.