Visibility of speech articulation enhances auditory phonetic convergence

Convergence in voice fundamental frequency during synchronous speech

Joint speech behaviours where speakers produce speech in unison are found in a variety of everyday settings, and have clinical relevance as a temporary fluency-enhancing technique for people who stutter. It is currently unknown whether such synchronisation of speech timing among two speakers is also accompanied by alignment in their vocal characteristics, for example in acoustic measures such as pitch. The current study investigated this by testing whether convergence in voice fundamental frequency (F0) between speakers could be demonstrated during synchronous speech. Sixty participants across two online experiments were audio recorded whilst reading a series of sentences, first on their own, and then in synchrony with another speaker (the accompanist) in a number of between-subject conditions. Experiment 1 demonstrated significant convergence in participants' F0 to a pre-recorded accompanist voice, in the form of both upward (high F0 accompanist condition) and downward (low and extra-low F0 accompanist conditions) changes in F0. Experiment 2 demonstrated that such convergence was not seen during a visual synchronous speech condition, in which participants spoke in synchrony with silent video recordings of the accompanist. An audiovisual condition in which participants were able to both see and hear the accompanist in pre-recorded videos did not result in greater convergence in F0 compared to synchronisation with the pre-recorded voice alone. These findings suggest the need for models of speech motor control to incorporate interactions between self- and other-speech feedback during speech production, and suggest a novel hypothesis for the mechanisms underlying the fluency-enhancing effects of synchronous speech in people who stutter.

Interlocutor accommodation of gradually altered nasal signal levels in a model speaker

BACKGROUND

Phonetic accommodation is observed when interacting speakers gradually converge (or diverge) on phonetic features over the course of a conversation. The present experiment investigated whether gradual changes in the nasal signal levels of a pre-recorded model speaker would lead to accommodation in the nasalance scores of the interlocutor in a speech-shadowing experiment.

METHODS

Twenty female speakers in two groups repeated sentences after a pre-recorded model speaker whose nasal signal level was gradually increased or decreased over the course of the experiment. Outcome measures were the mean nasalance scores at the initial baseline, maximum nasal signal level, minimum nasal signal level and final baseline conditions. The order of presentation of the maximum and minimum nasal signal levels was varied between the two groups.

RESULTS

The results showed a significant effect of condition in F(3) = 2.86, p = 0.045. Both groups of participants demonstrated lower nasalance scores in response to increased nasal signal levels in the model (phonetic divergence). The group that was first presented with the maximum nasal signal levels demonstrated lower nasalance scores for the minimum nasal signal level condition (phonetic convergence).

CONCLUSION

Speakers showed a consistent divergent reaction to a more nasal-sounding model speaker, but their response to a less nasal-sounding model may depend on the order of presentation of the manipulations. More research is needed to investigate the effects of increased versus decreased nasality in the speech of an interlocutor.

Sensitivity of occipito-temporal cortex, premotor and Broca's areas to visible speech gestures in a familiar language

When looking at a speaking person, the analysis of facial kinematics contributes to language discrimination and to the decoding of the time flow of visual speech. To disentangle these two factors, we investigated behavioural and fMRI responses to familiar and unfamiliar languages when observing speech gestures with natural or reversed kinematics. Twenty Italian volunteers viewed silent video-clips of speech shown as recorded (Forward, biological motion) or reversed in time (Backward, non-biological motion), in Italian (familiar language) or Arabic (non-familiar language). fMRI revealed that language (Italian/Arabic) and time-rendering (Forward/Backward) modulated distinct areas in the ventral occipito-temporal cortex, suggesting that visual speech analysis begins in this region, earlier than previously thought. Left premotor ventral (superior subdivision) and dorsal areas were preferentially activated with the familiar language independently of time-rendering, challenging the view that the role of these regions in speech processing is purely articulatory. The left premotor ventral region in the frontal operculum, thought to include part of the Broca's area, responded to the natural familiar language, consistent with the hypothesis of motor simulation of speech gestures.

Speaking to a common tune: Between-speaker convergence in voice fundamental frequency in a joint speech production task

Recent research on speech communication has revealed a tendency for speakers to imitate at least some of the characteristics of their interlocutor’s speech sound shape. This phenomenon, referred to as phonetic convergence, entails a moment-to-moment adaptation of the speaker’s speech targets to the perceived interlocutor’s speech. It is thought to contribute to setting up a conversational common ground between speakers and to facilitate mutual understanding. However, it remains uncertain to what extent phonetic convergence occurs in voice fundamental frequency (F₀), in spite of the major role played by pitch, F₀’s perceptual correlate, as a conveyor of both linguistic information and communicative cues associated with the speaker’s social/individual identity and emotional state. In the present work, we investigated to what extent two speakers converge towards each other with respect to variations in F₀ in a scripted dialogue. Pairs of speakers jointly performed a speech production task, in which they were asked to alternately read aloud a written story divided into a sequence of short reading turns. We devised an experimental set-up that allowed us to manipulate the speakers’ F₀ in real time across turns. We found that speakers tended to imitate each other’s changes in F₀ across turns that were both limited in amplitude and spread over large temporal intervals. This shows that, at the perceptual level, speakers monitor slow-varying movements in their partner’s F₀ with high accuracy and, at the production level, that speakers exert a very fine-tuned control on their laryngeal vibrator in order to imitate these F₀ variations. Remarkably, F₀ convergence across turns was found to occur in spite of the large melodic variations typically associated with reading turns. Our study sheds new light on speakers’ perceptual tracking of F₀ in speech processing, and the impact of this perceptual tracking on speech production.

Unintended imitation of fundamental frequency in female speakers

Objective: Former studies have shown that humans unintentionally adopt speech characteristics from their conversation partners, a phenomenon known as speech convergence. This study investigated imitation of fundamental frequency (F_o) in nine healthy females in a repetition task.Method: The design included two conditions; loud reading (baseline) and repetition after a model talker. The model talker was a female with a high-pitched voice and stimuli included both words and nonwords. All productions were analyzed with regard to mean and SD of F_o and compared between conditions and word types.Results: The participants showed different patterns regarding the effect on F_o, thus indicating that some individuals are more likely to change their speaking F_o in a repetition task than others. Group results showed somewhat but not significantly higher mean F_o in the repetition task as compared to baseline (reading). Moreover, nonwords were produced with a significantly higher mean F_o than real words, in both reading and repetition conditions. No interaction effect between condition and word type was found.

Vocal alignment to native and non-native speakers of English

Research on vocal alignment, the tendency for language users to match another individual's speech productions, suggests that multiple factors contribute to this behavior. Social and motivational goals, aspects of cognitive architecture, and linguistic flexibility may all affect the extent to which vocal alignment occurs, suggesting complex underlying mechanisms. The present study capitalized on the social and linguistic characteristics of Spanish-accented English to examine the relationship among these contributors to vocal alignment. American English-speaking adults participated in a shadowing task. Degree of vocal alignment was assessed by both acoustic measures and independent raters' judgments. Participants aligned to both native English and Spanish-accented productions, despite differences in attitudes to and intelligibility of the different accents. Individual differences in shadowers' vowel dispersion were also related to extent of vocal alignment, with greater dispersion associated with greater alignment. Acoustic measures were related to perceptual assessments of alignment and differed by accent type, suggesting that patterns of alignment may differ across accents. Overall, the current study demonstrates vocal alignment between talkers of differing language backgrounds and highlights the importance of acoustic and linguistic components of alignment behavior.

Auditory and Audiovisual Close Shadowing in Post-Lingually Deaf Cochlear-Implanted Patients and Normal-Hearing Elderly Adults

OBJECTIVES

The goal of this study was to determine the effect of auditory deprivation and age-related speech decline on perceptuo-motor abilities during speech processing in post-lingually deaf cochlear-implanted participants and in normal-hearing elderly (NHE) participants.

DESIGN

A close-shadowing experiment was carried out on 10 cochlear-implanted patients and on 10 NHE participants, with two groups of normal-hearing young participants as controls. To this end, participants had to categorize auditory and audiovisual syllables as quickly as possible, either manually or orally. Reaction times and percentages of correct responses were compared depending on response modes, stimulus modalities, and syllables.

RESULTS

Responses of cochlear-implanted subjects were globally slower and less accurate than those of both young and elderly normal-hearing people. Adding the visual modality was found to enhance performance for cochlear-implanted patients, whereas no significant effect was obtained for the NHE group. Critically, oral responses were faster than manual ones for all groups. In addition, for NHE participants, manual responses were more accurate than oral responses, as was the case for normal-hearing young participants when presented with noisy speech stimuli.

CONCLUSIONS

Faster reaction times were observed for oral than for manual responses in all groups, suggesting that perceptuo-motor relationships were somewhat successfully functional after cochlear implantation and remain efficient in the NHE group. These results are in agreement with recent perceptuo-motor theories of speech perception. They are also supported by the theoretical assumption that implicit motor knowledge and motor representations partly constrain auditory speech processing. In this framework, oral responses would have been generated at an earlier stage of a sensorimotor loop, whereas manual responses would appear late, leading to slower but more accurate responses. The difference between oral and manual responses suggests that the perceptuo-motor loop is still effective for NHE subjects and also for cochlear-implanted participants, despite degraded global performance.

Phonetic imitation of coarticulatory vowel nasalization

This study investigates the spontaneous phonetic imitation of coarticulatory vowel nasalization. Speakers produced monosyllabic words with a vowel-nasal sequence either from dense or sparse phonological neighborhoods in shadowing and word-naming tasks. During shadowing, they were exposed to target words that were modified to have either an artificially increased or decreased degree of coarticulatory vowel nasality. Increased nasality, which is communicatively more facilitative in that it provides robust predictive information about the upcoming nasal segment, was imitated more strongly during shadowing than decreased nasality. An effect of neighborhood density was also observed only in the increased nasality condition, where high neighborhood density words were imitated more robustly in early shadowing repetition. An effect of exposure to decreased nasality was observed during post-shadowing word-naming only. The observed imitation of coarticulatory nasality provides evidence that speakers and listeners are sensitive to the details of coarticulatory realization, and that imitation need not be mediated by abstract phonological representations. Neither a communicative account nor a representational account could single-handedly predict these observed patterns of imitation. As such, it is argued that these findings support both communicative and representational accounts of phonetic imitation.