Making and monitoring errors based on altered auditory feedback

Sensorimotor Adaptation to Formant-Shifted Auditory Feedback Is Predicted by Language-Specific Factors in L1 and L2 Speech Production

Auditory feedback plays an important role in the long-term updating and maintenance of speech motor control; thus, the current study explored the unresolved question of how sensorimotor adaptation is predicted by language-specific and domain-general factors in first-language (L1) and second-language (L2) production. Eighteen English-L1 speakers and 22 English-L2 speakers performed the same sensorimotor adaptation experiments and tasks, which measured language-specific and domain-general abilities. The experiment manipulated the language groups (English-L1 and English-L2) and experimental conditions (baseline, early adaptation, late adaptation, and end). Linear mixed-effects model analyses indicated that auditory acuity was significantly associated with sensorimotor adaptation in L1 and L2 speakers. Analysis of vocal responses showed that L1 speakers exhibited significant sensorimotor adaptation under the early adaptation, late adaptation, and end conditions, whereas L2 speakers exhibited significant sensorimotor adaptation only under the late adaptation condition. Furthermore, the domain-general factors of working memory and executive control were not associated with adaptation/aftereffects in either L1 or L2 production, except for the role of working memory in aftereffects in L2 production. Overall, the study empirically supported the hypothesis that sensorimotor adaptation is predicted by language-specific factors such as auditory acuity and language experience, whereas general cognitive abilities do not play a major role in this process.

Unlocking the musical brain: A proof-of-concept study on playing the piano in MRI scanner with naturalistic stimuli

Music is a universal human phenomenon, and can be studied for itself or as a window into the understanding of the brain. Few neuroimaging studies investigate actual playing in the MRI scanner, likely because of the lack of available experimental hardware and analysis tools. Here, we offer an innovative paradigm that addresses this issue in neuromusicology using naturalistic, polyphonic musical stimuli, presents a commercially available MRI-compatible piano, and a flexible approach to quantify participant's performance. We show how making errors while playing can be investigated using an altered auditory feedback paradigm. In the spirit of open science, we make our experimental paradigms and analysis tools available to other researchers studying pianists in MRI. Altogether, we present a proof-of-concept study which shows the feasibility of playing the novel piano in MRI, and a step towards using more naturalistic stimuli.

The Key to Singing Off-Key: The Trained Singer and Pitch Perception Distortion

OBJECTIVES

Pitch perception distortion (PPD) is a novel term describing a phenomenon in which an amplified, accompanied singer's perception of their sung pitch relative to band or accompaniment becomes ambiguous, leading to one of two conditions: a) the singer believes they are out of tune with the accompaniment, but are in tune as perceived by a listener, or b) the singer believes they are in tune with the accompaniment, but are not. This pilot study aims to investigate the existence and incidence of PPD among amplified, accompanied performers and identify associated variables.

DESIGN/METHODS

115 singers were recruited to participate in an online survey, which collected information on musical training, performance environment, and PPD experience.

RESULTS

Reported PPD incidence was 68%, with 92% of respondents indicating that PPD occurred rarely. The factors reported as most associated with PPD experiences included loud stage volume, poor song familiarity, singing outside one's habitual pitch range, and singing loudly. Contrary to previous studies and our hypotheses, no association was found between modality of auditory feedback (e.g., in-ears versus floor monitors) and incidence of PPD. Additionally, higher levels of training were found to be associated with higher incidence of PPD.

CONCLUSIONS

The reported incidence supports that PPD exists beyond chance and anecdotal experience. In light of the highly trained sample, the data suggest that pitch accuracy in accompanied, amplified performance may be more associated with aural environment-specifically loud stage volume-and a highly trained singer's tuning strategy in response to that environment rather than a singer's mastery of vocal intonation skills in isolation. Loud stage volume was implicated as a primary factor associated with PPD, which may be related to the stapedius reflex. Future investigations will target attempted elicitation of PPD in trained singers after establishing baseline auditory reflex thresholds and objective measurements of intonation accuracy.

Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming

Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech, while other natural action-induced sounds have been widely neglected. Here, we studied the detection of audiovisual asynchrony in three different whole-body actions with natural action-induced sounds–hurdling, tap dancing and drumming. In Study 1, we examined whether audiovisual asynchrony detection, assessed by a simultaneity judgment task, differs as a function of sound production intentionality. Based on previous findings, we expected that auditory and visual signals should be integrated over a wider temporal window for actions creating sounds intentionally (tap dancing), compared to actions creating sounds incidentally (hurdling). While percentages of perceived synchrony differed in the expected way, we identified two further factors, namely high event density and low rhythmicity, to induce higher synchrony ratings as well. Therefore, we systematically varied event density and rhythmicity in Study 2, this time using drumming stimuli to exert full control over these variables, and the same simultaneity judgment tasks. Results suggest that high event density leads to a bias to integrate rather than segregate auditory and visual signals, even at relatively large asynchronies. Rhythmicity had a similar, albeit weaker effect, when event density was low. Our findings demonstrate that shorter asynchronies and visual-first asynchronies lead to higher synchrony ratings of whole-body action, pointing to clear parallels with audiovisual integration in speech perception. Overconfidence in the naturally expected, that is, synchrony of sound and sight, was stronger for intentional (vs. incidental) sound production and for movements with high (vs. low) rhythmicity, presumably because both encourage predictive processes. In contrast, high event density appears to increase synchronicity judgments simply because it makes the detection of audiovisual asynchrony more difficult. More studies using real-life audiovisual stimuli with varying event densities and rhythmicities are needed to fully uncover the general mechanisms of audiovisual integration.

Auditory agnosia with anosognosia

A 66-year-old right-handed female medical doctor suffered two consecutive cardioembolic strokes, initially affecting the right frontal lobe and the right insula, followed by a lesion in the left temporal lobe. The patient presented with distinctive phenomenology of general auditory agnosia with anosognosia for the deficit. She did not understand verbal requests and her answers to oral questions were fluent but unrelated to the topic. However, she was able to correctly answer written questions, name objects, and fluently describe their purpose, which is characteristic for verbal auditory agnosia. She was also unable to recognise environmental sounds or to recognise and repeat any melody. These inabilities represent environmental sound agnosia and amusia, respectively. Surprisingly, she was not aware of the problem, not asking any questions regarding her symptoms, and avoiding discussing her inability to understand spoken language, which is indicative of anosognosia. The deficits in our patient followed a distinct pattern of recovery. The verbal auditory agnosia was the first to resolve, followed by environmental sound agnosia. Amusia persisted the longest. The patient was clinically assessed from the first day of symptom onset and the evolution of symptoms was video documented. We give a detailed account of the patient's behaviour and provide results of audiological and neuropsychological evaluations. We discuss the anatomy of auditory agnosia and anosognosia relevant to the case. This case study may serve to better understand auditory agnosia in clinical settings. It is important to distinguish auditory agnosia from Wernicke's aphasia, because use of written language may enable normal communication.

Too Late! Influence of Temporal Delay on the Neural Processing of One's Own Incidental and Intentional Action-Induced Sounds

The influence of delayed auditory feedback on action evaluation and execution of real-life action-induced sounds apart from language and music is still poorly understood. Here, we examined how a temporal delay impacted the behavioral evaluation and neural representation of hurdling and tap-dancing actions in a functional magnetic resonance imaging (fMRI) experiment, postulating that effects of delay diverge between the two, as we create action-induced sounds intentionally in tap dancing, but incidentally in hurdling. Based on previous findings, we expected that conditions differ regarding the engagement of the supplementary motor area (SMA), posterior superior temporal gyrus (pSTG), and primary auditory cortex (A1). Participants were videotaped during a 9-week training of hurdling and tap dancing; in the fMRI scanner, they were presented with point-light videos of their own training videos, including the original or the slightly delayed sound, and had to evaluate how well they performed on each single trial. For the undelayed conditions, we replicated A1 attenuation and enhanced pSTG and SMA engagement for tap dancing (intentionally generated sounds) vs. hurdling (incidentally generated sounds). Delayed auditory feedback did not negatively influence behavioral rating scores in general. Blood-oxygen-level-dependent (BOLD) response transiently increased and then adapted to repeated presentation of point-light videos with delayed sound in pSTG. This region also showed a significantly stronger correlation with the SMA under delayed feedback. Notably, SMA activation increased more for delayed feedback in the tap-dancing condition, covarying with higher rating scores. Findings suggest that action evaluation is more strongly based on top–down predictions from SMA when sounds of intentional action are distorted.

Adaptation to pitch-altered feedback is independent of one's own voice pitch sensitivity

Monitoring voice pitch is a fine-tuned process in daily conversations as conveying accurately the linguistic and affective cues in a given utterance depends on the precise control of phonation and intonation. This monitoring is thought to depend on whether the error is treated as self-generated or externally-generated, resulting in either a correction or inflation of errors. The present study reports on two separate paradigms of adaptation to altered feedback to explore whether participants could behave in a more cohesive manner once the error is of comparable size perceptually. The vocal behavior of normal-hearing and fluent speakers was recorded in response to a personalized size of pitch shift versus a non-specific size, one semitone. The personalized size of shift was determined based on the just-noticeable difference in fundamental frequency (F0) of each participant’s voice. Here we show that both tasks successfully demonstrated opposing responses to a constant and predictable F0 perturbation (on from the production onset) but these effects barely carried over once the feedback was back to normal, depicting a pattern that bears some resemblance to compensatory responses. Experiencing a F0 shift that is perceived as self-generated (because it was precisely just-noticeable) is not enough to force speakers to behave more consistently and more homogeneously in an opposing manner. On the contrary, our results suggest that the type of the response as well as the magnitude of the response do not depend in any trivial way on the sensitivity of participants to their own voice pitch. Based on this finding, we speculate that error correction could possibly occur even with a bionic ear, typically even when F0 cues are too subtle for cochlear implant users to detect accurately.

Incidental or Intentional? Different Brain Responses to One's Own Action Sounds in Hurdling vs. Tap Dancing

Most human actions produce concomitant sounds. Action sounds can be either part of the action goal (GAS, goal-related action sounds), as for instance in tap dancing, or a mere by-product of the action (BAS, by-product action sounds), as for instance in hurdling. It is currently unclear whether these two types of action sounds-incidental or intentional-differ in their neural representation and whether the impact on the performance evaluation of an action diverges between the two. We here examined whether during the observation of tap dancing compared to hurdling, auditory information is a more important factor for positive action quality ratings. Moreover, we tested whether observation of tap dancing vs. hurdling led to stronger attenuation in primary auditory cortex, and a stronger mismatch signal when sounds do not match our expectations. We recorded individual point-light videos of newly trained participants performing tap dancing and hurdling. In the subsequent functional magnetic resonance imaging (fMRI) session, participants were presented with the videos that displayed their own actions, including corresponding action sounds, and were asked to rate the quality of their performance. Videos were either in their original form or scrambled regarding the visual modality, the auditory modality, or both. As hypothesized, behavioral results showed significantly lower rating scores in the GAS condition compared to the BAS condition when the auditory modality was scrambled. Functional MRI contrasts between BAS and GAS actions revealed higher activation of primary auditory cortex in the BAS condition, speaking in favor of stronger attenuation in GAS, as well as stronger activation of posterior superior temporal gyri and the supplementary motor area in GAS. Results suggest that the processing of self-generated action sounds depends on whether we have the intention to produce a sound with our action or not, and action sounds may be more prone to be used as sensory feedback when they are part of the explicit action goal. Our findings contribute to a better understanding of the function of action sounds for learning and controlling sound-producing actions.

Tonal and textural influences on musical sight-reading

Two experiments investigated the impact of two structural factors-musical tonality and musical texture-on pianists' ability to play by sight without prior preparation, known as musical sight-reading. Tonality refers to the cognitive organization of tones around a central reference pitch, whereas texture refers to the organization of music in terms of the simultaneous versus successive onsets of tones as well as the number of hands (unimanual versus bimanual) involved in performance. Both experiments demonstrated that tonality and texture influenced sight-reading. For tonality, both studies found that errors in performance increased for passages with lesser perceived psychological stability (i.e., minor and atonal passages) relative to greater perceived stability (i.e., major passages). For texture, both studies found that errors in performance increased for passages that were more texturally complex, requiring two-handed versus one-handed performance, with some additional evidence that the relative simultaneity of note onsets (primarily simultaneous versus primarily successive) also influenced errors. These experiments are interpreted within a perception-action framework of music performance, highlighting influences of both top-down cognitive factors and bottom-up motoric processes on sight-reading behavior.

The influence of pitch feedback on learning of motor -timing and sequencing: A piano study with novices

Audio-motor coordination is a fundamental requirement in the learning and execution of sequential actions such as music performance. Predictive motor control mechanisms determine the sequential content and timing of upcoming tones and thereby facilitate accurate performance. To study the role of auditory-motor predictions at early stages of acquiring piano performance skills, we conducted an experiment in which non-musicians learned to play a musical sequence on the piano in synchrony with a metronome. Three experimental conditions compared errors and timing. The first consisted of normal auditory feedback using conventional piano key-to-tone mappings. The second employed fixed-pitch auditory feedback consisting of a single tone that was given with each key stroke. In the third condition, for each key stroke a tone was randomly drawn from the set of tones associated with the normal sequence. The results showed that when auditory feedback tones were randomly assigned, participants produced more sequencing errors (i.e., a higher percentage of incorrect key strokes) compared to when auditory feedback was normal or consisted of a single tone of fixed pitch. Furthermore, synchronization with the metronome was most accurate in the fixed-pitch single-tone condition. These findings suggest that predictive motor control mechanisms support sequencing and timing, and that these sensorimotor processes are dissociable even at early stages of acquiring complex motor skills such as music performance.

The effect of handedness on spatial and motor representation of pitch patterns in pianists

This study investigated the effect of handedness on pianists' abilities to adjust their keyboard performance skills to new spatial and motor mappings. Left- and right-handed pianists practiced simple melodies on a regular MIDI piano keyboard (practice) and were then asked to perform these with modified melodic contours (the same or reversed melodic contour causing a change of fingering) and on a reversed MIDI piano keyboard (test). The difference of performance duration between the practice and the test phase as well as the amount of errors played were used as test measures. Overall, a stronger effect for modified melodic contours than for the reversed keyboard was observed. Furthermore, we observed a trend of left-handed pianists to be quicker and more accurate in playing melodies when reversing their fingering with reversed contours in their left-hand performances. This suggests that handedness may influence pianists' skill to adjust to new spatial and motor mappings.

Online Adaptation to Altered Auditory Feedback Is Predicted by Auditory Acuity and Not by Domain-General Executive Control Resources

When a speaker's auditory feedback is altered, he adapts for the perturbation by altering his own production, which demonstrates the role of auditory feedback in speech motor control. In the present study, we explored the role of auditory acuity and executive control in this process. Based on the DIVA model and the major cognitive control models, we expected that higher auditory acuity, and better executive control skills would predict larger adaptation to the alteration. Thirty-six Spanish native speakers performed an altered auditory feedback experiment, executive control (numerical Stroop, Simon and Flanker) tasks, and auditory acuity tasks (loudness, pitch, and melody pattern discrimination). In the altered feedback experiment, participants had to produce the pseudoword “pep” (/pep/) while perceiving their auditory feedback in real time through earphones. The auditory feedback was first unaltered and then progressively altered in F1 and F2 dimensions until maximal alteration (F1 −150 Hz; F2 +300 Hz). The normalized distance of maximal adaptation ranged from 4 to 137 Hz (median of 75 ± 36). The different measures of auditory acuity were significant predictors of adaptation, while individual measures of cognitive function skills (obtained from the executive control tasks) were not. Better auditory discriminators adapted more to the alteration. We conclude that adaptation to altered auditory feedback is very well-predicted by general auditory acuity, as suggested by the DIVA model. In line with the framework of motor-control models, no specific claim on the implication of executive resources in speech motor control can be made.

Cingulate and cerebellar beta oscillations are engaged in the acquisition of auditory-motor sequences

Singing, music performance, and speech rely on the retrieval of complex sounds, which are generated by the corresponding actions and are organized into sequences. It is crucial in these forms of behavior that the serial organization (i.e., order) of both the actions and associated sounds be monitored and learned. To investigate the neural processes involved in the monitoring of serial order during the initial learning of sensorimotor sequences, we performed magnetoencephalographic recordings while participants explicitly learned short piano sequences under the effect of occasional alterations of auditory feedback (AAF). The main result was a prominent and selective modulation of beta (13-30 Hz) oscillations in cingulate and cerebellar regions during the processing of AAF that simulated serial order errors. Furthermore, the AAF-induced modulation of beta oscillations was associated with higher error rates, reflecting compensatory changes in sequence planning. This suggests that cingulate and cerebellar beta oscillations play a role in tracking serial order during initial sensorimotor learning and in updating the mapping of the sensorimotor representations. The findings support the notion that the modulation of beta oscillations is a candidate mechanism for the integration of sequential motor and auditory information during an early stage of skill acquisition in music performance. This has potential implications for singing and speech. Hum Brain Mapp 38:5161-5179, 2017. © 2017 Wiley Periodicals, Inc.

Musical training as an alternative and effective method for neuro-education and neuro-rehabilitation

In the last decade, important advances in the field of cognitive science, psychology, and neuroscience have largely contributed to improve our knowledge on brain functioning. More recently, a line of research has been developed that aims at using musical training and practice as alternative tools for boosting specific perceptual, motor, cognitive, and emotional skills both in healthy population and in neurologic patients. These findings are of great hope for a better treatment of language-based learning disorders or motor impairment in chronic non-communicative diseases. In the first part of this review, we highlight several studies showing that learning to play a musical instrument can induce substantial neuroplastic changes in cortical and subcortical regions of motor, auditory and speech processing networks in healthy population. In a second part, we provide an overview of the evidence showing that musical training can be an alternative, low-cost and effective method for the treatment of language-based learning impaired populations. We then report results of the few studies showing that training with musical instruments can have positive effects on motor, emotional, and cognitive deficits observed in patients with non-communicable diseases such as stroke or Parkinson Disease. Despite inherent differences between musical training in educational and rehabilitation contexts, these results favor the idea that the structural, multimodal, and emotional properties of musical training can play an important role in developing new, creative and cost-effective intervention programs for education and rehabilitation in the next future.

A Dual-Stream Neuroanatomy of Singing

Singing requires effortless and efficient use of auditory and motor systems that center around the perception and production of the human voice. Although perception and production are usually tightly coupled functions, occasional mismatches between the two systems inform us of dissociable pathways in the brain systems that enable singing. Here I review the literature on perception and production in the auditory modality, and propose a dual-stream neuroanatomical model that subserves singing. I will discuss studies surrounding the neural functions of feedforward, feedback, and efference systems that control vocal monitoring, as well as the white matter pathways that connect frontal and temporal regions that are involved in perception and production. I will also consider disruptions of the perception-production network that are evident in tone-deaf individuals and poor pitch singers. Finally, by comparing expert singers against other musicians and nonmusicians, I will evaluate the possibility that singing training might offer rehabilitation from these disruptions through neuroplasticity of the perception-production network. Taken together, the best available evidence supports a model of dorsal and ventral pathways in auditory-motor integration that enables singing and is shared with language, music, speech, and human interactions in the auditory environment.