Action representation of sound: audiomotor recognition network while listening to newly acquired actions

From music making to speaking: engaging the mirror neuron system in autism

Individuals with autism show impairments in emotional tuning, social interactions and communication. These are functions that have been attributed to the putative human mirror neuron system (MNS), which contains neurons that respond to the actions of self and others. It has been proposed that a dysfunction of that system underlies some of the characteristics of autism. Here, we review behavioral and imaging studies that implicate the MNS (or a brain network with similar functions) in sensory-motor integration and speech representation, and review data supporting the hypothesis that MNS activity could be abnormal in autism. In addition, we propose that an intervention designed to engage brain regions that overlap with the MNS may have significant clinical potential. We argue that this engagement could be achieved through forms of music making. Music making with others (e.g., playing instruments or singing) is a multi-modal activity that has been shown to engage brain regions that largely overlap with the human MNS. Furthermore, many children with autism thoroughly enjoy participating in musical activities. Such activities may enhance their ability to focus and interact with others, thereby fostering the development of communication and social skills. Thus, interventions incorporating methods of music making may offer a promising approach for facilitating expressive language in otherwise nonverbal children with autism.

Sensorimotor cortical response during motion reflecting audiovisual stimulation: evidence from fractal EEG analysis

Sensorimotor activity in response to motion reflecting audiovisual titillation is studied in this article. EEG recordings, and especially the Mu-rhythm over the sensorimotor cortex (C3, CZ, and C4 electrodes), were acquired and explored. An experiment was designed to provide auditory (Modest Mussorgsky's "Promenade" theme) and visual (synchronized human figure walking) stimuli to advanced music students (AMS) and non-musicians (NM) as a control subject group. EEG signals were analyzed using fractal dimension (FD) estimation (Higuchi's, Katz's and Petrosian's algorithms) and statistical methods. Experimental results from the midline electrode (CZ) based on the Higuchi method showed significant differences between the AMS and the NM groups, with the former displaying substantial sensorimotor response during auditory stimulation and stronger correlation with the acoustic stimulus than the latter. This observation was linked to mirror neuron system activity, a neurological mechanism that allows trained musicians to detect action-related meanings underlying the structural patterns in musical excerpts. Contrarily, the response of AMS and NM converged during audiovisual stimulation due to the dominant presence of human-like motion in the visual stimulus. These findings shed light upon music perception aspects, exhibiting the potential of FD to respond to different states of cortical activity.

Music playing and memory trace: evidence from event-related potentials

We examined the relationship between motor practice and auditory memory for sound sequences to evaluate the hypothesis that practice involving physical performance might enhance auditory memory. Participants learned two unfamiliar sound sequences using different training methods. Under the key-press condition, they learned a melody while pressing a key during auditory input. Under the no-key-press condition, they listened to another melody without any key pressing. The two melodies were presented alternately, and all participants were trained in both methods. Participants were instructed to pay attention under both conditions. After training, they listened to the two melodies again without pressing keys, and ERPs were recorded. During the ERP recordings, 10% of the tones in these melodies deviated from the originals. The grand-average ERPs showed that the amplitude of mismatch negativity (MMN) elicited by deviant stimuli was larger under the key-press condition than under the no-key-press condition. This effect appeared only in the high absolute pitch group, which included those with a pronounced ability to identify a note without external reference. This result suggests that the effect of training with key pressing was mediated by individual musical skills.

Revealing action representation processes in audio perception using fractal EEG analysis

Electroencephalogram (EEG) recordings, and especially the Mu-rhythm over the sensorimotor cortex that relates to the activation of the mirror neuron system (MNS), were acquired from two subject groups (orchestral musicians and nonmusicians), in order to explore action representation processes involved in the perception and performance of musical pieces. Two types of stimuli were used, i.e., an auditory one consisting of an excerpt of Beethoven's fifth symphony and a visual one presenting a conductor directing an orchestra performing the same excerpt of the piece. Three tasks were conducted including auditory stimulation, audiovisual stimulation, and visual stimulation only, and the acquired signals were processed using fractal [time-dependent fractal dimension (FD) estimation] and statistical analysis (analysis of variance, Mann-Whitney). Experimental results showed significant differences between the two groups while desychronization of the Mu-rhythm, which can be linked to MNS activation, was observed during all tasks for the musicians' group, as opposed to the nonmusicians' group who exhibited similar response only when the visual stimulus was present. The mobility of the conductor was also correlated to the estimated FD signals, showing significantly higher correlation for the case of musicians compared to nonmusicians' one. The present study sheds light upon the difference in action representation in auditory perception between musicians and nonmusicians and paves the way for better comprehension of the underlying mechanisms of the MNS.

The Therapeutic Effects of Singing in Neurological Disorders

Music making (playing an instrument or singing) is a multimodal activity that involves the integration of auditory and sensorimotor processes. The ability to sing in humans is evident from infancy, and does not depend on formal vocal training but can be enhanced by training. Given the behavioral similarities between singing and speaking, as well as the shared and distinct neural correlates of both, researchers have begun to examine whether singing can be used to treat some of the speech-motor abnormalities associated with various neurological conditions. This paper reviews recent evidence on the therapeutic effects of singing, and how it can potentially ameliorate some of the speech deficits associated with conditions such as stuttering, Parkinson's disease, acquired brain lesions, and autism. By reviewing the status quo, it is hoped that future research can help to disentangle the relative contribution of factors to why singing works. This may ultimately lead to the development of specialized or "gold-standard" treatments for these disorders, and to an improvement in the quality of life for patients.

Auditory selective attention to speech modulates activity in the visual word form area

Selective attention to speech versus nonspeech signals in complex auditory input could produce top-down modulation of cortical regions previously linked to perception of spoken, and even visual, words. To isolate such top-down attentional effects, we contrasted 2 equally challenging active listening tasks, performed on the same complex auditory stimuli (words overlaid with a series of 3 tones). Instructions required selectively attending to either the speech signals (in service of rhyme judgment) or the melodic signals (tone-triplet matching). Selective attention to speech, relative to attention to melody, was associated with blood oxygenation level-dependent (BOLD) increases during functional magnetic resonance imaging (fMRI) in left inferior frontal gyrus, temporal regions, and the visual word form area (VWFA). Further investigation of the activity in visual regions revealed overall deactivation relative to baseline rest for both attention conditions. Topographic analysis demonstrated that while attending to melody drove deactivation equivalently across all fusiform regions of interest examined, attending to speech produced a regionally specific modulation: deactivation of all fusiform regions, except the VWFA. Results indicate that selective attention to speech can topographically tune extrastriate cortex, leading to increased activity in VWFA relative to surrounding regions, in line with the well-established connectivity between areas related to spoken and visual word perception in skilled readers.

The plasticity of the superior longitudinal fasciculus as a function of musical expertise: a diffusion tensor imaging study

Previous neuroimaging studies have demonstrated that musical expertise leads to functional alterations in language processing. We utilized diffusion tensor imaging (DTI) to investigate white matter plasticity in musicians with absolute pitch (AP), relative pitch and non-musicians. Using DTI, we analysed the fractional anisotropy (FA) of the superior longitudinal fasciculus (SLF), which is considered the most primary pathway for processing and production of speech and music. In association with different levels of musical expertise, we found that AP is characterized by a greater left than right asymmetry of FA in core fibres of the SLF. A voxel-based analysis revealed three clusters within the left hemisphere SLF that showed significant positive correlations with error rates only for AP-musicians in an AP-test, but not for musicians without AP. We therefore conclude that the SLF architecture in AP musicians is related to AP acuity. In order to reconcile our observations with general aspects of development of fibre bundles, we introduce the Pioneer Axon Thesis, a theoretical approach to formalize axonal arrangements of major white matter pathways.

Tuning-in to the beat: Aesthetic appreciation of musical rhythms correlates with a premotor activity boost

Listening to music can induce us to tune in to its beat. Previous neuroimaging studies have shown that the motor system becomes involved in perceptual rhythm and timing tasks in general, as well as during preference-related responses to music. However, the role of preferred rhythm and, in particular, of preferred beat frequency (tempo) in driving activity in the motor system remains unknown. The goals of the present functional magnetic resonance imaging (fMRI) study were to determine whether the musical rhythms that are subjectively judged as beautiful boost activity in motor-related areas and if so, whether this effect is driven by preferred tempo, the underlying pulse people tune in to. On the basis of the subjects' judgments, individual preferences were determined for the different systematically varied constituents of the musical rhythms. Results demonstrate the involvement of premotor and cerebellar areas during preferred compared to not preferred musical rhythms and indicate that activity in the ventral premotor cortex (PMv) is enhanced by preferred tempo. Our findings support the assumption that the premotor activity increase during preferred tempo is the result of enhanced sensorimotor simulation of the beat frequency. This may serve as a mechanism that facilitates the tuning-in to the beat of appealing music.

The Two-Level Theory of verb meaning: An approach to integrating the semantics of action with the mirror neuron system

Verbs have two separate levels of meaning. One level reflects the uniqueness of every verb and is called the "root". The other level consists of a more austere representation that is shared by all the verbs in a given class and is called the "event structure template". We explore the following hypotheses about how, with specific reference to the motor features of action verbs, these two distinct levels of semantic representation might correspond to two distinct levels of the mirror neuron system. Hypothesis 1: Root-level motor features of verb meaning are partially subserved by somatotopically mapped mirror neurons in the left primary motor and/or premotor cortices. Hypothesis 2: Template-level motor features of verb meaning are partially subserved by representationally more schematic mirror neurons in Brodmann area 44 of the left inferior frontal gyrus. Evidence has been accumulating in support of the general neuroanatomical claims made by these two hypotheses-namely, that each level of verb meaning is associated with the designated cortical areas. However, as yet no studies have satisfied all the criteria necessary to support the more specific neurobiological claims made by the two hypotheses-namely, that each level of verb meaning is associated with mirror neurons in the pertinent brain regions. This would require demonstrating that within those regions the same neuronal populations are engaged during (a) the linguistic processing of particular motor features of verb meaning, (b) the execution of actions with the corresponding motor features, and (c) the observation of actions with the corresponding motor features.

Self versus other in piano performance: detectability of timing perturbations depends on personal playing style

Differences between recorded repetitions of one's own movements are detected more readily than are differences between repetitions of others' movements, suggesting improved visual discrimination due to heightened resonance in the observer's action system and/or relatively accurate internal action simulation (Daprati et al. in Conscious Cogn 16:178-188, 2007). In Experiment 1, we attempted to replicate this finding in the auditory modality. Pianists were recorded playing musical excerpts three times and later judged whether pairs of recordings were the same take or different takes of the same excerpt. They were no better at distinguishing different takes of their own playing than those of other pianists' playing, even though discrimination and self-recognition were well above chance. In Experiment 2, the same pianists tried to detect small local timing deviations that had been introduced artificially. They were better at detecting such deviations in their own performances than in those of another pianist, but only if the deviations were placed at points of a pre-existing self-other difference in local timing. In that case, pianists' ability to predict their own characteristic action pattern did aid their perception of temporal irregularity. These results do not support the perceptual sharpening hypothesis of Daprati et al. in the musical domain, but they do suggest that pianists listening to performances generate idiosyncratic temporal expectations, probably through internal action simulation.

Expanding the mirror: vicarious activity for actions, emotions, and sensations

We often empathically share the states of others. The discovery of 'mirror neurons' suggested a neural mechanism for monkeys to share the actions of others. Here we expand this view by showing that mirror neurons for actions not only exist in the premotor cortex or in monkeys and that vicarious activity can also be measured for the emotions and sensations of others. Although we still need to empirically explore the function and development of these vicarious activations, we should stop thinking of vicarious brain activity as a peculiar property of the premotor cortex: instead it seems to be a very common phenomenon which leads social stimuli to recruit a wide range of seemingly private neural systems.

Neural mechanisms of the influence of popularity on adolescent ratings of music

It is well-known that social influences affect consumption decisions. We used functional magnetic resonance imaging (fMRI) to elucidate the neural mechanisms associated with social influence with regard to a common consumer good: music. Our study population was adolescents, age 12-17. Music is a common purchase in this age group, and it is widely believed that adolescent behavior is influenced by perceptions of popularity in their reference group. Using 15-s clips of songs from MySpace.com, we obtained behavioral measures of preferences and neurobiological responses to the songs. The data were gathered with, and without, the overall popularity of the song revealed. Song popularity had a significant effect on the participants' likability ratings of the songs. fMRI results showed a strong correlation between the participants' rating and activity in the caudate nucleus, a region previously implicated in reward-driven actions. The tendency to change one's evaluation of a song was positively correlated with activation in the anterior insula and anterior cingulate, two regions that are associated with physiological arousal and negative affective states. Sensitivity to popularity was linked to lower activation levels in the middle temporal gyrus, suggesting a lower depth of musical semantic processing. Our results suggest that a principal mechanism whereby popularity ratings affect consumer choice is through the anxiety generated by the mismatch between one's own preferences and others'. This mismatch anxiety motivates people to switch their choices in the direction of the consensus. Our data suggest that this is a major force behind the conformity observed in music tastes in some teenagers.

How and when auditory action effects impair motor performance

Music performance is characterized by complex cross-modal interactions, offering a remarkable window into training-induced long-term plasticity and multimodal integration processes. Previous research with pianists has shown that playing a musical score is affected by the concurrent presentation of musical tones. We investigated the nature of this audio-motor coupling by evaluating how congruent and incongruent cross-modal auditory cues affect motor performance at different time intervals. We found facilitation if a congruent sound preceded motor planning with a large Stimulus Onset Asynchrony (SOA -300 and -200 ms), whereas we observed interference when an incongruent sound was presented with shorter SOAs (-200, -100 and 0 ms). Interference and facilitation, instead of developing through time as opposite effects of the same mechanism, showed dissociable time-courses suggesting their derivation from distinct processes. It seems that the motor preparation induced by the auditory cue has different consequences on motor performance according to the congruency with the future motor state the system is planning and the degree of asynchrony between the motor act and the sound presentation. The temporal dissociation we found contributes to the understanding of how perception meets action in the context of audio-motor integration.

Tone deafness: a new disconnection syndrome?

Communicating with one's environment requires efficient neural interaction between action and perception. Neural substrates of sound perception and production are connected by the arcuate fasciculus (AF). Although AF is known to be involved in language, its roles in non-linguistic functions are unexplored. Here, we show that tone-deaf people, with impaired sound perception and production, have reduced AF connectivity. Diffusion tensor tractography and psychophysics were assessed in tone-deaf individuals and matched controls. Abnormally reduced AF connectivity was observed in the tone deaf. Furthermore, we observed relationships between AF and auditory-motor behavior: superior and inferior AF branches predict psychophysically assessed pitch discrimination and sound production perception abilities, respectively. This neural abnormality suggests that tone deafness leads to a reduction in connectivity resulting in pitch-related impairments. Results support a dual-stream anatomy of sound production and perception implicated in vocal communications. By identifying white matter differences and their psychophysical correlates, results contribute to our understanding of how neural connectivity subserves behavior.

Melodic intonation therapy: shared insights on how it is done and why it might help

For more than 100 years, clinicians have noted that patients with nonfluent aphasia are capable of singing words that they cannot speak. Thus, the use of melody and rhythm has long been recommended for improving aphasic patients' fluency, but it was not until 1973 that a music-based treatment [Melodic Intonation Therapy (MIT)] was developed. Our ongoing investigation of MIT's efficacy has provided valuable insight into this therapy's effect on language recovery. Here we share those observations, our additions to the protocol that aim to enhance MIT's benefit, and the rationale that supports them.

A neuroscientific perspective on music therapy

During the last years, a number of studies demonstrated that music listening (and even more so music production) activates a multitude of brain structures involved in cognitive, sensorimotor, and emotional processing. For example, music engages sensory processes, attention, memory-related processes, perception-action mediation ("mirror neuron system" activity), multisensory integration, activity changes in core areas of emotional processing, processing of musical syntax and musical meaning, and social cognition. It is likely that the engagement of these processes by music can have beneficial effects on the psychological and physiological health of individuals, although the mechanisms underlying such effects are currently not well understood. This article gives a brief overview of factors contributing to the effects of music-therapeutic work. Then, neuroscientific studies using music to investigate emotion, perception-action mediation ("mirror function"), and social cognition are reviewed, including illustrations of the relevance of these domains for music therapy.

Broca's area in language, action, and music

The work of Paul Broca has been of pivotal importance in the localization of some higher cognitive brain functions. He first reported that lesions to the caudal part of the inferior frontal gyrus were associated with expressive deficits. Although most of his claims are still true today, the emergence of novel techniques as well as the use of comparative analyses prompts modern research for a revision of the role played by Broca's area. Here we review current research showing that the inferior frontal gyrus and the ventral premotor cortex are activated for tasks other than language production. Specifically, a growing number of studies report the involvement of these two regions in language comprehension, action execution and observation, and music execution and listening. Recently, the critical involvement of the same areas in representing abstract hierarchical structures has also been demonstrated. Indeed, language, action, and music share a common syntactic-like structure. We propose that these areas are tuned to detect and represent complex hierarchical dependencies, regardless of modality and use. We speculate that this capacity evolved from motor and premotor functions associated with action execution and understanding, such as those characterizing the mirror-neuron system.

Evidence for plasticity in white-matter tracts of patients with chronic Broca's aphasia undergoing intense intonation-based speech therapy

Recovery from aphasia can be achieved through recruitment of either perilesional brain regions in the affected hemisphere or homologous language regions in the nonlesional hemisphere. For patients with large left-hemisphere lesions, recovery through the right hemisphere may be the only possible path. The right-hemisphere regions most likely to play a role in this recovery process are the superior temporal lobe (important for auditory feedback control), premotor regions/posterior inferior frontal gyrus (important for planning and sequencing of motor actions and for auditory-motor mapping), and the primary motor cortex (important for execution of vocal motor actions). These regions are connected reciprocally via a major fiber tract called the arcuate fasciculus (AF), however, this tract is not as well developed in the right hemisphere as it is in the dominant left. We tested whether an intonation-based speech therapy (i.e., melodic intonation therapy [MIT]), which is typically administered in an intense fashion with 75-80 daily therapy sessions, would lead to changes in white-matter tracts, particularly the AF. Using diffusion tensor imaging (DTI), we found a significant increase in the number of AF fibers and AF volume comparing post- with pretreatment assessments in six patients that could not be attributed to scan-to-scan variability. This suggests that intense, long-term MIT leads to remodeling of the right AF and may provide an explanation for the sustained therapy effects that were seen in these six patients.

Evidence for mirror systems in emotions

Why do we feel tears well up when we see a loved one cry? Why do we wince when we see other people hurt themselves? This review addresses these questions from the perspective of embodied simulation: observing the actions and tactile sensations of others activates premotor, posterior parietal and somatosensory regions in the brain of the observer which are also active when performing similar movements and feeling similar sensations. We will show that seeing the emotions of others also recruits regions involved in experiencing similar emotions, although there does not seem to be a reliable mapping of particular emotions onto particular brain regions. Instead, emotion simulation seems to involve a mosaic of affective, motor and somatosensory components. The relative contributions of these components to a particular emotion and their interrelationship are largely unknown, although recent experimental evidence suggests that motor simulation may be a trigger for the simulation of associated feeling states. This mosaic of simulations may be necessary for generating the compelling insights we have into the feelings of others. Through their integration with, and modulation by, higher cognitive functions, they could be at the core of important social functions, including empathy, mind reading and social learning.

Perceiving pitch absolutely: comparing absolute and relative pitch possessors in a pitch memory task

BACKGROUND

The perceptual-cognitive mechanisms and neural correlates of Absolute Pitch (AP) are not fully understood. The aim of this fMRI study was to examine the neural network underlying AP using a pitch memory experiment and contrasting two groups of musicians with each other, those that have AP and those that do not.

RESULTS

We found a common activation pattern for both groups that included the superior temporal gyrus (STG) extending into the adjacent superior temporal sulcus (STS), the inferior parietal lobule (IPL) extending into the adjacent intraparietal sulcus (IPS), the posterior part of the inferior frontal gyrus (IFG), the pre-supplementary motor area (pre-SMA), and superior lateral cerebellar regions. Significant between-group differences were seen in the left STS during the early encoding phase of the pitch memory task (more activation in AP musicians) and in the right superior parietal lobule (SPL)/intraparietal sulcus (IPS) during the early perceptual phase (ITP 0-3) and later working memory/multimodal encoding phase of the pitch memory task (more activation in non-AP musicians). Non-significant between-group trends were seen in the posterior IFG (more in AP musicians) and the IPL (more anterior activations in the non-AP group and more posterior activations in the AP group).

CONCLUSION

Since the increased activation of the left STS in AP musicians was observed during the early perceptual encoding phase and since the STS has been shown to be involved in categorization tasks, its activation might suggest that AP musicians involve categorization regions in tonal tasks. The increased activation of the right SPL/IPS in non-AP musicians indicates either an increased use of regions that are part of a tonal working memory (WM) network, or the use of a multimodal encoding strategy such as the utilization of a visual-spatial mapping scheme (i.e., imagining notes on a staff or using a spatial coding for their relative pitch height) for pitch information.

Do we really need vision? How blind people "see" the actions of others

Observing and learning actions and behaviors from others, a mechanism crucial for survival and social interaction, engages the mirror neuron system. To determine whether vision is a necessary prerequisite for the human mirror system to develop and function, we used functional magnetic resonance imaging to compare brain activity in congenitally blind individuals during the auditory presentation of hand-executed actions or environmental sounds, and the motor pantomime of manipulation tasks, with that in sighted volunteers, who additionally performed a visual action recognition task. Congenitally blind individuals activated a premotor-temporoparietal cortical network in response to aurally presented actions that overlapped both with mirror system areas found in sighted subjects in response to visually and aurally presented stimuli, and with the brain response elicited by motor pantomime of the same actions. Furthermore, the mirror system cortex showed a significantly greater response to motor familiar than to unfamiliar action sounds in both sighted and blind individuals. Thus, the mirror system in humans can develop in the absence of sight. The results in blind individuals demonstrate that the sound of an action engages the mirror system for action schemas that have not been learned through the visual modality and that this activity is not mediated by visual imagery. These findings indicate that the mirror system is based on supramodal sensory representations of actions and, furthermore, that these abstract representations allow individuals with no visual experience to interact effectively with others.

Convergence and divergence in a neural architecture for recognition and memory

How does the brain represent external reality so that it can be perceived in the form of mental images? How are the representations stored in memory so that an approximation of their original content can be re-experienced during recall? A framework introduced in the late 1980s proposed that mental images arise from neural activity in early sensory cortices both during perception and recall. Neurons in the association cortices, by contrast, would not code explicit mental content; rather, they would hold the records needed to reconstruct an approximation of the original perceptual maps in early cortices. Several neurophysiological and neuroimaging studies now lend growing support to this proposal.

Different categories of living and non-living sound-sources activate distinct cortical networks

With regard to hearing perception, it remains unclear as to whether, or the extent to which, different conceptual categories of real-world sounds and related categorical knowledge are differentially represented in the brain. Semantic knowledge representations are reported to include the major divisions of living versus non-living things, plus more specific categories including animals, tools, biological motion, faces, and places-categories typically defined by their characteristic visual features. Here, we used functional magnetic resonance imaging (fMRI) to identify brain regions showing preferential activity to four categories of action sounds, which included non-vocal human and animal actions (living), plus mechanical and environmental sound-producing actions (non-living). The results showed a striking antero-posterior division in cortical representations for sounds produced by living versus non-living sources. Additionally, there were several significant differences by category, depending on whether the task was category-specific (e.g. human or not) versus non-specific (detect end-of-sound). In general, (1) human-produced sounds yielded robust activation in the bilateral posterior superior temporal sulci independent of task. Task demands modulated activation of left lateralized fronto-parietal regions, bilateral insular cortices, and sub-cortical regions previously implicated in observation-execution matching, consistent with "embodied" and mirror-neuron network representations subserving recognition. (2) Animal action sounds preferentially activated the bilateral posterior insulae. (3) Mechanical sounds activated the anterior superior temporal gyri and parahippocampal cortices. (4) Environmental sounds preferentially activated dorsal occipital and medial parietal cortices. Overall, this multi-level dissociation of networks for preferentially representing distinct sound-source categories provides novel support for grounded cognition models that may underlie organizational principles for hearing perception.

Instrumental music influences recognition of emotional body language

In everyday life, emotional events are perceived by multiple sensory systems. Research has shown that recognition of emotions in one modality is biased towards the emotion expressed in a simultaneously presented but task irrelevant modality. In the present study, we combine visual and auditory stimuli that convey similar affective meaning but have a low probability of co-occurrence in everyday life. Dynamic face-blurred whole body expressions of a person grasping an object while expressing happiness or sadness are presented in combination with fragments of happy or sad instrumental classical music. Participants were instructed to categorize the emotion expressed by the visual stimulus. The results show that recognition of body language is influenced by the auditory stimuli. These findings indicate that crossmodal influences as previously observed for audiovisual speech can also be obtained from the ignored auditory to the attended visual modality in audiovisual stimuli that consist of whole bodies and music.

Brain to music to brain!

It has been implicitly understood that culture and music as collective products of human brain in turn influence the brain itself. Now, imaging and anatomical data add substance to this notion. The impact of playing piano on the brain of musicians and its possible effects on cultural and neurological evolution are briefly discussed.

Listening to humans walking together activates the social brain circuitry

Human footsteps carry a vast amount of social information, which is often unconsciously noted. Using functional magnetic resonance imaging, we analyzed brain networks activated by footstep sounds of one or two persons walking. Listening to two persons walking together activated brain areas previously associated with affective states and social interaction, such as the subcallosal gyrus bilaterally, the right temporal pole, and the right amygdala. These areas seem to be involved in the analysis of persons' identity and complex social stimuli on the basis of auditory cues. Single footsteps activated only the biological motion area in the posterior STS region. Thus, hearing two persons walking together involved a more widespread brain network than did hearing footsteps from a single person.

Performed or observed keyboard actions affect pianists' judgements of relative pitch

Action can affect visual perception if the action's expected sensory effects resemble a concurrent unstable or deviant event. To determine whether action can also change auditory perception, participants were required to play pairs of octave-ambiguous tones by pressing successive keys on a piano or computer keyboard and to judge whether each pitch interval was rising or falling. Both pianists and nonpianist musicians gave significantly more "rising" responses when the order of key presses was left-to-right than when it was right-to-left, in accord with the pitch mapping of the piano. However, the effect was much larger in pianists. Pianists showed a similarly large effect when they passively observed the experimenter pressing keys on a piano keyboard, as long as the keyboard faced the participant. The results suggest that acquired action-effect associations can affect auditory perceptual judgement.

A little more conversation, a little less action--candidate roles for the motor cortex in speech perception

The motor theory of speech perception assumes that activation of the motor system is essential in the perception of speech. However, deficits in speech perception and comprehension do not arise from damage that is restricted to the motor cortex, few functional imaging studies reveal activity in the motor cortex during speech perception, and the motor cortex is strongly activated by many different sound categories. Here, we evaluate alternative roles for the motor cortex in spoken communication and suggest a specific role in sensorimotor processing in conversation. We argue that motor cortex activation is essential in joint speech, particularly for the timing of turn taking.

Effects of seeing and hearing speech on speech production: a response time study

Research demonstrates that listening to and viewing speech excites tongue and lip motor areas involved in speech production. This perceptual-motor relationship was investigated behaviourally by presenting video clips of a speaker producing vowel-consonant-vowel syllables in three conditions: visual-only, audio-only, and audiovisual. Participants identified target letters that were flashed over the mouth during the video, either manually or verbally as quickly as possible. Verbal responses were fastest when the target matched the speech stimuli in all modality conditions, yet optimal facilitation was observed when participants were presented with visual-only stimuli. Critically, no such facilitation occurred when participants were asked to identify the target manually. Our findings support previous research suggesting a close relationship between speech perception and production by demonstrating that viewing speech can 'prime' our motor system for subsequent speech production.

Brain activation during anticipation of sound sequences

Music consists of sound sequences that require integration over time. As we become familiar with music, associations between notes, melodies, and entire symphonic movements become stronger and more complex. These associations can become so tight that, for example, hearing the end of one album track can elicit a robust image of the upcoming track while anticipating it in total silence. Here, we study this predictive "anticipatory imagery" at various stages throughout learning and investigate activity changes in corresponding neural structures using functional magnetic resonance imaging. Anticipatory imagery (in silence) for highly familiar naturalistic music was accompanied by pronounced activity in rostral prefrontal cortex (PFC) and premotor areas. Examining changes in the neural bases of anticipatory imagery during two stages of learning conditional associations between simple melodies, however, demonstrates the importance of fronto-striatal connections, consistent with a role of the basal ganglia in "training" frontal cortex (Pasupathy and Miller, 2005). Another striking change in neural resources during learning was a shift between caudal PFC earlier to rostral PFC later in learning. Our findings regarding musical anticipation and sound sequence learning are highly compatible with studies of motor sequence learning, suggesting common predictive mechanisms in both domains.

Selective neurophysiologic responses to music in instrumentalists with different listening biographies

To appropriately adapt to constant sensory stimulation, neurons in the auditory system are tuned to various acoustic characteristics, such as center frequencies, frequency modulations, and their combinations, particularly those combinations that carry species-specific communicative functions. The present study asks whether such tunings extend beyond acoustic and communicative functions to auditory self-relevance and expertise. More specifically, we examined the role of the listening biography--an individual's long term experience with a particular type of auditory input--on perceptual-neural plasticity. Two groups of expert instrumentalists (violinists and flutists) listened to matched musical excerpts played on the two instruments (J.S. Bach Partitas for solo violin and flute) while their cerebral hemodynamic responses were measured using fMRI. Our experimental design allowed for a comprehensive investigation of the neurophysiology (cerebral hemodynamic responses as measured by fMRI) of auditory expertise (i.e., when violinists listened to violin music and when flutists listened to flute music) and nonexpertise (i.e., when subjects listened to music played on the other instrument). We found an extensive cerebral network of expertise, which implicates increased sensitivity to musical syntax (BA 44), timbre (auditory association cortex), and sound-motor interactions (precentral gyrus) when listening to music played on the instrument of expertise (the instrument for which subjects had a unique listening biography). These findings highlight auditory self-relevance and expertise as a mechanism of perceptual-neural plasticity, and implicate neural tuning that includes and extends beyond acoustic and communication-relevant structures.

Neuroanatomical correlates of musicianship as revealed by cortical thickness and voxel-based morphometry

We used a multimethod approach to investigate the neuroanatomical correlates of musicianship and absolute pitch (AP). Cortical thickness measures, interregional correlations applied to these thicknesses, and voxel-based morphometry (VBM) were applied to the same magnetic resonance imaging data set of 71 musicians (27 with AP) and 64 nonmusicians. Cortical thickness was greater in musicians with peaks in superior temporal and dorsolateral frontal regions. Correlations between 2 seed points, centered on peaks of thickness difference within the right frontal cortex, and all other points across the cortex showed greater specificity of significant correlations among musicians, with fewer and more discrete areas correlating with the frontal seeds, including the superior temporal cortex. VBM of gray matter (GM)-classified voxels yielded a strongly right-lateralized focus of greater GM concentration in musicians centered on the posterolateral aspect of Heschl's gyrus. Together, these results are consistent with functional evidence emphasizing the importance of a frontotemporal network of areas heavily relied upon in the performance of musical tasks. Among musicians, contrasts of AP possessors and nonpossessors showed significantly thinner cortex among possessors in a number of areas, including the posterior dorsal frontal cortices that have been previously implicated in the performance of AP tasks.

Neural dynamics of learning sound-action associations

A motor component is pre-requisite to any communicative act as one must inherently move to communicate. To learn to make a communicative act, the brain must be able to dynamically associate arbitrary percepts to the neural substrate underlying the pre-requisite motor activity. We aimed to investigate whether brain regions involved in complex gestures (ventral pre-motor cortex, Brodmann Area 44) were involved in mediating association between novel abstract auditory stimuli and novel gestural movements. In a functional resonance imaging (fMRI) study we asked participants to learn associations between previously unrelated novel sounds and meaningless gestures inside the scanner. We use functional connectivity analysis to eliminate the often present confound of ‘strategic covert naming’ when dealing with BA44 and to rule out effects of non-specific reductions in signal. Brodmann Area 44, a region incorporating Broca's region showed strong, bilateral, negative correlation of BOLD (blood oxygen level dependent) response with learning of sound-action associations during data acquisition. Left-inferior-parietal-lobule (l-IPL) and bilateral loci in and around visual area V5, right-orbital-frontal-gyrus, right-hippocampus, left-para-hippocampus, right-head-of-caudate, right-insula and left-lingual-gyrus also showed decreases in BOLD response with learning. Concurrent with these decreases in BOLD response, an increasing connectivity between areas of the imaged network as well as the right-middle-frontal-gyrus with rising learning performance was revealed by a psychophysiological interaction (PPI) analysis. The increasing connectivity therefore occurs within an increasingly energy efficient network as learning proceeds. Strongest learning related connectivity between regions was found when analysing BA44 and l-IPL seeds. The results clearly show that BA44 and l-IPL is dynamically involved in linking gesture and sound and therefore provides evidence that one of the mechanisms required for the evolution of human communication is found within these motor regions.

Cortical plasticity induced by short-term unimodal and multimodal musical training

Learning to play a musical instrument requires complex multimodal skills involving simultaneous perception of several sensory modalities: auditory, visual, somatosensory, as well as the motor system. Therefore, musical training provides a good and adequate neuroscientific model to study multimodal brain plasticity effects in humans. Here, we investigated the impact of short-term unimodal and multimodal musical training on brain plasticity. Two groups of nonmusicians were musically trained over the course of 2 weeks. One group [sensorimotor-auditory (SA)] learned to play a musical sequence on the piano, whereas the other group [auditory (A)] listened to and made judgments about the music that had been played by participants of the sensorimotor-auditory group. Training-induced cortical plasticity was assessed by recording the musically elicited mismatch negativity (MMNm) from magnetoencephalographic measurements before and after training. SA and A groups showed significantly different cortical responses after training. Specifically, the SA group showed significant enlargement of MMNm after training compared with the A group, reflecting greater enhancement of musical representations in auditory cortex after sensorimotor-auditory training compared with after mere auditory training. Thus, we have experimentally demonstrated that not only are sensorimotor and auditory systems connected, but also that sensorimotor-auditory training causes plastic reorganizational changes in the auditory cortex over and above changes introduced by auditory training alone.

Testing simulation theory with cross-modal multivariate classification of fMRI data

The discovery of mirror neurons has suggested a potential neural basis for simulation and common coding theories of action perception, theories which propose that we understand other people's actions because perceiving their actions activates some of our neurons in much the same way as when we perform the actions. We propose testing this model directly in humans with functional magnetic resonance imaging (fMRI) by means of cross-modal classification. Cross-modal classification evaluates whether a classifier that has learned to separate stimuli in the sensory domain can also separate the stimuli in the motor domain. Successful classification provides support for simulation theories because it means that the fMRI signal, and presumably brain activity, is similar when perceiving and performing actions. In this paper we demonstrate the feasibility of the technique by showing that classifiers which have learned to discriminate whether a participant heard a hand or a mouth action, based on the activity patterns in the premotor cortex, can also determine, without additional training, whether the participant executed a hand or mouth action. This provides direct evidence that, while perceiving others' actions, (1) the pattern of activity in premotor voxels with sensory properties is a significant source of information regarding the nature of these actions, and (2) that this information shares a common code with motor execution.

Empathy and identification in Von Donnersmarck's The Lives of Others

Florian Henckel von Donnersmarck's The Lives of Others, set in the German Democratic Republic in 1984, five years before the fall of the Berlin Wall, has been called the first accurate depiction of the psychological terror wielded by the Stasi, the East German secret police, who safeguarded the dictatorship of the proletariat. The film is about the psychological and political transformation of a Stasi officer, Wiesler, who undertakes the surveillance of a prominent playwright and his actress lover. The mechanisms through which Wiesler comes to empathize and identify with the subjects of his investigation, as he observes and listens in on the rich blend of passion, poetry, and politics that characterizes their lives, are explored in depth. Wiesler's transformation is based in part on the capacity to form implicit models of the behavior and experiences of others, based on the mirror neuron system, that Gallese and his colleagues call "embodied simulation." Underpinning the processes of empathy and identification so central to this film, embodied simulation is an unconscious and prereflexive mechanism through which the actions, emotions, and sensations we observe activate internal representations of the bodily and mental states of the other. Embodied simulation also expands our understanding of the power of the primal scene, which has long been identified as a major organizer of unconscious fantasies and conflicts throughout life, and which forms the central metaphor of the film. Embodied simulation scaffolds our aesthetic response to art, music, and literature, underlies the dynamics of spectatorship, and potentially catalyzes resistance to totalitarian mass movements.

The Diverging Force of Imitation: Integrating Cognitive Science and Hermeneutics

Recent research on infant and animal imitation and on mirror neuron systems has brought imitation back in focus in psychology and cognitive science. This topic has always been important for philosophical hermeneutics as well, focusing on theory and method of understanding. Unfortunately, relations between the scientific and the hermeneutic approaches to imitation and understanding have scarcely been investigated, to the loss of both disciplines. In contrast to the cognitive scientific emphasis on sharing and convergence of representations, the hermeneutic analysis emphasizes the indeterminacy and openness of action understanding due to preunderstanding, action configuration, and the processual nature of understanding. This article discusses empirical evidence in support of these aspects and concludes that hermeneutics can contribute to the scientific investigation of imitation and understanding. Since, conversely, some grounding—and constraining—aspects of hermeneutics may be derived from cognitive science, both should be integrated in a multilevel explanation of imitation and understanding. This holds also for explanations that are largely based on mirror neuron systems, since these appear to be sensitive to developmental and experiential factors, too.