The neuroanatomy of speech sequencing at the syllable level

Localized estimation of event-related neural source activity from simultaneous MEG-EEG with a recurrent neural network

Estimating intracranial current sources underlying the electromagnetic signals observed from extracranial sensors is a perennial challenge in non-invasive neuroimaging. Established solutions to this inverse problem treat time samples independently without considering the temporal dynamics of event-related brain processes. This paper describes current source estimation from simultaneously recorded magneto- and electro-encephalography (MEEG) using a recurrent neural network (RNN) that learns sequential relationships from neural data. The RNN was trained in two phases: (1) pre-training and (2) transfer learning with L1 regularization applied to the source estimation layer. Performance of using scaled labels derived from MEEG, magnetoencephalography (MEG), or electroencephalography (EEG) were compared, as were results from volumetric source space with free dipole orientation and surface source space with fixed dipole orientation. Exact low-resolution electromagnetic tomography (eLORETA) and mixed-norm L1/L2 (MxNE) source estimation methods were also applied to these data for comparison with the RNN method. The RNN approach outperformed other methods in terms of output signal-to-noise ratio, correlation and mean-squared error metrics evaluated against reference event-related field (ERF) and event-related potential (ERP) waveforms. Using MEEG labels with fixed-orientation surface sources produced the most consistent estimates. To estimate sources of ERF and ERP waveforms, the RNN generates temporal dynamics within its internal computational units, driven by sequential structure in neural data used as training labels. It thus provides a data-driven model of computational transformations from psychophysiological events into corresponding event-related neural signals, which is unique among MEEG source reconstruction solutions.

Considerations for identifying subtypes of speech sound disorder

BACKGROUND

Speech sound disorders (SSDs) in children are heterogeneous. Differentiating children with SSDs into distinct subtypes is important so that each child receives a treatment approach well suited to the particular difficulties they are experiencing.

AIMS

To study the distinct underlying processes that differentiate phonological processing, phonological planning or motor planning deficits.

METHOD

The literature on the nature of SSDs is reviewed to reveal diagnostic signs at the level of distal causes, proximal factors and surface characteristics.

MAIN CONTRIBUTION

Subtypes of SSDs may be identified by linking the surface characteristics of the children's speech to underlying explanatory proximal factors. The proximal factors may be revealed by measures of speech perception skills, phonological memory and speech-motor control. The evidence suggests that consistent phonological disorder (CPD) can be identified by predictable patterns of speech error associated with speech perception errors. Inconsistent phonological disorder (IPD) is associated with a deficit in the selection and sequencing of phonemes, that is, revealed as within-word inconsistency and poor phonological memory. The motor planning deficit that is specific to childhood apraxia of speech (CAS) is revealed by transcoding errors on the syllable repetition task and an inability to produce [pətəkə] accurately and rapidly.

CONCLUSIONS & IMPLICATIONS

Children with SSDs form a heterogeneous population. Surface characteristics overlap considerably among those with severe disorders, but certain signs are unique to particular subtypes. Careful attention to underlying causal factors will support the accurate diagnosis and selection of personalized treatment options.

WHAT THIS PAPER ADDS

What is already known on the subject SSD in children are heterogenous, with numerous subtypes of primary SSD proposed. Diagnosing the specific subtype of SSD is important in order to assign the most efficacious treatment approach for each child. Identifying the distinct subtype for each child is difficult because the surface characteristics of certain subtypes overlap among categories (e.g., CPD or IPD; CAS). What this paper adds to the existing knowledge The diagnostic challenge might be eased by systematic attention to explanatory factors in relation to the surface characteristics, using specific tests for this purpose. Word identification tasks tap speech perception skills; repetition of short versus long strings of nonsense syllables permits observation of phonological memory and phonological planning skills; and standard maximum performance tests provide considerable information about speech motor control. What are the potential or actual clinical implications of this work? Children with SSDs should receive comprehensive assessments of their phonological processing, phonological planning and motor planning skills frequently, alongside examinations of their error patterns in connected speech. Such assessments will serve to identify the child's primary challenges currently and as they change over developmental time.

Pars Opercularis Underlies Efferent Predictions and Successful Auditory Feedback Processing in Speech: Evidence From Left-Hemisphere Stroke

Hearing one's own speech allows for acoustic self-monitoring in real time. Left-hemisphere motor planning regions are thought to give rise to efferent predictions that can be compared to true feedback in sensory cortices, resulting in neural suppression commensurate with the degree of overlap between predicted and actual sensations. Sensory prediction errors thus serve as a possible mechanism of detection of deviant speech sounds, which can then feed back into corrective action, allowing for online control of speech acoustics. The goal of this study was to assess the integrity of this detection-correction circuit in persons with aphasia (PWA) whose left-hemisphere lesions may limit their ability to control variability in speech output. We recorded magnetoencephalography (MEG) while 15 PWA and age-matched controls spoke monosyllabic words and listened to playback of their utterances. From this, we measured speaking-induced suppression of the M100 neural response and related it to lesion profiles and speech behavior. Both speaking-induced suppression and cortical sensitivity to deviance were preserved at the group level in PWA. PWA with more spared tissue in pars opercularis had greater left-hemisphere neural suppression and greater behavioral correction of acoustically deviant pronunciations, whereas sparing of superior temporal gyrus was not related to neural suppression or acoustic behavior. In turn, PWA who made greater corrections had fewer overt speech errors in the MEG task. Thus, the motor planning regions that generate the efferent prediction are integral to performing corrections when that prediction is violated.

Decoding kinematic information from beta-band motor rhythms of speech motor cortex: a methodological/analytic approach using concurrent speech movement tracking and magnetoencephalography

Introduction

Articulography and functional neuroimaging are two major tools for studying the neurobiology of speech production. Until now, however, it has generally not been feasible to use both in the same experimental setup because of technical incompatibilities between the two methodologies.

Methods

Here we describe results from a novel articulography system dubbed Magneto-articulography for the Assessment of Speech Kinematics (MASK), which is technically compatible with magnetoencephalography (MEG) brain scanning systems. In the present paper we describe our methodological and analytic approach for extracting brain motor activities related to key kinematic and coordination event parameters derived from time-registered MASK tracking measurements. Data were collected from 10 healthy adults with tracking coils on the tongue, lips, and jaw. Analyses targeted the gestural landmarks of reiterated utterances/ipa/ and /api/, produced at normal and faster rates.

Results

The results show that (1) Speech sensorimotor cortex can be reliably located in peri-rolandic regions of the left hemisphere; (2) mu (8-12 Hz) and beta band (13-30 Hz) neuromotor oscillations are present in the speech signals and contain information structures that are independent of those present in higher-frequency bands; and (3) hypotheses concerning the information content of speech motor rhythms can be systematically evaluated with multivariate pattern analytic techniques.

Discussion

These results show that MASK provides the capability, for deriving subject-specific articulatory parameters, based on well-established and robust motor control parameters, in the same experimental setup as the MEG brain recordings and in temporal and spatial co-register with the brain data. The analytic approach described here provides new capabilities for testing hypotheses concerning the types of kinematic information that are encoded and processed within specific components of the speech neuromotor system.

Neurocomputational modeling of speech motor development

This review describes a computational approach for modeling the development of speech motor control in infants. We address the development of two levels of control: articulation of individual speech sounds (defined here as phonemes, syllables, or words for which there is an optimized motor program) and production of sound sequences such as phrases or sentences. We describe the DIVA model of speech motor control and its application to the problem of learning individual sounds in the infant's native language. Then we describe the GODIVA model, an extension of DIVA, and how chunking of frequently produced phoneme sequences is implemented within it.

Lateralization and Time-Course of Cortical Phonological Representations during Syllable Production

Spoken language contains information at a broad range of timescales, from phonetic distinctions on the order of milliseconds to semantic contexts which shift over seconds to minutes. It is not well understood how the brain's speech production systems combine features at these timescales into a coherent vocal output. We investigated the spatial and temporal representations in cerebral cortex of three phonological units with different durations: consonants, vowels, and syllables. Electrocorticography (ECoG) recordings were obtained from five participants while speaking single syllables. We developed a novel clustering and Kalman filter-based trend analysis procedure to sort electrodes into temporal response profiles. A linear discriminant classifier was used to determine how strongly each electrode's response encoded phonological features. We found distinct time-courses of encoding phonological units depending on their duration: consonants were represented more during speech preparation, vowels were represented evenly throughout trials, and syllables during production. Locations of strongly speech-encoding electrodes (the top 30% of electrodes) likewise depended on phonological element duration, with consonant-encoding electrodes left-lateralized, vowel-encoding hemispherically balanced, and syllable-encoding right-lateralized. The lateralization of speech-encoding electrodes depended on onset time, with electrodes active before or after speech production favoring left hemisphere and those active during speech favoring the right. Single-electrode speech classification revealed cortical areas with preferential encoding of particular phonemic elements, including consonant encoding in the left precentral and postcentral gyri and syllable encoding in the right middle frontal gyrus. Our findings support neurolinguistic theories of left hemisphere specialization for processing short-timescale linguistic units and right hemisphere processing of longer-duration units.

An fMRI study of inflectional encoding in spoken word production: Role of domain-general inhibition

A major issue concerning inflectional encoding in spoken word production is whether or not regular forms (e.g., past tense walked) are encoded by rule application and irregular forms (e.g., swam) by retrieval from associative memory and inhibition of the regular rule. We used functional magnetic resonance imaging (fMRI) to examine the involvement of domain-general inhibition, thought to be underpinned by right inferior frontal gyrus (IFG), right pre-supplementary motor area (SMA), and right basal ganglia. Participants were presented with infinitive verbs that take either regular or irregular past tense. They switched between producing the past tense of these regular and irregular verbs in one block, and between inflecting or reading these infinitive verbs aloud in another block. As concerns corticobasal areas, compared to reading, inflecting activated left IFG and left preSMA/SMA. Regulars yielded higher activation than irregulars in these frontal areas, both on switch and repeat trials, which did not differ in activation. Switching between inflecting and reading activated left preSMA/SMA. These results indicate that inflectional encoding, and switching between inflecting and reading, engage frontal areas in the left hemisphere, including left preSMA/SMA for both and left IFG for inflecting, without recruiting the domain-general inhibition circuitry in the right hemisphere. We advance an account of inflectional encoding in spoken word production that assumes a distinction between regulars and irregulars, but without engaging domain-general inhibition.

Disruption to left inferior frontal cortex modulates semantic prediction effects in reading and subsequent memory: Evidence from simultaneous TMS-EEG

Readers use prior context to predict features of upcoming words. When predictions are accurate, this increases the efficiency of comprehension. However, little is known about the fate of predictable and unpredictable words in memory or the neural systems governing these processes. Several theories suggest that the speech production system, including the left inferior frontal cortex (LIFC), is recruited for prediction but evidence that LIFC plays a causal role is lacking. We first examined the effects of predictability on memory and then tested the role of posterior LIFC using transcranial magnetic stimulation (TMS). In Experiment 1, participants read category cues, followed by a predictable, unpredictable, or incongruent target word for later recall. We observed a predictability benefit to memory, with predictable words remembered better than unpredictable words. In Experiment 2, participants performed the same task with electroencephalography (EEG) while undergoing event-related TMS over posterior LIFC using a protocol known to disrupt speech production, or over the right hemisphere homologue as an active control site. Under control stimulation, predictable words were better recalled than unpredictable words, replicating Experiment 1. This predictability benefit to memory was eliminated under LIFC stimulation. Moreover, while an a priori ROI-based analysis did not yield evidence for a reduction in the N400 predictability effect, mass-univariate analyses did suggest that the N400 predictability effect was reduced in spatial and temporal extent under LIFC stimulation. Collectively, these results provide causal evidence that the LIFC is recruited for prediction during silent reading, consistent with prediction-through-production accounts.

Beyond Broca: neural architecture and evolution of a dual motor speech coordination system

Classical neural architecture models of speech production propose a single system centred on Broca's area coordinating all the vocal articulators from lips to larynx. Modern evidence has challenged both the idea that Broca's area is involved in motor speech coordination and that there is only one coordination network. Drawing on a wide range of evidence, here we propose a dual speech coordination model in which laryngeal control of pitch-related aspects of prosody and song are coordinated by a hierarchically organized dorsolateral system while supralaryngeal articulation at the phonetic/syllabic level is coordinated by a more ventral system posterior to Broca's area. We argue further that these two speech production subsystems have distinguishable evolutionary histories and discuss the implications for models of language evolution.

The dual stream model of speech and language processing

The Wernicke-Lichtheim-Geschwind model of the neurology of language has served the field well despite its limited scope. More recent work has updated the basic architecture of the classical model and expanded its scope. This chapter briefly reviews the Wernicke-Lichtheim-Geschwind model and points out its shortcomings, then describes and motivates the dual stream model and how it solves several empirical shortcomings of the classical model. The chapter also (i) underscores how the dual stream model relates to the organization of nonlinguistic cortical networks, integrating language systems with the broader functional-anatomical landscape, (ii) describes recent work that further specifies the computational architecture and neural correlates of the dorsal speech production system, and (iii) summarizes recent extensions of the architectural framework to include syntax.

Utility of the Repeat and Point Test for Subtyping Patients With Primary Progressive Aphasia

BACKGROUND

Primary progressive aphasia (PPA) may present with three distinct clinical sybtypes: semantic variant PPA (svPPA), nonfluent/agrammatic variant PPA (nfvPPA), and logopenic variant PPA (lvPPA).

OBJECTIVE

The aim was to examine the utility of the German version of the Repeat and Point (R&P) Test for subtyping patients with PPA.

METHOD

During the R&P Test, the examiner reads out aloud a noun and the participants are asked to repeat the word and subsequently point to the corresponding picture. Data from 204 patients (68 svPPA, 85 nfvPPA, and 51 lvPPA) and 33 healthy controls were analyzed.

RESULTS

Controls completed both tasks with >90% accuracy. Patients with svPPA had high scores in repetition (mean=9.2±1.32) but low scores in pointing (mean=6±2.52). In contrast, patients with nfvPPA and lvPPA performed comparably in both tasks with lower scores in repetition (mean=7.4±2.7 for nfvPPA and 8.2±2.34 for lvPPA) but higher scores in pointing (mean=8.9±1.41 for nfvPPA and 8.6±1.62 for lvPPA). The R&P Test had high accuracy discriminating svPPA from nfvPPA (83% accuracy) and lvPPA (79% accuracy). However, there was low accuracy discriminating nfvPPA from lvPPA (<60%).

CONCLUSION

The R&P Test helps to differentiate svPPA from 2 nonsemantic variants (nfvPPA and lvPPA). However, additional tests are required for the differentiation of nfvPPA and lvPPA.

Extensive long-term verbal memory training is associated with brain plasticity

The human brain has a remarkable capacity to store a lifetime of information through visual or auditory routes. It excels and exceeds any artificial memory system in mixing and integrating multiple pieces of information encoded. In this study, a group of verbal memory experts was evaluated by multiple structural brain analysis methods to record the changes in the brain structure. The participants were professional Hindu pandits (priests/scholars) trained in reciting Vedas and other forms of Hindu scriptures. These professional Vedic priests are experts in memorization and recitation of oral texts with precise diction. Vedas are a collection of hymns. It is estimated that there are more than 20,000 mantras and shlokas in the four Vedas. The analysis included the measurement of the grey and white matter density, gyrification, and cortical thickness in a group of Vedic pandits and comparing these measures with a matched control group. The results revealed an increased grey matter (GM) and white matter (WM) in the midbrain, pons, thalamus, parahippocampus, and orbitofrontal regions in pandits. The whole-brain corelation analysis using length of post-training  teaching duration showed significant correlation with the left angular gyrus. We also found increased gyrification in the insula, supplementary motor area, medial frontal areas, and increased cortical thickness (CT) in the right temporal pole and caudate regions of the brain. These findings, collectively, provide unique information regarding the association between crucial memory regions in the brain and long-term practice of oral recitation of scriptures from memory with the proper diction that also involved controlled breathing.

Hybrid auditory fMRI: In pursuit of increasing data acquisition while decreasing the impact of scanner noise

BACKGROUND

Two challenges in auditory fMRI include the loud scanner noise during sound presentation and slow data acquisition. Here, we introduce a new auditory imaging protocol, termed "hybrid", that alleviates these obstacles.

NEW METHOD

We designed a within-subject experiment (N = 14) wherein language-driven activity was measured by hybrid, interleaved silent (ISSS), and continuous multiband acquisition. To determine the advantage of noise attenuation during sound presentation, hybrid was compared to multiband. To identify the benefits of increased temporal resolution, hybrid was compared to ISSS. Data were evaluated by whole-brain univariate general linear modeling (GLM) and multivariate pattern analysis (MVPA).

RESULTS

Comparison with existing methods: CONCLUSIONS: Our data revealed that hybrid imaging restored neural activity in the canonical language network that was absent due to the loud noise or slow sampling in the conventional imaging protocols. With its noise-attenuated sound presentation windows and increased acquisition speed, the hybrid protocol is well-suited for auditory fMRI research tracking neural activity pertaining to fast, time-varying acoustic events.

Behavioral and neural correlates of speech motor sequence learning in stuttering and neurotypical speakers: an fMRI investigation

Stuttering is a neurodevelopmental disorder characterized by impaired production of coordinated articulatory movements needed for fluent speech. It is currently unknown whether these abnormal production characteristics reflect disruptions to brain mechanisms underlying the acquisition and/or execution of speech motor sequences. To dissociate learning and control processes, we used a motor sequence learning paradigm to examine the behavioral and neural correlates of learning to produce novel phoneme sequences in adults who stutter (AWS) and neurotypical controls. Participants intensively practiced producing pseudowords containing non-native consonant clusters (e.g., "gvasf") over two days. The behavioral results indicated that although the two experimental groups showed comparable learning trajectories, AWS performed significantly worse on the task prior to and after speech motor practice. Using functional magnetic resonance imaging (fMRI), the authors compared brain activity during articulation of the practiced words and a set of novel pseudowords (matched in phonetic complexity). FMRI analyses revealed no differences between AWS and controls in cortical or subcortical regions; both groups showed comparable increases in activation in left-lateralized brain areas implicated in phonological working memory and speech motor planning during production of the novel sequences compared to the practiced sequences. Moreover, activation in left-lateralized basal ganglia sites was negatively correlated with in-scanner mean disfluency in AWS. Collectively, these findings demonstrate that AWS exhibit no deficit in constructing new speech motor sequences but do show impaired execution of these sequences before and after they have been acquired and consolidated.

Mental operations in rhythm: Motor-to-sensory transformation mediates imagined singing

What enables the mental activities of thinking verbally or humming in our mind? We hypothesized that the interaction between motor and sensory systems induces speech and melodic mental representations, and this motor-to-sensory transformation forms the neural basis that enables our verbal thinking and covert singing. Analogous with the neural entrainment to auditory stimuli, participants imagined singing lyrics of well-known songs rhythmically while their neural electromagnetic signals were recorded using magnetoencephalography (MEG). We found that when participants imagined singing the same song in similar durations across trials, the delta frequency band (1–3 Hz, similar to the rhythm of the songs) showed more consistent phase coherence across trials. This neural phase tracking of imagined singing was observed in a frontal-parietal-temporal network: the proposed motor-to-sensory transformation pathway, including the inferior frontal gyrus (IFG), insula (INS), premotor area, intra-parietal sulcus (IPS), temporal-parietal junction (TPJ), primary auditory cortex (Heschl’s gyrus [HG]), and superior temporal gyrus (STG) and sulcus (STS). These results suggest that neural responses can entrain the rhythm of mental activity. Moreover, the theta-band (4–8 Hz) phase coherence was localized in the auditory cortices. The mu (9–12 Hz) and beta (17–20 Hz) bands were observed in the right-lateralized sensorimotor systems that were consistent with the singing context. The gamma band was broadly manifested in the observed network. The coherent and frequency-specific activations in the motor-to-sensory transformation network mediate the internal construction of perceptual representations and form the foundation of neural computations for mental operations.

What enables our mental activities for thinking verbally or humming in our mind? Using an imagined singing paradigm with magnetoencephalography recordings, this study shows that neural oscillations in the motor-to-sensory transformation network tracked inner speech and covert singing.

The large-scale structural connectome of task-specific focal dystonia

The emerging view of dystonia is that of a large‐scale functional network disorder, in which the communication is disrupted between sensorimotor cortical areas, basal ganglia, thalamus, and cerebellum. The structural underpinnings of functional alterations in dystonia are, however, poorly understood. Notably, it is unclear whether structural changes form a larger‐scale dystonic network or rather remain focal to isolated brain regions, merely underlying their functional abnormalities. Using diffusion‐weighted imaging and graph theoretical analysis, we examined inter‐regional white matter connectivity of the whole‐brain structural network in two different forms of task‐specific focal dystonia, writer's cramp and laryngeal dystonia, compared to healthy individuals. We show that, in addition to profoundly altered functional network in focal dystonia, its structural connectome is characterized by large‐scale aberrations due to abnormal transfer of prefrontal and parietal nodes between neural communities and the reorganization of normal hub architecture, commonly involving the insula and superior frontal gyrus in patients compared to controls. Other prominent common changes involved the basal ganglia, parietal and cingulate cortical regions, whereas premotor and occipital abnormalities distinctly characterized the two forms of dystonia. We propose a revised pathophysiological model of focal dystonia as a disorder of both functional and structural connectomes, where dystonia form‐specific abnormalities underlie the divergent mechanisms in the development of distinct clinical symptomatology. These findings may guide the development of novel therapeutic strategies directed at targeted neuromodulation of pathophysiological brain regions for the restoration of their structural and functional connectivity.

The Cortical Organization of Syntax

Syntax, the structure of sentences, enables humans to express an infinite range of meanings through finite means. The neurobiology of syntax has been intensely studied but with little consensus. Two main candidate regions have been identified: the posterior inferior frontal gyrus (pIFG) and the posterior middle temporal gyrus (pMTG). Integrating research in linguistics, psycholinguistics, and neuroscience, we propose a neuroanatomical framework for syntax that attributes distinct syntactic computations to these regions in a unified model. The key theoretical advances are adopting a modern lexicalized view of syntax in which the lexicon and syntactic rules are intertwined, and recognizing a computational asymmetry in the role of syntax during comprehension and production. Our model postulates a hierarchical lexical-syntactic function to the pMTG, which interconnects previously identified speech perception and conceptual-semantic systems in the temporal and inferior parietal lobes, crucial for both sentence production and comprehension. These relational hierarchies are transformed via the pIFG into morpho-syntactic sequences, primarily tied to production. We show how this architecture provides a better account of the full range of data and is consistent with recent proposals regarding the organization of phonological processes in the brain.

Structured sequence processing and combinatorial binding: neurobiologically and computationally informed hypotheses

Understanding how the brain forms representations of structured information distributed in time is a challenging endeavour for the neuroscientific community, requiring computationally and neurobiologically informed approaches. The neural mechanisms for segmenting continuous streams of sensory input and establishing representations of dependencies remain largely unknown, as do the transformations and computations occurring between the brain regions involved in these aspects of sequence processing. We propose a blueprint for a neurobiologically informed and informing computational model of sequence processing (entitled: Vector-symbolic Sequencing of Binding INstantiating Dependencies, or VS-BIND). This model is designed to support the transformation of serially ordered elements in sensory sequences into structured representations of bound dependencies, readily operates on multiple timescales, and encodes or decodes sequences with respect to chunked items wherever dependencies occur in time. The model integrates established vector symbolic additive and conjunctive binding operators with neurobiologically plausible oscillatory dynamics, and is compatible with modern spiking neural network simulation methods. We show that the model is capable of simulating previous findings from structured sequence processing tasks that engage fronto-temporal regions, specifying mechanistic roles for regions such as prefrontal areas 44/45 and the frontal operculum during interactions with sensory representations in temporal cortex. Finally, we are able to make predictions based on the configuration of the model alone that underscore the importance of serial position information, which requires input from time-sensitive cells, known to reside in the hippocampus and dorsolateral prefrontal cortex. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Brain activation during non-habitual speech production: Revisiting the effects of simulated disfluencies in fluent speakers

Over the past decades, brain imaging studies in fluently speaking participants have greatly advanced our knowledge of the brain areas involved in speech production. In addition, complementary information has been provided by investigations of brain activation patterns associated with disordered speech. In the present study we specifically aimed to revisit and expand an earlier study by De Nil and colleagues, by investigating the effects of simulating disfluencies on the brain activation patterns of fluent speakers during overt and covert speech production. In contrast to the De Nil et al. study, the current findings show that the production of voluntary, self-generated disfluencies by fluent speakers resulted in increased recruitment and activation of brain areas involved in speech production. These areas show substantial overlap with the neural networks involved in motor sequence learning in general, and learning of speech production, in particular. The implications of these findings for the interpretation of brain imaging studies on disordered and non-habitual speech production are discussed.

Is It Speech or Song? Effect of Melody Priming on Pitch Perception of Modified Mandarin Speech

Tonal languages make use of pitch variation for distinguishing lexical semantics, and their melodic richness seems comparable to that of music. The present study investigated a novel priming effect of melody on the pitch processing of Mandarin speech. When a spoken Mandarin utterance is preceded by a musical melody, which mimics the melody of the utterance, the listener is likely to perceive this utterance as song. We used functional magnetic resonance imaging to examine the neural substrates of this speech-to-song transformation. Pitch contours of spoken utterances were modified so that these utterances can be perceived as either speech or song. When modified speech (target) was preceded by a musical melody (prime) that mimics the speech melody, a task of judging the melodic similarity between the target and prime was associated with increased activity in the inferior frontal gyrus (IFG) and superior/middle temporal gyrus (STG/MTG) during target perception. We suggest that the pars triangularis of the right IFG may allocate attentional resources to the multi-modal processing of speech melody, and the STG/MTG may integrate the phonological and musical (melodic) information of this stimulus. These results are discussed in relation to subvocal rehearsal, a speech-to-song illusion, and song perception.