Selective Amplification of Stimulus Differences during Categorical Processing of Speech

The phonological store of working memory: A critique and an alternative, perceptual-motor, approach to verbal short-term memory

A key quality of a good theory is its fruitfulness, one measure of which might be the degree to which it compels researchers to test it, refine it, or offer alternative explanations of the same empirical data. Perhaps the most fruitful element of Baddeley and Hitch's (1974) Working Memory framework has been the concept of a short-term phonological store, a discrete cognitive module dedicated to the passive storage of verbal material that is architecturally fractionated from perceptual, language, and articulatory systems. This review discusses how the phonological store construct has served as the main theoretical springboard for an alternative perceptual-motor approach in which serial-recall performance reflects the opportunistic co-opting of the articulatory-planning system and, when auditory material is involved, the products of obligatory auditory perceptual organisation. It is argued that this approach, which rejects the need to posit a distinct short-term store, provides a better account of the two putative empirical hallmarks of the phonological store-the phonological similarity effect and the irrelevant speech effect-and that it shows promise too in being able to account for nonword repetition and word-form learning, the supposed evolved function of the phonological store. The neuropsychological literature cited as strong additional support for the phonological store concept is also scrutinised through the lens of the perceptual-motor approach for the first time and a tentative articulatory-planning deficit hypothesis for the "short-term memory" patient profile is advanced. Finally, the relation of the perceptual-motor approach to other "emergent-property" accounts of short-term memory is briefly considered.

Sensitivity to and Control of Distraction: Distractor-Entrained Oscillation and Frontoparietal EEG Gamma Synchronization

While recent advancements have been made towards a better understanding of the involvement of the prefrontal cortex (PFC) in the context of cognitive control, the exact mechanism is still not fully understood. Successful behavior requires the correct detection of goal-relevant cues and resisting irrelevant distractions. Frontal parietal networks have been implicated as important for maintaining cognitive control in the face of distraction. The present study investigated the role of gamma-band power in distraction resistance and frontoparietal networks, as its increase is linked to cholinergic activity. We examined changes in gamma activity and their relationship to frontoparietal top-down modulation for distractor challenges and to bottom-up distractor processing. Healthy young adults were tested using a modified version of the distractor condition sustained attention task (dSAT) while wearing an EEG. The modified distractor was designed so that oscillatory activities could be entrained to it, and the strength of entrainment was used to assess the degree of distraction. Increased top-down control during the distractor challenge increased gamma power in the left parietal regions rather than the right prefrontal regions predicted from rodent studies. Specifically, left parietal gamma power increased in response to distraction where the amount of this increase was negatively correlated with the neural activity reflecting bottom-up distractor processing in the visual area. Variability in gamma power in right prefrontal regions was associated with increased response time variability during distraction. This may suggest that the right prefrontal region may contribute to the signaling needed for top-down control rather than its implementation.

The visuo-sensorimotor substrate of co-speech gesture processing

Co-speech gestures are integral to human communication and exhibit diverse forms, each serving a distinct communication function. However, existing literature has focused on individual gesture types, leaving a gap in understanding the comparative neural processing of these diverse forms. To address this, our study investigated the neural processing of two types of iconic gestures: those representing attributes or event knowledge of entity concepts, beat gestures enacting rhythmic manual movements without semantic information, and self-adaptors. During functional magnetic resonance imaging, systematic randomization and attentive observation of video stimuli revealed a general neural substrate for co-speech gesture processing primarily in the bilateral middle temporal and inferior parietal cortices, characterizing visuospatial attention, semantic integration of cross-modal information, and multisensory processing of manual and audiovisual inputs. Specific types of gestures and grooming movements elicited distinct neural responses. Greater activity in the right supramarginal and inferior frontal regions was specific to self-adaptors, and is relevant to the spatiomotor and integrative processing of speech and gestures. The semantic and sensorimotor regions were least active for beat gestures. The processing of attribute gestures was most pronounced in the left posterior middle temporal gyrus upon access to knowledge of entity concepts. This fMRI study illuminated the neural underpinnings of gesture-speech integration and highlighted the differential processing pathways for various co-speech gestures.

Beyond Broca's and Wernicke's: Functional Mapping of Ancillary Language Centers Prior to Brain Tumor Surgery

Functional MRI is a well-established tool used for pre-surgical planning to help the neurosurgeon have a roadmap of critical functional areas that should be avoided, if possible, during surgery to minimize morbidity for patients with brain tumors (though this also has applications for surgical resection of epileptogenic tissue and vascular lesions). This article reviews the locations of secondary language centers within the brain along with imaging findings to help improve our confidence in our knowledge on language lateralization. Brief overviews of these language centers and their contributions to the language networks will be discussed. These language centers include primary language centers of "Broca's Area" and "Wernicke's Area". However, there are multiple secondary language centers such as the dorsal lateral prefrontal cortex (DLPFC), frontal eye fields, pre- supplemental motor area (pre-SMA), Basal Temporal Language Area (BTLA), along with other areas of activation. Knowing these foci helps to increase self-assurance when discussing the nature of laterality with the neurosurgeon. By knowing secondary language centers for language lateralization, via fMRI, one can feel confident on providing neurosurgeon colleagues with appropriate information on the laterality of language in preparation for surgery.

Right Posterior Temporal Cortex Supports Integration of Phonetic and Talker Information

Though the right hemisphere has been implicated in talker processing, it is thought to play a minimal role in phonetic processing, at least relative to the left hemisphere. Recent evidence suggests that the right posterior temporal cortex may support learning of phonetic variation associated with a specific talker. In the current study, listeners heard a male talker and a female talker, one of whom produced an ambiguous fricative in /s/-biased lexical contexts (e.g., epi?ode) and one who produced it in /∫/-biased contexts (e.g., friend?ip). Listeners in a behavioral experiment (Experiment 1) showed evidence of lexically guided perceptual learning, categorizing ambiguous fricatives in line with their previous experience. Listeners in an fMRI experiment (Experiment 2) showed differential phonetic categorization as a function of talker, allowing for an investigation of the neural basis of talker-specific phonetic processing, though they did not exhibit perceptual learning (likely due to characteristics of our in-scanner headphones). Searchlight analyses revealed that the patterns of activation in the right superior temporal sulcus (STS) contained information about who was talking and what phoneme they produced. We take this as evidence that talker information and phonetic information are integrated in the right STS. Functional connectivity analyses suggested that the process of conditioning phonetic identity on talker information depends on the coordinated activity of a left-lateralized phonetic processing system and a right-lateralized talker processing system. Overall, these results clarify the mechanisms through which the right hemisphere supports talker-specific phonetic processing.

Hemispheric dissociations in regions supporting auditory sentence comprehension in older adults

We investigated how the aging brain copes with acoustic and syntactic challenges during spoken language comprehension. Thirty-eight healthy adults aged 54 - 80 years (M = 66 years) participated in an fMRI experiment wherein listeners indicated the gender of an agent in short spoken sentences that varied in syntactic complexity (object-relative vs subject-relative center-embedded clause structures) and acoustic richness (high vs low spectral detail, but all intelligible). We found widespread activity throughout a bilateral frontotemporal network during successful sentence comprehension. Consistent with prior reports, bilateral inferior frontal gyrus and left posterior superior temporal gyrus were more active in response to object-relative sentences than to subject-relative sentences. Moreover, several regions were significantly correlated with individual differences in task performance: Activity in right frontoparietal cortex and left cerebellum (Crus I & II) showed a negative correlation with overall comprehension. By contrast, left frontotemporal areas and right cerebellum (Lobule VII) showed a negative correlation with accuracy specifically for syntactically complex sentences. In addition, laterality analyses confirmed a lack of hemispheric lateralization in activity evoked by sentence stimuli in older adults. Importantly, we found different hemispheric roles, with a left-lateralized core language network supporting syntactic operations, and right-hemisphere regions coming into play to aid in general cognitive demands during spoken sentence processing. Together our findings support the view that high levels of language comprehension in older adults are maintained by a close interplay between a core left hemisphere language network and additional neural resources in the contralateral hemisphere.

Speech sound categorization: The contribution of non-auditory and auditory cortical regions

Which processes in the human brain lead to the categorical perception of speech sounds? Investigation of this question is hampered by the fact that categorical speech perception is normally confounded by acoustic differences in the stimulus. By using ambiguous sounds, however, it is possible to dissociate acoustic from perceptual stimulus representations. Twenty-seven normally hearing individuals took part in an fMRI study in which they were presented with an ambiguous syllable (intermediate between /da/ and /ga/) in one ear and with disambiguating acoustic feature (third formant, F3) in the other ear. Multi-voxel pattern searchlight analysis was used to identify brain areas that consistently differentiated between response patterns associated with different syllable reports. By comparing responses to different stimuli with identical syllable reports and identical stimuli with different syllable reports, we disambiguated whether these regions primarily differentiated the acoustics of the stimuli or the syllable report. We found that BOLD activity patterns in left perisylvian regions (STG, SMG), left inferior frontal regions (vMC, IFG, AI), left supplementary motor cortex (SMA/pre-SMA), and right motor and somatosensory regions (M1/S1) represent listeners' syllable report irrespective of stimulus acoustics. Most of these regions are outside of what is traditionally regarded as auditory or phonological processing areas. Our results indicate that the process of speech sound categorization implicates decision-making mechanisms and auditory-motor transformations.

Categorical perception and influence of attention on neural consistency in response to speech sounds in adults with dyslexia

Developmental dyslexia is a common neurodevelopmental disorder that is associated with alterations in the behavioral and neural processing of speech sounds, but the scope and nature of that association is uncertain. It has been proposed that more variable auditory processing could underlie some of the core deficits in this disorder. In the current study, magnetoencephalography (MEG) data were acquired from adults with and without dyslexia while they passively listened to or actively categorized tokens from a /ba/-/da/ consonant continuum. We observed no significant group difference in active categorical perception of this continuum in either of our two behavioral assessments. During passive listening, adults with dyslexia exhibited neural responses that were as consistent as those of typically reading adults in six cortical regions associated with auditory perception, language, and reading. However, they exhibited significantly less consistency in the left supramarginal gyrus, where greater inconsistency correlated significantly with worse decoding skills in the group with dyslexia. The group difference in the left supramarginal gyrus was evident only when neural data were binned with a high temporal resolution and was only significant during the passive condition. Interestingly, consistency significantly improved in both groups during active categorization versus passive listening. These findings suggest that adults with dyslexia exhibit typical levels of neural consistency in response to speech sounds with the exception of the left supramarginal gyrus and that this consistency increases during active versus passive perception of speech sounds similarly in the two groups.

Neural correlates of phonology-to-orthography mapping consistency effects on Chinese spoken word recognition

Previous studies have shown that reading experience reshapes speech processing. The orthography can be implemented in the brain by restructuring the phonological representations or being co-activated during spoken word recognition. This study utilized event-related functional magnetic resonance imaging and functional connectivity analysis to examine the neural mechanism underlying two types of orthographic effects in the Chinese auditory semantic category task, namely phonology-to-orthography consistency (POC) and homophone density (HD). We found that the POC effects originated from the speech network, suggesting that sublexical orthographic information could change the organization of preexisting phonological representations when learning to read. Meanwhile, the HD effects were localized to the left fusiform and lingual gyrus, suggesting that lexical orthographic knowledge may be activated online during spoken word recognition. These results demonstrated the different natures and neural mechanisms for the POC and HD effects on Chinese spoken word recognition.

Categorical Encoding of Vowels in Primary Auditory Cortex

Speech perception involves mapping from a continuous and variable acoustic speech signal to discrete, linguistically meaningful units. However, it is unclear where in the auditory processing stream speech sound representations cease to be veridical (faithfully encoding precise acoustic properties) and become categorical (encoding sounds as linguistic categories). In this study, we used functional magnetic resonance imaging and multivariate pattern analysis to determine whether tonotopic primary auditory cortex (PAC), defined as tonotopic voxels falling within Heschl's gyrus, represents one class of speech sounds-vowels-veridically or categorically. For each of 15 participants, 4 individualized synthetic vowel stimuli were generated such that the vowels were equidistant in acoustic space, yet straddled a categorical boundary (with the first 2 vowels perceived as [i] and the last 2 perceived as [i]). Each participant's 4 vowels were then presented in a block design with an irrelevant but attention-demanding level change detection task. We found that in PAC bilaterally, neural discrimination between pairs of vowels that crossed the categorical boundary was more accurate than neural discrimination between equivalently spaced vowel pairs that fell within a category. These findings suggest that PAC does not represent vowel sounds veridically, but that encoding of vowels is shaped by linguistically relevant phonemic categories.

Neural Decoding Reveals Concurrent Phonemic and Subphonemic Representations of Speech Across Tasks

Robust and efficient speech perception relies on the interpretation of acoustically variable phoneme realizations, yet prior neuroimaging studies are inconclusive regarding the degree to which subphonemic detail is maintained over time as categorical representations arise. It is also unknown whether this depends on the demands of the listening task. We addressed these questions by using neural decoding to quantify the (dis)similarity of brain response patterns evoked during two different tasks. We recorded magnetoencephalography (MEG) as adult participants heard isolated, randomized tokens from a /ba/-/da/ speech continuum. In the passive task, their attention was diverted. In the active task, they categorized each token as ba or da. We found that linear classifiers successfully decoded ba vs. da perception from the MEG data. Data from the left hemisphere were sufficient to decode the percept early in the trial, while the right hemisphere was necessary but not sufficient for decoding at later time points. We also decoded stimulus representations and found that they were maintained longer in the active task than in the passive task; however, these representations did not pattern more like discrete phonemes when an active categorical response was required. Instead, in both tasks, early phonemic patterns gave way to a representation of stimulus ambiguity that coincided in time with reliable percept decoding. Our results suggest that the categorization process does not require the loss of subphonemic detail, and that the neural representation of isolated speech sounds includes concurrent phonemic and subphonemic information.

A distributed dynamic brain network mediates linguistic tone representation and categorization

Successful categorization requires listeners to represent the incoming sensory information, resolve the "blooming, buzzing confusion" inherent to noisy sensory signals, and leverage the accumulated evidence towards making a decision. Despite decades of intense debate, the neural systems underlying speech categorization remain unresolved. Here we assessed the neural representation and categorization of lexical tones by native Mandarin speakers (N = 31) across a range of acoustic and contextual variabilities (talkers, perceptual saliences, and stimulus-contexts) using functional magnetic imaging (fMRI) and an evidence accumulation model of decision-making. Univariate activation and multivariate pattern analyses reveal that the acoustic-variability-tolerant representations of tone category are observed within the middle portion of the left superior temporal gyrus (STG). Activation patterns in the frontal and parietal regions also contained category-relevant information that was differentially sensitive to various forms of variability. The robustness of neural representations of tone category in a distributed fronto-temporoparietal network is associated with trial-by-trial decision-making parameters. These findings support a hybrid model involving a representational core within the STG that operates dynamically within an extensive frontoparietal network to support the representation and categorization of linguistic pitch patterns.

Lexical Information Guides Retuning of Neural Patterns in Perceptual Learning for Speech

A listener's interpretation of a given speech sound can vary probabilistically from moment to moment. Previous experience (i.e., the contexts in which one has encountered an ambiguous sound) can further influence the interpretation of speech, a phenomenon known as perceptual learning for speech. This study used multivoxel pattern analysis to query how neural patterns reflect perceptual learning, leveraging archival fMRI data from a lexically guided perceptual learning study conducted by Myers and Mesite [Myers, E. B., & Mesite, L. M. Neural systems underlying perceptual adjustment to non-standard speech tokens. Journal of Memory and Language, 76, 80-93, 2014]. In that study, participants first heard ambiguous /s/-/∫/ blends in either /s/-biased lexical contexts (epi_ode) or /∫/-biased contexts (refre_ing); subsequently, they performed a phonetic categorization task on tokens from an /asi/-/a∫i/ continuum. In the current work, a classifier was trained to distinguish between phonetic categorization trials in which participants heard unambiguous productions of /s/ and those in which they heard unambiguous productions of /∫/. The classifier was able to generalize this training to ambiguous tokens from the middle of the continuum on the basis of individual participants' trial-by-trial perception. We take these findings as evidence that perceptual learning for speech involves neural recalibration, such that the pattern of activation approximates the perceived category. Exploratory analyses showed that left parietal regions (supramarginal and angular gyri) and right temporal regions (superior, middle, and transverse temporal gyri) were most informative for categorization. Overall, our results inform an understanding of how moment-to-moment variability in speech perception is encoded in the brain.

Neuroanatomical changes associated with age-related hearing loss and listening effort

Age-related hearing loss is associated with a decrease in hearing abilities for high frequencies and therefore leads to impairments in understanding speech—in particular, under adverse listening conditions. Growing evidence suggests that age-related hearing loss is related to various neural changes, for instance, affecting auditory and frontal brain regions. How the decreased auditory input and the increased listening effort in daily life are associated with structural changes is less clear, since previous evidence is scarce and mostly involved low sample sizes. Hence, the aim of the current study was to investigate the impact of age-related untreated hearing loss and subjectively rated daily life listening effort on grey matter and white matter changes in a large sample of participants (n = 71). For that aim, we conducted anatomical MRI and diffusion tensor imaging (DTI) in elderly hard-of-hearing and age-matched normal-hearing participants. Our results showed significantly lower grey matter volume in the middle frontal cortex in hard-of-hearing compared to normal-hearing participants. Further, higher listening effort was associated with lower grey matter volume and cortical thickness in the orbitofrontal cortex and lower grey matter volume in the inferior frontal cortex. No significant relations between hearing abilities or listening effort were obtained for white matter integrity in tracts connecting auditory and prefrontal as well as visual areas. These findings provide evidence that hearing impairment as well as daily life listening effort seems to be associated with grey matter loss in prefrontal brain regions. We further conclude that alterations in cortical thickness seem to be linked to the increased listening effort rather than the hearing loss itself.

Task load modulates tDCS effects on brain network for phonological processing

Motor participation in phonological processing can be modulated by task nature across the speech perception to speech production range. The pars opercularis of the left inferior frontal gyrus (LIFG) would be increasingly active across this range, because of changing motor demands. Here, we investigated with simultaneous tDCS and fMRI whether the task load modulation of tDCS effects translates into predictable patterns of functional connectivity. Findings were analysed under the "multi-node framework", according to which task load and the network structure underlying cognitive functions are modulators of tDCS effects. In a within-subject study, participants (N = 20) performed categorical perception, lexical decision and word naming tasks [which differentially recruit the target of stimulation (LIFG)], which were repeatedly administered in three tDCS sessions (anodal, cathodal and sham). The LIFG, left superior temporal gyrus and their right homologues formed the target network subserving phonological processing. C-tDCS inhibition and A-tDCS excitation should increase with task load. Correspondingly, the larger the task load, the larger the relevance of the target for the task and smaller the room for compensation of C-tDCS inhibition by less relevant nodes. Functional connectivity analyses were performed with partial correlations, and network compensation globally inferred by comparing the relative number of significant connections each condition induced relative to sham. Overall, simultaneous tDCS and fMRI was adequate to show that motor participation in phonological processing is modulated by task nature. Network responses induced by C-tDCS across phonological processing tasks matched predictions. A-tDCS effects were attributed to optimisation of network efficiency.

Age-related hearing loss influences functional connectivity of auditory cortex for the McGurk illusion

Age-related hearing loss affects hearing at high frequencies and is associated with difficulties in understanding speech. Increased audio-visual integration has recently been found in age-related hearing impairment, the brain mechanisms that contribute to this effect are however unclear. We used functional magnetic resonance imaging in elderly subjects with normal hearing and mild to moderate uncompensated hearing loss. Audio-visual integration was studied using the McGurk task. In this task, an illusionary fused percept can occur if incongruent auditory and visual syllables are presented. The paradigm included unisensory stimuli (auditory only, visual only), congruent audio-visual and incongruent (McGurk) audio-visual stimuli. An illusionary precept was reported in over 60% of incongruent trials. These McGurk illusion rates were equal in both groups of elderly subjects and correlated positively with speech-in-noise perception and daily listening effort. Normal-hearing participants showed an increased neural response in left pre- and postcentral gyri and right middle frontal gyrus for incongruent stimuli (McGurk) compared to congruent audio-visual stimuli. Activation patterns were however not different between groups. Task-modulated functional connectivity differed between groups showing increased connectivity from auditory cortex to visual, parietal and frontal areas in hard of hearing participants as compared to normal-hearing participants when comparing incongruent stimuli (McGurk) with congruent audio-visual stimuli. These results suggest that changes in functional connectivity of auditory cortex rather than activation strength during processing of audio-visual McGurk stimuli accompany age-related hearing loss.

Reading and spelling skills are differentially related to phonological processing: Behavioral and fMRI study

The manuscript reports a study on a large sample (N = 170) of Polish speaking 8-13 year old children, whose brain activation was measured in relation to tasks that require auditory phonological processing. We aimed to relate brain activation to individual differences in reading and spelling. We found that individual proficiency in both reading and spelling significantly correlated with activation of the left ventral occipito-temporal cortex encompassing the Visual Word Form Area which has been implicated in automatic orthographic activations. Reading but not spelling was found to correlate with activation in the left anterior dorsal stream (anterior supramarginal and postcentral gyri). Our results indicate that the level of both reading and spelling is related to activity in areas involved in the storage of fine-grained orthographic representations. However, only the reading level is uniquely related to activity of regions responsible for the articulation, motor planning and grapheme-to-phoneme correspondence, which form the basis for effective decoding skill.

Sensory-to-Category Transformation via Dynamic Reorganization of Ensemble Structures in Mouse Auditory Cortex

The ability to group physical stimuli into behaviorally relevant categories is fundamental to perception and cognition. Despite a large body of work on stimulus categorization at the behavioral and cognitive levels, little is known about the underlying mechanisms at the neuronal level. Here, combining mouse auditory psychophysical behavior and in vivo two-photon imaging from the auditory cortex, we investigate how sensory-to-category transformation is implemented by cortical neurons during a stimulus categorization task. Distinct from responses during passive listening, many neurons exhibited emergent selectivity to stimuli near the category boundary during task performance, reshaping local tuning maps; other neurons became more selective to category membership of stimuli. At the population level, local cortical ensembles robustly encode category information and predict trial-by-trial decisions during task performance. Our data uncover a task-dependent dynamic reorganization of cortical response patterns serving as a neural mechanism for sensory-to-category transformation during perceptual decision-making.

Interaction of the effects associated with auditory-motor integration and attention-engaging listening tasks

A number of previous studies have implicated regions in posterior auditory cortex (AC) in auditory-motor integration during speech production. Other studies, in turn, have shown that activation in AC and adjacent regions in the inferior parietal lobule (IPL) is strongly modulated during active listening and depends on task requirements. The present fMRI study investigated whether auditory-motor effects interact with those related to active listening tasks in AC and IPL. In separate task blocks, our subjects performed either auditory discrimination or 2-back memory tasks on phonemic or nonphonemic vowels. They responded to targets by either overtly repeating the last vowel of a target pair, overtly producing a given response vowel, or by pressing a response button. We hypothesized that the requirements for auditory-motor integration, and the associated activation, would be stronger during repetition than production responses and during repetition of nonphonemic than phonemic vowels. We also hypothesized that if auditory-motor effects are independent of task-dependent modulations, then the auditory-motor effects should not differ during discrimination and 2-back tasks. We found that activation in AC and IPL was significantly modulated by task (discrimination vs. 2-back), vocal-response type (repetition vs. production), and motor-response type (vocal vs. button). Motor-response and task effects interacted in IPL but not in AC. Overall, the results support the view that regions in posterior AC are important in auditory-motor integration. However, the present study shows that activation in wide AC and IPL regions is modulated by the motor requirements of active listening tasks in a more general manner. Further, the results suggest that activation modulations in AC associated with attention-engaging listening tasks and those associated with auditory-motor performance are mediated by independent mechanisms.

The parietal lobe and language

Although the parietal lobe was considered by many of the earliest investigators of disordered language to be a major component of the neural systems instantiating language, most views of the anatomic substrate of language emphasize the role of temporal and frontal lobes in language processing. We review evidence from lesion studies as well as functional neuroimaging, demonstrating that the left parietal lobe is also crucial for several aspects of language. First, we argue that the parietal lobe plays a major role in semantic processing, particularly for "thematic" relationships in which information from multiple sensory and motor domains is integrated. Additionally, we review a number of accounts that emphasize the role of the left parietal lobe in phonologic processing. Although the accounts differ somewhat with respect to the nature of the linguistic computations subserved by the parietal lobe, they share the view that the parietal lobe is essential for the processes by which sound-based representations are transcoded into a format that can drive action systems. We suggest that investigations of the linguistic capacities of the parietal lobe constrained by the understanding of the parietal lobe in action and multimodal sensory integration may serve to enhance not only our understanding of language, but also the relationship between language and more basic brain functions.

Left Inferior Frontal Gyrus Sensitivity to Phonetic Competition in Receptive Language Processing: A Comparison of Clear and Conversational Speech

The speech signal is rife with variations in phonetic ambiguity. For instance, when talkers speak in a conversational register, they demonstrate less articulatory precision, leading to greater potential for confusability at the phonetic level compared with a clear speech register. Current psycholinguistic models assume that ambiguous speech sounds activate more than one phonological category and that competition at prelexical levels cascades to lexical levels of processing. Imaging studies have shown that the left inferior frontal gyrus (LIFG) is modulated by phonetic competition between simultaneously activated categories, with increases in activation for more ambiguous tokens. Yet, these studies have often used artificially manipulated speech and/or metalinguistic tasks, which arguably may recruit neural regions that are not critical for natural speech recognition. Indeed, a prominent model of speech processing, the dual-stream model, posits that the LIFG is not involved in prelexical processing in receptive language processing. In the current study, we exploited natural variation in phonetic competition in the speech signal to investigate the neural systems sensitive to phonetic competition as listeners engage in a receptive language task. Participants heard nonsense sentences spoken in either a clear or conversational register as neural activity was monitored using fMRI. Conversational sentences contained greater phonetic competition, as estimated by measures of vowel confusability, and these sentences also elicited greater activation in a region in the LIFG. Sentence-level phonetic competition metrics uniquely correlated with LIFG activity as well. This finding is consistent with the hypothesis that the LIFG responds to competition at multiple levels of language processing and that recruitment of this region does not require an explicit phonological judgment.

Cooperative cortical network for categorical processing of Chinese lexical tone

The variation of pitch in speech not only creates the intonation for affective communication but also signals different meaning of a word in tonal languages, like Chinese. Due to its subtle and brisk pitch contour distinction between tone categories, the underlying neural processing mechanism is largely unknown. Using direct recordings of the human brain, we found categorical neural responses to lexical tones over a distributed cooperative network that included not only the auditory areas in the temporal cortex but also motor areas in the frontal cortex. Strong causal links from the temporal cortex to the motor cortex were discovered, which provides new evidence of top-down influence and sensory–motor interaction during speech perception.

In tonal languages such as Chinese, lexical tone with varying pitch contours serves as a key feature to provide contrast in word meaning. Similar to phoneme processing, behavioral studies have suggested that Chinese tone is categorically perceived. However, its underlying neural mechanism remains poorly understood. By conducting cortical surface recordings in surgical patients, we revealed a cooperative cortical network along with its dynamics responsible for this categorical perception. Based on an oddball paradigm, we found amplified neural dissimilarity between cross-category tone pairs, rather than between within-category tone pairs, over cortical sites covering both the ventral and dorsal streams of speech processing. The bilateral superior temporal gyrus (STG) and the middle temporal gyrus (MTG) exhibited increased response latencies and enlarged neural dissimilarity, suggesting a ventral hierarchy that gradually differentiates the acoustic features of lexical tones. In addition, the bilateral motor cortices were also found to be involved in categorical processing, interacting with both the STG and the MTG and exhibiting a response latency in between. Moreover, the motor cortex received enhanced Granger causal influence from the semantic hub, the anterior temporal lobe, in the right hemisphere. These unique data suggest that there exists a distributed cooperative cortical network supporting the categorical processing of lexical tone in tonal language speakers, not only encompassing a bilateral temporal hierarchy that is shared by categorical processing of phonemes but also involving intensive speech–motor interactions over the right hemisphere, which might be the unique machinery responsible for the reliable discrimination of tone identities.

Reading-induced shifts of perceptual speech representations in auditory cortex

Learning to read requires the formation of efficient neural associations between written and spoken language. Whether these associations influence the auditory cortical representation of speech remains unknown. Here we address this question by combining multivariate functional MRI analysis and a newly-developed ‘text-based recalibration’ paradigm. In this paradigm, the pairing of visual text and ambiguous speech sounds shifts (i.e. recalibrates) the perceptual interpretation of the ambiguous sounds in subsequent auditory-only trials. We show that it is possible to retrieve the text-induced perceptual interpretation from fMRI activity patterns in the posterior superior temporal cortex. Furthermore, this auditory cortical region showed significant functional connectivity with the inferior parietal lobe (IPL) during the pairing of text with ambiguous speech. Our findings indicate that reading-related audiovisual mappings can adjust the auditory cortical representation of speech in typically reading adults. Additionally, they suggest the involvement of the IPL in audiovisual and/or higher-order perceptual processes leading to this adjustment. When applied in typical and dyslexic readers of different ages, our text-based recalibration paradigm may reveal relevant aspects of perceptual learning and plasticity during successful and failing reading development.

Fronto-Parietal Contributions to Phonological Processes in Successful Artificial Grammar Learning

Sensitivity to regularities plays a crucial role in the acquisition of various linguistic features from spoken language input. Artificial grammar learning paradigms explore pattern recognition abilities in a set of structured sequences (i.e., of syllables or letters). In the present study, we investigated the functional underpinnings of learning phonological regularities in auditorily presented syllable sequences. While previous neuroimaging studies either focused on functional differences between the processing of correct vs. incorrect sequences or between different levels of sequence complexity, here the focus is on the neural foundation of the actual learning success. During functional magnetic resonance imaging (fMRI), participants were exposed to a set of syllable sequences with an underlying phonological rule system, known to ensure performance differences between participants. We expected that successful learning and rule application would require phonological segmentation and phoneme comparison. As an outcome of four alternating learning and test fMRI sessions, participants split into successful learners and non-learners. Relative to non-learners, successful learners showed increased task-related activity in a fronto-parietal network of brain areas encompassing the left lateral premotor cortex as well as bilateral superior and inferior parietal cortices during both learning and rule application. These areas were previously associated with phonological segmentation, phoneme comparison, and verbal working memory. Based on these activity patterns and the phonological strategies for rule acquisition and application, we argue that successful learning and processing of complex phonological rules in our paradigm is mediated via a fronto-parietal network for phonological processes.

Is the Sensorimotor Cortex Relevant for Speech Perception and Understanding? An Integrative Review

In the neuroscience of language, phonemes are frequently described as multimodal units whose neuronal representations are distributed across perisylvian cortical regions, including auditory and sensorimotor areas. A different position views phonemes primarily as acoustic entities with posterior temporal localization, which are functionally independent from frontoparietal articulatory programs. To address this current controversy, we here discuss experimental results from functional magnetic resonance imaging (fMRI) as well as transcranial magnetic stimulation (TMS) studies. On first glance, a mixed picture emerges, with earlier research documenting neurofunctional distinctions between phonemes in both temporal and frontoparietal sensorimotor systems, but some recent work seemingly failing to replicate the latter. Detailed analysis of methodological differences between studies reveals that the way experiments are set up explains whether sensorimotor cortex maps phonological information during speech perception or not. In particular, acoustic noise during the experiment and ‘motor noise’ caused by button press tasks work against the frontoparietal manifestation of phonemes. We highlight recent studies using sparse imaging and passive speech perception tasks along with multivariate pattern analysis (MVPA) and especially representational similarity analysis (RSA), which succeeded in separating acoustic-phonological from general-acoustic processes and in mapping specific phonological information on temporal and frontoparietal regions. The question about a causal role of sensorimotor cortex on speech perception and understanding is addressed by reviewing recent TMS studies. We conclude that frontoparietal cortices, including ventral motor and somatosensory areas, reflect phonological information during speech perception and exert a causal influence on language understanding.

Processing of Phonemic Consonant Length: Semantic and Fragment Priming Evidence from Bengali

Six cross-modal lexical decision tasks with priming probed listeners' processing of the geminate-singleton contrast in Bengali, where duration alone leads to phonemic contrast ([pata] 'leaf' vs. [pat:a] 'whereabouts'), in order to investigate the phonological representation of consonantal duration in the lexicon. Four form-priming experiments (auditory fragment primes and visual targets) were designed to investigate listeners' sensitivity to segments of conflicting duration. Each prime derived from a real word ([k(h)[symbol: see text]m]/[g(h)en:]) was matched with a mispronunciation of the opposite duration (*[k(h)[symbol: see text]m:]/*[g(h)en]) and both were used to prime the full words [k(h)[symbol: see text]ma] ('forgiveness') and [g(h)en:a] ('disgust') respectively. Although all fragments led to priming, the results showed an asymmetric pattern. The fragments of words with singletons mispronounced as geminates led to equal priming, while those with geminates mispronounced as singletons showed a difference. The priming effect of the real-word geminate fragment was significantly greater than that of its corresponding nonword singleton fragment. In two subsequent semantic priming tasks with full-word primes a stronger asymmetry was found: nonword geminates (*[k(h)[symbol: see text]m:a]) primed semantically related words ([marjona] 'forgiveness') but singleton nonword primes (*[ghena]) did not show priming. This overall asymmetry in the tolerance of geminate nonwords in place of singleton words is attributed to a representational mismatch and points towards a moraic representation of duration. While geminates require a mora which cannot be derived from singleton input, the additional information in geminate nonwords does not create a similar mismatch.

Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions

The brain's circuitry for perceiving and producing speech may show a notable level of overlap that is crucial for normal development and behavior. The extent to which sensorimotor integration plays a role in speech perception remains highly controversial, however. Methodological constraints related to experimental designs and analysis methods have so far prevented the disentanglement of neural responses to acoustic versus articulatory speech features. Using a passive listening paradigm and multivariate decoding of single-trial fMRI responses to spoken syllables, we investigated brain-based generalization of articulatory features (place and manner of articulation, and voicing) beyond their acoustic (surface) form in adult human listeners. For example, we trained a classifier to discriminate place of articulation within stop syllables (e.g., /pa/ vs /ta/) and tested whether this training generalizes to fricatives (e.g., /fa/ vs /sa/). This novel approach revealed generalization of place and manner of articulation at multiple cortical levels within the dorsal auditory pathway, including auditory, sensorimotor, motor, and somatosensory regions, suggesting the representation of sensorimotor information. Additionally, generalization of voicing included the right anterior superior temporal sulcus associated with the perception of human voices as well as somatosensory regions bilaterally. Our findings highlight the close connection between brain systems for speech perception and production, and in particular, indicate the availability of articulatory codes during passive speech perception.

SIGNIFICANCE STATEMENT

Sensorimotor integration is central to verbal communication and provides a link between auditory signals of speech perception and motor programs of speech production. It remains highly controversial, however, to what extent the brain's speech perception system actively uses articulatory (motor), in addition to acoustic/phonetic, representations. In this study, we examine the role of articulatory representations during passive listening using carefully controlled stimuli (spoken syllables) in combination with multivariate fMRI decoding. Our approach enabled us to disentangle brain responses to acoustic and articulatory speech properties. In particular, it revealed articulatory-specific brain responses of speech at multiple cortical levels, including auditory, sensorimotor, and motor regions, suggesting the representation of sensorimotor information during passive speech perception.

Early-latency categorical speech sound representations in the left inferior frontal gyrus

Efficient speech perception requires the mapping of highly variable acoustic signals to distinct phonetic categories. How the brain overcomes this many-to-one mapping problem has remained unresolved. To infer the cortical location, latency, and dependency on attention of categorical speech sound representations in the human brain, we measured stimulus-specific adaptation of neuromagnetic responses to sounds from a phonetic continuum. The participants attended to the sounds while performing a non-phonetic listening task and, in a separate recording condition, ignored the sounds while watching a silent film. Neural adaptation indicative of phoneme category selectivity was found only during the attentive condition in the pars opercularis (POp) of the left inferior frontal gyrus, where the degree of selectivity correlated with the ability of the participants to categorize the phonetic stimuli. Importantly, these category-specific representations were activated at an early latency of 115-140 ms, which is compatible with the speed of perceptual phonetic categorization. Further, concurrent functional connectivity was observed between POp and posterior auditory cortical areas. These novel findings suggest that when humans attend to speech, the left POp mediates phonetic categorization through integration of auditory and motor information via the dorsal auditory stream.

Hierarchical Organization of Auditory and Motor Representations in Speech Perception: Evidence from Searchlight Similarity Analysis

How humans extract the identity of speech sounds from highly variable acoustic signals remains unclear. Here, we use searchlight representational similarity analysis (RSA) to localize and characterize neural representations of syllables at different levels of the hierarchically organized temporo-frontal pathways for speech perception. We asked participants to listen to spoken syllables that differed considerably in their surface acoustic form by changing speaker and degrading surface acoustics using noise-vocoding and sine wave synthesis while we recorded neural responses with functional magnetic resonance imaging. We found evidence for a graded hierarchy of abstraction across the brain. At the peak of the hierarchy, neural representations in somatomotor cortex encoded syllable identity but not surface acoustic form, at the base of the hierarchy, primary auditory cortex showed the reverse. In contrast, bilateral temporal cortex exhibited an intermediate response, encoding both syllable identity and the surface acoustic form of speech. Regions of somatomotor cortex associated with encoding syllable identity in perception were also engaged when producing the same syllables in a separate session. These findings are consistent with a hierarchical account of how variable acoustic signals are transformed into abstract representations of the identity of speech sounds.

Distributed neural representations of phonological features during speech perception

A fundamental goal of the human auditory system is to map complex acoustic signals onto stable internal representations of the basic sound patterns of speech. Phonemes and the distinctive features that they comprise constitute the basic building blocks from which higher-level linguistic representations, such as words and sentences, are formed. Although the neural structures underlying phonemic representations have been well studied, there is considerable debate regarding frontal-motor cortical contributions to speech as well as the extent of lateralization of phonological representations within auditory cortex. Here we used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis to investigate the distributed patterns of activation that are associated with the categorical and perceptual similarity structure of 16 consonant exemplars in the English language used in Miller and Nicely's (1955) classic study of acoustic confusability. Participants performed an incidental task while listening to phonemes in the MRI scanner. Neural activity in bilateral anterior superior temporal gyrus and supratemporal plane was correlated with the first two components derived from a multidimensional scaling analysis of a behaviorally derived confusability matrix. We further showed that neural representations corresponding to the categorical features of voicing, manner of articulation, and place of articulation were widely distributed throughout bilateral primary, secondary, and association areas of the superior temporal cortex, but not motor cortex. Although classification of phonological features was generally bilateral, we found that multivariate pattern information was moderately stronger in the left compared with the right hemisphere for place but not for voicing or manner of articulation.

Neural pathways for visual speech perception

This paper examines the questions, what levels of speech can be perceived visually, and how is visual speech represented by the brain? Review of the literature leads to the conclusions that every level of psycholinguistic speech structure (i.e., phonetic features, phonemes, syllables, words, and prosody) can be perceived visually, although individuals differ in their abilities to do so; and that there are visual modality-specific representations of speech qua speech in higher-level vision brain areas. That is, the visual system represents the modal patterns of visual speech. The suggestion that the auditory speech pathway receives and represents visual speech is examined in light of neuroimaging evidence on the auditory speech pathways. We outline the generally agreed-upon organization of the visual ventral and dorsal pathways and examine several types of visual processing that might be related to speech through those pathways, specifically, face and body, orthography, and sign language processing. In this context, we examine the visual speech processing literature, which reveals widespread diverse patterns of activity in posterior temporal cortices in response to visual speech stimuli. We outline a model of the visual and auditory speech pathways and make several suggestions: (1) The visual perception of speech relies on visual pathway representations of speech qua speech. (2) A proposed site of these representations, the temporal visual speech area (TVSA) has been demonstrated in posterior temporal cortex, ventral and posterior to multisensory posterior superior temporal sulcus (pSTS). (3) Given that visual speech has dynamic and configural features, its representations in feedforward visual pathways are expected to integrate these features, possibly in TVSA.

Acoustical and categorical tasks differently modulate activations of human auditory cortex to vowels

The present study compared activations to prototype, nonprototype, nonphonemic, and cross-category vowel pairs during vowel discrimination, category discrimination, 2-back, and visual tasks. Our results support previous findings that areas of the superior temporal gyrus (STG) are sensitive to the speech-level difference between prototype vs. nonprototype and phonemic vs. nonphonemic vowels. Further, consistent with previous studies, we found enhanced activations in anterior-posterior STG and inferior parietal lobule (IPL) during the vowel discrimination and 2-back tasks, respectively. Unlike the vowel discrimination task, the category discrimination task was associated with strong IPL activations. Our results provide evidence that activations in STG and IPL strongly depend on whether the task requires analysis of detailed acoustical information or operations on categorical representations. Based on previous studies investigating activations during categorical pitch and spatial tasks, we argue that this distinction is probably not specific to speech.

Inferior parietal lobule contributions to visual word recognition

This study investigated how the left inferior parietal lobule (IPL) contributes to visual word recognition. We used repetitive TMS to temporarily disrupt neural information processing in two anatomical fields of the IPL, namely, the angular (ANG) and supramarginal (SMG) gyri, and observed the effects on reading tasks that focused attention on either the meaning or sounds of written words. Relative to no TMS, stimulation of the left ANG selectively slowed responses in the meaning, but not sound, task, whereas stimulation of the left SMG affected responses in the sound, but not meaning, task. These results demonstrate that ANG and SMG doubly dissociate in their contributions to visual word recognition. We suggest that this functional division of labor may be understood in terms of the distinct patterns of cortico-cortical connectivity resulting in separable functional circuits.

tDCS to temporoparietal cortex during familiarisation enhances the subsequent phonological coherence of nonwords in immediate serial recall

Research has shown that direct current stimulation (tDCS) over left temporoparietal cortex - a region implicated in phonological processing - aids new word learning. The locus of this effect remains unclear since (i) experiments have not empirically separated the acquisition of phonological forms from lexical-semantic links and (ii) outcome measures have focused on learnt associations with a referent rather than phonological stability. We tested the hypothesis that left temporoparietal tDCS would strengthen the acquisition of phonological forms, even in the absence of the opportunity to acquire lexical-semantic associations. Participants were familiarised with nonwords paired with (i) photographs of concrete referents or (ii) blurred images where no clear features were visible. Nonword familiarisation proceeded under conditions of anodal tDCS and sham stimulation in different sessions. We examined the impact of these manipulations on the stability of the phonological trace in an immediate serial recall (ISR) task the following day, ensuring that any effects were due to the influence of tDCS on long-term learning and not a direct consequence of short-term changes in neural excitability. We found that only a few exposures to the phonological forms of nonwords were sufficient to enhance nonword ISR overall compared to entirely novel items. Anodal tDCS during familiarisation further enhanced the acquisition of phonological forms, producing a specific reduction in the frequency of phoneme migrations when sequences of nonwords were maintained in verbal short-term memory. More of the phonemes that were recalled were bound together as a whole correct nonword following tDCS. These data show that tDCS to left temporoparietal cortex can facilitate word learning by strengthening the acquisition of long-term phonological forms, irrespective of the availability of a concrete referent, and that the consequences of this learning can be seen beyond the learning task as strengthened phonological coherence in verbal short-term memory.

The functional organization of the left STS: a large scale meta-analysis of PET and fMRI studies of healthy adults

The superior temporal sulcus (STS) in the left hemisphere is functionally diverse, with sub-areas implicated in both linguistic and non-linguistic functions. However, the number and boundaries of distinct functional regions remain to be determined. Here, we present new evidence, from meta-analysis of a large number of positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) studies, of different functional specificity in the left STS supporting a division of its middle to terminal extent into at least three functional areas. The middle portion of the left STS stem (fmSTS) is highly specialized for speech perception and the processing of language material. The posterior portion of the left STS stem (fpSTS) is highly versatile and involved in multiple functions supporting semantic memory and associative thinking. The fpSTS responds to both language and non-language stimuli but the sensitivity to non-language material is greater. The horizontal portion of the left STS stem and terminal ascending branches (ftSTS) display intermediate functional specificity, with the anterior-dorsal ascending branch (fatSTS) supporting executive functions and motor planning and showing greater sensitivity to language material, and the horizontal stem and posterior-ventral ascending branch (fptSTS) supporting primarily semantic processing and displaying greater sensitivity to non-language material. We suggest that the high functional specificity of the left fmSTS for speech is an important means by which the human brain achieves exquisite affinity and efficiency for native speech perception. In contrast, the extreme multi-functionality of the left fpSTS reflects the role of this area as a cortical hub for semantic processing and the extraction of meaning from multiple sources of information. Finally, in the left ftSTS, further functional differentiation between the dorsal and ventral aspect is warranted.

Emergence of category-level sensitivities in non-native speech sound learning

Over the course of development, speech sounds that are contrastive in one's native language tend to become perceived categorically: that is, listeners are unaware of variation within phonetic categories while showing excellent sensitivity to speech sounds that span linguistically meaningful phonetic category boundaries. The end stage of this developmental process is that the perceptual systems that handle acoustic-phonetic information show special tuning to native language contrasts, and as such, category-level information appears to be present at even fairly low levels of the neural processing stream. Research on adults acquiring non-native speech categories offers an avenue for investigating the interplay of category-level information and perceptual sensitivities to these sounds as speech categories emerge. In particular, one can observe the neural changes that unfold as listeners learn not only to perceive acoustic distinctions that mark non-native speech sound contrasts, but also to map these distinctions onto category-level representations. An emergent literature on the neural basis of novel and non-native speech sound learning offers new insight into this question. In this review, I will examine this literature in order to answer two key questions. First, where in the neural pathway does sensitivity to category-level phonetic information first emerge over the trajectory of speech sound learning? Second, how do frontal and temporal brain areas work in concert over the course of non-native speech sound learning? Finally, in the context of this literature I will describe a model of speech sound learning in which rapidly-adapting access to categorical information in the frontal lobes modulates the sensitivity of stable, slowly-adapting responses in the temporal lobes.

Mapping phonemic processing zones along human perisylvian cortex: an electro-corticographic investigation

The auditory system is organized such that progressively more complex features are represented across successive cortical hierarchical stages. Just when and where the processing of phonemes, fundamental elements of the speech signal, is achieved in this hierarchy remains a matter of vigorous debate. Non-invasive measures of phonemic representation have been somewhat equivocal. While some studies point to a primary role for middle/anterior regions of the superior temporal gyrus (STG), others implicate the posterior STG. Differences in stimulation, task and inter-individual anatomical/functional variability may account for these discrepant findings. Here, we sought to clarify this issue by mapping phonemic representation across left perisylvian cortex, taking advantage of the excellent sampling density afforded by intracranial recordings in humans. We asked whether one or both major divisions of the STG were sensitive to phonemic transitions. The high signal-to-noise characteristics of direct intracranial recordings allowed for analysis at the individual participant level, circumventing issues of inter-individual anatomic and functional variability that may have obscured previous findings at the group level of analysis. The mismatch negativity (MMN), an electrophysiological response elicited by changes in repetitive streams of stimulation, served as our primary dependent measure. Oddball configurations of pairs of phonemes, spectro-temporally matched non-phonemes, and simple tones were presented. The loci of the MMN clearly differed as a function of stimulus type. Phoneme representation was most robust over middle/anterior STG/STS, but was also observed over posterior STG/SMG. These data point to multiple phonemic processing zones along perisylvian cortex, both anterior and posterior to primary auditory cortex. This finding is considered within the context of a dual stream model of auditory processing in which functionally distinct ventral and dorsal auditory processing pathways may be engaged by speech stimuli.

Neural correlates of successful semantic processing during propofol sedation

Sedation has a graded effect on brain responses to auditory stimuli: perceptual processing persists at sedation levels that attenuate more complex processing. We used fMRI in healthy volunteers sedated with propofol to assess changes in neural responses to spoken stimuli. Volunteers were scanned awake, sedated, and during recovery, while making perceptual or semantic decisions about nonspeech sounds or spoken words respectively. Sedation caused increased error rates and response times, and differentially affected responses to words in the left inferior frontal gyrus (LIFG) and the left inferior temporal gyrus (LITG). Activity in LIFG regions putatively associated with semantic processing, was significantly reduced by sedation despite sedated volunteers continuing to make accurate semantic decisions. Instead, LITG activity was preserved for words greater than nonspeech sounds and may therefore be associated with persistent semantic processing during the deepest levels of sedation. These results suggest functionally distinct contributions of frontal and temporal regions to semantic decision making. These results have implications for functional imaging studies of language, for understanding mechanisms of impaired speech comprehension in postoperative patients with residual levels of anesthetic, and may contribute to the development of frameworks against which EEG based monitors could be calibrated to detect awareness under anesthesia.

How learning to abstract shapes neural sound representations

The transformation of acoustic signals into abstract perceptual representations is the essence of the efficient and goal-directed neural processing of sounds in complex natural environments. While the human and animal auditory system is perfectly equipped to process the spectrotemporal sound features, adequate sound identification and categorization require neural sound representations that are invariant to irrelevant stimulus parameters. Crucially, what is relevant and irrelevant is not necessarily intrinsic to the physical stimulus structure but needs to be learned over time, often through integration of information from other senses. This review discusses the main principles underlying categorical sound perception with a special focus on the role of learning and neural plasticity. We examine the role of different neural structures along the auditory processing pathway in the formation of abstract sound representations with respect to hierarchical as well as dynamic and distributed processing models. Whereas most fMRI studies on categorical sound processing employed speech sounds, the emphasis of the current review lies on the contribution of empirical studies using natural or artificial sounds that enable separating acoustic and perceptual processing levels and avoid interference with existing category representations. Finally, we discuss the opportunities of modern analyses techniques such as multivariate pattern analysis (MVPA) in studying categorical sound representations. With their increased sensitivity to distributed activation changes—even in absence of changes in overall signal level—these analyses techniques provide a promising tool to reveal the neural underpinnings of perceptually invariant sound representations.

Representations of Invariant Musical Categories Are Decodable by Pattern Analysis of Locally Distributed BOLD Responses in Superior Temporal and Intraparietal Sulci

In categorical perception (CP), continuous physical signals are mapped to discrete perceptual bins: mental categories not found in the physical world. CP has been demonstrated across multiple sensory modalities and, in audition, for certain over-learned speech and musical sounds. The neural basis of auditory CP, however, remains ambiguous, including its robustness in nonspeech processes and the relative roles of left/right hemispheres; primary/nonprimary cortices; and ventral/dorsal perceptual processing streams. Here, highly trained musicians listened to 2-tone musical intervals, which they perceive categorically while undergoing functional magnetic resonance imaging. Multivariate pattern analyses were performed after grouping sounds by interval quality (determined by frequency ratio between tones) or pitch height (perceived noncategorically, frequency ratios remain constant). Distributed activity patterns in spheres of voxels were used to determine sound sample identities. For intervals, significant decoding accuracy was observed in the right superior temporal and left intraparietal sulci, with smaller peaks observed homologously in contralateral hemispheres. For pitch height, no significant decoding accuracy was observed, consistent with the non-CP of this dimension. These results suggest that similar mechanisms are operative for nonspeech categories as for speech; espouse roles for 2 segregated processing streams; and support hierarchical processing models for CP.

Neural dynamics of phonological processing in the dorsal auditory stream

Neuroanatomical models hypothesize a role for the dorsal auditory pathway in phonological processing as a feedforward efferent system (Davis and Johnsrude, 2007; Rauschecker and Scott, 2009; Hickok et al., 2011). But the functional organization of the pathway, in terms of time course of interactions between auditory, somatosensory, and motor regions, and the hemispheric lateralization pattern is largely unknown. Here, ambiguous duplex syllables, with elements presented dichotically at varying interaural asynchronies, were used to parametrically modulate phonological processing and associated neural activity in the human dorsal auditory stream. Subjects performed syllable and chirp identification tasks, while event-related potentials and functional magnetic resonance images were concurrently collected. Joint independent component analysis was applied to fuse the neuroimaging data and study the neural dynamics of brain regions involved in phonological processing with high spatiotemporal resolution. Results revealed a highly interactive neural network associated with phonological processing, composed of functional fields in posterior temporal gyrus (pSTG), inferior parietal lobule (IPL), and ventral central sulcus (vCS) that were engaged early and almost simultaneously (at 80-100 ms), consistent with a direct influence of articulatory somatomotor areas on phonemic perception. Left hemispheric lateralization was observed 250 ms earlier in IPL and vCS than pSTG, suggesting that functional specialization of somatomotor (and not auditory) areas determined lateralization in the dorsal auditory pathway. The temporal dynamics of the dorsal auditory pathway described here offer a new understanding of its functional organization and demonstrate that temporal information is essential to resolve neural circuits underlying complex behaviors.

Smoothness without smoothing: why Gaussian naive Bayes is not naive for multi-subject searchlight studies

Spatial smoothness is helpful when averaging fMRI signals across multiple subjects, as it allows different subjects' corresponding brain areas to be pooled together even if they are slightly misaligned. However, smoothing is usually not applied when performing multivoxel pattern-based analyses (MVPA), as it runs the risk of blurring away the information that fine-grained spatial patterns contain. It would therefore be desirable, if possible, to carry out pattern-based analyses which take unsmoothed data as their input but which produce smooth images as output. We show here that the Gaussian Naive Bayes (GNB) classifier does precisely this, when it is used in “searchlight” pattern-based analyses. We explain why this occurs, and illustrate the effect in real fMRI data. Moreover, we show that analyses using GNBs produce results at the multi-subject level which are statistically robust, neurally plausible, and which replicate across two independent data sets. By contrast, SVM classifiers applied to the same data do not generate a replication, even if the SVM-derived searchlight maps have smoothing applied to them. An additional advantage of GNB classifiers for searchlight analyses is that they are orders of magnitude faster to compute than more complex alternatives such as SVMs. Collectively, these results suggest that Gaussian Naive Bayes classifiers may be a highly non-naive choice for multi-subject pattern-based fMRI studies.