Abstract phoneme representations in the left temporal cortex: magnetic mismatch negativity study

Cortical Generators and Connections Underlying Phoneme Perception: A Mismatch Negativity and P300 Investigation

The cortical generators of the pure tone MMN and P300 have been thoroughly studied. Their nature and interaction with respect to phoneme perception, however, is poorly understood. Accordingly, the cortical sources and functional connections that underlie the MMN and P300 in relation to passive and active speech sound perception were identified. An inattentive and attentive phonemic oddball paradigm, eliciting a MMN and P300 respectively, were administered in 60 healthy adults during simultaneous high-density EEG recording. For both the MMN and P300, eLORETA source reconstruction was performed. The maximal cross-correlation was calculated between ROI-pairs to investigate inter-regional functional connectivity specific to passive and active deviant processing. MMN activation clusters were identified in the temporal (insula, superior temporal gyrus and temporal pole), frontal (rostral middle frontal and pars opercularis) and parietal (postcentral and supramarginal gyrus) cortex. Passive discrimination of deviant phonemes was aided by a network connecting right temporoparietal cortices to left frontal areas. For the P300, clusters with significantly higher activity were found in the frontal (caudal middle frontal and precentral), parietal (precuneus) and cingulate (posterior and isthmus) cortex. Significant intra- and interhemispheric connections between parietal, cingulate and occipital regions constituted the network governing active phonemic target detection. A predominantly bilateral network was found to underly both the MMN and P300. While passive phoneme discrimination is aided by a fronto-temporo-parietal network, active categorization calls on a network entailing fronto-parieto-cingulate cortices. Neural processing of phonemic contrasts, as reflected by the MMN and P300, does not appear to show pronounced lateralization to the language-dominant hemisphere.

Does accentual variation in the pronunciation of French words influence their recognition? It depends on the ear of presentation

This repetition priming study examined how word accentual variation in French is represented and processed during spoken word recognition. Mismatched primes in the accentual pattern were less effective than matched primes in facilitating target word recognition when the targets were presented in the left ear but not in the right ear. This indicates that in French, the accentual pattern of words influences their recognition when processing is constrained in the right hemisphere. This study pleads in favor of two memory systems, the one retaining words in an abstract format and the other retaining words in their various forms.

Formant-invariant voice and pitch representations are pre-attentively formed from constantly varying speech and non-speech stimuli

The present study investigated whether listeners can form abstract voice representations while ignoring constantly changing phonological information and if they can use the resulting information to facilitate voice change detection. Further, the study aimed at understanding whether the use of abstraction is restricted to the speech domain or can be deployed also in non‐speech contexts. We ran an electroencephalogram (EEG) experiment including one passive and one active oddball task, each featuring a speech and a rotated speech condition. In the speech condition, participants heard constantly changing vowels uttered by a male speaker (standard stimuli) which were infrequently replaced by vowels uttered by a female speaker with higher pitch (deviant stimuli). In the rotated speech condition, participants heard rotated vowels, in which the natural formant structure of speech was disrupted. In the passive task, the mismatch negativity was elicited after the presentation of the deviant voice in both conditions, indicating that listeners could successfully group together different stimuli into a formant‐invariant voice representation. In the active task, participants showed shorter reaction times (RTs), higher accuracy and a larger P3b in the speech condition with respect to the rotated speech condition. Results showed that whereas at a pre‐attentive level the cognitive system can track pitch regularities while presumably ignoring constantly changing formant information both in speech and in rotated speech, at an attentive level the use of such information is facilitated for speech. This facilitation was also testified by a stronger synchronisation in the theta band (4–7 Hz), potentially pointing towards differences in encoding/retrieval processes.

Automatic Speech Discrimination Assessment Methods Based on Event-Related Potentials (ERP)

Speech discrimination is used by audiologists in diagnosing and determining treatment for hearing loss patients. Usually, assessing speech discrimination requires subjective responses. Using electroencephalography (EEG), a method that is based on event-related potentials (ERPs), could provide objective speech discrimination. In this work we proposed a visual-ERP-based method to assess speech discrimination using pictures that represent word meaning. The proposed method was implemented with three strategies, each with different number of pictures and test sequences. Machine learning was adopted to classify between the task conditions based on features that were extracted from EEG signals. The results from the proposed method were compared to that of a similar visual-ERP-based method using letters and a method that is based on the auditory mismatch negativity (MMN) component. The P3 component and the late positive potential (LPP) component were observed in the two visual-ERP-based methods while MMN was observed during the MMN-based method. A total of two out of three strategies of the proposed method, along with the MMN-based method, achieved approximately 80% average classification accuracy by a combination of support vector machine (SVM) and common spatial pattern (CSP). Potentially, these methods could serve as a pre-screening tool to make speech discrimination assessment more accessible, particularly in areas with a shortage of audiologists.

Later but Not Weaker: Neural Categorization of Native Vowels of Children at Familial Risk of Dyslexia

Although allophonic speech processing has been hypothesized to be a contributing factor in developmental dyslexia, experimental evidence is limited and inconsistent. The current study compared the categorization of native similar sounding vowels of typically developing (TD) children and children at familial risk (FR) of dyslexia. EEG response was collected in a non-attentive passive oddball paradigm from 35 TD and 35 FR Dutch 20-month-old infants who were matched on vocabulary. The children were presented with two nonwords “giep” [ɣip] and “gip” [ɣIp] that contrasted solely with respect to the vowel. In the multiple-speaker condition, both nonwords were produced by twelve different speakers while in the single-speaker condition, single tokens of each word were used as stimuli. For both conditions and for both groups, infant positive mismatch response (p-MMR) was elicited, and the p-MMR amplitude was comparable between the two groups, although the FR children had a later p-MMR peak than the TD children in the multiple-speaker condition. These findings indicate that FR children are able to categorize speech sounds, but that they may do so in a more effortful way than TDs.

Early differentiation of memory retrieval processes for newly learned voices and phonemes as indexed by the MMN

Linguistic and vocal information are thought to be differentially processed since the early stages of speech perception, but it remains unclear if this differentiation also concerns automatic processes of memory retrieval. The aim of this ERP study was to compare the automatic retrieval processes for newly learned voices vs phonemes. In a longitudinal experiment, two groups of participants were trained in learning either a new phoneme or a new voice. The MMN elicited by the presentation of the two was measured before and after the training. An enhanced MMN was elicited by the presentation of the learned phoneme, reflecting the activation of an automatic memory retrieval process. Instead, a reduced MMN was elicited by the learned voice, indicating that the voice was perceived as a typical member of the learned voice identity. This suggests that the automatic processes that retrieve linguistic and vocal information are differently affected by experience.

The neural basis of counting sequences

Sequence processing is critical for complex behavior, and counting sequences hold a unique place underlying human numerical development. Despite this, the neural bases of counting sequences remain unstudied. We hypothesized that counting sequences in adults would involve representations in sensory, order, magnitude, and linguistic codes that implicate regions in auditory, supplementary motor, posterior parietal, and inferior frontal areas, respectively. In an fMRI scanner, participants heard four-number sequences in a 2 × 2 × 2 design. The sequences were adjacent or not (e.g., 5, 6, 7, 8 vs. 5, 6, 7, 9), ordered or not (e.g., 5, 6, 7, 8 vs. 8, 5, 7, 6), and were spoken by a voice of consistent or variable identity. Then, neural substrates of counting sequences were identified by testing for the effect of consecutiveness (ordered nonadjacent versus ordered adjacent, e.g., 5, 6, 7, 9 > 5, 6, 7, 8) in the hypothesized brain regions. Violations to consecutiveness elicited brain activity in the right inferior frontal gyrus (IFG) and the supplementary motor area (SMA). In contrast, no such activation was observed in the auditory cortex, despite violations in voice identity recruiting strong activity in that region. Also, no activation was observed in the inferior parietal lobule, despite a robust effect of orderedness observed in that brain region. These findings indicate that listening to counting sequences do not automatically elicit sensory or magnitude codes but suggest that the precise increments in the sequence are tracked by the mechanism for processing ordered associations in the SMA and by the mechanism for binding individual lexical items into a cohesive whole in the IFG.

Perceptual Asymmetries and Auditory Processing of Estonian Quantities

Similar to visual perception, auditory perception also has a clearly described “pop-out” effect, where an element with some extra feature is easier to detect among elements without an extra feature. This phenomenon is better known as auditory perceptual asymmetry. We investigated such asymmetry between shorter or longer duration, and level or falling of pitch of linguistic stimuli that carry a meaning in one language (Estonian), but not in another (Russian). For the mismatch negativity (MMN) experiment, we created four different types of stimuli by modifying the duration of the first vowel [ɑ] (170, 290 ms) and pitch contour (level vs. falling pitch) of the stimuli words (‘SATA,’ ‘SAKI’). The stimuli were synthesized from Estonian words (‘SATA,’ ‘SAKI’) and follow the Estonian language three-way quantity system, which incorporates tonal features (falling pitch contour) together with temporal patterns. This made the meaning of the word dependent on the combination of both features and allows us to compare the relative contribution of duration and pitch contour in discrimination of language stimuli in the brain via MMN generation. The participants of the experiment were 12 Russian native speakers with little or no experience in Estonian and living in Estonia short-term, and 12 Estonian native speakers (age 18–27 years). We found that participants’ perception of the linguistic stimuli differed not only according to the physical features but also according to their native language, confirming that the meaning of the word interferes with the early automatic processing of phonological features. The GAMM and ANOVA analysis of the reversed design results showed that the deviant with longer duration among shorter standards elicited a MMN response with greater amplitude than the short deviant among long standards, while changes in pitch contour (falling vs. level pitch) produced neither strong MMN nor asymmetry. Thus, we demonstrate the effect of language background on asymmetric perception of linguistic stimuli that aligns with those of previous studies (Jaramillo et al., 2000), and contributes to the growing body of knowledge supporting auditory perceptual asymmetry.

N1 Repetition-Attenuation for Acoustically Variable Speech and Spectrally Rotated Speech

The amplitude of the event-related N1 wave decreases with repeated stimulation. This repetition-attenuation has not previously been investigated in response to variable auditory stimuli, nor has the relative impact of acoustic vs. perceptual category repetition been studied. In the present study, N1 repetition-attenuation was investigated for speech and spectrally rotated speech with varying degrees of acoustic and perceptual category variation. In the speech condition, participants (n = 19) listened to stimulus trains consisting of either the same vowel exemplar (no variability condition), different exemplars of the same vowel (low variability condition), or different exemplars of two different vowels (high variability condition). In the rotated speech condition, the spectrally rotated counterparts of the vowels were presented. Findings show N1 repetition-attenuation in the face of acoustic and perceptual category variability, but no impact of the degree of variability on the degree of N1 attenuation. Speech stimuli resulted in less attenuation than the acoustically matched non-speech stimuli, which is in line with previous findings. It remains unclear if the attenuation of the N1 wave is reduced as a result of stimuli being perceived as belonging to perceptual categories or as a result of some other characteristic of speech.

The Influence of Form- and Meaning-Based Predictions on Cortical Speech Processing Under Challenging Listening Conditions: A MEG Study

Under adverse listening conditions, prior linguistic knowledge about the form (i.e., phonology) and meaning (i.e., semantics) help us to predict what an interlocutor is about to say. Previous research has shown that accurate predictions of incoming speech increase speech intelligibility, and that semantic predictions enhance the perceptual clarity of degraded speech even when exact phonological predictions are possible. In addition, working memory (WM) is thought to have specific influence over anticipatory mechanisms by actively maintaining and updating the relevance of predicted vs. unpredicted speech inputs. However, the relative impact on speech processing of deviations from expectations related to form and meaning is incompletely understood. Here, we use MEG to investigate the cortical temporal processing of deviations from the expected form and meaning of final words during sentence processing. Our overall aim was to observe how deviations from the expected form and meaning modulate cortical speech processing under adverse listening conditions and investigate the degree to which this is associated with WM capacity. Results indicated that different types of deviations are processed differently in the auditory N400 and Mismatch Negativity (MMN) components. In particular, MMN was sensitive to the type of deviation (form or meaning) whereas the N400 was sensitive to the magnitude of the deviation rather than its type. WM capacity was associated with the ability to process phonological incoming information and semantic integration.

Neuromagnetic speech discrimination responses are associated with reading-related skills in dyslexic and typical readers

Poor neural speech discrimination has been connected to dyslexia, and may represent phonological processing deficits that are hypothesized to be the main cause for reading impairments. Thus far, neural speech discrimination impairments have rarely been investigated in adult dyslexics, and even less by examining sources of neuromagnetic responses. We compared neuromagnetic speech discrimination in dyslexic and typical readers with mismatch fields (MMF) and determined the associations between MMFs and reading-related skills. We expected weak and atypically lateralized MMFs in dyslexic readers, and positive associations between reading-related skills and MMF strength. MMFs were recorded to a repeating pseudoword /ta-ta/ with occasional changes in vowel identity, duration, or syllable frequency from 43 adults, 21 with confirmed dyslexia. Phonetic (vowel and duration) changes elicited left-lateralized MMFs in the auditory cortices. Contrary to our hypothesis, MMF source strengths or lateralization did not differ between groups. However, better verbal working memory was associated with stronger left-hemispheric MMFs to duration changes across groups, and better reading was associated with stronger right-hemispheric late MMFs across speech-sound changes in dyslexic readers. This suggests a link between neural speech processing and reading-related skills, in line with previous work. Furthermore, our findings suggest a right-hemispheric compensatory mechanism for language processing in dyslexia. The results obtained promote the use of MMFs in investigating reading-related brain processes.

Poor neural and perceptual phoneme discrimination during acoustic variation in dyslexia

Whereas natural acoustic variation in speech does not compromise phoneme discrimination in healthy adults, it was hypothesized to be a challenge for developmental dyslexics. We investigated dyslexics’ neural and perceptual discrimination of native language phonemes during acoustic variation. Dyslexics and non-dyslexics heard /æ/ and /i/ phonemes in a context with f_o variation and then in a context without it. Mismatch negativity (MMN) and P3a responses to phoneme changes were recorded with electroencephalogram to compare groups during ignore and attentive listening. Perceptual phoneme discrimination in the variable context was evaluated with hit-ratios and reaction times. MMN/N2bs were diminished in dyslexics in the variable context. Hit-ratios were smaller in dyslexics than controls. MMNs did not differ between groups in the context without variation. These results suggest that even distinctive vowels are challenging to discriminate for dyslexics when the context resembles natural variability of speech. This most likely reflects poor categorical perception of phonemes in dyslexics. Difficulties to detect linguistically relevant invariant information during acoustic variation in speech may contribute to dyslexics’ deficits in forming native language phoneme representations during infancy. Future studies should acknowledge that simple experimental paradigms with repetitive stimuli can be insensitive to dyslexics’ speech processing deficits.

Normal categorical perception to syllable-like stimuli in long term and in first episode schizophrenia

Schizophrenia is associated with deficits in language processing that are evident even at first-episode. However, there is debate as to how early in the processing stream the linguistic deficits appear. We measured categorical processing of artificial syllables that varied in voice-onset time (VOT), and how sensory biasing impacts categorical perception. VOT varied in 5 ms increments from 0 ms (strong /ba/) to 40 ms (strong /pa/). Participants chose whether a syllable sounded more like /ba/ or /pa/. Twenty-two individuals with long-term schizophrenia (Sz) were compared to 21 controls (HCSz), and 17 individuals at their first-episode of schizophrenia (FE) were compared to 19 controls (HCFE). There were three conditions: equiprobable - each syllable had an equal probability of being presented; /ba/-biased - 0 ms VOT (strong /ba/) presented 70% of the time; /pa/-biased - 40 ms VOT (strong /pa/) presented 70% of the time. All groups showed categorical perception and category shifts during biased conditions. Sz and FE were statistically indistinguishable from controls in the point of categorical shift, slope of their response function, and the VOT needed to reliably perceive /pa/. Together, this suggests intact ability to map acoustic stimuli to phonetic categories when based on timing differences in voiced information, both early and late in the disease.

Cortical Areas Associated With Mismatch Negativity: A Connectivity Study Using Propofol Anesthesia

Auditory mismatch negativity (MMN) is an event-related potential (ERP) waveform induced by rare deviant stimuli that occur in a stream of regular auditory stimuli. The generators of MMN are believed to include several different cortical regions like the bilateral temporal and the right inferior frontal gyrus (IFG). However, exact cortical regions associated with MMN remain controversial. In this study, we compared the number of long-distance connections induced by the standard and deviant stimuli during awake state and propofol anesthesia state to identify the cortical areas associated with the generation of MMN. In awake state, we find that deviant stimuli synchronize more information between the right frontal and temporal than standard stimuli. Moreover, we find that the deviant stimuli in awake state activate the bilateral frontal, central areas, the left temporal and parietal areas as compared to the anesthesia state, whereas the standard stimuli do not. These results suggest that, in addition to the bilateral temporal and the right IFG, the bilateral frontal and centro-parietal regions also contribute to the generation of MMN.

Neural discrimination of speech sound changes in a variable context occurs irrespective of attention and explicit awareness

To process complex stimuli like language, our auditory system must tolerate large acoustic variance, like speaker variability, and still be sensitive enough to discriminate between phonemes and to detect complex sound relationships in, e.g., prosodic cues. Our study determined discrimination of speech sounds in input mimicking natural speech variability, and detection of deviations in regular pitch relationships (rule violations) between speech sounds. We investigated the automaticity and the influence of attention and explicit awareness on these changes by recording the neurophysiological mismatch negativity (MMN) and P3a as well as task performance from 21 adults. The results showed neural discrimination of phonemes and rule violations as indicated by MMN and P3a, regardless of whether the sounds were attended or not, even when participants could not explicitly describe the rule. While small sample size precluded statistical analysis of some outcomes, we still found preliminary associations between the MMN amplitudes, task performance, and emerging explicit awareness of the rule. Our results highlight the automaticity of processing complex aspects of speech as a basis for the emerging conscious perception and explicit awareness of speech properties. While MMN operates at the implicit processing level, P3a appears to work at the borderline of implicit and explicit.

Using rotated speech to approximate the acoustic mismatch negativity response to speech

The mismatch negativity (MMN) response is influenced by the magnitude of the acoustic difference between standard and deviant, and the response is typically larger to linguistically relevant changes than to linguistically irrelevant changes. Linguistically relevant changes between standard and deviant typically co-occur with differences between the two acoustic signals. It is therefore not straightforward to determine the contribution of each of those two factors to the MMN response. This study investigated whether spectrally rotated speech can be used to determine the impact of the acoustic difference on the MMN response to a combined linguistic and acoustic change between standard and deviant. Changes between rotated vowels elicited an MMN of comparable amplitude to the one elicited by a within-category vowel change, whereas the between-category vowel change resulted in an MMN amplitude of greater magnitude. A change between rotated vowels resulted in an MMN ampltude more similar to that of a within-vowel change than a complex tone change did. This suggests that the MMN amplitude reflecting the acoustic difference between two speech sounds can be well approximated by the MMN amplitude elicited in response to their rotated counterparts, in turn making it possible to estimate the part of the response specific to the linguistic difference.

N1 response attenuation and the mismatch negativity (MMN) to within- and across-category phonetic contrasts

According to the neural adaptation model of the mismatch negativity (MMN), the sensitivity of this event-related response to both acoustic and categorical information in speech sounds can be accounted for by assuming that (a) the degree of overlapping between neural representations of two sounds depends on both the acoustic difference between them and whether or not they belong to distinct phonetic categories, and (b) a release from stimulus-specific adaptation causes an enhanced N1 obligatory response to infrequent deviant stimuli. On the basis of this view, we tested in Experiment 1 whether the N1 response to the second sound of a pair (S₂ ) would be more attenuated in pairs of identical vowels compared with pairs of different vowels, and in pairs of exemplars of the same vowel category compared with pairs of exemplars of different categories. The psychoacoustic distance between S₁ and S₂ was the same for all within-category and across-category pairs. While N1 amplitudes decreased markedly from S₁ to S₂ , responses to S₂ were quite similar across pair types, indicating that the attenuation effect in such conditions is not stimulus specific. In Experiment 2, a pronounced MMN was elicited by a deviant vowel sound in an across-category oddball sequence, but not when the exact same deviant vowel was presented in a within-category oddball sequence. This adds evidence that MMN reflects categorical phonetic processing. Taken together, the results suggest that different neural processes underlie the attenuation of the N1 response to S₂ and the MMN to vowels.

When Hearing Is Tricky: Speech Processing Strategies in Prelingually Deafened Children and Adolescents with Cochlear Implants Having Good and Poor Speech Performance

Cochlear implants provide individuals who are deaf with access to speech. Although substantial advancements have been made by novel technologies, there still is high variability in language development during childhood, depending on adaptation and neural plasticity. These factors have often been investigated in the auditory domain, with the mismatch negativity as an index for sensory and phonological processing. Several studies have demonstrated that the MMN is an electrophysiological correlate for hearing improvement with cochlear implants. In this study, two groups of cochlear implant users, both with very good basic hearing abilities but with non-overlapping speech performance (very good or very poor speech performance), were matched according to device experience and age at implantation. We tested the perception of phonemes in the context of specific other phonemes from which they were very hard to discriminate (e.g., the vowels in /bu/ vs. /bo/). The most difficult pair was individually determined for each participant. Using behavioral measures, both cochlear implants groups performed worse than matched controls, and the good performers performed better than the poor performers. Cochlear implant groups and controls did not differ during time intervals typically used for the mismatch negativity, but earlier: source analyses revealed increased activity in the region of the right supramarginal gyrus (220-260 ms) in good performers. Poor performers showed increased activity in the left occipital cortex (220-290 ms), which may be an index for cross-modal perception. The time course and the neural generators differ from data from our earlier studies, in which the same phonemes were assessed in an easy-to-discriminate context. The results demonstrate that the groups used different language processing strategies, depending on the success of language development and the particular language context. Overall, our data emphasize the role of neural plasticity and use of adaptive strategies for successful language development with cochlear implants.

Can an auditory multi-feature optimal paradigm be used for the study of processes associated with attention capture in passive listeners?

OBJECTIVE

A rarely occurring and highly relevant auditory stimulus occurring outside of the current focus of attention can cause a switching of attention. Such attention capture is often studied in oddball paradigms consisting of a frequently occurring "standard" stimulus which is changed at odd times to form a "deviant". The deviant may result in the capturing of attention. An auditory ERP, the P3a, is often associated with this process. To collect a sufficient amount of data is however very time-consuming. A more multi-feature "optimal" paradigm has been proposed but it is not known if it is appropriate for the study of attention capture.

METHODS

An optimal paradigm was run in which 6 different rare deviants (p=.08) were separated by a standard stimulus (p=.50) and compared to results when 4 oddball paradigms were also run.

RESULTS

A large P3a was elicited by some of the deviants in the optimal paradigm but not by others. However, very similar results were observed when separate oddball paradigms were run.

CONCLUSIONS & SIGNIFICANCE

The present study indicates that the optimal paradigm provides a very time-saving method to study attention capture and the P3a.

Cortical encoding and neurophysiological tracking of intensity and pitch cues signaling English stress patterns in native and nonnative speakers

We examined cross-language differences in neural encoding and tracking of intensity and pitch cues signaling English stress patterns. Auditory mismatch negativities (MMNs) were recorded in English and Mandarin listeners in response to contrastive English pseudowords whose primary stress occurred either on the first or second syllable (i.e., "nocTICity" vs. "NOCticity"). The contrastive syllable stress elicited two consecutive MMNs in both language groups, but English speakers demonstrated larger responses to stress patterns than Mandarin speakers. Correlations between the amplitude of ERPs and continuous changes in the running intensity and pitch of speech assessed how well each language group's brain activity tracked these salient acoustic features of lexical stress. We found that English speakers' neural responses tracked intensity changes in speech more closely than Mandarin speakers (higher brain-acoustic correlation). Findings demonstrate more robust and precise processing of English stress (intensity) patterns in early auditory cortical responses of native relative to nonnative speakers.

Neurobiological evidence for voicing underspecification in English

In long-term memory, the phoneme units that make up words are coded for the distinctive features and feature values that are necessary to distinguish between words in the mental lexicon. Underspecification theory says that the phonemes that have unmarked feature values are even more abstract in that the feature is omitted from the representation altogether. This makes phoneme representations in words more sparse than the fully specified phonetic representations of the same words. Eulitz and Lahiri (2004) demonstrated that this theory predicts certain asymmetries in the Mismatch Negativity (MMN) response to phoneme contrasts. We expand on this research by demonstrating underspecification-driven asymmetry in the brain response to laryngeal feature contrasts in English (i.e. what makes /d/ and /t/ different). We add a new test by showing that the asymmetry disappears if the MMN paradigm is modified to encourage the formation of phonetic memory traces instead of phonemic memory traces. This result adds further neurobiological evidence that long-term phonological representations are more sparsely represented than phonetic representations.

Present and past: Can writing abilities in school children be associated with their auditory discrimination capacities in infancy?

Literacy acquisition is highly associated with auditory processing abilities, such as auditory discrimination. The event-related potential Mismatch Response (MMR) is an indicator for cortical auditory discrimination abilities and it has been found to be reduced in individuals with reading and writing impairments and also in infants at risk for these impairments. The goal of the present study was to analyze the relationship between auditory speech discrimination in infancy and writing abilities at school age within subjects, and to determine when auditory speech discrimination differences, relevant for later writing abilities, start to develop. We analyzed the MMR registered in response to natural syllables in German children with and without writing problems at two points during development, that is, at school age and at infancy, namely at age 1 month and 5 months. We observed MMR related auditory discrimination differences between infants with and without later writing problems, starting to develop at age 5 months-an age when infants begin to establish language-specific phoneme representations. At school age, these children with and without writing problems also showed auditory discrimination differences, reflected in the MMR, confirming a relationship between writing and auditory speech processing skills. Thus, writing problems at school age are, at least, partly grounded in auditory discrimination problems developing already during the first months of life.

Functionally integrated neural processing of linguistic and talker information: An event-related fMRI and ERP study

Speech signals contain information of both linguistic content and a talker's voice. Conventionally, linguistic and talker processing are thought to be mediated by distinct neural systems in the left and right hemispheres respectively, but there is growing evidence that linguistic and talker processing interact in many ways. Previous studies suggest that talker-related vocal tract changes are processed integrally with phonetic changes in the bilateral posterior superior temporal gyrus/superior temporal sulcus (STG/STS), because the vocal tract parameter influences the perception of phonetic information. It is yet unclear whether the bilateral STG is also activated by the integral processing of another parameter - pitch, which influences the perception of lexical tone information and is related to talker differences in tone languages. In this study, we conducted separate functional magnetic resonance imaging (fMRI) and event-related potential (ERP) experiments to examine the spatial and temporal loci of interactions of lexical tone and talker-related pitch processing in Cantonese. We found that the STG was activated bilaterally during the processing of talker changes when listeners attended to lexical tone changes in the stimuli and during the processing of lexical tone changes when listeners attended to talker changes, suggesting that lexical tone and talker processing are functionally integrated in the bilateral STG. It extends the previous study, providing evidence for a general neural mechanism of integral phonetic and talker processing in the bilateral STG. The ERP results show interactions of lexical tone and talker processing 500-800ms after auditory word onset (a simultaneous posterior P3b and a frontal negativity). Moreover, there is some asymmetry in the interaction, such that unattended talker changes affect linguistic processing more than vice versa, which may be related to the ambiguity that talker changes cause in speech perception and/or attention bias to talker changes. Our findings have implications for understanding the neural encoding of linguistic and talker information.

Electrophysiologic auditory tests

Auditory stimulation elicits a complex series of electric signals in the ear and nervous system which can be used for hearing assessment, audiologic and neurologic diagnosis, intraoperative monitoring, and neurophysiologic research. The earliest components comprise the electrocochleogram. The cochlear microphonic arises from receptor potentials from cochlear hair cells, and is used to assess hair cell function in patients with auditory neuropathy. The summating potential becomes larger in Ménière's disease. The eighth-nerve compound action potential is useful for objective audiometry. Brainstem auditory evoked potentials, a series of components generated in the eighth nerve and the brainstem auditory pathways, can be used for diagnostic assessment and intraoperative monitoring of the ears and of the auditory pathways up through the mesencephalon. They are relatively easy to record, highly consistent in normal subjects, and little unaffected by surgical anesthesia. Middle-latency and long-latency auditory evoked potentials are generated in multiple areas of cerebral cortex that are activated by auditory stimulation. Anesthetic effects limit their utility for intraoperative monitoring, and substantial intersubject variability limits their utility as a diagnostic test in individual patients, but they are important research tools for the study of memory processes and the way in which the brain analyzes auditory stimuli.

Statistical learning of music- and language-like sequences and tolerance for spectral shifts

In our previous study (Daikoku, Yatomi, & Yumoto, 2014), we demonstrated that the N1m response could be a marker for the statistical learning process of pitch sequence, in which each tone was ordered by a Markov stochastic model. The aim of the present study was to investigate how the statistical learning of music- and language-like auditory sequences is reflected in the N1m responses based on the assumption that both language and music share domain generality. By using vowel sounds generated by a formant synthesizer, we devised music- and language-like auditory sequences in which higher-ordered transitional rules were embedded according to a Markov stochastic model by controlling fundamental (F0) and/or formant frequencies (F1-F2). In each sequence, F0 and/or F1-F2 were spectrally shifted in the last one-third of the tone sequence. Neuromagnetic responses to the tone sequences were recorded from 14 right-handed normal volunteers. In the music- and language-like sequences with pitch change, the N1m responses to the tones that appeared with higher transitional probability were significantly decreased compared with the responses to the tones that appeared with lower transitional probability within the first two-thirds of each sequence. Moreover, the amplitude difference was even retained within the last one-third of the sequence after the spectral shifts. However, in the language-like sequence without pitch change, no significant difference could be detected. The pitch change may facilitate the statistical learning in language and music. Statistically acquired knowledge may be appropriated to process altered auditory sequences with spectral shifts. The relative processing of spectral sequences may be a domain-general auditory mechanism that is innate to humans.

Semantic advantage for learning new phonological form representations

Learning a new word requires discrimination between a novel sequence of sounds and similar known words. We investigated whether semantic information facilitates the acquisition of new phonological representations in adults and whether this learning enhancement is modulated by overnight consolidation. Participants learned novel spoken words either consistently associated with a visual referent or with no consistent meaning. An auditory oddball task tested discrimination of these newly learned phonological forms from known words. The MMN, an electrophysiological measure of auditory discrimination, was only elicited for words learned with a consistent semantic association. Immediately after training, this semantic benefit on auditory discrimination was linked to explicit learning of the associations, where participants with greater semantic learning exhibited a larger MMN. However, although the semantic-associated words continued to show greater auditory discrimination than nonassociated words after consolidation, the MMN was no longer related to performance in learning the semantic associations. We suggest that the provision of semantic systematicity directly impacts upon the development of new phonological representations and that a period of offline consolidation may promote the abstraction of these representations.

Implicit and explicit statistical learning of tone sequences across spectral shifts

We investigated how the statistical learning of auditory sequences is reflected in neuromagnetic responses in implicit and explicit learning conditions. Complex tones with fundamental frequencies (F0s) in a five-tone equal temperament were generated by a formant synthesizer. The tones were subsequently ordered with the constraint that the probability of the forthcoming tone was statistically defined (80% for one tone; 5% for the other four) by the latest two successive tones (second-order Markov chains). The tone sequence consisted of 500 tones and 250 successive tones with a relative shift of F0s based on the same Markov transitional matrix. In explicit and implicit learning conditions, neuromagnetic responses to the tone sequence were recorded from fourteen right-handed participants. The temporal profiles of the N1m responses to the tones with higher and lower transitional probabilities were compared. In the explicit learning condition, the N1m responses to tones with higher transitional probability were significantly decreased compared with responses to tones with lower transitional probability in the latter half of the 500-tone sequence. Furthermore, this difference was retained even after the F0s were relatively shifted. In the implicit learning condition, N1m responses to tones with higher transitional probability were significantly decreased only for the 250 tones following the relative shift of F0s. The delayed detection of learning effects across the sound-spectral shift in the implicit condition may imply that learning may progress earlier in explicit learning conditions than in implicit learning conditions. The finding that the learning effects were retained across spectral shifts regardless of the learning modality indicates that relative pitch processing may be an essential ability for humans.

Assimilation of L2 vowels to L1 phonemes governs L2 learning in adulthood: a behavioral and ERP study

According to the Perceptual Assimilation Model (PAM), articulatory similarity/dissimilarity between sounds of the second language (L2) and the native language (L1) governs L2 learnability in adulthood and predicts L2 sound perception by naïve listeners. We performed behavioral and neurophysiological experiments on two groups of university students at the first and fifth years of the English language curriculum and on a group of naïve listeners. Categorization and discrimination tests, as well as the mismatch negativity (MMN) brain response to L2 sound changes, showed that the discriminatory capabilities of the students did not significantly differ from those of the naïve subjects. In line with the PAM model, we extend the findings of previous behavioral studies showing that, at the neural level, classroom instruction in adulthood relies on assimilation of L2 vowels to L1 phoneme categories and does not trigger improvement in L2 phonetic discrimination. Implications for L2 classroom teaching practices are discussed.

The neural basis of sublexical speech and corresponding nonspeech processing: a combined EEG-MEG study

We addressed the neural organization of speech versus nonspeech sound processing by investigating preattentive cortical auditory processing of changes in five features of a consonant-vowel syllable (consonant, vowel, sound duration, frequency, and intensity) and their acoustically matched nonspeech counterparts in a simultaneous EEG-MEG recording of mismatch negativity (MMN/MMNm). Overall, speech-sound processing was enhanced compared to nonspeech sound processing. This effect was strongest for changes which affect word meaning (consonant, vowel, and vowel duration) in the left and for the vowel identity change in the right hemisphere also. Furthermore, in the right hemisphere, speech-sound frequency and intensity changes were processed faster than their nonspeech counterparts, and there was a trend for speech-enhancement in frequency processing. In summary, the results support the proposed existence of long-term memory traces for speech sounds in the auditory cortices, and indicate at least partly distinct neural substrates for speech and nonspeech sound processing.

Neural correlates of speech processing in prelingually deafened children and adolescents with cochlear implants

Prelingually deafened children with cochlear implants stand a good chance of developing satisfactory speech performance. Nevertheless, their eventual language performance is highly variable and not fully explainable by the duration of deafness and hearing experience. In this study, two groups of cochlear implant users (CI groups) with very good basic hearing abilities but non-overlapping speech performance (very good or very bad speech performance) were matched according to hearing age and age at implantation. We assessed whether these CI groups differed with regard to their phoneme discrimination ability and auditory sensory memory capacity, as suggested by earlier studies. These functions were measured behaviorally and with the Mismatch Negativity (MMN). Phoneme discrimination ability was comparable in the CI group of good performers and matched healthy controls, which were both better than the bad performers. Source analyses revealed larger MMN activity (155-225 ms) in good than in bad performers, which was generated in the frontal cortex and positively correlated with measures of working memory. For the bad performers, this was followed by an increased activation of left temporal regions from 225 to 250 ms with a focus on the auditory cortex. These results indicate that the two CI groups developed different auditory speech processing strategies and stress the role of phonological functions of auditory sensory memory and the prefrontal cortex in positively developing speech perception and production.

Toward a neural chronometry for the aesthetic experience of music

Music is often studied as a cognitive domain alongside language. The emotional aspects of music have also been shown to be important, but views on their nature diverge. For instance, the specific emotions that music induces and how they relate to emotional expression are still under debate. Here we propose a mental and neural chronometry of the aesthetic experience of music initiated and mediated by external and internal contexts such as intentionality, background mood, attention, and expertise. The initial stages necessary for an aesthetic experience of music are feature analysis, integration across modalities, and cognitive processing on the basis of long-term knowledge. These stages are common to individuals belonging to the same musical culture. The initial emotional reactions to music include the startle reflex, core "liking," and arousal. Subsequently, discrete emotions are perceived and induced. Presumably somatomotor processes synchronizing the body with the music also come into play here. The subsequent stages, in which cognitive, affective, and decisional processes intermingle, require controlled cross-modal neural processes to result in aesthetic emotions, aesthetic judgments, and conscious liking. These latter aesthetic stages often require attention, intentionality, and expertise for their full actualization.

Infants' brain responses for speech sound changes in fast multifeature MMN paradigm

OBJECTIVE

We investigated whether newborn speech-sound discrimination can be studied in 40 min using fast multifeature mismatch negativity (MMN) paradigm and do the results differ from those obtained with the traditional oddball paradigm.

METHODS

Newborns' MMN responses to five types of changes (consonant identity, F0, intensity, vowel duration and vowel identity) were recorded in the multifeature group (N=15) and vowel duration and vowel identity changes in the oddball group (N=13), after which the MMNs from both groups were compared with each others.

RESULTS

Statistically significant MMNs in the 190-600 ms time range from the stimulus onset were found for most change types in both paradigms. Newborn MMN responses were predominantly positive but a small number of participants elicited negative MMNs instead. MMN amplitudes did not differ between the multifeature and oddball groups.

CONCLUSIONS

Newborn speech-sound discrimination can be assessed in a short recording time using the fast multifeature paradigm.

SIGNIFICANCE

The paradigm presented here can be used to record extensive auditory discrimination profiles in newborns and assess development of speech-sound discrimination and its difficulties.

Attention modulates the use of spectral attributes in vowel discrimination: behavioral and event-related potential evidence

Speech contains a variety of acoustic cues to auditory and phonetic contrasts that are exploited by the listener in decoding the acoustic signal. In three experiments, we tried to elucidate whether listeners rely on formant peak frequencies or whole spectrum attributes in vowel discrimination. We created two vowel continua in which the acoustic distance in formant frequencies was constant but the continua differed in spectral moments (i.e., the whole spectrum modeled as a probability density function). In Experiment 1, we measured reaction times and response accuracy while listeners performed a go/no-go discrimination task. The results indicated that the performance of the listeners was based on the spectral moments (especially the first and second moments), and not on formant peaks. Behavioral results in Experiment 2 showed that, when the stimuli were presented in noise eliminating differences in spectral moments between the two continua, listeners employed formant peak frequencies. In Experiment 3, using the same listeners and stimuli as in Experiment 1, we measured an automatic brain potential, the mismatch negativity (MMN), when listeners did not attend to the auditory stimuli. Results showed that the MMN reflects sensitivity only to the formant structure of the vowels. We suggest that the auditory cortex automatically and pre-attentively encodes formant peak frequencies, whereas attention can be deployed for processing additional spectral information, such as spectral moments, to enhance vowel discrimination.

The role of stimulus cross-splicing in an event-related potentials study. Misleading formant transitions hinder automatic phonological processing

The mental organization of linguistic knowledge and its involvement in speech processing can be investigated using the mismatch negativity (MMN) component of the auditory event-related potential. A contradiction arises, however, between the technical need for strict control of acoustic stimulus properties and the quest for naturalness and acoustic variability of the stimuli. Here, two methods of preparing speech stimulus material were compared. Focussing on the automatic processing of a phonotactic restriction in German, two corresponding sets of various vowel-fricative syllables were used as stimuli. The former syllables were naturally spoken while the latter ones were created by means of cross-splicing. Phonetically, natural and spliced syllables differed with respect to the appropriateness of coarticulatory information about the forthcoming fricative within the vowels. Spliced syllables containing clearly misleading phonetic information were found to elicit larger N2 responses compared to their natural counterparts. Furthermore, MMN results found for the natural syllables could not be replicated with these spliced stimuli. These findings indicate that the automatic processing of the stimuli was considerably affected by the stimulus preparation method. Thus, in spite of its unquestioned benefits for MMN experiments, the splicing technique may lead to interference effects on the linguistic factors under investigation.

Preattentive extraction of abstract auditory rules in speech sound stream: a mismatch negativity study using lexical tones

BACKGROUND

Extraction of linguistically relevant auditory features is critical for speech comprehension in complex auditory environments, in which the relationships between acoustic stimuli are often abstract and constant while the stimuli per se are varying. These relationships are referred to as the abstract auditory rule in speech and have been investigated for their underlying neural mechanisms at an attentive stage. However, the issue of whether or not there is a sensory intelligence that enables one to automatically encode abstract auditory rules in speech at a preattentive stage has not yet been thoroughly addressed.

METHODOLOGY/PRINCIPAL FINDINGS

We chose Chinese lexical tones for the current study because they help to define word meaning and hence facilitate the fabrication of an abstract auditory rule in a speech sound stream. We continuously presented native Chinese speakers with Chinese vowels differing in formant, intensity, and level of pitch to construct a complex and varying auditory stream. In this stream, most of the sounds shared flat lexical tones to form an embedded abstract auditory rule. Occasionally the rule was randomly violated by those with a rising or falling lexical tone. The results showed that the violation of the abstract auditory rule of lexical tones evoked a robust preattentive auditory response, as revealed by whole-head electrical recordings of the mismatch negativity (MMN), though none of the subjects acquired explicit knowledge of the rule or became aware of the violation.

CONCLUSIONS/SIGNIFICANCE

Our results demonstrate that there is an auditory sensory intelligence in the perception of Chinese lexical tones. The existence of this intelligence suggests that the humans can automatically extract abstract auditory rules in speech at a preattentive stage to ensure speech communication in complex and noisy auditory environments without drawing on conscious resources.

The mismatch negativity (MMN)--a unique window to disturbed central auditory processing in ageing and different clinical conditions

In this article, we review clinical research using the mismatch negativity (MMN), a change-detection response of the brain elicited even in the absence of attention or behavioural task. In these studies, the MMN was usually elicited by employing occasional frequency, duration or speech-sound changes in repetitive background stimulation while the patient was reading or watching videos. It was found that in a large number of different neuropsychiatric, neurological and neurodevelopmental disorders, as well as in normal ageing, the MMN amplitude was attenuated and peak latency prolonged. Besides indexing decreased discrimination accuracy, these effects may also reflect, depending on the specific stimulus paradigm used, decreased sensory-memory duration, abnormal perception or attention control or, most importantly, cognitive decline. In fact, MMN deficiency appears to index cognitive decline irrespective of the specific symptomatologies and aetiologies of the different disorders involved.

Distinct neural responses to chord violations: a multiple source analysis study

The human brain is constantly predicting the auditory environment by representing sequential similarities and extracting temporal regularities. It has been proposed that simple auditory regularities are extracted at lower stations of the auditory cortex and more complex ones at other brain regions, such as the prefrontal cortex. Deviations from auditory regularities elicit a family of early negative electric potentials distributed over the frontal regions of the scalp. In this study, we wished to disentangle the brain processes associated with sequential vs. hierarchical auditory regularities in a musical context by studying the event-related potentials (ERPs), the behavioral responses to violations of these regularities, and the localization of the underlying ERP generators using two different source analysis algorithms. To this aim, participants listened to musical cadences constituted by seven chords, each containing either harmonically congruous chords, harmonically incongruous chords, or harmonically congruous but mistuned chords. EEG was recorded and multiple source analysis was performed. Incongruous chords violating the rules of harmony elicited a bilateral ERAN, whereas mistuned chords within chord sequences elicited a right-lateralized MMN. We found that the dominant cortical sources for the ERAN were localized around Broca's area and its right homolog, whereas the MMN generators were localized around the primary auditory cortex. These findings suggest a predominant role of the auditory cortices in detecting sequential scale regularities and the posterior prefrontal cortex in parsing hierarchical regularities in music.

Interplay between acoustic/phonetic and semantic processes during spoken sentence comprehension: an ERP study

When listening to speech in everyday-life situations, our cognitive system must often cope with signal instabilities such as sudden breaks, mispronunciations, interfering noises or reverberations potentially causing disruptions at the acoustic/phonetic interface and preventing efficient lexical access and semantic integration. The physiological mechanisms allowing listeners to react instantaneously to such fast and unexpected perturbations in order to maintain intelligibility of the delivered message are still partly unknown. The present electroencephalography (EEG) study aimed at investigating the cortical responses to real-time detection of a sudden acoustic/phonetic change occurring in connected speech and how these mechanisms interfere with semantic integration. Participants listened to sentences in which final words could contain signal reversals along the temporal dimension (time-reversed speech) of varying durations and could have either a low- or high-cloze probability within sentence context. Results revealed that early detection of the acoustic/phonetic change elicited a fronto-central negativity shortly after the onset of the manipulation that matched the spatio-temporal features of the Mismatch Negativity (MMN) recorded in the same participants during an oddball paradigm. Time reversal also affected late event-related potentials (ERPs) reflecting semantic expectancies (N400) differently when words were predictable or not from the sentence context. These findings are discussed in the context of brain signatures to transient acoustic/phonetic variations in speech. They contribute to a better understanding of natural speech comprehension as they show that acoustic/phonetic information and semantic knowledge strongly interact under adverse conditions.

Neuromagnetic evidence for a featural distinction of English consonants: sensor- and source-space data

Speech sounds can be classified on the basis of their underlying articulators or on the basis of the acoustic characteristics resulting from particular articulatory positions. Research in speech perception suggests that distinctive features are based on both articulatory and acoustic information. In recent years, neuroelectric and neuromagnetic investigations provided evidence for the brain's early sensitivity to distinctive features and their acoustic consequences, particularly for place of articulation distinctions. Here, we compare English consonants in a Mismatch Field design across two broad and distinct places of articulation - labial and coronal - and provide further evidence that early evoked auditory responses are sensitive to these features. We further add to the findings of asymmetric consonant processing, although we do not find support for coronal underspecification. Labial glides (Experiment 1) and fricatives (Experiment 2) elicited larger Mismatch responses than their coronal counterparts. Interestingly, their M100 dipoles differed along the anterior/posterior dimension in the auditory cortex that has previously been found to spatially reflect place of articulation differences. Our results are discussed with respect to acoustic and articulatory bases of featural speech sound classifications and with respect to a model that maps distinctive phonetic features onto long-term representations of speech sounds.

Native language experience influences the topography of the mismatch negativity to speech

The ability to learn second language speech sound categories declines during development. We examined this phenomenon by studying the mismatch negativity (MMN) to the /r/ - /l/ distinction in native English speakers and learners of English as a second language who are native speakers of Japanese. Previous studies have suggested that the MMN is remarkably plastic when evaluated as a waveform at a central electrode. We replicated this finding: analyses of the MMN at a typical electrode location (Fz) revealed only small, non-significant differences between groups, despite large behavioral differences in the ability to discriminate these sounds from one another. Topographic analyses, however, revealed reliable differences in lateralization of the MMN, such that native English speakers' responses were left-lateralized relative to native Japanese speakers' responses.

Pre-attentive and attentive processing of French vowels

This study aimed at investigating the effects of acoustic distance and of speaker variability on the pre-attentive and attentive perception of French vowels by French adult speakers. The electroencephalogram (EEG) was recorded while participants watched a silent movie (Passive condition) and discriminated deviant vowels (Active condition). The auditory sequence included 4 French vowels, /u/ (standard) and /o/, /y/ and /ø/ as deviants, produced by 3 different speakers. As the vowel /o/ is closer to /u/ than the other deviants in acoustic distance, we predicted smaller mismatch negativity (MMN) and smaller N1 component, as well as higher error rate and longer reaction times. Results were in line with these predictions. Moreover, the MMN was elicited by all deviant vowels independently of speaker variability. By contrast, the Vowel by Speaker interaction was significant in the Active listening condition thereby showing that subtle within-category differences are processed at the attentive level. These results suggest that while vowels are categorized pre-attentively according to phonemic representations and independently of speaker variability, participants are sensitive to between-speaker differences when they focus attention on vowel processing.