Literacy Affects Spoken Language in a Non-Linguistic Task: An ERP Study

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing

Both visual articulatory gestures and orthography provide information on the phonological content of speech. This EEG study investigated the integration between speech and these two visual inputs. A comparison of skilled readers' brain responses elicited by a spoken word presented alone versus synchronously with a static image of a viseme or a grapheme of the spoken word's onset showed that while neither visual input induced audiovisual integration on N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted in P350 time-window and generalized to all electrodes. The finding suggests a strong impact of spelling knowledge on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive value present in natural lip movements but not in static visemes is particularly critical to the contribution of visual articulatory gestures to speech processing.

Phonemic Training Modulates Early Speech Processing in Pre-reading Children

Phonemic awareness and rudimentary grapheme knowledge concurrently develop in pre-school age. In a training study, we tried to disentangle the role of both precursor functions of reading for spoken word recognition. Two groups of children exercised with phonemic materials, but only one of both groups learnt corresponding letters to trained phonemes. A control group exercised finger-number associations (non-linguistic training). After the training, we tested how sensitive children were to prime-target variation in word onset priming. A group of young adults took part in the same experiment to provide data from experienced readers. While decision latencies to the targets suggested fine-grained spoken word processing in all groups, event-related potentials (ERPs) indicated that both phonemic training groups processed phonemic variation in more detail than the non-linguistic training group and young adults at early stages of speech processing. Our results indicate temporal plasticity of implicit speech processing in pre-school age as a function of explicit phonemic training.

How much does orthography influence the processing of reduced word forms? Evidence from novel-word learning about French schwa deletion

This study examines the influence of orthography on the processing of reduced word forms. For this purpose, we compared the impact of phonological variation with the impact of spelling-sound consistency on the processing of words that may be produced with or without the vowel schwa. Participants learnt novel French words in which the vowel schwa was present or absent in the first syllable. In Experiment 1, the words were consistently produced without schwa or produced in a variable manner (i.e., sometimes produced with and sometimes produced without schwa). In Experiment 2, words were always produced in a consistent manner, but an orthographic exposure phase was included in which words that were produced without schwa were either spelled with or without the letter <e>. Results from naming and eye-tracking tasks suggest that both phonological variation and spelling-sound consistency influence the processing of spoken novel words. However, the influence of phonological variation outweighs the effect of spelling-sound consistency. Our findings therefore suggest that the influence of orthography on the processing of reduced word forms is relatively small.

Orthographic influence on spoken word identification: Behavioral and fMRI evidence

The current study investigated behavioral and neuroimaging evidence for orthographic influences on auditory word identification. To assess such influences, the proportion of similar sounding words (i.e. phonological neighbors) that were also spelled similarly (i.e., orthographic neighbors) was computed for each auditorily presented word as the Orthographic-to-Phonological Overlap Ratio (OPOR). Speech intelligibility was manipulated by presenting monosyllabic words in multi-talker babble at two signal-to-noise ratios: + 3 and + 10 dB SNR. Identification rates were lower for high overlap words in the challenging + 3 dB SNR condition. In addition, BOLD contrast increased with OPOR at the more difficult SNR, and decreased with OPOR under more favorable SNR conditions. Both voxel-based and region of interest analyses demonstrated robust effects of OPOR in several cingulo-opercular regions. However, contrary to prior theoretical accounts, no task-related activity was observed in posterior regions associated with phonological or orthographic processing. We suggest that, when processing is difficult, orthographic-to-phonological feature overlap increases the availability of competing responses, which then requires additional support from domain general performance systems in order to produce a single response.

Orthographic effects in spoken word recognition: Evidence from Chinese

Extensive evidence from alphabetic languages demonstrates a role of orthography in the processing of spoken words. Because alphabetic systems explicitly code speech sounds, such effects are perhaps not surprising. However, it is less clear whether orthographic codes are involuntarily accessed from spoken words in languages with non-alphabetic systems, in which the sound-spelling correspondence is largely arbitrary. We investigated the role of orthography via a semantic relatedness judgment task: native Mandarin speakers judged whether or not spoken word pairs were related in meaning. Word pairs were either semantically related, orthographically related, or unrelated. Results showed that relatedness judgments were made faster for word pairs that were semantically related than for unrelated word pairs. Critically, orthographic overlap on semantically unrelated word pairs induced a significant increase in response latencies. These findings indicate that orthographic information is involuntarily accessed in spoken-word recognition, even in a non-alphabetic language such as Chinese.

Word and object recognition during reading acquisition: MEG evidence

Studies on adults suggest that reading-induced brain changes might not be limited to linguistic processes. It is still unclear whether these results can be generalized to reading development. The present study shows to which extent neural responses to verbal and nonverbal stimuli are reorganized while children learn to read. MEG data of thirty Basque children (4-8y) were collected while they were presented with written words, spoken words and visual objects. The evoked fields elicited by the experimental stimuli were compared to their scrambled counterparts. Visual words elicited left posterior (200-300ms) and temporal activations (400-800ms). The size of these effects increased as reading performance improved, suggesting a reorganization of children's visual word responses. Spoken words elicited greater left temporal responses relative to scrambles (300-700ms). No evidence for the influence of reading expertise was observed. Brain responses to objects were greater than to scrambles in bilateral posterior regions (200-500ms). There was a greater left hemisphere involvement as reading errors decreased, suggesting a strengthened verbal decoding of visual configurations with reading acquisition. The present results reveal that learning to read not only influences written word processing, but also affects visual object recognition, suggesting a non-language specific impact of reading on children's neural mechanisms.

Effects of orthographic consistency and homophone density on Chinese spoken word recognition

Studies of alphabetic language have shown that orthographic knowledge influences phonological processing during spoken word recognition. This study utilized the Event-Related Potentials (ERPs) to differentiate two types of phonology-to-orthography (P-to-O) mapping consistencies in Chinese, namely homophone density and orthographic consistency. The ERP data revealed an orthographic consistency effect in the frontal-centrally distributed N400, and a homophone density effect in central-posteriorly distributed late positive component (LPC). Further source analyses using the standardized low-resolution electromagnetic tomography (sLORETA) demonstrated that the orthographic effect was not only localized in the frontal and temporal-parietal regions for phonological processing, but also in the posterior visual cortex for orthographic processing, while the homophone density effect was found in middle temporal gyrus for lexical-semantic selection, and in the temporal-occipital junction for orthographic processing. These results suggest that orthographic information not only shapes the nature of phonological representations, but may also be activated during on-line spoken word recognition.

Neural Correlates of Morphological Processing: Evidence from Chinese

Morphological decomposition is an important part of complex word processing. In Chinese, this requires a comprehensive consideration of phonological, orthographic and morphemic information. The left inferior frontal gyrus (L-IFG) has been implicated in this process in alphabetic languages. However, it is unclear whether the neural mechanisms underlying morphological processing in alphabetic languages would be the same in Chinese, a logographic language. To investigate the neural basis of morphological processing in Chinese compound words, an fMRI experiment was conducted using an explicit auditory morphological judgment task. Results showed the L-IFG to be a core area in Chinese morphological processing, consistent with research in alphabetic languages. Additionally, a broad network consisting of the L-MTG, the bilateral STG and the L-FG that taps phonological, orthographic, and semantic information was found to be involved. These results provide evidence that the L-IFG plays an important role in morphological processing even in languages that are typologically different.

Unattentive speech processing is influenced by orthographic knowledge: evidence from mismatch negativity

How far can acquired knowledge such as orthographic knowledge affect pre-existing abilities such as speech perception? This controversial issue was addressed by investigating the automaticity of the influence of orthographic knowledge on speech processing. Many studies demonstrated this influence in active, lexico-semantic speech processing tasks. However, it has never been observed when speech is unattended. Here, the Mismatch Negativity (MMN), an automatic index of experience-dependent auditory traces, was recorded in an unattended oddball paradigm manipulating the orthographic congruency between frequent and deviant spoken riming words. Both orthographically congruent and incongruent deviant words elicited a typical MMN over the fronto-central regions, with a stronger response in the incongruent condition. The finding showed that the orthographic dimension of spoken words influences a physiological marker of speech processing although participants were required not to attend to the auditory input. This provides evidence for an impact of acquiring a written code on speech processing.

Putting an “End” to the Motor Cortex Representations of Action Words

Language processing is an example of implicit learning of multiple statistical cues that provide probabilistic information regarding word structure and use. Much of the current debate about language embodiment is devoted to how action words are represented in the brain, with motor cortex activity evoked by these words assumed to selectively reflect conceptual content and/or its simulation. We investigated whether motor cortex activity evoked by manual action words (e.g., caress) might reflect sensitivity to probabilistic orthographic–phonological cues to grammatical category embedded within individual words. We first review neuroimaging data demonstrating that nonwords evoke activity much more reliably than action words along the entire motor strip, encompassing regions proposed to be action category specific. Using fMRI, we found that disyllabic words denoting manual actions evoked increased motor cortex activity compared with non-body-part-related words (e.g., canyon), activity which overlaps that evoked by observing and executing hand movements. This result is typically interpreted in support of language embodiment. Crucially, we also found that disyllabic nonwords containing endings with probabilistic cues predictive of verb status (e.g., -eve) evoked increased activity compared with nonwords with endings predictive of noun status (e.g., -age) in the identical motor area. Thus, motor cortex responses to action words cannot be assumed to selectively reflect conceptual content and/or its simulation. Our results clearly demonstrate motor cortex activity reflects implicit processing of ortho-phonological statistical regularities that help to distinguish a word's grammatical class.