The Long Road to Automation: Neurocognitive Development of Letter–Speech Sound Processing

Time course analyses of orthographic and phonological priming effects in developing readers

It has been assumed that fluent reading requires efficient integration of orthographic and phonological codes. However, it is thus far unclear how this integration process develops when children learn to become fluent readers. Therefore, we used masked priming to investigate time courses of orthographic and phonological code activation in children at incremental levels of reading development (second, fourth and sixth grade). The first study used targets with small phonological differences between phonological and orthographic primes, which are typical in transparent orthographies. The second study manipulated the strength of the phonological difference between prime and target to clarify whether phonological difference influences phonological priming effects. Results in both studies showed that orthographic priming effects became facilitative at increasingly short durations during reading development, but phonological priming was absent. These results are taken to suggest that development of reading fluency is accompanied by increased automatization of orthographic representations. The absence of phonological priming suggests that developing readers cannot yet activate phonological codes automatically.

Atypical White Matter Connectivity in Dyslexic Readers of a Fairly Transparent Orthography

Atypical structural properties of the brain's white matter bundles have been associated with failing reading acquisition in developmental dyslexia. Because these white matter properties may show dynamic changes with age and orthographic depth, we examined fractional anisotropy (FA) along 16 white matter tracts in 8- to 11-year-old dyslexic (DR) and typically reading (TR) children learning to read in a fairly transparent orthography (Dutch). Our results showed higher FA values in the bilateral anterior thalamic radiations of DRs and FA values of the left thalamic radiation scaled with behavioral reading-related scores. Furthermore, DRs tended to have atypical FA values in the bilateral arcuate fasciculi. Children's age additionally predicted FA values along the tracts. Together, our findings suggest differential contributions of cortical and thalamo-cortical pathways to the developing reading network in dyslexic and typical readers, possibly indicating prolonged letter-by-letter reading or increased attentional and/or working memory demands in dyslexic children during reading.

A Selective Deficit in Phonetic Recalibration by Text in Developmental Dyslexia

Upon hearing an ambiguous speech sound, listeners may adjust their perceptual interpretation of the speech input in accordance with contextual information, like accompanying text or lipread speech (i.e., phonetic recalibration; Bertelson et al., 2003). As developmental dyslexia (DD) has been associated with reduced integration of text and speech sounds, we investigated whether this deficit becomes manifest when text is used to induce this type of audiovisual learning. Adults with DD and normal readers were exposed to ambiguous consonants halfway between /aba/ and /ada/ together with text or lipread speech. After this audiovisual exposure phase, they categorized auditory-only ambiguous test sounds. Results showed that individuals with DD, unlike normal readers, did not use text to recalibrate their phoneme categories, whereas their recalibration by lipread speech was spared. Individuals with DD demonstrated similar deficits when ambiguous vowels (halfway between /wIt/ and /wet/) were recalibrated by text. These findings indicate that DD is related to a specific letter-speech sound association deficit that extends over phoneme classes (vowels and consonants), but – as lipreading was spared – does not extend to a more general audio–visual integration deficit. In particular, these results highlight diminished reading-related audiovisual learning in addition to the commonly reported phonological problems in developmental dyslexia.

Reading-induced shifts of perceptual speech representations in auditory cortex

Learning to read requires the formation of efficient neural associations between written and spoken language. Whether these associations influence the auditory cortical representation of speech remains unknown. Here we address this question by combining multivariate functional MRI analysis and a newly-developed ‘text-based recalibration’ paradigm. In this paradigm, the pairing of visual text and ambiguous speech sounds shifts (i.e. recalibrates) the perceptual interpretation of the ambiguous sounds in subsequent auditory-only trials. We show that it is possible to retrieve the text-induced perceptual interpretation from fMRI activity patterns in the posterior superior temporal cortex. Furthermore, this auditory cortical region showed significant functional connectivity with the inferior parietal lobe (IPL) during the pairing of text with ambiguous speech. Our findings indicate that reading-related audiovisual mappings can adjust the auditory cortical representation of speech in typically reading adults. Additionally, they suggest the involvement of the IPL in audiovisual and/or higher-order perceptual processes leading to this adjustment. When applied in typical and dyslexic readers of different ages, our text-based recalibration paradigm may reveal relevant aspects of perceptual learning and plasticity during successful and failing reading development.

Connections are not enough for membership: Letter/non-letter distinction persists through phonological association learning

In compound, hierarchical stimuli (also known as Navon figures), a Global Precedence Effect (GPE) can reliably be observed for both letters and non-letters. However, when presentation conditions sufficiently resemble those of reading, the GPE for letters has occasionally been found to disappear. We corroborate this effect in a study with a large group of participants. In addition, in-between two sessions, participants were trained in associating the non-letters with either phonological or non-phonological sounds. We reasoned that learning distinctive phonological associations might be akin to the acquisition of letter knowledge. This might eliminate the GPE also for the non-letters. However, the GPE persisted for the trained non-letters in both conditions. The large number of participants in this study revealed additional effects in the letter condition, which enabled further insights in the processing dissociation between letters and non-letter shapes.

Deficits in Letter-Speech Sound Associations but Intact Visual Conflict Processing in Dyslexia: Results from a Novel ERP-Paradigm

The reading and spelling deficits characteristic of developmental dyslexia (dyslexia) have been related to problems in phonological processing and in learning associations between letters and speech-sounds. Even when children with dyslexia have learned the letters and their corresponding speech sounds, letter-speech sound associations might still be less automatized compared to children with age-adequate literacy skills. In order to examine automaticity in letter-speech sound associations and to overcome some of the disadvantages associated with the frequently used visual-auditory oddball paradigm, we developed a novel electrophysiological letter-speech sound interference paradigm. This letter-speech sound interference paradigm was applied in a group of 9-year-old children with dyslexia (n = 36) and a group of typically developing (TD) children of similar age (n = 37). Participants had to indicate whether two letters look visually the same. In the incongruent condition (e.g., the letter pair A-a) there was a conflict between the visual information and the automatically activated phonological information; although the visual appearance of the two letters is different, they are both associated with the same speech sound. This conflict resulted in slower response times (RTs) in the incongruent than in the congruent (e.g., the letter pair A-e) condition. Furthermore, in the TD control group, the conflict resulted in fast and strong event-related potential (ERP) effects reflected in less negative N1 amplitudes and more positive conflict slow potentials (cSP) in the incongruent than in the congruent condition. However, the dyslexic group did not show any conflict-related ERP effects, implying that letter-speech sound associations are less automatized in this group. Furthermore, we examined general visual conflict processing in a control visual interference task, using false fonts. The conflict in this experiment was based purely on the visual similarity of the presented objects. Visual conflict resulted in slower RTs, less negative N2 amplitudes and more positive cSP in both groups. Thus, on a general, basic level, visual conflict processing does not seem to be affected in children with dyslexia.

Cross-modal noise compensation in audiovisual words

Perceiving linguistic input is vital for human functioning, but the process is complicated by the fact that the incoming signal is often degraded. However, humans can compensate for unimodal noise by relying on simultaneous sensory input from another modality. Here, we investigated noise-compensation for spoken and printed words in two experiments. In the first behavioral experiment, we observed that accuracy was modulated by reaction time, bias and sensitivity, but noise compensation could nevertheless be explained via accuracy differences when controlling for RT, bias and sensitivity. In the second experiment, we also measured Event Related Potentials (ERPs) and observed robust electrophysiological correlates of noise compensation starting at around 350 ms after stimulus onset, indicating that noise compensation is most prominent at lexical/semantic processing levels.

Serious games for health: three steps forwards

Serious games are educational tools which are more and more used in patient and health professional education. In this article, we discuss three main points that developers and educators need to address during the development of a serious game for health. We first explain how to develop motivating serious games by finding a point where the intrinsic and extrinsic motivations of end users can converge. Then, we propose to identify the features of serious games which enhance their learning effectiveness on the basis of a framework derived from cognitive science and called "the four pillars of learning." Finally, we discuss issues and solutions related to the evaluation of serious games.

Contributions of Letter-Speech Sound Learning and Visual Print Tuning to Reading Improvement: Evidence from Brain Potential and Dyslexia Training Studies

We use a neurocognitive perspective to discuss the contribution of learning letter-speech sound (L-SS) associations and visual specialization in the initial phases of reading in dyslexic children. We review findings from associative learning studies on related cognitive skills important for establishing and consolidating L-SS associations. Then we review brain potential studies, including our own, that yielded two markers associated with reading fluency. Here we show that the marker related to visual specialization (N170) predicts word and pseudoword reading fluency in children who received additional practice in the processing of morphological word structure. Conversely, L-SS integration (indexed by mismatch negativity (MMN)) may only remain important when direct orthography to semantic conversion is not possible, such as in pseudoword reading. In addition, the correlation between these two markers supports the notion that multisensory integration facilitates visual specialization. Finally, we review the role of implicit learning and executive functions in audiovisual learning in dyslexia. Implications for remedial research are discussed and suggestions for future studies are presented.

The potential of neuroscience for health sciences education: towards convergence of evidence and resisting seductive allure

Since emergence of the field 'Educational Neuroscience' (EN) in the late nineties of the previous century, a debate has emerged about the potential this field holds to influence teaching and learning in the classroom. By now, most agree that the original claims promising direct translations to teaching and learning were too strong. I argue here that research questions in (health professions) education require multi-methodological approaches, including neuroscience, while carefully weighing what (combination of) approaches are most suitable given a research question. Only through a multi-methodological approach will convergence of evidence emerge, which is so desperately needed for improving teaching and learning in the classroom. However, both researchers and teachers should become aware of the so-called 'seductive allure' of EN; that is, the demonstrable physical location and apparent objectivity of the measurements can be interpreted as yielding more powerful evidence and warranting stronger conclusions than, e.g., behavioral experiments, where in fact oftentimes the reverse is the case. I conclude that our tendency as researchers to commit ourselves to one methodological approach and to addressing educational research questions from a single methodological perspective is limiting progress in educational science and in translation to education.

Phonetic recalibration of speech by text

Listeners adjust their phonetic categories to cope with variations in the speech signal (phonetic recalibration). Previous studies have shown that lipread speech (and word knowledge) can adjust the perception of ambiguous speech and can induce phonetic adjustments (Bertelson, Vroomen, & de Gelder in Psychological Science, 14(6), 592–597, 2003; Norris, McQueen, & Cutler in Cognitive Psychology, 47(2), 204–238, 2003). We examined whether orthographic information (text) also can induce phonetic recalibration. Experiment 1 showed that after exposure to ambiguous speech sounds halfway between /b/ and /d/ that were combined with text (b or d) participants were more likely to categorize auditory-only test sounds in accordance with the exposed letters. Experiment 2 replicated this effect with a very short exposure phase. These results show that listeners adjust their phonetic boundaries in accordance with disambiguating orthographic information and that these adjustments show a rapid build-up.

Neural initialization of audiovisual integration in prereaders at varying risk for developmental dyslexia

Learning letter-speech sound correspondences is a major step in reading acquisition and is severely impaired in children with dyslexia. Up to now, it remains largely unknown how quickly neural networks adopt specific functions during audiovisual integration of linguistic information when prereading children learn letter-speech sound correspondences. Here, we simulated the process of learning letter-speech sound correspondences in 20 prereading children (6.13-7.17 years) at varying risk for dyslexia by training artificial letter-speech sound correspondences within a single experimental session. Subsequently, we acquired simultaneously event-related potentials (ERP) and functional magnetic resonance imaging (fMRI) scans during implicit audiovisual presentation of trained and untrained pairs. Audiovisual integration of trained pairs correlated with individual learning rates in right superior temporal, left inferior temporal, and bilateral parietal areas and with phonological awareness in left temporal areas. In correspondence, a differential left-lateralized parietooccipitotemporal ERP at 400 ms for trained pairs correlated with learning achievement and familial risk. Finally, a late (650 ms) posterior negativity indicating audiovisual congruency of trained pairs was associated with increased fMRI activation in the left occipital cortex. Taken together, a short (<30 min) letter-speech sound training initializes audiovisual integration in neural systems that are responsible for processing linguistic information in proficient readers. To conclude, the ability to learn grapheme-phoneme correspondences, the familial history of reading disability, and phonological awareness of prereading children account for the degree of audiovisual integration in a distributed brain network. Such findings on emerging linguistic audiovisual integration could allow for distinguishing between children with typical and atypical reading development. Hum Brain Mapp 38:1038-1055, 2017. © 2016 Wiley Periodicals, Inc.

Are the literacy difficulties that characterize developmental dyslexia associated with a failure to integrate letters and speech sounds?

The ‘automatic letter‐sound integration hypothesis’ (Blomert, 2011) proposes that dyslexia results from a failure to fully integrate letters and speech sounds into automated audio‐visual objects. We tested this hypothesis in a sample of English‐speaking children with dyslexic difficulties (N = 13) and samples of chronological‐age‐matched (CA; N = 17) and reading‐age‐matched controls (RA; N = 17) aged 7–13 years. Each child took part in two priming experiments in which speech sounds were preceded by congruent visual letters (congruent condition) or Greek letters (baseline). In a behavioural experiment, responses to speech sounds in the two conditions were compared using reaction times. These data revealed faster reaction times in the congruent condition in all three groups. In a second electrophysiological experiment, responses to speech sounds in the two conditions were compared using event‐related potentials (ERPs). These data revealed a significant effect of congruency on (1) the P1 ERP over left frontal electrodes in the CA group and over fronto‐central electrodes in the dyslexic group and (2) the P2 ERP in the dyslexic and RA control groups. These findings suggest that our sample of English‐speaking children with dyslexic difficulties demonstrate a degree of letter‐sound integration that is appropriate for their reading level, which challenges the letter‐sound integration hypothesis.

Cognitive and familial risk evidence converged: A data-driven identification of distinct and homogeneous subtypes within the heterogeneous sample of reading disabled children

The evident degree of heterogeneity observed in reading disabled children has puzzled reading researchers for decades. Recent advances in the genetic underpinnings of reading disability have indicated that the heritable, familial risk for dyslexia is a major risk factor. The present data-driven, classification attempt aims to revisit the possibility of identifying distinct cognitive deficit profiles in a large sample of second to fourth grade reading disabled children. In this sample, we investigated whether genetic and environmental risk factors are able to distinguish between poor reader subtypes. In this profile, we included reading-related measures of phonemic awareness, letter-speech sound processing and rapid naming, known as candidate vulnerability markers associated with dyslexia and familial risk for dyslexia, as well as general cognitive abilities (non-verbal IQ and vocabulary). Clustering was based on a 200 multi-start K-means approach. Results revealed four emerging subtypes of which the first subtype showed no cognitive deficits underlying their poor reading skills (Reading-only impaired poor readers). The other three subtypes shared a core phonological deficit (PA) with a variable and discriminative expression across the other underlying vulnerability markers. More specific, type 2 showed low to poor performance across all reading-related and general cognitive abilities (general poor readers), type 3 showed a specific letter-speech sound mapping deficit next to a PA deficit (PA-LS specific poor readers) and type 4 showed a specific rapid naming deficit complementing their phonological weakness (PA-RAN specific poor readers). The first three poor reader profiles were more characterized by variable environmental risk factor, while the fourth, PA-RAN poor reader subtype showed a significantly strong familial risk for dyslexia. Overall, when we zoom in on the heterogeneous phenomenon of reading disability, unique and distinct cognitive subtypes can be identified, distinguishing between those poor readers more influences by the role of genes and those more influenced by environmental risk factors. Taking into account this diversity of distinct cognitive subtypes, instead of looking at the reading disabled sample as a whole, will help tailor future diagnostic and intervention efforts more specifically to the needs of children with such a specific deficit and risk pattern, as well as providing a more promising way forward for genetic studies of dyslexia.

Out-of-synchrony speech entrainment in developmental dyslexia

Developmental dyslexia is a reading disorder often characterized by reduced awareness of speech units. Whether the neural source of this phonological disorder in dyslexic readers results from the malfunctioning of the primary auditory system or damaged feedback communication between higher-order phonological regions (i.e., left inferior frontal regions) and the auditory cortex is still under dispute. Here we recorded magnetoencephalographic (MEG) signals from 20 dyslexic readers and 20 age-matched controls while they were listening to ∼10-s-long spoken sentences. Compared to controls, dyslexic readers had (1) an impaired neural entrainment to speech in the delta band (0.5-1 Hz); (2) a reduced delta synchronization in both the right auditory cortex and the left inferior frontal gyrus; and (3) an impaired feedforward functional coupling between neural oscillations in the right auditory cortex and the left inferior frontal regions. This shows that during speech listening, individuals with developmental dyslexia present reduced neural synchrony to low-frequency speech oscillations in primary auditory regions that hinders higher-order speech processing steps. The present findings, thus, strengthen proposals assuming that improper low-frequency acoustic entrainment affects speech sampling. This low speech-brain synchronization has the strong potential to cause severe consequences for both phonological and reading skills. Interestingly, the reduced speech-brain synchronization in dyslexic readers compared to normal readers (and its higher-order consequences across the speech processing network) appears preserved through the development from childhood to adulthood. Thus, the evaluation of speech-brain synchronization could possibly serve as a diagnostic tool for early detection of children at risk of dyslexia. Hum Brain Mapp 37:2767-2783, 2016. © 2016 Wiley Periodicals, Inc.

The COGs (context, object, and goals) in multisensory processing

Our understanding of how perception operates in real-world environments has been substantially advanced by studying both multisensory processes and "top-down" control processes influencing sensory processing via activity from higher-order brain areas, such as attention, memory, and expectations. As the two topics have been traditionally studied separately, the mechanisms orchestrating real-world multisensory processing remain unclear. Past work has revealed that the observer's goals gate the influence of many multisensory processes on brain and behavioural responses, whereas some other multisensory processes might occur independently of these goals. Consequently, other forms of top-down control beyond goal dependence are necessary to explain the full range of multisensory effects currently reported at the brain and the cognitive level. These forms of control include sensitivity to stimulus context as well as the detection of matches (or lack thereof) between a multisensory stimulus and categorical attributes of naturalistic objects (e.g. tools, animals). In this review we discuss and integrate the existing findings that demonstrate the importance of such goal-, object- and context-based top-down control over multisensory processing. We then put forward a few principles emerging from this literature review with respect to the mechanisms underlying multisensory processing and discuss their possible broader implications.

ERPs Reveal the Time-Course of Aberrant Visual-Phonological Binding in Developmental Dyslexia

New evidence is accumulating for a deficit in binding visual-orthographic information with the corresponding phonological code in developmental dyslexia. Here, we identify the mechanisms underpinning this deficit using event-related brain potentials (ERPs) in dyslexic and control adult readers performing a letter-matching task. In each trial, a printed letter was presented synchronously with an auditory letter name. Incongruent (mismatched), frequent trials were interleaved with congruent (matched) infrequent target pairs, which participants were asked to report by pressing a button. In critical trials, incongruent letter pairs were mismatched but confusable in terms of their visual or phonological features. Typical readers showed early detection of deviant trials, indicated by larger modulation in the range of the phonological mismatch negativity (PMN) compared with standard trials. This was followed by stronger modulation of the P3b wave for visually confusable deviants and an increased lateralized readiness potential (LRP) for phonological deviants, compared with standards. In contrast, dyslexic readers showed reduced sensitivity to deviancy in the PMN range. Responses to deviants in the P3b range indicated normal letter recognition processes, but the LRP calculation revealed a specific impairment for visual-orthographic information during response selection in dyslexia. In a follow-up experiment using an analogous non-lexical task in the same participants, we found no reading-group differences, indicating a degree of specificity to over-learnt visual-phonological binding. Our findings indicate early insensitivity to visual-phonological binding in developmental dyslexia, coupled with difficulty selecting the correct orthographic code.

A Randomized Controlled Trial on The Beneficial Effects of Training Letter-Speech Sound Integration on Reading Fluency in Children with Dyslexia

A recent account of dyslexia assumes that a failure to develop automated letter-speech sound integration might be responsible for the observed lack of reading fluency. This study uses a pre-test-training-post-test design to evaluate the effects of a training program based on letter-speech sound associations with a special focus on gains in reading fluency. A sample of 44 children with dyslexia and 23 typical readers, aged 8 to 9, was recruited. Children with dyslexia were randomly allocated to either the training program group (n = 23) or a waiting-list control group (n = 21). The training intensively focused on letter-speech sound mapping and consisted of 34 individual sessions of 45 minutes over a five month period. The children with dyslexia showed substantial reading gains for the main word reading and spelling measures after training, improving at a faster rate than typical readers and waiting-list controls. The results are interpreted within the conceptual framework assuming a multisensory integration deficit as the most proximal cause of dysfluent reading in dyslexia.

TRIAL REGISTRATION

ISRCTN register ISRCTN12783279.

Crossmodal deficit in dyslexic children: practice affects the neural timing of letter-speech sound integration

A failure to build solid letter-speech sound associations may contribute to reading impairments in developmental dyslexia. Whether this reduced neural integration of letters and speech sounds changes over time within individual children and how this relates to behavioral gains in reading skills remains unknown. In this research, we examined changes in event-related potential (ERP) measures of letter-speech sound integration over a 6-month period during which 9-year-old dyslexic readers (n = 17) followed a training in letter-speech sound coupling next to their regular reading curriculum. We presented the Dutch spoken vowels /a/ and /o/ as standard and deviant stimuli in one auditory and two audiovisual oddball conditions. In one audiovisual condition (AV0), the letter “a” was presented simultaneously with the vowels, while in the other (AV200) it was preceding vowel onset for 200 ms. Prior to the training (T1), dyslexic readers showed the expected pattern of typical auditory mismatch responses, together with the absence of letter-speech sound effects in a late negativity (LN) window. After the training (T2), our results showed earlier (and enhanced) crossmodal effects in the LN window. Most interestingly, earlier LN latency at T2 was significantly related to higher behavioral accuracy in letter-speech sound coupling. On a more general level, the timing of the earlier mismatch negativity (MMN) in the simultaneous condition (AV0) measured at T1, significantly related to reading fluency at both T1 and T2 as well as with reading gains. Our findings suggest that the reduced neural integration of letters and speech sounds in dyslexic children may show moderate improvement with reading instruction and training and that behavioral improvements relate especially to individual differences in the timing of this neural integration.

The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities

Behavior, perception and cognition are strongly shaped by the synthesis of information across the different sensory modalities. Such multisensory integration often results in performance and perceptual benefits that reflect the additional information conferred by having cues from multiple senses providing redundant or complementary information. The spatial and temporal relationships of these cues provide powerful statistical information about how these cues should be integrated or "bound" in order to create a unified perceptual representation. Much recent work has examined the temporal factors that are integral in multisensory processing, with many focused on the construct of the multisensory temporal binding window - the epoch of time within which stimuli from different modalities is likely to be integrated and perceptually bound. Emerging evidence suggests that this temporal window is altered in a series of neurodevelopmental disorders, including autism, dyslexia and schizophrenia. In addition to their role in sensory processing, these deficits in multisensory temporal function may play an important role in the perceptual and cognitive weaknesses that characterize these clinical disorders. Within this context, focus on improving the acuity of multisensory temporal function may have important implications for the amelioration of the "higher-order" deficits that serve as the defining features of these disorders.

Reduced neural integration of letters and speech sounds in dyslexic children scales with individual differences in reading fluency

The acquisition of letter-speech sound associations is one of the basic requirements for fluent reading acquisition and its failure may contribute to reading difficulties in developmental dyslexia. Here we investigated event-related potential (ERP) measures of letter-speech sound integration in 9-year-old typical and dyslexic readers and specifically test their relation to individual differences in reading fluency. We employed an audiovisual oddball paradigm in typical readers (n = 20), dysfluent (n = 18) and severely dysfluent (n = 18) dyslexic children. In one auditory and two audiovisual conditions the Dutch spoken vowels/a/and/o/were presented as standard and deviant stimuli. In audiovisual blocks, the letter ‘a’ was presented either simultaneously (AV0), or 200 ms before (AV200) vowel sound onset. Across the three children groups, vowel deviancy in auditory blocks elicited comparable mismatch negativity (MMN) and late negativity (LN) responses. In typical readers, both audiovisual conditions (AV0 and AV200) led to enhanced MMN and LN amplitudes. In both dyslexic groups, the audiovisual LN effects were mildly reduced. Most interestingly, individual differences in reading fluency were correlated with MMN latency in the AV0 condition. A further analysis revealed that this effect was driven by a short-lived MMN effect encompassing only the N1 window in severely dysfluent dyslexics versus a longer MMN effect encompassing both the N1 and P2 windows in the other two groups. Our results confirm and extend previous findings in dyslexic children by demonstrating a deficient pattern of letter-speech sound integration depending on the level of reading dysfluency. These findings underscore the importance of considering individual differences across the entire spectrum of reading skills in addition to group differences between typical and dyslexic readers.

Unattentive speech processing is influenced by orthographic knowledge: evidence from mismatch negativity

How far can acquired knowledge such as orthographic knowledge affect pre-existing abilities such as speech perception? This controversial issue was addressed by investigating the automaticity of the influence of orthographic knowledge on speech processing. Many studies demonstrated this influence in active, lexico-semantic speech processing tasks. However, it has never been observed when speech is unattended. Here, the Mismatch Negativity (MMN), an automatic index of experience-dependent auditory traces, was recorded in an unattended oddball paradigm manipulating the orthographic congruency between frequent and deviant spoken riming words. Both orthographically congruent and incongruent deviant words elicited a typical MMN over the fronto-central regions, with a stronger response in the incongruent condition. The finding showed that the orthographic dimension of spoken words influences a physiological marker of speech processing although participants were required not to attend to the auditory input. This provides evidence for an impact of acquiring a written code on speech processing.

Impairments of multisensory integration and cross-sensory learning as pathways to dyslexia

Two sensory systems are intrinsic to learning to read. Written words enter the brain through the visual system and associated sounds through the auditory system. The task before the beginning reader is quite basic. She must learn correspondences between orthographic tokens and phonemic utterances, and she must do this to the point that there is seamless automatic 'connection' between these sensorially distinct units of language. It is self-evident then that learning to read requires formation of cross-sensory associations to the point that deeply encoded multisensory representations are attained. While the majority of individuals manage this task to a high degree of expertise, some struggle to attain even rudimentary capabilities. Why do dyslexic individuals, who learn well in myriad other domains, fail at this particular task? Here, we examine the literature as it pertains to multisensory processing in dyslexia. We find substantial support for multisensory deficits in dyslexia, and make the case that to fully understand its neurological basis, it will be necessary to thoroughly probe the integrity of auditory-visual integration mechanisms.

Reading as functional coordination: not recycling but a novel synthesis

The Functional Coordination approach describes the processes involved in learning to read as a form of procedural learning in which pre-existing skills, mainly from the visual, and auditory domain, are (1) recruited, (2) modified, and (3) coordinated to create the procedures for reading text, which form the basis of subsequent (4) automatization. In this context, we discuss evidence relating to the emerging prevalence of analytic processing in letter perception. We argue that the process of learning to read does not have to lead to a loss of perceptual skill as consequence of a "cultural recycling"; learning to read just leads to a novel synthesis of functions, which are coordinated for reading and then automatized as a package over several years. Developmental dyslexia is explained within this framework as a Functional Coordination Deficit (Lachmann, 2002), since the coordination level is assumed to be most liable to manifest deficiencies. This is because, at this level, the greatest degree of fine tuning of complex functions is required. Thus, developmental dyslexia is not seen as a consequence of a deficient automatization per se, but of automatization of abnormally developed functional coordination.

Event-related potentials to tones show differences between children with multiple risk factors for dyslexia and control children before the onset of formal reading instruction

Multiple risk factors can affect the development of specific reading problems or dyslexia. In addition to the most prevalent and studied risk factor, phonological processing, auditory discrimination problems have also been found in children and adults with reading difficulties. The present study examined 37 children between the ages of 5 and 6, 11 of which had multiple risk factors for developing reading problems. The children participated in a passive oddball EEG experiment with sinusoidal sounds with changes in sound frequency, duration, or intensity. The responses to the standard stimuli showed a negative voltage shift in children at risk for reading problems compared to control children at 107-215 ms in frontocentral areas corresponding to P1 offset and N250 onset. Source analyses showed that the difference originated from the left and right auditory cortices. Additionally, the children at risk for reading problems had a larger late discriminative negativity (LDN) response in amplitude for sound frequency change than the control children. The amplitudes at the P1-N250 time window showed correlations to letter knowledge and phonological identification whereas the amplitudes at the LDN time window correlated with verbal short-term memory and rapid naming. These results support the view that problems in basic auditory processing abilities precede the onset of reading instruction and can act as one of the risk factors for dyslexia.

Orthographic dependency in the neural correlates of reading: evidence from audiovisual integration in English readers

Reading skills are indispensible in modern technological societies. In transparent alphabetic orthographies, such as Dutch, reading skills build on associations between letters and speech sounds (LS pairs). Previously, we showed that the superior temporal cortex (STC) of Dutch readers is sensitive to the congruency of LS pairs. Here, we used functional magnetic resonance imaging to investigate whether a similar congruency sensitivity exists in STC of readers of the more opaque English orthography, where the relation among LS pairs is less reliable. Eighteen subjects passively perceived congruent and incongruent audiovisual pairs of different levels of transparency in English: letters and speech sounds (LS; irregular), letters and letter names (LN; fairly transparent), and numerals and number names (NN; transparent). In STC, we found congruency effects for NN and LN, but no effects in the predicted direction (congruent > incongruent) for LS pairs. These findings contrast with previous results obtained from Dutch readers. These data indicate that, through education, the STC becomes tuned to the congruency of transparent audiovisual pairs, but suggests a different neural processing of irregular mappings. The orthographic dependency of LS integration underscores cross-linguistic differences in the neural basis of reading and potentially has important implications for dyslexia interventions across languages.

How reading acquisition changes children's spoken language network

To examine the influence of age and reading proficiency on the development of the spoken language network, we tested 6- and 9-years-old children listening to native and foreign sentences in a slow event-related fMRI paradigm. We observed a stable organization of the peri-sylvian areas during this time period with a left dominance in the superior temporal sulcus and inferior frontal region. A year of reading instruction was nevertheless sufficient to increase activation in regions involved in phonological representations (posterior superior temporal region) and sentence integration (temporal pole and pars orbitalis). A top-down activation of the left inferior temporal cortex surrounding the visual word form area, was also observed but only in 9year-olds (3years of reading practice) listening to their native language. These results emphasize how a successful cultural practice, reading, slots in the biological constraints of the innate spoken language network.

Cross-modal integration in the brain is related to phonological awareness only in typical readers, not in those with reading difficulty

Fluent reading requires successfully mapping between visual orthographic and auditory phonological representations and is thus an intrinsically cross-modal process, though reading difficulty has often been characterized as a phonological deficit. However, recent evidence suggests that orthographic information influences phonological processing in typical developing (TD) readers, but that this effect may be blunted in those with reading difficulty (RD), suggesting that the core deficit underlying reading difficulties may be a failure to integrate orthographic and phonological information. Twenty-six (13 TD and 13 RD) children between 8 and 13 years of age participated in a functional magnetic resonance imaging (fMRI) experiment designed to assess the role of phonemic awareness in cross-modal processing. Participants completed a rhyme judgment task for word pairs presented unimodally (auditory only) and cross-modally (auditory followed by visual). For typically developing children, correlations between elision and neural activation were found for the cross-modal but not unimodal task, whereas in children with RD, no correlation was found. The results suggest that elision taps both phonemic awareness and cross-modal integration in typically developing readers, and that these processes are decoupled in children with reading difficulty.

Multimodal lexical processing in auditory cortex is literacy skill dependent

Literacy is a uniquely human cross-modal cognitive process wherein visual orthographic representations become associated with auditory phonological representations through experience. Developmental studies provide insight into how experience-dependent changes in brain organization influence phonological processing as a function of literacy. Previous investigations show a synchrony-dependent influence of letter presentation on individual phoneme processing in superior temporal sulcus; others demonstrate recruitment of primary and associative auditory cortex during cross-modal processing. We sought to determine whether brain regions supporting phonological processing of larger lexical units (monosyllabic words) over larger time windows is sensitive to cross-modal information, and whether such effects are literacy dependent. Twenty-two children (age 8-14 years) made rhyming judgments for sequentially presented word and pseudoword pairs presented either unimodally (auditory- or visual-only) or cross-modally (audiovisual). Regression analyses examined the relationship between literacy and congruency effects (overlapping orthography and phonology vs. overlapping phonology-only). We extend previous findings by showing that higher literacy is correlated with greater congruency effects in auditory cortex (i.e., planum temporale) only for cross-modal processing. These skill effects were specific to known words and occurred over a large time window, suggesting that multimodal integration in posterior auditory cortex is critical for fluent reading.

How silent is silent reading? Intracerebral evidence for top-down activation of temporal voice areas during reading

As you might experience it while reading this sentence, silent reading often involves an imagery speech component: we can hear our own "inner voice" pronouncing words mentally. Recent functional magnetic resonance imaging studies have associated that component with increased metabolic activity in the auditory cortex, including voice-selective areas. It remains to be determined, however, whether this activation arises automatically from early bottom-up visual inputs or whether it depends on late top-down control processes modulated by task demands. To answer this question, we collaborated with four epileptic human patients recorded with intracranial electrodes in the auditory cortex for therapeutic purposes, and measured high-frequency (50-150 Hz) "gamma" activity as a proxy of population level spiking activity. Temporal voice-selective areas (TVAs) were identified with an auditory localizer task and monitored as participants viewed words flashed on screen. We compared neural responses depending on whether words were attended or ignored and found a significant increase of neural activity in response to words, strongly enhanced by attention. In one of the patients, we could record that response at 800 ms in TVAs, but also at 700 ms in the primary auditory cortex and at 300 ms in the ventral occipital temporal cortex. Furthermore, single-trial analysis revealed a considerable jitter between activation peaks in visual and auditory cortices. Altogether, our results demonstrate that the multimodal mental experience of reading is in fact a heterogeneous complex of asynchronous neural responses, and that auditory and visual modalities often process distinct temporal frames of our environment at the same time.

Behavioral and ERP evidence of word and pseudoword superiority effects in 7- and 11-year-olds

In groups of 7-year-olds and 11-year-olds, event-related potentials (ERPs) were recorded to briefly presented, masked letter strings that included real word (DARK/PARK), pronounceable pseudoword (DARL/PARL), unpronounceable nonword (RDKA/RPKA), and letter-in-xs (DXXX, PXXX) stimuli in a variant of the Reicher-Wheeler paradigm. Behaviorally, participants decided which of two letters occurred at a given position in each string (here, forced-choice alternatives D and P). Both groups showed evidence of behavioral word (more accurate choices for letters in words than in baseline nonwords or letter-in-xs) and pseudoword (more accurate choices for letters in pseudowords than in baseline nonwords or letter-in-xs) superiority effects. Electrophysiologically, 11-year-olds evidenced superiority effects on P150 and N400 peak amplitude, while 7-year-olds showed effects only on N400 amplitude. These findings suggest that the mechanisms underlying the observed behavioral superiority effects may be lexical in younger children but both sublexical and lexical in older children. These results are consistent with a lengthy developmental time course for automatic sublexical orthographic specialization, extending beyond the age of 11.

Behavioral and ERP evidence of word and pseudoword superiority effects in 7- and 11-year-olds

In groups of 7-year-olds and 11-year-olds, event-related potentials (ERPs) were recorded to briefly presented, masked letter strings that included real word (DARK/PARK), pronounceable pseudoword (DARL/PARL), unpronounceable nonword (RDKA/RPKA), and letter-in-xs (DXXX, PXXX) stimuli in a variant of the Reicher-Wheeler paradigm. Behaviorally, participants decided which of two letters occurred at a given position in each string (here, forced-choice alternatives D and P). Both groups showed evidence of behavioral word (more accurate choices for letters in words than in baseline nonwords or letter-in-xs) and pseudoword (more accurate choices for letters in pseudowords than in baseline nonwords or letter-in-xs) superiority effects. Electrophysiologically, 11-year-olds evidenced superiority effects on P150 and N400 peak amplitude, while 7-year-olds showed effects only on N400 amplitude. These findings suggest that the mechanisms underlying the observed behavioral superiority effects may be lexical in younger children but both sublexical and lexical in older children. These results are consistent with a lengthy developmental time course for automatic sublexical orthographic specialization, extending beyond the age of 11.

The neurophysiological basis of the integration of written and heard syllables in dyslexic adults

OBJECTIVE

Letter-speech sound integration in fluent readers takes place automatically and is dependent on temporal synchrony between letters and sounds. In developmental dyslexia, however, letter-speech sound associations are hard to learn, compromising accurate and fluent reading. We studied the effect of printed text on processing speech sounds in dyslexic and fluent adult readers.

METHODS

Visual stimuli were presented with sequences of spoken syllables including vowel or consonant changes, or changes in syllable intensity, frequency, or vowel duration. As visual material, written syllables or their scrambled images were used. The auditory stimuli were presented either synchronously with the visual stimuli or time delayed. The mismatch negativity (MMN), an index of automatic neural change detection, was recorded.

RESULTS

MMN amplitudes were larger to syllable changes in combination with written syllables than with scrambled images in fluent readers. However, dyslexic readers showed no difference between syllables vs. scrambled image condition. Furthermore, MMNs to consonant and frequency changes peaked later in dyslexic than fluent readers.

CONCLUSIONS

Our results suggest deficient and sluggish audiovisual integration in dyslexic individuals, which is not dependent on the phonological relevance of the deviant type.

SIGNIFICANCE

Unlike previous studies, our study included several different types of syllable changes presented with concurrent print, enabling us to determine in more detail the nature of the audiovisual deficit in dyslexia.

Evidence for the late MMN as a neurophysiological endophenotype for dyslexia

Dyslexia affects 5–10% of school-aged children and is therefore one of the most common learning disorders. Research on auditory event related potentials (AERP), particularly the mismatch negativity (MMN) component, has revealed anomalies in individuals with dyslexia to speech stimuli. Furthermore, candidate genes for this disorder were found through molecular genetic studies. A current challenge for dyslexia research is to understand the interaction between molecular genetics and brain function, and to promote the identification of relevant endophenotypes for dyslexia. The present study examines MMN, a neurophysiological correlate of speech perception, and its potential as an endophenotype for dyslexia in three groups of children. The first group of children was clinically diagnosed with dyslexia, whereas the second group of children was comprised of their siblings who had average reading and spelling skills and were therefore “unaffected” despite having a genetic risk for dyslexia. The third group consisted of control children who were not related to the other groups and were also unaffected. In total, 225 children were included in the study. All children showed clear MMN activity to/da/−/ba/contrasts that could be separated into three distinct MMN components. Whilst the first two MMN components did not differentiate the groups, the late MMN component (300–700 ms) revealed significant group differences. The mean area of the late MMN was attenuated in both the dyslexic children and their unaffected siblings in comparison to the control children. This finding is indicative of analogous alterations of neurophysiological processes in children with dyslexia and those with a genetic risk for dyslexia, without a manifestation of the disorder. The present results therefore further suggest that the late MMN might be a potential endophenotype for dyslexia.

The mismatch negativity (MMN)--a unique window to disturbed central auditory processing in ageing and different clinical conditions

In this article, we review clinical research using the mismatch negativity (MMN), a change-detection response of the brain elicited even in the absence of attention or behavioural task. In these studies, the MMN was usually elicited by employing occasional frequency, duration or speech-sound changes in repetitive background stimulation while the patient was reading or watching videos. It was found that in a large number of different neuropsychiatric, neurological and neurodevelopmental disorders, as well as in normal ageing, the MMN amplitude was attenuated and peak latency prolonged. Besides indexing decreased discrimination accuracy, these effects may also reflect, depending on the specific stimulus paradigm used, decreased sensory-memory duration, abnormal perception or attention control or, most importantly, cognitive decline. In fact, MMN deficiency appears to index cognitive decline irrespective of the specific symptomatologies and aetiologies of the different disorders involved.