The neurophysiological basis of the integration of written and heard syllables in dyslexic adults

Children with developmental dyslexia have equivalent audiovisual speech perception performance but their perceptual weights differ

As reading is inherently a multisensory, audiovisual (AV) process where visual symbols (i.e., letters) are connected to speech sounds, the question has been raised whether individuals with reading difficulties, like children with developmental dyslexia (DD), have broader impairments in multisensory processing. This question has been posed before, yet it remains unanswered due to (a) the complexity and contentious etiology of DD along with (b) lack of consensus on developmentally appropriate AV processing tasks. We created an ecologically valid task for measuring multisensory AV processing by leveraging the natural phenomenon that speech perception improves when listeners are provided visual information from mouth movements (particularly when the auditory signal is degraded). We designed this AV processing task with low cognitive and linguistic demands such that children with and without DD would have equal unimodal (auditory and visual) performance. We then collected data in a group of 135 children (age 6.5-15) with an AV speech perception task to answer the following questions: (1) How do AV speech perception benefits manifest in children, with and without DD? (2) Do children all use the same perceptual weights to create AV speech perception benefits, and (3) what is the role of phonological processing in AV speech perception? We show that children with and without DD have equal AV speech perception benefits on this task, but that children with DD rely less on auditory processing in more difficult listening situations to create these benefits and weigh both incoming information streams differently. Lastly, any reported differences in speech perception in children with DD might be better explained by differences in phonological processing than differences in reading skills. RESEARCH HIGHLIGHTS: Children with versus without developmental dyslexia have equal audiovisual speech perception benefits, regardless of their phonological awareness or reading skills. Children with developmental dyslexia rely less on auditory performance to create audiovisual speech perception benefits. Individual differences in speech perception in children might be better explained by differences in phonological processing than differences in reading skills.

The influence of temporal asynchrony on character-speech integration in Chinese children with and without dyslexia: An ERP study

Dyslexic readers have been reported to show abnormal temporal acuity and multisensory integration deficiency. Here, we investigated the influence of temporal intervals on Chinese character-speech integration in children with and without dyslexia. Visual characters were presented synchronously to the onset of speech sounds (AV0) or before speech sound by 300 ms (AV300). Event-related potentials (ERP) evoked by congruent condition (speech sounds presented with congruent Chinese characters) and by baseline condition (speech sounds presented with Korean characters) were compared. Typically developing (TD) children exhibited congruency effect in AV0 condition, whereas dyslexic children exhibited congruency effect in AV300 condition. Moreover, congruency effect in TD children was due to enhanced neural activation to congruent trials, congruency effect in dyslexic children was contributed by neural suppression for baseline trials. These results suggested that different underlying mechanisms were involved in character-speech integration for typical and dyslexic children.

The Neural Representation of a Repeated Standard Stimulus in Dyslexia

The neural representation of a repeated stimulus is the standard against which a deviant stimulus is measured in the brain, giving rise to the well-known mismatch response. It has been suggested that individuals with dyslexia have poor implicit memory for recently repeated stimuli, such as the train of standards in an oddball paradigm. Here, we examined how the neural representation of a standard emerges over repetitions, asking whether there is less sensitivity to repetition and/or less accrual of "standardness" over successive repetitions in dyslexia. We recorded magnetoencephalography (MEG) as adults with and without dyslexia were passively exposed to speech syllables in a roving-oddball design. We performed time-resolved multivariate decoding of the MEG sensor data to identify the neural signature of standard vs. deviant trials, independent of stimulus differences. This "multivariate mismatch" was equally robust and had a similar time course in the two groups. In both groups, standards generated by as few as two repetitions were distinct from deviants, indicating normal sensitivity to repetition in dyslexia. However, only in the control group did standards become increasingly different from deviants with repetition. These results suggest that many of the mechanisms that give rise to neural adaptation as well as mismatch responses are intact in dyslexia, with the possible exception of a putatively predictive mechanism that successively integrates recent sensory information into feedforward processing.

ERP Correlates of Altered Orthographic-Phonological Processing in Dyslexia

Automatic visual word recognition requires not only well-established phonological and orthographic representations but also efficient audio-visual integration of these representations. One possibility is that in developmental dyslexia, inefficient orthographic processing might underlie poor reading. Alternatively, reading deficit could be due to inefficient phonological processing or inefficient integration of orthographic and phonological information. In this event-related potential study, participants with dyslexia (N = 25) and control readers (N = 27) were presented with pairs of words and pseudowords in an implicit same-different task. The reference-target pairs could be identical, or different in the identity or the position of the letters. To test the orthographic-phonological processing, target stimuli were presented in visual-only and audiovisual conditions. Participants with and without dyslexia processed the reference stimuli similarly; however, group differences emerged in the processing of target stimuli, especially in the audiovisual condition where control readers showed greater N1 responses for words than for pseudowords, but readers with dyslexia did not show such difference. Moreover, after 300 ms lexicality effect exhibited a more focused frontal topographic distribution in readers with dyslexia. Our results suggest that in developmental dyslexia, phonological processing and audiovisual processing deficits are more pronounced than orthographic processing deficits.

Audiovisual processing and selective attention in adult dyslexic readers: An event-related potential study

Developmental dyslexia is a specific reading disability characterized by decoding and spelling deficits that continue into adulthood. Because reading involves both attentional functions and audiovisual (AV) processing, the aim of this study was to explore the associations between the two factors in adult dyslexic readers. Adult non-impaired and dyslexic readers undertook alphabetic and non-alphabetic tasks, each composed of three experimental blocks. Two experimental blocks contained left and right spatial selective attention manipulations, and another block included central presentations of the stimuli. Event-related potential (ERP) and behavioural parameters were collected and analysed, particularly with respect to the N1-P2 ERP complex. The dyslexic readers showed deviant patterns of amplitudes when it came to alphabetic stimuli processing. However, there was no difference between the two groups with regard to the non-alphabetic stimuli. These results imply that adult dyslexic readers allocate altered attentional resources when it comes to the processing of AV alphabetic stimuli.

Speech discrimination in infants at family risk of dyslexia: Group and individual-based analyses

Deficiencies in discriminating and identifying speech sounds have been widely attested in individuals with dyslexia as well as in young children at family risk (FR) of dyslexia. A speech perception deficit has been hypothesized to be causally related to reading and spelling difficulties. So far, however, early speech perception of FR infants has not been assessed at different ages within a single experimental design. Furthermore, a combination of group- and individual-based analyses has not been made. In this cross-sectional study, vowel discrimination of 6-, 8-, and 10-month-old Dutch FR infants and their nonrisk (no-FR) peers was assessed. Infants (N = 196) were tested on a native English /aː/-/eː/ and non-native English /ɛ/-/æ/ contrast using a hybrid visual habituation paradigm. Frequentist analyses were used to interpret group differences. Bayesian hierarchical modeling was used to classify individuals as speech sound discriminators. FR and no-FR infants discriminated the native contrast at all ages. However, individual classification of the no-FR infants suggests improved discrimination with age, but not for the FR infants. No-FR infants discriminated the non-native contrast at 6 and 10 months, but not at 8 months. FR infants did not show evidence of discriminating the contrast at any of the ages, with 0% being classified as discriminators. The group- and individual-based data are complementary and together point toward speech perception differences between the groups. The findings also indicate that conducting individual analyses on hybrid visual habituation outcomes is possible. These outcomes form a fruitful avenue for gaining more understanding of development, group differences, and prospective relationships.

Neuromagnetic speech discrimination responses are associated with reading-related skills in dyslexic and typical readers

Poor neural speech discrimination has been connected to dyslexia, and may represent phonological processing deficits that are hypothesized to be the main cause for reading impairments. Thus far, neural speech discrimination impairments have rarely been investigated in adult dyslexics, and even less by examining sources of neuromagnetic responses. We compared neuromagnetic speech discrimination in dyslexic and typical readers with mismatch fields (MMF) and determined the associations between MMFs and reading-related skills. We expected weak and atypically lateralized MMFs in dyslexic readers, and positive associations between reading-related skills and MMF strength. MMFs were recorded to a repeating pseudoword /ta-ta/ with occasional changes in vowel identity, duration, or syllable frequency from 43 adults, 21 with confirmed dyslexia. Phonetic (vowel and duration) changes elicited left-lateralized MMFs in the auditory cortices. Contrary to our hypothesis, MMF source strengths or lateralization did not differ between groups. However, better verbal working memory was associated with stronger left-hemispheric MMFs to duration changes across groups, and better reading was associated with stronger right-hemispheric late MMFs across speech-sound changes in dyslexic readers. This suggests a link between neural speech processing and reading-related skills, in line with previous work. Furthermore, our findings suggest a right-hemispheric compensatory mechanism for language processing in dyslexia. The results obtained promote the use of MMFs in investigating reading-related brain processes.

Temporal representation impairment in developmental dyslexia for unisensory and multisensory stimuli

Dyslexia has been associated with a problem in visual-audio integration mechanisms. Here, we investigate for the first time the contribution of unisensory cues on multisensory audio and visual integration in 32 dyslexic children by modelling results using the Bayesian approach. Non-linguistic stimuli were used. Children performed a temporal task: they had to report whether the middle of three stimuli was closer in time to the first one or to the last one presented. Children with dyslexia, compared with typical children, exhibited poorer unimodal thresholds, requiring greater temporal distance between items for correct judgements, while multisensory thresholds were well predicted by the Bayesian model. This result suggests that the multisensory deficit in dyslexia is due to impaired audio and visual inputs rather than impaired multisensory processing per se. We also observed that poorer temporal skills correlated with lower reading skills in dyslexic children, suggesting that this temporal capability can be linked to reading abilities.

Multisensory Integration as a Window into Orderly and Disrupted Cognition and Communication

During our everyday lives, we are confronted with a vast amount of information from several sensory modalities. This multisensory information needs to be appropriately integrated for us to effectively engage with and learn from our world. Research carried out over the last half century has provided new insights into the way such multisensory processing improves human performance and perception; the neurophysiological foundations of multisensory function; the time course for its development; how multisensory abilities differ in clinical populations; and, most recently, the links between multisensory processing and cognitive abilities. This review summarizes the extant literature on multisensory function in typical and atypical circumstances, discusses the implications of the work carried out to date for theory and research, and points toward next steps for advancing the field.

Disrupted left fusiform response to print in beginning kindergartners is associated with subsequent reading

Dyslexia is a common neurobiological disorder in which a child fails to acquire typical word reading skills despite adequate opportunity and intelligence. The visual word form area (VWFA) is a region within the left fusiform gyrus that specializes for print over the course of reading acquisition and is often hypoactivated in individuals with dyslexia. It is currently unknown whether atypicalities in this brain region are already present in kindergarten children who will subsequently develop dyslexia. Here, we measured fMRI activation in response to letters and false fonts in bilateral fusiform gyrus in children with and without risk for dyslexia (defined by family history or low scores on assessments of pre-reading skills, such as phonological awareness). We then followed these children longitudinally through the end of second grade to evaluate whether brain activation patterns in kindergarten were related to second-grade reading outcomes. Compared to typical readers who exhibited no risk factors for reading impairment in kindergarten, there was significant hypoactivation to both letters and false-fonts in the left fusiform gyrus in at-risk children who subsequently developed reading impairment, but not in at-risk children who developed typical reading skills. There were no significant differences in letter- or false-font responses in the right fusiform gyrus among the groups. The finding that hypoactivation to print in the VWFA is present in children who subsequently develop reading impairment even prior to the onset of formal reading instruction suggests that atypical responses to print play an early role in the development of reading impairments such as dyslexia.

A Selective Deficit in Phonetic Recalibration by Text in Developmental Dyslexia

Upon hearing an ambiguous speech sound, listeners may adjust their perceptual interpretation of the speech input in accordance with contextual information, like accompanying text or lipread speech (i.e., phonetic recalibration; Bertelson et al., 2003). As developmental dyslexia (DD) has been associated with reduced integration of text and speech sounds, we investigated whether this deficit becomes manifest when text is used to induce this type of audiovisual learning. Adults with DD and normal readers were exposed to ambiguous consonants halfway between /aba/ and /ada/ together with text or lipread speech. After this audiovisual exposure phase, they categorized auditory-only ambiguous test sounds. Results showed that individuals with DD, unlike normal readers, did not use text to recalibrate their phoneme categories, whereas their recalibration by lipread speech was spared. Individuals with DD demonstrated similar deficits when ambiguous vowels (halfway between /wIt/ and /wet/) were recalibrated by text. These findings indicate that DD is related to a specific letter-speech sound association deficit that extends over phoneme classes (vowels and consonants), but – as lipreading was spared – does not extend to a more general audio–visual integration deficit. In particular, these results highlight diminished reading-related audiovisual learning in addition to the commonly reported phonological problems in developmental dyslexia.

A General Audiovisual Temporal Processing Deficit in Adult Readers With Dyslexia

PURPOSE

Because reading is an audiovisual process, reading impairment may reflect an audiovisual processing deficit. The aim of the present study was to test the existence and scope of such a deficit in adult readers with dyslexia.

METHOD

We tested 39 typical readers and 51 adult readers with dyslexia on their sensitivity to the simultaneity of audiovisual speech and nonspeech stimuli, their time window of audiovisual integration for speech (using incongruent /aCa/ syllables), and their audiovisual perception of phonetic categories.

RESULTS

Adult readers with dyslexia showed less sensitivity to audiovisual simultaneity than typical readers for both speech and nonspeech events. We found no differences between readers with dyslexia and typical readers in the temporal window of integration for audiovisual speech or in the audiovisual perception of phonetic categories.

CONCLUSIONS

The results suggest an audiovisual temporal deficit in dyslexia that is not specific to speech-related events. But the differences found for audiovisual temporal sensitivity did not translate into a deficit in audiovisual speech perception. Hence, there seems to be a hiatus between simultaneity judgment and perception, suggesting a multisensory system that uses different mechanisms across tasks. Alternatively, it is possible that the audiovisual deficit in dyslexia is only observable when explicit judgments about audiovisual simultaneity are required.

ERPs Reveal the Time-Course of Aberrant Visual-Phonological Binding in Developmental Dyslexia

New evidence is accumulating for a deficit in binding visual-orthographic information with the corresponding phonological code in developmental dyslexia. Here, we identify the mechanisms underpinning this deficit using event-related brain potentials (ERPs) in dyslexic and control adult readers performing a letter-matching task. In each trial, a printed letter was presented synchronously with an auditory letter name. Incongruent (mismatched), frequent trials were interleaved with congruent (matched) infrequent target pairs, which participants were asked to report by pressing a button. In critical trials, incongruent letter pairs were mismatched but confusable in terms of their visual or phonological features. Typical readers showed early detection of deviant trials, indicated by larger modulation in the range of the phonological mismatch negativity (PMN) compared with standard trials. This was followed by stronger modulation of the P3b wave for visually confusable deviants and an increased lateralized readiness potential (LRP) for phonological deviants, compared with standards. In contrast, dyslexic readers showed reduced sensitivity to deviancy in the PMN range. Responses to deviants in the P3b range indicated normal letter recognition processes, but the LRP calculation revealed a specific impairment for visual-orthographic information during response selection in dyslexia. In a follow-up experiment using an analogous non-lexical task in the same participants, we found no reading-group differences, indicating a degree of specificity to over-learnt visual-phonological binding. Our findings indicate early insensitivity to visual-phonological binding in developmental dyslexia, coupled with difficulty selecting the correct orthographic code.

Reduced neural integration of letters and speech sounds in dyslexic children scales with individual differences in reading fluency

The acquisition of letter-speech sound associations is one of the basic requirements for fluent reading acquisition and its failure may contribute to reading difficulties in developmental dyslexia. Here we investigated event-related potential (ERP) measures of letter-speech sound integration in 9-year-old typical and dyslexic readers and specifically test their relation to individual differences in reading fluency. We employed an audiovisual oddball paradigm in typical readers (n = 20), dysfluent (n = 18) and severely dysfluent (n = 18) dyslexic children. In one auditory and two audiovisual conditions the Dutch spoken vowels/a/and/o/were presented as standard and deviant stimuli. In audiovisual blocks, the letter ‘a’ was presented either simultaneously (AV0), or 200 ms before (AV200) vowel sound onset. Across the three children groups, vowel deviancy in auditory blocks elicited comparable mismatch negativity (MMN) and late negativity (LN) responses. In typical readers, both audiovisual conditions (AV0 and AV200) led to enhanced MMN and LN amplitudes. In both dyslexic groups, the audiovisual LN effects were mildly reduced. Most interestingly, individual differences in reading fluency were correlated with MMN latency in the AV0 condition. A further analysis revealed that this effect was driven by a short-lived MMN effect encompassing only the N1 window in severely dysfluent dyslexics versus a longer MMN effect encompassing both the N1 and P2 windows in the other two groups. Our results confirm and extend previous findings in dyslexic children by demonstrating a deficient pattern of letter-speech sound integration depending on the level of reading dysfluency. These findings underscore the importance of considering individual differences across the entire spectrum of reading skills in addition to group differences between typical and dyslexic readers.

Impairments of multisensory integration and cross-sensory learning as pathways to dyslexia

Two sensory systems are intrinsic to learning to read. Written words enter the brain through the visual system and associated sounds through the auditory system. The task before the beginning reader is quite basic. She must learn correspondences between orthographic tokens and phonemic utterances, and she must do this to the point that there is seamless automatic 'connection' between these sensorially distinct units of language. It is self-evident then that learning to read requires formation of cross-sensory associations to the point that deeply encoded multisensory representations are attained. While the majority of individuals manage this task to a high degree of expertise, some struggle to attain even rudimentary capabilities. Why do dyslexic individuals, who learn well in myriad other domains, fail at this particular task? Here, we examine the literature as it pertains to multisensory processing in dyslexia. We find substantial support for multisensory deficits in dyslexia, and make the case that to fully understand its neurological basis, it will be necessary to thoroughly probe the integrity of auditory-visual integration mechanisms.

Letters in the forest: global precedence effect disappears for letters but not for non-letters under reading-like conditions

Normally skilled reading involves special processing strategies for letters, which are habitually funneled into an abstract letter code. On the basis of previous studies we argue that this habit leads to the preferred usage of an analytic strategy for the processing of letters, while non-letters are preferably processed via a holistic strategy. The well-known global precedence effect (GPE) seems to contradict to this assumption, since, with compound, hierarchical figures, including letter items, faster responses are observed to the global than to the local level of the figure, as well as an asymmetric interference effect from global to local level. We argue that with letters these effects depend on presentation conditions; only when they elicit the processing strategies automatized for reading, an analytic strategy for letters in contrast to non-letters is to be expected. We compared the GPE for letters and non-letters in central viewing, with the global stimulus size close to the functional visual field in whole word reading (6.5° of visual angle) and local stimuli close to the critical size for fluent reading of individual letters (0.5° of visual angle). Under these conditions, the GPE remained robust for non-letters. For letters, however, it disappeared: letters showed no overall response time advantage for the global level and symmetric congruence effects (local-to-global as well as global-to-local interference). We interpret these results as according to the view that reading is based on resident analytic visual processing strategies for letters.

Individual differences in crossmodal brain activity predict arcuate fasciculus connectivity in developing readers

Crossmodal integration of auditory and visual information, such as phonemes and graphemes, is a critical skill for fluent reading. Previous work has demonstrated that white matter connectivity along the arcuate fasciculus (AF) is predicted by reading skill and that crossmodal processing particularly activates the posterior STS (pSTS). However, the relationship between this crossmodal activation and white matter integrity has not been previously reported. We investigated the interrelationship of crossmodal integration, both in terms of behavioral performance and pSTS activity, with AF tract coherence using a rhyme judgment task in a group of 47 children with a range of reading abilities. We demonstrate that both response accuracy and pSTS activity for crossmodal (auditory-visual) rhyme judgments was predictive of fractional anisotropy along the left AF. Unimodal (auditory-only or visual-only) pSTS activity was not significantly related to AF connectivity. Furthermore, activity in other reading-related ROIs did not show the same AV-only AF coherence relationship, and AV pSTS activity was not related to connectivity along other language-related tracts. This study is the first to directly show that crossmodal brain activity is specifically related to connectivity in the AF, supporting its role in phoneme-grapheme integration ability. More generally, this study helps to define an interdependent neural network for reading-related integration.