Working memory, deafness and sign language

Effect of visual presentation format and recall direction on letter span and error patterns in Deaf signing and hearing adults

Deaf signers consistently show shorter memory spans than hearing nonsigners, but the scope and nature of this difference remain unclear. The present study tested whether Deaf signers are biased toward flexible use of visual aspects of linguistic items. Matched samples of adult Deaf signers (N = 33) and hearing nonsigners (N = 32) performed a letter-span task with visual serial presentation, to bias phonological processing, and a simultaneous presentation, to bias visuospatial processing. We also manipulated short-term memory by varying recall direction (forward, backward). Analyses revealed reduced spans for Deaf signers compared with hearing nonsigners, backward compared with forward recall, and sequential compared with simultaneous presentation. Item-level responses indicated that Deaf signers made more errors than hearing nonsigners across three error types. Deaf signers also showed reduced item position binding compared with hearing nonsigners, which indicates differences related to item order and sequencing in tasks with printed, linguistic stimuli. Deaf signers were the only group who demonstrated reduced omission errors when switching from sequential to simultaneous presentation, suggesting flexible processing mechanisms. No group differences were found for a secondary spatial span test, indicating the scope of group differences for ordered information was limited to verbal items. Overall, results are consistent with flexible use of different memory cues in Deaf signers. A core area for future research includes evaluating reduced activation of phonological representations of linguistic items in Deaf signers. These results amplify a novel M³ model approach for evaluating how errors contribute to short-term memory differences in Deaf signers.

Working Memory-Based Cognitive Rehabilitation: Spoken Language of Deaf and Hard-of-Hearing Children

This research examined the effect of the Working Memory-based Cognitive Rehabilitation (WMCR) intervention on the spoken language development of deaf and hard-of-hearing (DHH) children. In this clinical trial study, 28 DHH children aged between 5 and 6 years were selected by random sampling method. The participants were randomly assigned to experimental and control groups. The experimental group participated in the WMCR intervention involving 11 sessions. All participants were assessed pre-and postintervention. Data were collected by the Newsha Development Scale and analyzed through MANCOVA. The results revealed a significant difference between the scores of the receptive and expressive language of the experimental group that were exposed to the WMCR intervention compared with the control group. The receptive and expressive language skills of the experimental group indicated a significant improvement after the intervention. Therefore, the WMCR intervention is an effective method that affects the spoken language skills of DHH children. These findings have critical implications for teachers, parents, and therapists in supporting DHH young children to develop their language skills.

Enhanced biological motion perception in deaf native signers

We conducted two studies to test how deaf signed language users perceive biological motions. We created 18 Biological Motion point-light displays (PLDs) depicting everyday human actions, and 18 Scrambled control PLDs. First, we conducted an online behavioral rating survey, in which deaf and hearing raters identified the biological motion PLDs and rated how easy it was for them to identify the actions. Then, we conducted an EEG study in which Deaf Signers and Hearing Non-Signers watched both the Biological Motion PLDs and the Scrambled PLDs, and we computed the time-frequency responses within the theta, alpha, and beta EEG rhythms. From the behavioral rating task, we show that the deaf raters reported significantly less effort required for identifying the Biological motion PLDs, across all stimuli. The EEG results showed that the Deaf Signers showed theta, mu, and beta differentiation between Scrambled and Biological PLDs earlier and more consistently than Hearing Non-Signers. We conclude that native ASL users exhibit experience-dependent neuroplasticity in the domain of biological human motion perception.

Visuospatial and Tactile Working Memory in Individuals with Congenital Deafness

Studies examining visuospatial working memory (WM) in individuals with congenital deafness have yielded inconsistent results, and tactile WM has rarely been examined. The current study examined WM span tasks in the two modalities among 20 individuals with congenital deafness and 20 participants with typical hearing. The congenital deafness group had longer forward and backward spans than typical hearing participants in a computerized Corsi block-tapping test (Visuospatial Span), whereas no such difference was found in the Tactual Span (tactile WM). In the congenital deafness group, age of sign language acquisition was not correlated with either condition of the visuospatial task, and Tactual and Visuospatial Spans scores were correlated in the backward but not the forward condition. The typical hearing group showed no correlation between the tasks. The findings suggest that early deafness leads to visuospatial but not tactile superiority in WM, specifically with respect to the storage component. More broadly, it appears that deafness-related compensation mechanisms in WM do not affect the other modalities in a uniform manner.

Evidence of an Effect of Gaming Experience on Visuospatial Attention in Deaf but Not in Hearing Individuals

Auditory cortex in congenitally deaf early sign language users reorganizes to support cognitive processing in the visual domain. However, evidence suggests that the potential benefits of this reorganization are largely unrealized. At the same time, there is growing evidence that experience of playing computer and console games improves visual cognition, in particular visuospatial attentional processes. In the present study, we investigated in a group of deaf early signers whether those who reported recently playing computer or console games (deaf gamers) had better visuospatial attentional control than those who reported not playing such games (deaf non-gamers), and whether any such effect was related to cognitive processing in the visual domain. Using a classic test of attentional control, the Eriksen Flanker task, we found that deaf gamers performed on a par with hearing controls, while the performance of deaf non-gamers was poorer. Among hearing controls there was no effect of gaming. This suggests that deaf gamers may have better visuospatial attentional control than deaf non-gamers, probably because they are less susceptible to parafoveal distractions. Future work should examine the robustness of this potential gaming benefit and whether it is associated with neural plasticity in early deaf signers, as well as whether gaming intervention can improve visuospatial cognition in deaf people.

Working Memory for Linguistic and Non-linguistic Manual Gestures: Evidence, Theory, and Application

Linguistic manual gestures are the basis of sign languages used by deaf individuals. Working memory and language processing are intimately connected and thus when language is gesture-based, it is important to understand related working memory mechanisms. This article reviews work on working memory for linguistic and non-linguistic manual gestures and discusses theoretical and applied implications. Empirical evidence shows that there are effects of load and stimulus degradation on working memory for manual gestures. These effects are similar to those found for working memory for speech-based language. Further, there are effects of pre-existing linguistic representation that are partially similar across language modalities. But above all, deaf signers score higher than hearing non-signers on an n-back task with sign-based stimuli, irrespective of their semantic and phonological content, but not with non-linguistic manual actions. This pattern may be partially explained by recent findings relating to cross-modal plasticity in deaf individuals. It suggests that in linguistic gesture-based working memory, semantic aspects may outweigh phonological aspects when processing takes place under challenging conditions. The close association between working memory and language development should be taken into account in understanding and alleviating the challenges faced by deaf children growing up with cochlear implants as well as other clinical populations.

The relation between working memory and language comprehension in signers and speakers

This study investigated the relation between linguistic and spatial working memory (WM) resources and language comprehension for signed compared to spoken language. Sign languages are both linguistic and visual-spatial, and therefore provide a unique window on modality-specific versus modality-independent contributions of WM resources to language processing. Deaf users of American Sign Language (ASL), hearing monolingual English speakers, and hearing ASL-English bilinguals completed several spatial and linguistic serial recall tasks. Additionally, their comprehension of spatial and non-spatial information in ASL and spoken English narratives was assessed. Results from the linguistic serial recall tasks revealed that the often reported advantage for speakers on linguistic short-term memory tasks does not extend to complex WM tasks with a serial recall component. For English, linguistic WM predicted retention of non-spatial information, and both linguistic and spatial WM predicted retention of spatial information. For ASL, spatial WM predicted retention of spatial (but not non-spatial) information, and linguistic WM did not predict retention of either spatial or non-spatial information. Overall, our findings argue against strong assumptions of independent domain-specific subsystems for the storage and processing of linguistic and spatial information and furthermore suggest a less important role for serial encoding in signed than spoken language comprehension.

Domain-Generality of Timing-Based Serial Order Processes in Short-Term Memory: New Insights from Musical and Verbal Domains

Several models in the verbal domain of short-term memory (STM) consider a dissociation between item and order processing. This view is supported by data demonstrating that different types of time-based interference have a greater effect on memory for the order of to-be-remembered items than on memory for the items themselves. The present study investigated the domain-generality of the item versus serial order dissociation by comparing the differential effects of time-based interfering tasks, such as rhythmic interference and articulatory suppression, on item and order processing in verbal and musical STM domains. In Experiment 1, participants had to maintain sequences of verbal or musical information in STM, followed by a probe sequence, this under different conditions of interference (no-interference, rhythmic interference, articulatory suppression). They were required to decide whether all items of the probe list matched those of the memory list (item condition) or whether the order of the items in the probe sequence matched the order in the memory list (order condition). In Experiment 2, participants performed a serial order probe recognition task for verbal and musical sequences ensuring sequential maintenance processes, under no-interference or rhythmic interference conditions. For Experiment 1, serial order recognition was not significantly more impacted by interfering tasks than was item recognition, this for both verbal and musical domains. For Experiment 2, we observed selective interference of the rhythmic interference condition on both musical and verbal order STM tasks. Overall, the results suggest a similar and selective sensitivity to time-based interference for serial order STM in verbal and musical domains, but only when the STM tasks ensure sequential maintenance processes.

Detecting Memory Impairment in Deaf People: A New Test of Verbal Learning and Memory in British Sign Language

OBJECTIVE

Most existing tests of memory and verbal learning in adults were created for spoken languages, and are unsuitable for assessing deaf people who rely on signed languages. In response to this need for sign language measures, the British Sign Language Verbal Learning and Memory Test (BSL-VLMT) was developed. It follows the format of the English language Hopkins Verbal Learning Test Revised, using standardized video-presentation with novel stimuli and instructions wholly in British Sign Language, and no English language requirement.

METHOD

Data were collected from 223 cognitively healthy deaf signers aged 50-89 and 12 deaf patients diagnosed with dementia. Normative data percentiles were derived for clinical use, and receiver-operating characteristic curves computed to explore the clinical potential and diagnostic sensitivity and specificity.

RESULTS

The test showed good discrimination between the normative and clinical samples, providing preliminary evidence of clinical utility for identifying learning and memory impairment in older deaf signers with neurodegeneration.

CONCLUSIONS

This innovative video testing approach transforms the ability to accurately detect memory impairments in deaf people and avoids the problems of using interpreters, with international potential for adapting similar tests into other signed languages.

Effects of Hearing Status and Sign Language Use on Working Memory

Deaf individuals have been found to score lower than hearing individuals across a variety of memory tasks involving both verbal and nonverbal stimuli, particularly those requiring retention of serial order. Deaf individuals who are native signers, meanwhile, have been found to score higher on visual-spatial memory tasks than on verbal-sequential tasks and higher on some visual-spatial tasks than hearing nonsigners. However, hearing status and preferred language modality (signed or spoken) frequently are confounded in such studies. That situation is resolved in the present study by including deaf students who use spoken language and sign language interpreting students (hearing signers) as well as deaf signers and hearing nonsigners. Three complex memory span tasks revealed overall advantages for hearing signers and nonsigners over both deaf signers and deaf nonsigners on 2 tasks involving memory for verbal stimuli (letters). There were no differences among the groups on the task involving visual-spatial stimuli. The results are consistent with and extend recent findings concerning the effects of hearing status and language on memory and are discussed in terms of language modality, hearing status, and cognitive abilities among deaf and hearing individuals.

Monitoring Different Phonological Parameters of Sign Language Engages the Same Cortical Language Network but Distinctive Perceptual Ones

The study of signed languages allows the dissociation of sensorimotor and cognitive neural components of the language signal. Here we investigated the neurocognitive processes underlying the monitoring of two phonological parameters of sign languages: handshape and location. Our goal was to determine if brain regions processing sensorimotor characteristics of different phonological parameters of sign languages were also involved in phonological processing, with their activity being modulated by the linguistic content of manual actions. We conducted an fMRI experiment using manual actions varying in phonological structure and semantics: (1) signs of a familiar sign language (British Sign Language), (2) signs of an unfamiliar sign language (Swedish Sign Language), and (3) invented nonsigns that violate the phonological rules of British Sign Language and Swedish Sign Language or consist of nonoccurring combinations of phonological parameters. Three groups of participants were tested: deaf native signers, deaf nonsigners, and hearing nonsigners. Results show that the linguistic processing of different phonological parameters of sign language is independent of the sensorimotor characteristics of the language signal. Handshape and location were processed by different perceptual and task-related brain networks but recruited the same language areas. The semantic content of the stimuli did not influence this process, but phonological structure did, with nonsigns being associated with longer RTs and stronger activations in an action observation network in all participants and in the supramarginal gyrus exclusively in deaf signers. These results suggest higher processing demands for stimuli that contravene the phonological rules of a signed language, independently of previous knowledge of signed languages. We suggest that the phonological characteristics of a language may arise as a consequence of more efficient neural processing for its perception and production.

Load and distinctness interact in working memory for lexical manual gestures

The Ease of Language Understanding model (Rönnberg et al., 2013) predicts that decreasing the distinctness of language stimuli increases working memory load; in the speech domain this notion is supported by empirical evidence. Our aim was to determine whether such an over-additive interaction can be generalized to sign processing in sign-naïve individuals and whether it is modulated by experience of computer gaming. Twenty young adults with no knowledge of sign language performed an n-back working memory task based on manual gestures lexicalized in sign language; the visual resolution of the signs and working memory load were manipulated. Performance was poorer when load was high and resolution was low. These two effects interacted over-additively, demonstrating that reducing the resolution of signed stimuli increases working memory load when there is no pre-existing semantic representation. This suggests that load and distinctness are handled by a shared amodal mechanism which can be revealed empirically when stimuli are degraded and load is high, even without pre-existing semantic representation. There was some evidence that the mechanism is influenced by computer gaming experience. Future work should explore how the shared mechanism is influenced by pre-existing semantic representation and sensory factors together with computer gaming experience.

Cross-modal activation of auditory regions during visuo-spatial working memory in early deafness

Early deafness can reshape deprived auditory regions to enable the processing of signals from the remaining intact sensory modalities. Cross-modal activation has been observed in auditory regions during non-auditory tasks in early deaf subjects. In hearing subjects, visual working memory can evoke activation of the visual cortex, which further contributes to behavioural performance. In early deaf subjects, however, whether and how auditory regions participate in visual working memory remains unclear. We hypothesized that auditory regions may be involved in visual working memory processing and activation of auditory regions may contribute to the superior behavioural performance of early deaf subjects. In this study, 41 early deaf subjects (22 females and 19 males, age range: 20-26 years, age of onset of deafness < 2 years) and 40 age- and gender-matched hearing controls underwent functional magnetic resonance imaging during a visuo-spatial delayed recognition task that consisted of encoding, maintenance and recognition stages. The early deaf subjects exhibited faster reaction times on the spatial working memory task than did the hearing controls. Compared with hearing controls, deaf subjects exhibited increased activation in the superior temporal gyrus bilaterally during the recognition stage. This increased activation amplitude predicted faster and more accurate working memory performance in deaf subjects. Deaf subjects also had increased activation in the superior temporal gyrus bilaterally during the maintenance stage and in the right superior temporal gyrus during the encoding stage. These increased activation amplitude also predicted faster reaction times on the spatial working memory task in deaf subjects. These findings suggest that cross-modal plasticity occurs in auditory association areas in early deaf subjects. These areas are involved in visuo-spatial working memory. Furthermore, amplitudes of cross-modal activation during the maintenance stage were positively correlated with the age of onset of hearing aid use and were negatively correlated with the percentage of lifetime hearing aid use in deaf subjects. These findings suggest that earlier and longer hearing aid use may inhibit cross-modal reorganization in early deaf subjects. Granger causality analysis revealed that, compared to the hearing controls, the deaf subjects had an enhanced net causal flow from the frontal eye field to the superior temporal gyrus. These findings indicate that a top-down mechanism may better account for the cross-modal activation of auditory regions in early deaf subjects.See MacSweeney and Cardin (doi:10/1093/awv197) for a scientific commentary on this article.

Making sense of an unexpected detrimental effect of sign language use in a visual task

What supports deaf signers advantage over nonsigners on visuospatial short-term memory (STM) tasks is still a matter of debate. We compared the performance of 18 deaf Italian Sign Language (LIS) users with that of a matched group of Italian hearing nonsigners in three different tasks: two versions of the Corsi Block test, namely span forward and span backward, and the Visual Pattern Test (VPT). Although the Corsi forward and backward are dynamic and mainly involve a spatial component, the VPT is static and taps primarily the visual component of STM. Signers significantly outperformed nonsigners on both versions of the Corsi Block test, whereas they performed significantly worse on the VPT. We suggest that the source of the different pattern lies in the static nature of the VPT versus the dynamic nature of the Corsi spans.

The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances

Working memory is important for online language processing during conversation. We use it to maintain relevant information, to inhibit or ignore irrelevant information, and to attend to conversation selectively. Working memory helps us to keep track of and actively participate in conversation, including taking turns and following the gist. This paper examines the Ease of Language Understanding model (i.e., the ELU model, Rönnberg, 2003; Rönnberg et al., 2008) in light of new behavioral and neural findings concerning the role of working memory capacity (WMC) in uni-modal and bimodal language processing. The new ELU model is a meaning prediction system that depends on phonological and semantic interactions in rapid implicit and slower explicit processing mechanisms that both depend on WMC albeit in different ways. It is based on findings that address the relationship between WMC and (a) early attention processes in listening to speech, (b) signal processing in hearing aids and its effects on short-term memory, (c) inhibition of speech maskers and its effect on episodic long-term memory, (d) the effects of hearing impairment on episodic and semantic long-term memory, and finally, (e) listening effort. New predictions and clinical implications are outlined. Comparisons with other WMC and speech perception models are made.

The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances

Working memory is important for online language processing during conversation. We use it to maintain relevant information, to inhibit or ignore irrelevant information, and to attend to conversation selectively. Working memory helps us to keep track of and actively participate in conversation, including taking turns and following the gist. This paper examines the Ease of Language Understanding model (i.e., the ELU model, Rönnberg, 2003; Rönnberg et al., 2008) in light of new behavioral and neural findings concerning the role of working memory capacity (WMC) in uni-modal and bimodal language processing. The new ELU model is a meaning prediction system that depends on phonological and semantic interactions in rapid implicit and slower explicit processing mechanisms that both depend on WMC albeit in different ways. It is based on findings that address the relationship between WMC and (a) early attention processes in listening to speech, (b) signal processing in hearing aids and its effects on short-term memory, (c) inhibition of speech maskers and its effect on episodic long-term memory, (d) the effects of hearing impairment on episodic and semantic long-term memory, and finally, (e) listening effort. New predictions and clinical implications are outlined. Comparisons with other WMC and speech perception models are made.

Developmental neuroplasticity after cochlear implantation

Cortical development is dependent on stimulus-driven learning. The absence of sensory input from birth, as occurs in congenital deafness, affects normal growth and connectivity needed to form a functional sensory system, resulting in deficits in oral language learning. Cochlear implants bypass cochlear damage by directly stimulating the auditory nerve and brain, making it possible to avoid many of the deleterious effects of sensory deprivation. Congenitally deaf animals and children who receive implants provide a platform to examine the characteristics of cortical plasticity in the auditory system. In this review, we discuss the existence of time limits for, and mechanistic constraints on, sensitive periods for cochlear implantation and describe the effects of multimodal and cognitive reorganization that result from long-term auditory deprivation.

Early experience of Cued Speech enhances speechreading performance in deaf

It is known that deaf individuals usually outperform normal hearing subjects in speechreading; however, the underlying reasons remain unclear. In the present study, speechreading performance was assessed in normal hearing participants (NH), deaf participants who had been exposed to the Cued Speech (CS) system early and intensively, and deaf participants exposed to oral language without Cued Speech (NCS). Results show a gradation in performance with highest performance in CS, then in NCS, and finally NH participants. Moreover, error analysis suggests that speechreading processing is more accurate in the CS group than in the other groups. Given that early and intensive CS has been shown to promote development of accurate phonological processing, we propose that the higher speechreading results in Cued Speech users are linked to a better capacity in phonological decoding of visual articulators.

Short-term memory stages in sign vs. speech: the source of the serial span discrepancy

Speakers generally outperform signers when asked to recall a list of unrelated verbal items. This phenomenon is well established, but its source has remained unclear. In this study, we evaluate the relative contribution of the three main processing stages of short-term memory--perception, encoding, and recall--in this effect. The present study factorially manipulates whether American Sign Language (ASL) or English is used for perception, memory encoding, and recall in hearing ASL-English bilinguals. Results indicate that using ASL during both perception and encoding contributes to the serial span discrepancy. Interestingly, performing recall in ASL slightly increased span, ruling out the view that signing is in general a poor choice for short-term memory. These results suggest that despite the general equivalence of sign and speech in other memory domains, speech-based representations are better suited for the specific task of perception and memory encoding of a series of unrelated verbal items in serial order through the phonological loop. This work suggests that interpretation of performance on serial recall tasks in English may not translate straightforwardly to serial tasks in sign language.

Looking for an explanation for the low sign span. Is order involved?

Although signed and speech-based languages have a similar internal organization of verbal short-term memory, sign span is lower than word span. We investigated whether this is due to the fact that signs are not suited for serial recall, as proposed by Bavelier, Newport, Hall, Supalla, and Boutla (2008. Ordered short-term memory differs in signers and speakers: Implications for models of short-term memory. Cognition, 107, 433-459). We administered a serial recall task with stimuli in Italian Sign Language to 12 deaf people, and we compared their performance with that of twelve age-, gender-, and education-matched hearing participants who performed the task in Italian. The results do not offer evidence for the hypothesis that serial order per se is a detrimental factor for deaf participants. An alternative explanation for the lower sign span based on signs being phonologically heavier than words is considered.

Effects of age on the temporal organization of working memory in deaf signers

Deaf native signers have a general working memory (WM) capacity similar to that of hearing non-signers but are less sensitive to the temporal order of stored items at retrieval. General WM capacity declines with age, but little is known of how cognitive aging affects WM function in deaf signers. We investigated WM function in elderly deaf signers (EDS) and an age-matched comparison group of hearing non-signers (EHN) using a paradigm designed to highlight differences in temporal and spatial processing of item and order information. EDS performed worse than EHN on both item and order recognition using a temporal style of presentation. Reanalysis together with earlier data showed that with the temporal style of presentation, order recognition performance for EDS was also lower than for young adult deaf signers. Older participants responded more slowly than younger participants. These findings suggest that apart from age-related slowing irrespective of sensory and language status, there is an age-related difference specific to deaf signers in the ability to retain order information in WM when temporal processing demands are high. This may be due to neural reorganisation arising from sign language use. Concurrent spatial information with the Mixed style of presentation resulted in enhanced order processing for all groups, suggesting that concurrent temporal and spatial cues may enhance learning for both deaf and hearing groups. These findings support and extend the WM model for Ease of Language Understanding.

Is there an alternative cerebral network associated with enhanced phonological processing in deaf speech-users? An exceptional case

Most people born deaf and exposed to oral language show scant evidence of sensitivity to the phonology of speech when processing written language. In this respect they differ from hearing people. However, occasionally, a prelingually deaf person can achieve good processing of written language in terms of phonological sensitivity and awareness, and in this respect appears exceptional. We report the pattern of event-related fMRI activation in such a deaf reader while performing a rhyme-judgment on written words with similar spelling endings that do not provide rhyme clues. The left inferior frontal gyrus pars opercularis and the left inferior parietal lobe showed greater activation for this task than for a letter-string identity matching task. This participant was special in this regard, showing significantly greater activation in these regions than a group of hearing participants with a similar level of phonological and reading skill. In addition, SR showed activation in the left mid-fusiform gyrus; a region which did not show task-specific activation in the other respondents. The pattern of activation in this exceptional deaf reader was also unique compared with three deaf readers who showed limited phonological processing. We discuss the possibility that this pattern of activation may be critical in relation to phonological decoding of the written word in good deaf readers whose phonological reading skills are indistinguishable from those of hearing readers.