Neural correlates of working memory for sign language

Differentiating Language for Students Who Are Deaf or Hard of Hearing: A Practice-Informed Framework for Auditory and Visual Supports

PURPOSE

Speech-language pathologists (SLPs) serving students who are d/Deaf or hard of hearing (Deaf/hh) and their deaf education counterparts must navigate complexities in language that include modalities that are spoken or signed and proficiency, which is often compromised. This tutorial describes a practice-informed framework that conceptualizes and organizes a continuum of auditory and visual language supports with the aim of informing the practice of the SLP whose training is more inherently focused on spoken language alone, as well as the practice of the teacher of the Deaf/hh (TDHH) who may focus more on visual language supports.

METHOD

This product resulted from a need within interdisciplinary, graduate programs for SLPs and TDHHs. Both cohorts required preparation to address the needs of diverse language learners who are Deaf/hh. This tutorial includes a brief review of the challenges in developing language proficiency and describes the complexities of effective service delivery. The process of developing a practice-informed framework for language supports is summarized, referencing established practices in auditory-based and visually based methodologies, identifying parallel practices, and summarizing the practices within a multitiered framework called the Framework of Differentiated Practices for Language Support. Recommendations for use of the framework include guidance on the identification of a student's language modality/ies and proficiency to effectively match students' needs and target supports.

CONCLUSIONS

An examination of established practices in language supports across auditory and visual modalities reveals clear parallels that can be organized into a tiered framework. The result is a reference for differentiating language for the interdisciplinary school team. The parallel supports also provide evidence of similarities in practice across philosophical boundaries as professionals work collaboratively.

The cognitive hearing science perspective on perceiving, understanding, and remembering language: The ELU model

The review gives an introductory description of the successive development of data patterns based on comparisons between hearing-impaired and normal hearing participants' speech understanding skills, later prompting the formulation of the Ease of Language Understanding (ELU) model. The model builds on the interaction between an input buffer (RAMBPHO, Rapid Automatic Multimodal Binding of PHOnology) and three memory systems: working memory (WM), semantic long-term memory (SLTM), and episodic long-term memory (ELTM). RAMBPHO input may either match or mismatch multimodal SLTM representations. Given a match, lexical access is accomplished rapidly and implicitly within approximately 100-400 ms. Given a mismatch, the prediction is that WM is engaged explicitly to repair the meaning of the input - in interaction with SLTM and ELTM - taking seconds rather than milliseconds. The multimodal and multilevel nature of representations held in WM and LTM are at the center of the review, being integral parts of the prediction and postdiction components of language understanding. Finally, some hypotheses based on a selective use-disuse of memory systems mechanism are described in relation to mild cognitive impairment and dementia. Alternative speech perception and WM models are evaluated, and recent developments and generalisations, ELU model tests, and boundaries are discussed.

Phonological store and speechreading performance of Chinese students with hearing impairment

This study presents three experiments to examine the role of the phonological store component of working memory in the speechreading performance of students with hearing impairment (HI) in China. In Experiment 1, 86 high school students with HI completed an immediate serial recall task with four lists of monosyllabic words that differed in phonological and visual similarities. In Experiment 2 and Experiment 3, 40 participants divided into high or low phonological store capacity (PS) and 40 participants divided into high or low visual phonological story capacity (VPS) completed a speechreading test at the word, phrase and sentence levels. Results revealed that (1) immediate serial recall showed effects of phonological and visual similarity and their interaction; (2) there was no significant effect of phonological store capacities on speechreading; and (3) there was a significant effect of visual phonological store capacities on accuracy but not speed of speechreading. These findings point to a general phonological store system for visual orthographic coding and phonological coding that students with HI engage in speechreading in Chinese. It provides evidence for the contention that the visual-based coding has a more direct impact on speechreading performance of Chinese students with HI than the speech-based coding.

The genetic architecture of language functional connectivity

Language is a unique trait of the human species, of which the genetic architecture remains largely unknown. Through language disorders studies, many candidate genes were identified. However, such complex and multifactorial trait is unlikely to be driven by only few genes and case-control studies, suffering from a lack of power, struggle to uncover significant variants. In parallel, neuroimaging has significantly contributed to the understanding of structural and functional aspects of language in the human brain and the recent availability of large scale cohorts like UK Biobank have made possible to study language via image-derived endophenotypes in the general population. Because of its strong relationship with task-based fMRI (tbfMRI) activations and its easiness of acquisition, resting-state functional MRI (rsfMRI) have been more popularised, making it a good surrogate of functional neuronal processes. Taking advantage of such a synergistic system by aggregating effects across spatially distributed traits, we performed a multivariate genome-wide association study (mvGWAS) between genetic variations and resting-state functional connectivity (FC) of classical brain language areas in the inferior frontal (pars opercularis, triangularis and orbitalis), temporal and inferior parietal lobes (angular and supramarginal gyri), in 32,186 participants from UK Biobank. Twenty genomic loci were found associated with language FCs, out of which three were replicated in an independent replication sample. A locus in 3p11.1, regulating EPHA3 gene expression, is found associated with FCs of the semantic component of the language network, while a locus in 15q14, regulating THBS1 gene expression is found associated with FCs of the perceptual-motor language processing, bringing novel insights into the neurobiology of language.

New Perspectives on the Neurobiology of Sign Languages

The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.

Is Addenbrooke's Cognitive Examination III Sensitive Enough to Detect Cognitive Dysfunctions in Patients with Focal Cerebellar Lesions?

OBJECTIVE

The main aim of the study was to evaluate whether the available brief test of mental functions Addenbrooke's cognitive examination III (ACE III) detects cognitive impairment in patients with cerebellar damage. The second goal was to show the ACE III cognitive impairment profile of patients with focal cerebellar lesions.

METHOD

The study sample consisted of 31 patients with focal cerebellar lesions, 78 patients with supratentorial brain damage, and 31 subjects after spine surgery or with spine degeneration considered as control group, free of organic brain damage. The ACE III was used.

RESULTS

Patients with cerebellar damage obtained significantly lower results in the ACE III total score and in several subscales: attention, fluency, language, and visuospatial domains than healthy controls without brain damage. With the cut-off level of 89 points, the ACE III was characterized by the sensitivity of 71%, specificity of 72%, and accuracy of 72%. The cerebellar cognitive impairment profile was found to be "frontal-like" and similar to that observed in patients with anterior supratentorial brain damage, with decreased ability to retrieve previously learned material and its preserved recognition, impaired word fluency, and executive dysfunction. The results are consistent with cerebellar cognitive affective syndrome.

CONCLUSIONS

The ACE III can be used as a sensitive screening tool to detect cognitive impairments in patients with cerebellar damage.

Cognitive Hearing Science: Three Memory Systems, Two Approaches, and the Ease of Language Understanding Model

Purpose The purpose of this study was to conceptualize the subtle balancing act between language input and prediction (cognitive priming of future input) to achieve understanding of communicated content. When understanding fails, reconstructive postdiction is initiated. Three memory systems play important roles: working memory (WM), episodic long-term memory (ELTM), and semantic long-term memory (SLTM). The axiom of the Ease of Language Understanding (ELU) model is that explicit WM resources are invoked by a mismatch between language input-in the form of rapid automatic multimodal binding of phonology-and multimodal phonological and lexical representations in SLTM. However, if there is a match between rapid automatic multimodal binding of phonology output and SLTM/ELTM representations, language processing continues rapidly and implicitly. Method and Results In our first ELU approach, we focused on experimental manipulations of signal processing in hearing aids and background noise to cause a mismatch with LTM representations; both resulted in increased dependence on WM. Our second-and main approach relevant for this review article-focuses on the relative effects of age-related hearing loss on the three memory systems. According to the ELU, WM is predicted to be frequently occupied with reconstruction of what was actually heard, resulting in a relative disuse of phonological/lexical representations in the ELTM and SLTM systems. The prediction and results do not depend on test modality per se but rather on the particular memory system. This will be further discussed. Conclusions Related to the literature on ELTM decline as precursors of dementia and the fact that the risk for Alzheimer's disease increases substantially over time due to hearing loss, there is a possibility that lowered ELTM due to hearing loss and disuse may be part of the causal chain linking hearing loss and dementia. Future ELU research will focus on this possibility.

The role of the superior parietal lobule in lexical processing of sign language: Insights from fMRI and TMS

There is strong evidence that neuronal bases for language processing are remarkably similar for sign and spoken languages. However, as meanings and linguistic structures of sign languages are coded in movement and space and decoded through vision, differences are also present, predominantly in occipitotemporal and parietal areas, such as superior parietal lobule (SPL). Whether the involvement of SPL reflects domain-general visuospatial attention or processes specific to sign language comprehension remains an open question. Here we conducted two experiments to investigate the role of SPL and the laterality of its engagement in sign language lexical processing. First, using unique longitudinal and between-group designs we mapped brain responses to sign language in hearing late learners and deaf signers. Second, using transcranial magnetic stimulation (TMS) in both groups we tested the behavioural relevance of SPL's engagement and its lateralisation during sign language comprehension. SPL activation in hearing participants was observed in the right hemisphere before and bilaterally after the sign language course. Additionally, after the course hearing learners exhibited greater activation in the occipital cortex and left SPL than deaf signers. TMS applied to the right SPL decreased accuracy in both hearing learners and deaf signers. Stimulation of the left SPL decreased accuracy only in hearing learners. Our results suggest that right SPL might be involved in visuospatial attention while left SPL might support phonological decoding of signs in non-proficient signers.

Multimodal imaging of brain reorganization in hearing late learners of sign language

The neural plasticity underlying language learning is a process rather than a single event. However, the dynamics of training-induced brain reorganization have rarely been examined, especially using a multimodal magnetic resonance imaging approach, which allows us to study the relationship between functional and structural changes. We focus on sign language acquisition in hearing adults who underwent an 8-month long course and five neuroimaging sessions. We assessed what neural changes occurred as participants learned a new language in a different modality-as reflected by task-based activity, connectivity changes, and co-occurring structural alterations. Major changes in the activity pattern appeared after just 3 months of learning, as indicated by increases in activation within the modality-independent perisylvian language network, together with increased activation in modality-dependent parieto-occipital, visuospatial and motion-sensitive regions. Despite further learning, no alterations in activation were detected during the following months. However, enhanced coupling between left-lateralized occipital and inferior frontal regions was observed as the proficiency increased. Furthermore, an increase in gray matter volume was detected in the left inferior frontal gyrus which peaked at the end of learning. Overall, these results showed complexity and temporal distinctiveness of various aspects of brain reorganization associated with learning of new language in different sensory modality.

Professional or Amateur? The Phonological Output Buffer as a Working Memory Operator

The Phonological Output Buffer (POB) is thought to be the stage in language production where phonemes are held in working memory and assembled into words. The neural implementation of the POB remains unclear despite a wealth of phenomenological data. Individuals with POB impairment make phonological errors when they produce words and non-words, including phoneme omissions, insertions, transpositions, substitutions and perseverations. Errors can apply to different kinds and sizes of units, such as phonemes, number words, morphological affixes, and function words, and evidence from POB impairments suggests that units tend to substituted with units of the same kind—e.g., numbers with numbers and whole morphological affixes with other affixes. This suggests that different units are processed and stored in the POB in the same stage, but perhaps separately in different mini-stores. Further, similar impairments can affect the buffer used to produce Sign Language, which raises the question of whether it is instantiated in a distinct device with the same design. However, what appear as separate buffers may be distinct regions in the activity space of a single extended POB network, connected with a lexicon network. The self-consistency of this idea can be assessed by studying an autoassociative Potts network, as a model of memory storage distributed over several cortical areas, and testing whether the network can represent both units of word and signs, reflecting the types and patterns of errors made by individuals with POB impairment.

Working Memory for Linguistic and Non-linguistic Manual Gestures: Evidence, Theory, and Application

Linguistic manual gestures are the basis of sign languages used by deaf individuals. Working memory and language processing are intimately connected and thus when language is gesture-based, it is important to understand related working memory mechanisms. This article reviews work on working memory for linguistic and non-linguistic manual gestures and discusses theoretical and applied implications. Empirical evidence shows that there are effects of load and stimulus degradation on working memory for manual gestures. These effects are similar to those found for working memory for speech-based language. Further, there are effects of pre-existing linguistic representation that are partially similar across language modalities. But above all, deaf signers score higher than hearing non-signers on an n-back task with sign-based stimuli, irrespective of their semantic and phonological content, but not with non-linguistic manual actions. This pattern may be partially explained by recent findings relating to cross-modal plasticity in deaf individuals. It suggests that in linguistic gesture-based working memory, semantic aspects may outweigh phonological aspects when processing takes place under challenging conditions. The close association between working memory and language development should be taken into account in understanding and alleviating the challenges faced by deaf children growing up with cochlear implants as well as other clinical populations.

The phonological loop: Some answers and some questions

Jones, Hughes, and Macken (2007) claim that their data and our own are inconsistent with a multicomponent working-memory model. We explain in greater detail how the model can account for the data and can address their more specific criticisms. Both sides accept that data relating to the presence of a phonological similarity effect throughout the list depend on list length. We accept that, at this point, all explanations of their interaction are speculative and require further empirical investigation. We examine J, H, & M's interpretation of their and our results in terms of an auditory modality effect, observing that their interpretation of this effect is not well supported by the literature. We suggest that their account assumes a very narrow basis for a general theory of short-term retention, in contrast to a phonological loop interpretation, which forms part of a well-developed and articulated model of working memory.

Neural Activity During Mental Rotation in Deaf Signers: The Influence of Long-Term Sign Language Experience

OBJECTIVES

Mental rotation is the brain's visuospatial understanding of what objects are and where they belong. Previous research indicated that deaf signers showed behavioral enhancement for nonlinguistic visual tasks, including mental rotation. In this study, we investigated the neural difference of mental rotation processing between deaf signers and hearing nonsigners using blood oxygen level-dependent (BOLD) functional magnetic resonance imaging (fMRI).

DESIGN

The participants performed a block-designed experiment, consisting of alternating blocks of comparison and rotation periods, separated by a baseline or fixation period. Mental rotation tasks were performed using three-dimensional figures. fMRI images were acquired during the entire experiment, and the fMRI data were analyzed with Analysis of Functional NeuroImages. A factorial design analysis of variance was designed for fMRI analyses. The differences of activation were analyzed for the main effects of group and task, as well as for the interaction of group by task.

RESULTS

The study showed differences in activated areas between deaf signers and hearing nonsigners on the mental rotation of three-dimensional figures. Subtracting activations of fixation from activations of rotation, both groups showed consistent activation in bilateral occipital lobe, bilateral parietal lobe, and bilateral posterior temporal lobe. There were different main effects of task (rotation versus comparison) with significant activation clusters in the bilateral precuneus, the right middle frontal gyrus, the bilateral medial frontal gyrus, the right interior frontal gyrus, the right superior frontal gyrus, the right anterior cingulate, and the bilateral posterior cingulate. There were significant interaction effects of group by task in the bilateral anterior cingulate, the right inferior frontal gyrus, the left superior frontal gyrus, the left posterior cingulate, the left middle temporal gyrus, and the right inferior parietal lobe. In simple effects of deaf and hearing groups with rotation minus comparison, deaf signers mainly showed activity in the right hemisphere, while hearing nonsigners showed bilateral activity. In the simple effects of rotation task, decreased activities were shown for deaf signers compared with hearing nonsigners throughout several regions, including the bilateral parahippocampal gyrus, the left posterior cingulate cortex, the right anterior cingulate cortex, and the right inferior parietal lobe.

CONCLUSION

Decreased activations in several brain regions of deaf signers when compared to hearing nonsigners reflected increased neural efficiency and a precise functional circuitry, which was generated through long-term experience with sign language processing. In addition, we inferred tentatively that there may be a lateralization pattern to the right hemisphere for deaf signers when performing mental rotation tasks.

Phonological memory in sign language relies on the visuomotor neural system outside the left hemisphere language network

Sign language is an essential medium for everyday social interaction for deaf people and plays a critical role in verbal learning. In particular, language development in those people should heavily rely on the verbal short-term memory (STM) via sign language. Most previous studies compared neural activations during signed language processing in deaf signers and those during spoken language processing in hearing speakers. For sign language users, it thus remains unclear how visuospatial inputs are converted into the verbal STM operating in the left-hemisphere language network. Using functional magnetic resonance imaging, the present study investigated neural activation while bilinguals of spoken and signed language were engaged in a sequence memory span task. On each trial, participants viewed a nonsense syllable sequence presented either as written letters or as fingerspelling (4-7 syllables in length) and then held the syllable sequence for 12 s. Behavioral analysis revealed that participants relied on phonological memory while holding verbal information regardless of the type of input modality. At the neural level, this maintenance stage broadly activated the left-hemisphere language network, including the inferior frontal gyrus, supplementary motor area, superior temporal gyrus and inferior parietal lobule, for both letter and fingerspelling conditions. Interestingly, while most participants reported that they relied on phonological memory during maintenance, direct comparisons between letters and fingers revealed strikingly different patterns of neural activation during the same period. Namely, the effortful maintenance of fingerspelling inputs relative to letter inputs activated the left superior parietal lobule and dorsal premotor area, i.e., brain regions known to play a role in visuomotor analysis of hand/arm movements. These findings suggest that the dorsal visuomotor neural system subserves verbal learning via sign language by relaying gestural inputs to the classical left-hemisphere language network.

The relation between working memory and language comprehension in signers and speakers

This study investigated the relation between linguistic and spatial working memory (WM) resources and language comprehension for signed compared to spoken language. Sign languages are both linguistic and visual-spatial, and therefore provide a unique window on modality-specific versus modality-independent contributions of WM resources to language processing. Deaf users of American Sign Language (ASL), hearing monolingual English speakers, and hearing ASL-English bilinguals completed several spatial and linguistic serial recall tasks. Additionally, their comprehension of spatial and non-spatial information in ASL and spoken English narratives was assessed. Results from the linguistic serial recall tasks revealed that the often reported advantage for speakers on linguistic short-term memory tasks does not extend to complex WM tasks with a serial recall component. For English, linguistic WM predicted retention of non-spatial information, and both linguistic and spatial WM predicted retention of spatial information. For ASL, spatial WM predicted retention of spatial (but not non-spatial) information, and linguistic WM did not predict retention of either spatial or non-spatial information. Overall, our findings argue against strong assumptions of independent domain-specific subsystems for the storage and processing of linguistic and spatial information and furthermore suggest a less important role for serial encoding in signed than spoken language comprehension.

Age-Related Declines in Early Sensory Memory: Identification of Rapid Auditory and Visual Stimulus Sequences

Age-related temporal-processing declines of rapidly presented sequences may involve contributions of sensory memory. This study investigated recall for rapidly presented auditory (vowel) and visual (letter) sequences presented at six different stimulus onset asynchronies (SOA) that spanned threshold SOAs for sequence identification. Younger, middle-aged, and older adults participated in all tasks. Results were investigated at both equivalent performance levels (i.e., SOA threshold) and at identical physical stimulus values (i.e., SOAs). For four-item sequences, results demonstrated best performance for the first and last items in the auditory sequences, but only the first item for visual sequences. For two-item sequences, adults identified the second vowel or letter significantly better than the first. Overall, when temporal-order performance was equated for each individual by testing at SOA thresholds, recall accuracy for each position across the age groups was highly similar. These results suggest that modality-specific processing declines of older adults primarily determine temporal-order performance for rapid sequences. However, there is some evidence for a second amodal processing decline in older adults related to early sensory memory for final items in a sequence. This selective deficit was observed particularly for longer sequence lengths and was not accounted for by temporal masking.

Simultaneous EEG-fMRI for working memory of the human brain

Memory plays an important role in human life. Memory can be divided into two categories, i.e., long term memory and short term memory (STM). STM or working memory (WM) stores information for a short span of time and it is used for information manipulations and fast response activities. WM is generally involved in the higher cognitive functions of the brain. Different studies have been carried out by researchers to understand the WM process. Most of these studies were based on neuroimaging modalities like fMRI, EEG, MEG etc., which use standalone processes. Each neuroimaging modality has some pros and cons. For example, EEG gives high temporal resolution but poor spatial resolution. On the other hand, the fMRI results have a high spatial resolution but poor temporal resolution. For a more in depth understanding and insight of what is happening inside the human brain during the WM process or during cognitive tasks, high spatial as well as high temporal resolution is desirable. Over the past decade, researchers have been working to combine different modalities to achieve a high spatial and temporal resolution at the same time. Developments of MRI compatible EEG equipment in recent times have enabled researchers to combine EEG-fMRI successfully. The research publications in simultaneous EEG-fMRI have been increasing tremendously. This review is focused on the WM research involving simultaneous EEG-fMRI data acquisition and analysis. We have covered the simultaneous EEG-fMRI application in WM and data processing. Also, it adds to potential fusion methods which can be used for simultaneous EEG-fMRI for WM and cognitive tasks.

Neural Language Processing in Adolescent First-Language Learners: Longitudinal Case Studies in American Sign Language

One key question in neurolinguistics is the extent to which the neural processing system for language requires linguistic experience during early life to develop fully. We conducted a longitudinal anatomically constrained magnetoencephalography (aMEG) analysis of lexico-semantic processing in 2 deaf adolescents who had no sustained language input until 14 years of age, when they became fully immersed in American Sign Language. After 2 to 3 years of language, the adolescents' neural responses to signed words were highly atypical, localizing mainly to right dorsal frontoparietal regions and often responding more strongly to semantically primed words (Ferjan Ramirez N, Leonard MK, Torres C, Hatrak M, Halgren E, Mayberry RI. 2014. Neural language processing in adolescent first-language learners. Cereb Cortex. 24 (10): 2772-2783). Here, we show that after an additional 15 months of language experience, the adolescents' neural responses remained atypical in terms of polarity. While their responses to less familiar signed words still showed atypical localization patterns, the localization of responses to highly familiar signed words became more concentrated in the left perisylvian language network. Our findings suggest that the timing of language experience affects the organization of neural language processing; however, even in adolescence, language representation in the human brain continues to evolve with experience.

Load and distinctness interact in working memory for lexical manual gestures

The Ease of Language Understanding model (Rönnberg et al., 2013) predicts that decreasing the distinctness of language stimuli increases working memory load; in the speech domain this notion is supported by empirical evidence. Our aim was to determine whether such an over-additive interaction can be generalized to sign processing in sign-naïve individuals and whether it is modulated by experience of computer gaming. Twenty young adults with no knowledge of sign language performed an n-back working memory task based on manual gestures lexicalized in sign language; the visual resolution of the signs and working memory load were manipulated. Performance was poorer when load was high and resolution was low. These two effects interacted over-additively, demonstrating that reducing the resolution of signed stimuli increases working memory load when there is no pre-existing semantic representation. This suggests that load and distinctness are handled by a shared amodal mechanism which can be revealed empirically when stimuli are degraded and load is high, even without pre-existing semantic representation. There was some evidence that the mechanism is influenced by computer gaming experience. Future work should explore how the shared mechanism is influenced by pre-existing semantic representation and sensory factors together with computer gaming experience.

How sensory-motor systems impact the neural organization for language: direct contrasts between spoken and signed language

To investigate the impact of sensory-motor systems on the neural organization for language, we conducted an H₂¹⁵O-PET study of sign and spoken word production (picture-naming) and an fMRI study of sign and audio-visual spoken language comprehension (detection of a semantically anomalous sentence) with hearing bilinguals who are native users of American Sign Language (ASL) and English. Directly contrasting speech and sign production revealed greater activation in bilateral parietal cortex for signing, while speaking resulted in greater activation in bilateral superior temporal cortex (STC) and right frontal cortex, likely reflecting auditory feedback control. Surprisingly, the language production contrast revealed a relative increase in activation in bilateral occipital cortex for speaking. We speculate that greater activation in visual cortex for speaking may actually reflect cortical attenuation when signing, which functions to distinguish self-produced from externally generated visual input. Directly contrasting speech and sign comprehension revealed greater activation in bilateral STC for speech and greater activation in bilateral occipital-temporal cortex for sign. Sign comprehension, like sign production, engaged bilateral parietal cortex to a greater extent than spoken language. We hypothesize that posterior parietal activation in part reflects processing related to spatial classifier constructions in ASL and that anterior parietal activation may reflect covert imitation that functions as a predictive model during sign comprehension. The conjunction analysis for comprehension revealed that both speech and sign bilaterally engaged the inferior frontal gyrus (with more extensive activation on the left) and the superior temporal sulcus, suggesting an invariant bilateral perisylvian language system. We conclude that surface level differences between sign and spoken languages should not be dismissed and are critical for understanding the neurobiology of language.

Individual differences in distractibility: An update and a model

This paper reviews the current literature on individual differences in susceptibility to the effects of background sound on visual-verbal task performance. A large body of evidence suggests that individual differences in working memory capacity (WMC) underpin individual differences in susceptibility to auditory distraction in most tasks and contexts. Specifically, high WMC is associated with a more steadfast locus of attention (thus overruling the call for attention that background noise may evoke) and a more constrained auditory-sensory gating (i.e., less processing of the background sound). The relation between WMC and distractibility is a general framework that may also explain distractibility differences between populations that differ along variables that covary with WMC (such as age, developmental disorders, and personality traits). A neurocognitive task-engagement/distraction trade-off (TEDTOFF) model that summarizes current knowledge is outlined and directions for future research are proposed.

The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances

Working memory is important for online language processing during conversation. We use it to maintain relevant information, to inhibit or ignore irrelevant information, and to attend to conversation selectively. Working memory helps us to keep track of and actively participate in conversation, including taking turns and following the gist. This paper examines the Ease of Language Understanding model (i.e., the ELU model, Rönnberg, 2003; Rönnberg et al., 2008) in light of new behavioral and neural findings concerning the role of working memory capacity (WMC) in uni-modal and bimodal language processing. The new ELU model is a meaning prediction system that depends on phonological and semantic interactions in rapid implicit and slower explicit processing mechanisms that both depend on WMC albeit in different ways. It is based on findings that address the relationship between WMC and (a) early attention processes in listening to speech, (b) signal processing in hearing aids and its effects on short-term memory, (c) inhibition of speech maskers and its effect on episodic long-term memory, (d) the effects of hearing impairment on episodic and semantic long-term memory, and finally, (e) listening effort. New predictions and clinical implications are outlined. Comparisons with other WMC and speech perception models are made.

The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances

Working memory is important for online language processing during conversation. We use it to maintain relevant information, to inhibit or ignore irrelevant information, and to attend to conversation selectively. Working memory helps us to keep track of and actively participate in conversation, including taking turns and following the gist. This paper examines the Ease of Language Understanding model (i.e., the ELU model, Rönnberg, 2003; Rönnberg et al., 2008) in light of new behavioral and neural findings concerning the role of working memory capacity (WMC) in uni-modal and bimodal language processing. The new ELU model is a meaning prediction system that depends on phonological and semantic interactions in rapid implicit and slower explicit processing mechanisms that both depend on WMC albeit in different ways. It is based on findings that address the relationship between WMC and (a) early attention processes in listening to speech, (b) signal processing in hearing aids and its effects on short-term memory, (c) inhibition of speech maskers and its effect on episodic long-term memory, (d) the effects of hearing impairment on episodic and semantic long-term memory, and finally, (e) listening effort. New predictions and clinical implications are outlined. Comparisons with other WMC and speech perception models are made.

Signed language working memory capacity of signed language interpreters and deaf signers

This study investigated the effects of hearing status and age of signed language acquisition on signed language working memory capacity. Professional Auslan (Australian sign language)/English interpreters (hearing native signers and hearing nonnative signers) and deaf Auslan signers (deaf native signers and deaf nonnative signers) completed an Auslan working memory (WM) span task. The results revealed that the hearing signers (i.e., the professional interpreters) significantly outperformed the deaf signers on the Auslan WM span task. However, the results showed no significant differences between the native signers and the nonnative signers in their Auslan working memory capacity. Furthermore, there was no significant interaction between hearing status and age of signed language acquisition. Additionally, the study found no significant differences between the deaf native signers (adults) and the deaf nonnative signers (adults) in their Auslan working memory capacity. The findings are discussed in relation to the participants' memory strategies and their early language experience. The findings present challenges for WM theories.

Levels of processing and language modality specificity in working memory

Neural networks underpinning working memory demonstrate sign language specific components possibly related to differences in temporary storage mechanisms. A processing approach to memory systems suggests that the organisation of memory storage is related to type of memory processing as well. In the present study, we investigated for the first time semantic, phonological and orthographic processing in working memory for sign- and speech-based language. During fMRI we administered a picture-based 2-back working memory task with Semantic, Phonological, Orthographic and Baseline conditions to 11 deaf signers and 20 hearing non-signers. Behavioural data showed poorer and slower performance for both groups in Phonological and Orthographic conditions than in the Semantic condition, in line with depth-of-processing theory. An exclusive masking procedure revealed distinct sign-specific neural networks supporting working memory components at all three levels of processing. The overall pattern of sign-specific activations may reflect a relative intermodality difference in the relationship between phonology and semantics influencing working memory storage and processing.

Routes to short-term memory indexing: lessons from deaf native users of American Sign Language

Models of working memory (WM) have been instrumental in understanding foundational cognitive processes and sources of individual differences. However, current models cannot conclusively explain the consistent group differences between deaf signers and hearing speakers on a number of short-term memory (STM) tasks. Here we take the perspective that these results are not due to a temporal order-processing deficit in deaf individuals, but rather reflect different biases in how different types of memory cues are used to do a given task. We further argue that the main driving force behind the shifts in relative biasing is a consequence of language modality (sign vs. speech) and the processing they afford, and not deafness, per se.

Age of acquisition effects on the functional organization of language in the adult brain

Using functional magnetic resonance imaging (fMRI), we neuroimaged deaf adults as they performed two linguistic tasks with sentences in American Sign Language, grammatical judgment and phonemic-hand judgment. Participants' age-onset of sign language acquisition ranged from birth to 14 years; length of sign language experience was substantial and did not vary in relation to age of acquisition. For both tasks, a more left lateralized pattern of activation was observed, with activity for grammatical judgment being more anterior than that observed for phonemic-hand judgment, which was more posterior by comparison. Age of acquisition was linearly and negatively related to activation levels in anterior language regions and positively related to activation levels in posterior visual regions for both tasks.

Working memory: theories, models, and controversies

I present an account of the origins and development of the multicomponent approach to working memory, making a distinction between the overall theoretical framework, which has remained relatively stable, and the attempts to build more specific models within this framework. I follow this with a brief discussion of alternative models and their relationship to the framework. I conclude with speculations on further developments and a comment on the value of attempting to apply models and theories beyond the laboratory studies on which they are typically based.

Separable developmental trajectories for the abilities to detect auditory amplitude and frequency modulation

Amplitude modulation (AM) and frequency modulation (FM) are inherent components of most natural sounds. The ability to detect these modulations, considered critical for normal auditory and speech perception, improves over the course of development. However, the extent to which the development of AM and FM detection skills follow different trajectories, and therefore can be attributed to the maturation of separate processes, remains unclear. Here we explored the relationship between the developmental trajectories for the detection of sinusoidal AM and FM in a cross-sectional design employing children aged 8-10 and 11-12 years and adults. For FM of tonal carriers, both average performance (mean) and performance consistency (within-listener standard deviation) were adult-like in the 8-10 y/o. In contrast, in the same listeners, average performance for AM of wideband noise carriers was still not adult-like in the 11-12 y/o, though performance consistency was already mature in the 8-10 y/o. Among the children there were no significant correlations for either measure between the degrees of maturity for AM and FM detection. These differences in developmental trajectory between the two modulation cues and between average detection thresholds and performance consistency suggest that at least partially distinct processes may underlie the development of AM and FM detection as well as the abilities to detect modulation and to do so consistently.

Cognitive hearing science: the legacy of Stuart Gatehouse

Stuart Gatehouse was one of the pioneers of cognitive hearing science. The ease of language understanding (ELU) model (Rönnberg) is one example of a cognitive hearing science model where the interplay between memory systems and signal processing is emphasized. The mismatch notion is central to ELU and concerns how phonological information derived from the signal, matches/mismatches phonological representations in lexical and semantic long-term memory (LTM). When signals match, processing is rapid, automatic and implicit, and lexical activation proceeds smoothly. Given a mismatch, lexical activation fails, and working or short-term memory (WM/STM) is assumed to be invoked to engage in explicit repair strategies to disambiguate what was said in the conversation. In a recent study, negative long-term consequences of mismatch were found by means of relating hearing loss to episodic LTM in a sample of old hearing-aid wearers. STM was intact (Rönnberg et al.). Beneficial short-term consequences of a binary masking noise reduction scheme on STM was obtained in 4-talker babble for individuals with high WM capacity, but not in stationary noise backgrounds (Ng et al.). This suggests that individuals high on WM capacity inhibit semantic auditory distraction in 4-talker babble while exploiting the phonological benefits in terms of speech quality provided by binary masking (Wang). Both long-term and short-term mismatch effects, apparent in data sets including behavioral as well as subjective (Rudner et al.) data, need to be taken into account in the design of future hearing instruments.

Short-term memory stages in sign vs. speech: the source of the serial span discrepancy

Speakers generally outperform signers when asked to recall a list of unrelated verbal items. This phenomenon is well established, but its source has remained unclear. In this study, we evaluate the relative contribution of the three main processing stages of short-term memory--perception, encoding, and recall--in this effect. The present study factorially manipulates whether American Sign Language (ASL) or English is used for perception, memory encoding, and recall in hearing ASL-English bilinguals. Results indicate that using ASL during both perception and encoding contributes to the serial span discrepancy. Interestingly, performing recall in ASL slightly increased span, ruling out the view that signing is in general a poor choice for short-term memory. These results suggest that despite the general equivalence of sign and speech in other memory domains, speech-based representations are better suited for the specific task of perception and memory encoding of a series of unrelated verbal items in serial order through the phonological loop. This work suggests that interpretation of performance on serial recall tasks in English may not translate straightforwardly to serial tasks in sign language.

Looking for an explanation for the low sign span. Is order involved?

Although signed and speech-based languages have a similar internal organization of verbal short-term memory, sign span is lower than word span. We investigated whether this is due to the fact that signs are not suited for serial recall, as proposed by Bavelier, Newport, Hall, Supalla, and Boutla (2008. Ordered short-term memory differs in signers and speakers: Implications for models of short-term memory. Cognition, 107, 433-459). We administered a serial recall task with stimuli in Italian Sign Language to 12 deaf people, and we compared their performance with that of twelve age-, gender-, and education-matched hearing participants who performed the task in Italian. The results do not offer evidence for the hypothesis that serial order per se is a detrimental factor for deaf participants. An alternative explanation for the lower sign span based on signs being phonologically heavier than words is considered.

The signal-cognition interface: interactions between degraded auditory signals and cognitive processes

A hearing loss leads to problems with speech perception; this is exacerbated when competing noise is present. The speech signal is recognized by the cognitive system of the listener; noise and distortion tax the cognitive system when interpreting it. The auditory system must interact with the cognitive system for optimal signal decoding. This article discusses this interaction between the signal and cognitive system based on two models: an auditory model describing signal transmission and degeneration due to a hearing loss and a cognitive model for Ease of Language Understanding. The signal distortion depends on the specifics of the hearing impairment and thus differently distorted signals can affect the cognitive system in different ways. Consequently, the severity of a hearing loss may not only depend on the lesion itself but also on the cognitive recourses required to interpret the signal.

Effects of age on the temporal organization of working memory in deaf signers

Deaf native signers have a general working memory (WM) capacity similar to that of hearing non-signers but are less sensitive to the temporal order of stored items at retrieval. General WM capacity declines with age, but little is known of how cognitive aging affects WM function in deaf signers. We investigated WM function in elderly deaf signers (EDS) and an age-matched comparison group of hearing non-signers (EHN) using a paradigm designed to highlight differences in temporal and spatial processing of item and order information. EDS performed worse than EHN on both item and order recognition using a temporal style of presentation. Reanalysis together with earlier data showed that with the temporal style of presentation, order recognition performance for EDS was also lower than for young adult deaf signers. Older participants responded more slowly than younger participants. These findings suggest that apart from age-related slowing irrespective of sensory and language status, there is an age-related difference specific to deaf signers in the ability to retain order information in WM when temporal processing demands are high. This may be due to neural reorganisation arising from sign language use. Concurrent spatial information with the Mixed style of presentation resulted in enhanced order processing for all groups, suggesting that concurrent temporal and spatial cues may enhance learning for both deaf and hearing groups. These findings support and extend the WM model for Ease of Language Understanding.

Dissociable neural circuits for encoding and retrieval of object locations during active navigation in humans

Several cortical and subcortical circuits have been implicated in object location memory and navigation. Uncertainty remains, however, about which neural circuits are involved in the distinct processes of encoding and retrieval during active navigation through three-dimensional space. We used functional magnetic resonance imaging (fMRI) to measure neural responses as participants learned the location of a single target object relative to a small set of landmarks. Following a delay, the target was removed and participants were required to navigate back to its original position. The relative and absolute locations of landmarks and the target object were changed on every trial, so that participants had to learn a novel arrangement for each spatial scene. At encoding, greater activity within the right hippocampus and the parahippocampal gyrus bilaterally predicted more accurate navigation to the hidden target object in the retrieval phase. By contrast, during the retrieval phase, more accurate performance was associated with increased activity in the left hippocampus and the striatum bilaterally. Dividing participants into good and poor navigators, based upon behavioural performance, revealed greater striatal activity in good navigators during retrieval, perhaps reflecting superior procedural learning in these individuals. By contrast, the poor navigators showed stronger left hippocampal activity, suggesting reliance on a less effective verbal or symbolic code by this group. Our findings suggest separate neural substrates for the encoding and retrieval stages of object location memory during active navigation, which are further modulated by participants' overall navigational ability.

Neural organization of linguistic short-term memory is sensory modality-dependent: evidence from signed and spoken language

Despite decades of research, there is still disagreement regarding the nature of the information that is maintained in linguistic short-term memory (STM). Some authors argue for abstract phonological codes, whereas others argue for more general sensory traces. We assess these possibilities by investigating linguistic STM in two distinct sensory-motor modalities, spoken and signed language. Hearing bilingual participants (native in English and American Sign Language) performed equivalent STM tasks in both languages during functional magnetic resonance imaging. Distinct, sensory-specific activations were seen during the maintenance phase of the task for spoken versus signed language. These regions have been previously shown to respond to nonlinguistic sensory stimulation, suggesting that linguistic STM tasks recruit sensory-specific networks. However, maintenance-phase activations common to the two languages were also observed, implying some form of common process. We conclude that linguistic STM involves sensory-dependent neural networks, but suggest that sensory-independent neural networks may also exist.

Phonological mismatch and explicit cognitive processing in a sample of 102 hearing-aid users

Rudner et al (2008) showed that when compression release settings are manipulated in the hearing instruments of Swedish habitual users, the resulting mismatch between the phonological form of the input speech signal and representations stored in long-term memory leads to greater engagement of explicit cognitive processing under taxing listening conditions. The mismatch effect is manifest in significant correlations between performance on cognitive tests and aided-speech-recognition performance in modulated noise and/or with fast compression release settings. This effect is predicted by the ELU model (Rönnberg et al, 2008). In order to test whether the mismatch effect can be generalized across languages, we examined two sets of aided speech recognition data collected from a Danish population where two cognitive tests, reading span and letter monitoring, had been administered. A reanalysis of all three datasets, including 102 participants, demonstrated the mismatch effect. These findings suggest that the effect of phonological mismatch, as predicted by the ELU model (Rönnberg et al, this issue) and tapped by the reading span test, is a stable phenomenon across these two Scandinavian languages.

Explicit processing demands reveal language modality-specific organization of working memory

The working memory model for Ease of Language Understanding (ELU) predicts that processing differences between language modalities emerge when cognitive demands are explicit. This prediction was tested in three working memory experiments with participants who were Deaf Signers (DS), Hearing Signers (HS), or Hearing Nonsigners (HN). Easily nameable pictures were used as stimuli to avoid confounds relating to sensory modality. Performance was largely similar for DS, HS, and HN, suggesting that previously identified intermodal differences may be due to differences in retention of sensory information. When explicit processing demands were high, differences emerged between DS and HN, suggesting that although working memory storage in both groups is sensitive to temporal organization, retrieval is not sensitive to temporal organization in DS. A general effect of semantic similarity was also found. These findings are discussed in relation to the ELU model.