Neural correlates of working memory for sign language

Ordered short-term memory differs in signers and speakers: implications for models of short-term memory

Capacity limits in linguistic short-term memory (STM) are typically measured with forward span tasks in which participants are asked to recall lists of words in the order presented. Using such tasks, native signers of American Sign Language (ASL) exhibit smaller spans than native speakers ([Boutla, M., Supalla, T., Newport, E. L., & Bavelier, D. (2004). Short-term memory span: Insights from sign language. Nature Neuroscience, 7(9), 997-1002]). Here, we test the hypothesis that this population difference reflects differences in the way speakers and signers maintain temporal order information in short-term memory. We show that native signers differ from speakers on measures of short-term memory that require maintenance of temporal order of the tested materials, but not on those in which temporal order is not required. In addition, we show that, in a recall task with free order, bilingual subjects are more likely to recall in temporal order when using English than ASL. We conclude that speakers and signers do share common short-term memory processes. However, whereas short-term memory for spoken English is predominantly organized in terms of temporal order, we argue that this dimension does not play as great a role in signers' short-term memory. Other factors that may affect STM processes in signers are discussed.

Encoding, rehearsal, and recall in signers and speakers: shared network but differential engagement

Short-term memory (STM), or the ability to hold verbal information in mind for a few seconds, is known to rely on the integrity of a frontoparietal network of areas. Here, we used functional magnetic resonance imaging to ask whether a similar network is engaged when verbal information is conveyed through a visuospatial language, American Sign Language, rather than speech. Deaf native signers and hearing native English speakers performed a verbal recall task, where they had to first encode a list of letters in memory, maintain it for a few seconds, and finally recall it in the order presented. The frontoparietal network described to mediate STM in speakers was also observed in signers, with its recruitment appearing independent of the modality of the language. This finding supports the view that signed and spoken STM rely on similar mechanisms. However, deaf signers and hearing speakers differentially engaged key structures of the frontoparietal network as the stages of STM unfold. In particular, deaf signers relied to a greater extent than hearing speakers on passive memory storage areas during encoding and maintenance, but on executive process areas during recall. This work opens new avenues for understanding similarities and differences in STM performance in signers and speakers.

Cerebellar information processing and visuospatial functions

Although there are consistent reports of altered visuospatial abilities in subjects with cerebellar pathologies, and although experimental evidence indicates the importance of this part of the brain in spatial processing, the role of the cerebellum in this area remains elusive. In the present essay, experimental and clinical studies from our group, focussing relations between cerebellum and visuospatial functions are reviewed. Explorative behaviour, visuospatial abilities and sequential spatial processing functions are analyzed to focus cerebellar involvement in spatial data processing. Reviewed evidence enlightens the importance of the cerebellum for scanning sensory data to extract relevant spatial information and for the acquisition of spatial-related procedures. This hypothesis is discussed within the general framework of cerebellar involvement in cognition.

Neural representation of binding lexical signs and words in the episodic buffer of working memory

The episodic buffer accommodates formation and maintenance of unitary multidimensional representations based on information in different codes from different sources. Formation, based on submorphemic units, engages posterior brain regions, while maintenance engages frontal regions. Using a hybrid fMRI design, that allows separate analysis of transient and sustained components, an n-back task and an experimental group of 13 hearing native signers, with experience of Swedish Sign Language and Swedish since birth, we investigated binding of lexical signs and words in working memory. Results show that the transient component of these functions is supported by a buffer-specific network of posterior regions including the right middle temporal lobe, possibly relating to binding of phonological loop representations with semantic representations in long-term memory, as well as a loop-specific network, in line with predictions of a functional relationship between loop and buffer. The left hippocampus was engaged in transient and sustained components of buffer processing, possibly reflecting the meaningful nature of the stimuli. Only a minor role was found for executive functions in line with other recent work. A novel representation of the sustained component of working memory for audiovisual language in the right inferior temporal lobe may be related to perception of speech-related facial gestures. Previous findings of sign and speech loop representation in working memory were replicated and extended. Together, these findings support the notion of a module that mediates between codes and sources, such as the episodic buffer, and further our understanding of its nature.

Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing

The advent of functional neuroimaging has allowed tremendous advances in our understanding of brain-language relationships, in addition to generating substantial empirical data on this subject in the form of thousands of activation peak coordinates reported in a decade of language studies. We performed a large-scale meta-analysis of this literature, aimed at defining the composition of the phonological, semantic, and sentence processing networks in the frontal, temporal, and inferior parietal regions of the left cerebral hemisphere. For each of these language components, activation peaks issued from relevant component-specific contrasts were submitted to a spatial clustering algorithm, which gathered activation peaks on the basis of their relative distance in the MNI space. From a sample of 730 activation peaks extracted from 129 scientific reports selected among 260, we isolated 30 activation clusters, defining the functional fields constituting three distributed networks of frontal and temporal areas and revealing the functional organization of the left hemisphere for language. The functional role of each activation cluster is discussed based on the nature of the tasks in which it was involved. This meta-analysis sheds light on several contemporary issues, notably on the fine-scale functional architecture of the inferior frontal gyrus for phonological and semantic processing, the evidence for an elementary audio-motor loop involved in both comprehension and production of syllables including the primary auditory areas and the motor mouth area, evidence of areas of overlap between phonological and semantic processing, in particular at the location of the selective human voice area that was the seat of partial overlap of the three language components, the evidence of a cortical area in the pars opercularis of the inferior frontal gyrus dedicated to syntactic processing and in the posterior part of the superior temporal gyrus a region selectively activated by sentence and text processing, and the hypothesis that different working memory perception-actions loops are identifiable for the different language components. These results argue for large-scale architecture networks rather than modular organization of language in the left hemisphere.

Cerebellar activation patterns in deaf participants for perception of sign language and written text

To address the influence of language presentation on cerebellar activation, hearing volunteers were compared with deaf individuals using functional magnetic resonance imaging during observation of German sign language and text reading. In deaf participants, German sign language revealed activation of the auditory and language-related areas including the medial temporal gyrus (BA 21) and Crus I. In normal hearing participants not understanding German sign language, Crus I was less activated during observation of German sign language but strongly activated during text reading and accompanied by activation of the medial temporal gyrus. Our results indicate that cerebellar activation in Crus I corresponds to language perception as correlated with cortical activation in language-related areas and is not dependent on the mode of language presentation.