The hands and mouth do not always slip together in British sign language: dissociating articulatory channels in the lexicon

Sociolinguistic Variation in Mouthings in British Sign Language: A Corpus-Based Study

Mouth activity forms a key component of all sign languages. This can be divided into mouthings, which originate from words in the ambient spoken language, and mouth gestures, which do not. This study examines the relationship between the distribution of mouthings co-occurring with verb signs in British Sign Language (BSL) and various linguistic and social factors, using the BSL Corpus. We find considerable variation between participants and a lack of homogeneity in mouth actions with particular signs. This accords with previous theories that mouthings constitute code-blending between spoken and signed languages-similar to code-switching or code-mixing in spoken languages-rather than being a phonologically or lexically compulsory part of the sign. We also find a strong association between production of plain verbs (which are body-anchored and cannot be modified spatially) and increased mouthing. In addition, we observe significant effects of region (signers from the south of the United Kingdom mouth more than those from the north), gender (women mouth more than men), and age (signers aged 16-35 years produce fewer mouthings than older participants). We find no significant effect of language background (deaf vs. hearing family). Based on these findings, we argue that the multimodal, multilingual, and simultaneous nature of code-blending in sign languages fits well within the paradigm of translanguaging. We discuss implications of this for concepts of translanguaging, code-switching, code-mixing, and related phenomena, highlighting the need to consider not just modality and linguistic codes but also sequential versus simultaneous patterning.

Ongoing Sign Processing Facilitates Written Word Recognition in Deaf Native Signing Children

Signed and written languages are intimately related in proficient signing readers. Here, we tested whether deaf native signing beginning readers are able to make rapid use of ongoing sign language to facilitate recognition of written words. Deaf native signing children (mean 10 years, 7 months) received prime target pairs with sign word onsets as primes and written words as targets. In a control group of hearing children (matched in their reading abilities to the deaf children, mean 8 years, 8 months), spoken word onsets were instead used as primes. Targets (written German words) either were completions of the German signs or of the spoken word onsets. Task of the participants was to decide whether the target word was a possible German word. Sign onsets facilitated processing of written targets in deaf children similarly to spoken word onsets facilitating processing of written targets in hearing children. In both groups, priming elicited similar effects in the simultaneously recorded event related potentials (ERPs), starting as early as 200 ms after the onset of the written target. These results suggest that beginning readers can use ongoing lexical processing in their native language - be it signed or spoken - to facilitate written word recognition. We conclude that intimate interactions between sign and written language might in turn facilitate reading acquisition in deaf beginning readers.

New Insights Into Mouthings: Evidence From a Corpus-Based Study of Russian Sign Language

While some aspects of mouthings have been previously investigated, many topics in the use of this cross-modal contact phenomenon in sign languages remain un(der)studied, and not much is known about mouthings in Russian Sign Language (RSL), in particular. This article examines various aspects of mouthings as these are used by native RSL signers and aims to contribute new insights into the use and origin of mouthings in this sign language. Based on novel data from the online RSL Corpus alongside additional elicited data, we describe the distribution, forms, functions and spreading patterns of mouthings. Our findings furthermore show that sign languages exhibit more extensive variation in the use of mouthings than has previously been thought. Moreover, we – thus far uniquely – describe mouthings also as a written-language-based contact phenomenon. This study has the potential to provide a better understanding of the nature of such contact-induced features as mouthings in sign languages in general and reveals a complex interplay of the modalities of signed, spoken and written languages.

Psycholinguistic norms for more than 300 lexical signs in German Sign Language (DGS)

Sign language offers a unique perspective on the human faculty of language by illustrating that linguistic abilities are not bound to speech and writing. In studies of spoken and written language processing, lexical variables such as, for example, age of acquisition have been found to play an important role, but such information is not as yet available for German Sign Language (Deutsche Gebärdensprache, DGS). Here, we present a set of norms for frequency, age of acquisition, and iconicity for more than 300 lexical DGS signs, derived from subjective ratings by 32 deaf signers. We also provide additional norms for iconicity and transparency for the same set of signs derived from ratings by 30 hearing non-signers. In addition to empirical norming data, the dataset includes machine-readable information about a sign's correspondence in German and English, as well as annotations of lexico-semantic and phonological properties: one-handed vs. two-handed, place of articulation, most likely lexical class, animacy, verb type, (potential) homonymy, and potential dialectal variation. Finally, we include information about sign onset and offset for all stimulus clips from automated motion-tracking data. All norms, stimulus clips, data, as well as code used for analysis are made available through the Open Science Framework in the hope that they may prove to be useful to other researchers: https://doi.org/10.17605/OSF.IO/MZ8J4.

Making Sense of the Hands and Mouth: The Role of "Secondary" Cues to Meaning in British Sign Language and English

Successful face-to-face communication involves multiple channels, notably hand gestures in addition to speech for spoken language, and mouth patterns in addition to manual signs for sign language. In four experiments, we assess the extent to which comprehenders of British Sign Language (BSL) and English rely, respectively, on cues from the hands and the mouth in accessing meaning. We created congruent and incongruent combinations of BSL manual signs and mouthings and English speech and gesture by video manipulation and asked participants to carry out a picture-matching task. When participants were instructed to pay attention only to the primary channel, incongruent "secondary" cues still affected performance, showing that these are reliably used for comprehension. When both cues were relevant, the languages diverged: Hand gestures continued to be used in English, but mouth movements did not in BSL. Moreover, non-fluent speakers and signers varied in the use of these cues: Gestures were found to be more important for non-native than native speakers; mouth movements were found to be less important for non-fluent signers. We discuss the results in terms of the information provided by different communicative channels, which combine to provide meaningful information.

Manual and Spoken Cues in French Sign Language's Lexical Access: Evidence From Mouthing in a Sign-Picture Priming Paradigm

Although Sign Languages are gestural languages, the fact remains that some linguistic information can also be conveyed by spoken components as mouthing. Mouthing usually tend to reproduce the more relevant phonetic part of the equivalent spoken word matching with the manual sign. Therefore, one crucial issue in sign language is to understand whether mouthing is part of the signs themselves or not, and to which extent it contributes to the construction of signs meaning. Another question is to know whether mouthing patterns constitute a phonological or a semantic cue in the lexical sign entry. This study aimed to investigate the role of mouthing on the processing of lexical signs in French Sign Language (LSF), according the type of bilingualism (intramodal vs. bimodal). For this purpose, a behavioral sign-picture lexical decision experiment was designed. Intramodal signers (native deaf adults) and Bimodal signers (fluent hearing adults) have to decide as fast as possible whether a picture matched with the sign seen just before. Five experimental conditions in which the pair sign-mouthing were congruent or incongruent were created. Our results showed a strong interference effect when the sign-mouthing matching was incongruent, reflected by higher error rates and lengthened reaction times compared with the congruent condition. This finding suggests that both groups of signers use the available lexical information contained in mouthing during accessing the sign meaning. In addition, deaf intramodal signers were strongly interfered than hearing bimodal signers. Taken together, our data indicate that mouthing is a determining factor in LSF lexical access, specifically in deaf signers.

Let's not forget the role of deafness in sign/speech bilingualism

Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) address the fascinating question of what can be learnt about language, cognition and the brain from the unique group of people who have grown up learning both a signed and a spoken language. The focus of their review is hearing individuals – referred to as hearing bimodal bilinguals. The review presents an excellent overview of research in this field and highlights the unique insights that this population can provide.

On the Conventionalization of Mouth Actions in Australian Sign Language

This study investigates the conventionalization of mouth actions in Australian Sign Language. Signed languages were once thought of as simply manual languages because the hands produce the signs which individually and in groups are the symbolic units most easily equated with the words, phrases and clauses of spoken languages. However, it has long been acknowledged that non-manual activity, such as movements of the body, head and the face play a very important role. In this context, mouth actions that occur while communicating in signed languages have posed a number of questions for linguists: are the silent mouthings of spoken language words simply borrowings from the respective majority community spoken language(s)? Are those mouth actions that are not silent mouthings of spoken words conventionalized linguistic units proper to each signed language, culturally linked semi-conventional gestural units shared by signers with members of the majority speaking community, or even gestures and expressions common to all humans? We use a corpus-based approach to gather evidence of the extent of the use of mouth actions in naturalistic Australian Sign Language-making comparisons with other signed languages where data is available--and the form/meaning pairings that these mouth actions instantiate.

Psycholinguistic, cognitive, and neural implications of bimodal bilingualism

Bimodal bilinguals, fluent in a signed and a spoken language, exhibit a unique form of bilingualism because their two languages access distinct sensory-motor systems for comprehension and production. Differences between unimodal and bimodal bilinguals have implications for how the brain is organized to control, process, and represent two languages. Evidence from code-blending (simultaneous production of a word and a sign) indicates that the production system can access two lexical representations without cost, and the comprehension system must be able to simultaneously integrate lexical information from two languages. Further, evidence of cross-language activation in bimodal bilinguals indicates the necessity of links between languages at the lexical or semantic level. Finally, the bimodal bilingual brain differs from the unimodal bilingual brain with respect to the degree and extent of neural overlap for the two languages, with less overlap for bimodal bilinguals.

With or without semantic mediation: retrieval of lexical representations in sign production

How are lexical representations retrieved during sign production? Similar to spoken languages, lexical representation in sign language must be accessed through semantics when naming pictures. However, it remains an open issue whether lexical representations in sign language can be accessed via routes that bypass semantics when retrieval is elicited by written words. Here we address this issue by exploring under which circumstances sign retrieval is sensitive to semantic context. To this end we replicate in sign language production the cumulative semantic cost: The observation that naming latencies increase monotonically with each additional within-category item that is named in a sequence of pictures. In the experiment reported here, deaf participants signed sequences of pictures or signed sequences of Italian written words using Italian Sign Language. The results showed a cumulative semantic cost in picture naming but, strikingly, not in word naming. This suggests that only picture naming required access to semantics, whereas deaf signers accessed the sign language lexicon directly (i.e., bypassing semantics) when naming written words. The implications of these findings for the architecture of the sign production system are discussed in the context of current models of lexical access in spoken language production.

Syntactic priming in American Sign Language

Psycholinguistic studies of sign language processing provide valuable opportunities to assess whether language phenomena, which are primarily studied in spoken language, are fundamentally shaped by peripheral biology. For example, we know that when given a choice between two syntactically permissible ways to express the same proposition, speakers tend to choose structures that were recently used, a phenomenon known as syntactic priming. Here, we report two experiments testing syntactic priming of a noun phrase construction in American Sign Language (ASL). Experiment 1 shows that second language (L2) signers with normal hearing exhibit syntactic priming in ASL and that priming is stronger when the head noun is repeated between prime and target (the lexical boost effect). Experiment 2 shows that syntactic priming is equally strong among deaf native L1 signers, deaf late L1 learners, and hearing L2 signers. Experiment 2 also tested for, but did not find evidence of, phonological or semantic boosts to syntactic priming in ASL. These results show that despite the profound differences between spoken and signed languages in terms of how they are produced and perceived, the psychological representation of sentence structure (as assessed by syntactic priming) operates similarly in sign and speech.

Bimodal bilingualism and the frequency-lag hypothesis

The frequency-lag hypothesis proposes that bilinguals have slowed lexical retrieval relative to monolinguals and in their nondominant language relative to their dominant language, particularly for low-frequency words. These effects arise because bilinguals divide their language use between 2 languages and use their nondominant language less frequently. We conducted a picture-naming study with hearing American Sign Language (ASL)-English bilinguals (bimodal bilinguals), deaf signers, and English-speaking monolinguals. As predicted by the frequency-lag hypothesis, bimodal bilinguals were slower, less accurate, and exhibited a larger frequency effect when naming pictures in ASL as compared with English (their dominant language) and as compared with deaf signers. For English there was no difference in naming latencies, error rates, or frequency effects for bimodal bilinguals as compared with monolinguals. Neither age of ASL acquisition nor interpreting experience affected the results; picture-naming accuracy and frequency effects were equivalent for deaf signers and English monolinguals. Larger frequency effects in ASL relative to English for bimodal bilinguals suggests that they are affected by a frequency lag in ASL. The absence of a lag for English could reflect the use of mouthing and/or code-blending, which may shield bimodal bilinguals from the lexical slowing observed for spoken language bilinguals in the dominant language.

Brain activations associated with sign production using word and picture inputs in deaf signers

Effective literacy education in deaf students calls for psycholinguistic research revealing the cognitive and neural mechanisms underlying their written language processing. When learning a written language, deaf students are often instructed to sign out printed text. The present fMRI study was intended to reveal the neural substrates associated with word signing by comparing it with picture signing. Native deaf signers were asked to overtly sign in Chinese Sign Language (CSL) common objects indicated with written words or presented as pictures. Except in left inferior frontal gyrus and inferior parietal lobule where word signing elicited greater activation than picture signing, the two tasks engaged a highly overlapping set of brain regions previously implicated in sign production. The results suggest that word signing in the deaf signers relies on meaning activation from printed visual forms, followed by similar production processes from meaning to signs as in picture signing. The present study also documents the basic brain activation pattern for sign production in CSL and supports the notion of a universal core neural network for sign production across different sign languages.