Asymetric Event-Related Potential Priming Effects Between English Letters and American Sign Language Fingerspelling Fonts

Letter recognition plays an important role in reading and follows different phases of processing, from early visual feature detection to the access of abstract letter representations. Deaf ASL-English bilinguals experience orthography in two forms: English letters and fingerspelling. However, the neurobiological nature of fingerspelling representations, and the relationship between the two orthographies, remains unexplored. We examined the temporal dynamics of single English letter and ASL fingerspelling font processing in an unmasked priming paradigm with centrally presented targets for 200 ms preceded by 100 ms primes. Event-related brain potentials were recorded while participants performed a probe detection task. Experiment 1 examined English letter-to-letter priming in deaf signers and hearing non-signers. We found that English letter recognition is similar for deaf and hearing readers, extending previous findings with hearing readers to unmasked presentations. Experiment 2 examined priming effects between English letters and ASL fingerspelling fonts in deaf signers only. We found that fingerspelling fonts primed both fingerspelling fonts and English letters, but English letters did not prime fingerspelling fonts, indicating a priming asymmetry between letters and fingerspelling fonts. We also found an N400-like priming effect when the primes were fingerspelling fonts which might reflect strategic access to the lexical names of letters. The studies suggest that deaf ASL-English bilinguals process English letters and ASL fingerspelling differently and that the two systems may have distinct neural representations. However, the fact that fingerspelling fonts can prime English letters suggests that the two orthographies may share abstract representations to some extent.

Fingerspelling and Its Role in Translanguaging

Fingerspelling is a critical component of many sign languages. This manual representation of orthographic code is one key way in which signers engage in translanguaging, drawing from all of their linguistic and semiotic resources to support communication. Translanguaging in bimodal bilinguals is unique because it involves drawing from languages in different modalities, namely a signed language like American Sign Language and a spoken language like English (or its written form). Fingerspelling can be seen as a unique product of the unified linguistic system that translanguaging theories purport, as it blends features of both sign and print. The goals of this paper are twofold: to integrate existing research on fingerspelling in order to characterize it as a cognitive-linguistic phenomenon and to discuss the role of fingerspelling in translanguaging and communication. We will first review and synthesize research from linguistics and cognitive neuroscience to summarize our current understanding of fingerspelling, its production, comprehension, and acquisition. We will then discuss how fingerspelling relates to translanguaging theories and how it can be incorporated into translanguaging practices to support literacy and other communication goals.

Are palm reversals the pronoun reversals of sign language? Evidence from a fingerspelling task

Acquisition of pronominal forms by children with autism spectrum disorder (ASD) continues to garner significant attention due to the unusual ways that such children produce and comprehend them. In particular, pronoun reversal errors (e.g., using the 2nd-person pronoun "you" to refer to oneself) have been noted in the speech of children with ASD since the very first report of the disorder. In more recent years, investigations of the signing of deaf children with ASD have documented a different phenomenon: palm orientation reversals, such that signs typically produced with an outward-facing palm are produced with the palm towards the signer, or vice versa. At the same time, true pronoun reversals have yet to be documented in the signing of deaf children on the autism spectrum. These two curious facts have led us to ask if there is evidence that palm orientation reversals in signed languages and pronoun reversals in spoken languages could be surface manifestations of the same underlying differences present in ASD. In this paper we seek to establish whether there is evidence for such an analogy, by comparing the ages at which the two phenomena appear in both typically-developing (TD) children and those with ASD, the frequency and consistency with which they appear, and their relationships with other linguistic and cognitive skills. Data are presented from a fingerspelling task given to a sample of 17 native-signing children with ASD and 24 native-signing TD children. We conclude that there are provocative parallels between pronoun reversals in spoken languages and palm reversals in signed languages, though more research is needed to definitively answer these questions.

New Perspectives on the Neurobiology of Sign Languages

The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.

Recognition of Fingerspelling Sequences in Polish Sign Language Using Point Clouds Obtained from Depth Images

The paper presents a method for recognizing sequences of static letters of the Polish finger alphabet using the point cloud descriptors: viewpoint feature histogram, eigenvalues-based descriptors, ensemble of shape functions, and global radius-based surface descriptor. Each sequence is understood as quick highly coarticulated motions, and the classification is performed by networks of hidden Markov models trained by transitions between postures corresponding to particular letters. Three kinds of the left-to-right Markov models of the transitions, two networks of the transition models—independent and dependent on a dictionary—as well as various combinations of point cloud descriptors are examined on a publicly available dataset of 4200 executions (registered as depth map sequences) prepared by the authors. The hand shape representation proposed in our method can also be applied for recognition of hand postures in single frames. We confirmed this using a known, challenging American finger alphabet dataset with about 60,000 depth images.

Differences in praxis performance and receptive language during fingerspelling between deaf children with and without autism spectrum disorder

Children with autism spectrum disorder present with a variety of social communication deficits such as atypicalities in social gaze and verbal and non-verbal communication delays as well as perceptuo-motor deficits like motor incoordination and dyspraxia. In this study, we had the unique opportunity to study praxis performance in deaf children with and without autism spectrum disorder in a fingerspelling context using American Sign Language. A total of 11 deaf children with autism spectrum disorder and 11 typically developing deaf children aged between 5 and 14 years completed a fingerspelling task. Children were asked to fingerspell 15 different words shown on an iPad. We coded various praxis errors and fingerspelling time. The deaf children with autism spectrum disorder had greater errors in pace, sequence precision, accuracy, and body part use and also took longer to fingerspell each word. Additionally, the deaf children with autism spectrum disorder had poor receptive language skills and this strongly correlated with their praxis performance and autism severity. These findings extend the evidence for dyspraxia in hearing children with autism spectrum disorder to deaf children with autism spectrum disorder. Poor sign language production in children with autism spectrum disorder may contribute to their poor gestural learning/comprehension and vice versa. Our findings have therapeutic implications for children with autism spectrum disorder when teaching sign language.

Synesthesia for manual alphabet letters and numeral signs in second-language users of signed languages

Many synesthetes experience colors when viewing letters or digits. We document, for the first time, an analogous phenomenon among users of signed languages who showed color synesthesia for fingerspelled letters and signed numerals. Four synesthetes experienced colors when they viewed manual letters and numerals (in two cases, colors were subjectively projected on to the hands). There was a correspondence between the colors experienced for written graphemes and their manual counterparts, suggesting that the development of these two types of synesthesia is interdependent despite the fact that these systems are superficially distinct and rely on different perceptual recognition mechanisms in the brain.

Coarticulation in fluent fingerspelling

In speech, the phenomenon of coarticulation (differentiation of phoneme production depending on the preceding or following phonemes) suggests an organization of movement sequences that is not strictly serial. In the skeletal motor system, however, evidence for comparable fluency has been lacking. Thus the present study was designed to quantify coarticulation in the hand movement sequences of sign language interpreters engaged in fingerspelling. Records of 17 measured joint angles were subjected to discriminant and correlation analyses to determine to what extent and in what manner the hand shape for a particular letter was influenced by the hand shapes for the preceding or the following letters. Substantial evidence of coarticulation was found, revealing both forward and reverse influences across letters. These influences could be further categorized as assimilation (tending to reduce the differences between sequential hand shapes) or dissimilation (tending to emphasize the differences between sequential hand shapes). The proximal interphalangeal (PIP) joints of the index and middle fingers tended to show dissimilation, whereas at the same time (i.e., during the spelling of the same letters) the joints of the wrist and thumb tended to show assimilation. The index and middle finger PIP joints have been shown previously to be among the most important joints for computer recognition of the 26 letter shapes, and therefore the dissimilation may have served to enhance visual discrimination. The simultaneous occurrence of dissimilation in some joints and assimilation in others demonstrates an unprecedented level of parallel control of individual joint rotations in an essentially serial task.