With or without semantic mediation: retrieval of lexical representations in sign production

Characterizing language production across modalities

ABSTRACTThis study investigates factors influencing lexical access in language production across modalities (signed and oral). Data from deaf and hearing signers were reanalyzed (Baus and Costa, 2015, On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC). Brain Research, 1609(1), 40-53. https://doi.org/10.1016/j.brainres.2015.03.013; Gimeno-Martínez and Baus, 2022, Iconicity in sign language production: Task matters. Neuropsychologia, 167, 108166. https://doi.org/10.1016/j.neuropsychologia.2022.108166) testing the influence of psycholinguistic variables and ERP mean amplitudes on signing and naming latencies. Deaf signers' signing latencies were influenced by sign iconicity in the picture signing task, and by spoken psycholinguistic variables in the word-to-sign translation task. Additionally, ERP amplitudes before response influenced signing but not translation latencies. Hearing signers' latencies, both signing and naming, were influenced by sign iconicity and word frequency, with early ERP amplitudes predicting only naming latencies. These findings highlight general and modality-specific determinants of lexical access in language production.

How (and why) does iconicity effect lexical access: An electrophysiological study of American sign language

Prior research has found that iconicity facilitates sign production in picture-naming paradigms and has effects on ERP components. These findings may be explained by two separate hypotheses: (1) a task-specific hypothesis that suggests these effects occur because visual features of the iconic sign form can map onto the visual features of the pictures, and (2) a semantic feature hypothesis that suggests that the retrieval of iconic signs results in greater semantic activation due to the robust representation of sensory-motor semantic features compared to non-iconic signs. To test these two hypotheses, iconic and non-iconic American Sign Language (ASL) signs were elicited from deaf native/early signers using a picture-naming task and an English-to-ASL translation task, while electrophysiological recordings were made. Behavioral facilitation (faster response times) and reduced negativities were observed for iconic signs (both prior to and within the N400 time window), but only in the picture-naming task. No ERP or behavioral differences were found between iconic and non-iconic signs in the translation task. This pattern of results supports the task-specific hypothesis and provides evidence that iconicity only facilitates sign production when the eliciting stimulus and the form of the sign can visually overlap (a picture-sign alignment effect).

Exploring the ventral white matter language network in bimodal and unimodal bilinguals

We used diffusion magnetic resonance imaging tractography to investigate the effect of language modality on the anatomy of the ventral white matter language network by comparing unimodal (Italian/English) and bimodal bilinguals (Italian/Italian Sign Language). We extracted the diffusion tractography measures of the Inferior Longitudinal fasciculus (ILF), Uncinate fasciculus (UF) and Inferior Fronto-Occipital fasciculus (IFOF) and we correlated them with the degree of bilingualism and the individual performance in fluency tasks. For both groups of bilinguals, the microstructural properties of the right ILF were correlated with individual level of proficiency in L2, confirming the involvement of this tract in bilingualism. In addition, we found that the degree of left lateralization of the ILF predicted the performance in semantic fluency in L1. The microstructural properties of the right UF correlated with performance in phonological fluency in L1, only for bimodal bilinguals. Overall, the pattern shows both similarities and differences between the two groups of bilinguals.

New Perspectives on the Neurobiology of Sign Languages

The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.

Cumulative semantic cost without successful naming

Accessing semantic information has negative consequences for successive recovering attempts of similar information. For instance, in the course of picture-naming tasks, the time required to name an object is determined by the total number of items from the same category that have already been named; naming latencies increase proportionally to the total number of semantically related words named previously. This phenomenon is called cumulative semantic cost (or interference). Two picture-naming experiments with children (4–11 years old, 229 participants) investigate whether having successfully named the previous within-category items is a necessary condition for the cumulative semantic cost to appear. We anticipated that younger children would have a larger rate of nonresponses compared with older children, reflecting the fact that younger children have not yet consolidated many lexical representations. Our results confirmed this prediction. Critically, we also observed that cumulative semantic cost was independent of having successfully retrieved previous within-category lexical items. Furthermore, picture trials for which the previous within-category item elicited a nonresponse showed the same amount of cost as those picture trials for which the previous within-category item elicited a correct naming event. Our findings indicate that it is the attempt to retrieve a lexical unit, and not the successful retrieval of a specific lexical unit, that causes semantic cost in picture naming. This cost can be explained by a mechanism of weakening the semantic-to-lexical mappings of semantic coordinate words. The findings are also discussed in the context of retrieval-induced forgetting effects in memory recall research.

Psycholinguistic, cognitive, and neural implications of bimodal bilingualism

Bimodal bilinguals, fluent in a signed and a spoken language, exhibit a unique form of bilingualism because their two languages access distinct sensory-motor systems for comprehension and production. Differences between unimodal and bimodal bilinguals have implications for how the brain is organized to control, process, and represent two languages. Evidence from code-blending (simultaneous production of a word and a sign) indicates that the production system can access two lexical representations without cost, and the comprehension system must be able to simultaneously integrate lexical information from two languages. Further, evidence of cross-language activation in bimodal bilinguals indicates the necessity of links between languages at the lexical or semantic level. Finally, the bimodal bilingual brain differs from the unimodal bilingual brain with respect to the degree and extent of neural overlap for the two languages, with less overlap for bimodal bilinguals.

What can Written-Words Tell us About Lexical Retrieval in Speech Production?

In recent decades, researchers have exploited semantic context effects in picture naming tasks in order to investigate the mechanisms involved in the retrieval of words from the mental lexicon. In the blocked naming paradigm, participants name target pictures that are either blocked or not blocked by semantic category. In the continuous naming task, participants name a sequence of target pictures that are drawn from multiple semantic categories. Semantic context effects in both tasks are a highly reliable phenomenon. The empirical evidence is, however, sparse and inconsistent when the target stimuli are printed-words instead of pictures. In the first part of the present study we review the empirical evidence regarding semantic context effects with written-word stimuli in the blocked and continuous naming tasks. In the second part, we empirically test whether semantic context effects are transferred from picture naming trials to word reading trials, and from word reading trials to picture naming trials. The results indicate a transfer of semantic context effects from picture naming to subsequently read within-category words. There is no transfer of semantic effects from target words that were read to subsequently named within-category pictures. These results replicate previous findings (Navarrete et al., 2010) and are contrary to predictions from a recent theoretical analysis by Belke (2013). The empirical evidence reported in the literature together with the present results, are discussed in relation to current accounts of semantic context effects in speech production.

Evidence of an association between sign language phonological awareness and word reading in deaf and hard-of-hearing children

BACKGROUND AND AIMS

Children with good phonological awareness (PA) are often good word readers. Here, we asked whether Swedish deaf and hard-of-hearing (DHH) children who are more aware of the phonology of Swedish Sign Language, a language with no orthography, are better at reading words in Swedish.

METHODS AND PROCEDURES

We developed the Cross-modal Phonological Awareness Test (C-PhAT) that can be used to assess PA in both Swedish Sign Language (C-PhAT-SSL) and Swedish (C-PhAT-Swed), and investigated how C-PhAT performance was related to word reading as well as linguistic and cognitive skills. We validated C-PhAT-Swed and administered C-PhAT-Swed and C-PhAT-SSL to DHH children who attended Swedish deaf schools with a bilingual curriculum and were at an early stage of reading.

OUTCOMES AND RESULTS

C-PhAT-SSL correlated significantly with word reading for DHH children. They performed poorly on C-PhAT-Swed and their scores did not correlate significantly either with C-PhAT-SSL or word reading, although they did correlate significantly with cognitive measures.

CONCLUSIONS AND IMPLICATIONS

These results provide preliminary evidence that DHH children with good sign language PA are better at reading words and show that measures of spoken language PA in DHH children may be confounded by individual differences in cognitive skills.