Lexical access versus lexical decision processes for auditory, visual, and audiovisual items: Insights from behavioral and neural measures

Event-related Potential Measures of Visual Word Processing in Monolingual and Bilingual Children and Adults: A Focus on Word Frequency Effects

How does language background influence the neural correlates of visual word recognition in children? To address this question, we used an ERP lexical decision task to examine first-language (L1) and second-language (L2) visual word processing in monolingual and bilingual school-aged children and young adults (n = 123). In particular, we focused on the effects of word frequency (an index of lexical accessibility) on RTs and the N400 ERP component. Behaviorally, we found larger L1 versus L2 word frequency effects among bilingual children, driven by faster and more accurate responses to higher-frequency words (no other language or age group differences were observed). Neurophysiologically, we found larger L1 word frequency effects in bilinguals versus monolinguals (across both age groups), reflected in more negative ERP amplitudes to lower-frequency words. However, the bilingual groups processed L1 and L2 words similarly, despite lower levels of subjective and objective L2 proficiency. Taken together, our findings suggest that divided L1 experience (but not L2 experience) influences the neural correlates of visual word recognition across childhood and adulthood.

Characterizing language production across modalities

ABSTRACTThis study investigates factors influencing lexical access in language production across modalities (signed and oral). Data from deaf and hearing signers were reanalyzed (Baus and Costa, 2015, On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC). Brain Research, 1609(1), 40-53. https://doi.org/10.1016/j.brainres.2015.03.013; Gimeno-Martínez and Baus, 2022, Iconicity in sign language production: Task matters. Neuropsychologia, 167, 108166. https://doi.org/10.1016/j.neuropsychologia.2022.108166) testing the influence of psycholinguistic variables and ERP mean amplitudes on signing and naming latencies. Deaf signers' signing latencies were influenced by sign iconicity in the picture signing task, and by spoken psycholinguistic variables in the word-to-sign translation task. Additionally, ERP amplitudes before response influenced signing but not translation latencies. Hearing signers' latencies, both signing and naming, were influenced by sign iconicity and word frequency, with early ERP amplitudes predicting only naming latencies. These findings highlight general and modality-specific determinants of lexical access in language production.

Sensory modality affects the spatiotemporal dynamics of alpha and theta oscillations associated with prospective memory

BACKGROUND

The maintenance of an intention in memory (Prospective Memory, PM) while performing a task is associated with a cost in terms of both performance (longer response times and lower accuracy) and neurophysiological modulations, which extent depends on several features of the stimuli.

AIM

This study explores the neural patterns associated with PM in different sensory modalities, to identify differences depending on this variable and discuss their functional meaning.

METHOD

Data were collected using a High-Density EEG during a baseline and a PM condition, proposed in a visual and an auditory version. Theta and alpha oscillations were compared between the two conditions within each modality using a cluster-based permutation approach.

RESULTS

PM conditions were associated with clusters of decreased alpha and theta activity in both modalities. However, different spatiotemporal dynamics were elicited as a function of sensory modality: alpha decreases displayed an overlapping onset between modalities, but different durations, lasting longer in the auditory modality. Conversely, the clusters of decreased theta activity presented similar durations between modalities, but different temporal and spatial onsets, appearing at different moments over the respective sensory areas.

CONCLUSIONS

The similar spatiotemporal properties of alpha suppression between modalities indicate that such oscillations may represent a supramodal, top-down process, presumably reflecting the external direction of attention to successfully detect the prospective cue (strategic monitoring). In theta, the clusters showed more modality-specific differences, which temporal and spatial properties correspond to the ones necessary to perform the ongoing task, suggesting a shift in resource allocation in favor of the PM task.

The effect of unisensory and multisensory information on lexical decision and free recall in young and older adults

Studies using simple low-level stimuli show that multisensory stimuli lead to greater improvements in processing speed for older adults than young adults. However, there is insufficient evidence to explain how these benefits influence performance for more complex processes such as judgement and memory tasks. This study examined how presenting stimuli in multiple sensory modalities (audio-visual) instead of one (audio-only or visual-only) may help older adults to improve their memory and cognitive processing compared to young adults. Young and older adults completed lexical decision (real word vs. pseudoword judgement) and word recall tasks, either independently, or in combination (dual-task), with and without perceptual noise. Older adults were better able to remember words when encoding independently. In contrast, young adults were better able to remember words when encoding in combination with lexical decisions. Both young and older adults had better word recall in the audio-visual condition compared with the audio-only condition. The findings indicate significant age differences when dealing with multiple tasks during encoding. Crucially, there is no greater multisensory benefit for older adults compared to young adults in more complex processes, rather multisensory stimuli can be useful in enhancing cognitive performance for both young and older adults.

Neurophysiological Correlates of Reading Difficulties in Elementary School Children

The present study examined the event-related potentials (ERPs) and reading-language skills of elementary school children with and without reading difficulties. Typically developing children showed an N400 effect characterized by significantly larger N400 amplitudes elicited by nonwords than real words. Their meaning processing shown by the N400 systematically differed by lexicality. On the other hand, the N400 effect was absent in children with reading difficultiesExploratory analyses were conducted with the N1 and Late Positive Component. Additionally, the relationships between ERPs and reading-language skills were examined; sight word efficiency and phonemic decoding efficiency accounted for significant variance in the N400 effect.

Tracking Lexical Access and Code-Switching in Multilingual Participants with Different Degrees of Simultaneous Interpretation Expertise

With the worldwide increase in people speaking more than one language, a better understanding of the behavioural and neural mechanisms governing lexical selection, lexical access in multiple languages and code switching has attracted widespread interest from several disciplines. Previous studies documented higher costs when processing a non‐native (L2) than a native (L1) language or when switching from L2 to L1. However, studies on auditory language reception are still scarce and did not take into account the degree of switching experience. Accordingly, in the present study, we combined behavioural and electrophysiological measurements to assess lexical access in L1 and L2 as well as code switching in professional simultaneous interpreters, trainee interpreters, foreign language teachers and Anglistics students, while the participants performed a bilingual auditory lexical decision task. The purpose of this study was to expand the knowledge on code switching in auditory language processing and examine whether the degree of simultaneous interpretation experience might reduce switching costs. As a main result, we revealed that L2 compared to L1 trials, as well as switch compared to non‐switch trials, generally resulted in lower accuracies, longer reaction times and increased N400 amplitudes in all groups of participants. Otherwise, we did not reveal any influence of switching direction and interpretation expertise on N400 parameters. Taken together, these results suggest that a late age of L2 acquisition leads to switching costs, irrespective of proficiency level. Furthermore, we provided first evidence that simultaneous interpretation training does not diminish switching costs, at least when focusing on lexical access.

EXPRESS: The Impact of Sensory Modality on Prospective Memory: Differences between Visual and Auditory Processing

Prospective memory (PM) is the ability to perform an intended action when the appropriate conditions occur. Several features play a role in the successful retrieval of an intention: The activity we are concurrently engaged in, the number of intentions we are maintaining, where our attention is focused (outward vs. to inner states), and how outstanding the trigger of the intention is. Another factor that may play a crucial role is sensory modality: Do auditory and visual stimuli prompt PM processing in the same way? In this study, we explored for the first time the nature of PM for auditory stimuli and the presence of modality-dependent differences in PM processing. To do so, an identical paradigm composed of multiple PM tasks was administered in two versions, one with auditory stimuli and one with visual ones. Each PM task differed for features such as focality, saliency, and number of intentions (factors that are known in literature to modulate the monitoring and maintenance requests of PM) to explore the impact of sensory modality on a broad variety of classical PM tasks. In general, PM processing showed similar patterns between modalities, especially for low demanding prospective instructions. Conversely, substantial differences were found when the prospective load was increased and monitoring requests enhanced, as participants were significantly slower and less accurate with acoustic stimuli. These results represent the first evidence that modality-dependent effects arise in PM processing, especially in its interaction with features such as the difficulty of the task and the increased monitoring load.

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing

Both visual articulatory gestures and orthography provide information on the phonological content of speech. This EEG study investigated the integration between speech and these two visual inputs. A comparison of skilled readers' brain responses elicited by a spoken word presented alone versus synchronously with a static image of a viseme or a grapheme of the spoken word's onset showed that while neither visual input induced audiovisual integration on N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted in P350 time-window and generalized to all electrodes. The finding suggests a strong impact of spelling knowledge on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive value present in natural lip movements but not in static visemes is particularly critical to the contribution of visual articulatory gestures to speech processing.

Lexico-semantic access and audiovisual integration in the aging brain: Insights from mixed-effects regression analyses of event-related potentials

We investigated how aging modulates lexico-semantic processes in the visual (seeing written items), auditory (hearing spoken items) and audiovisual (seeing written items while hearing congruent spoken items) modalities. Participants were young and older adults who performed a delayed lexical decision task (LDT) presented in blocks of visual, auditory, and audiovisual stimuli. Event-related potentials (ERPs) revealed differences between young and older adults despite older adults' ability to identify words and pseudowords as accurately as young adults. The observed differences included more focalized lexico-semantic access in the N400 time window in older relative to young adults, stronger re-instantiation and/or more widespread activity of the lexicality effect at the time of responding, and stronger multimodal integration for older relative to young adults. Our results offer new insights into how functional neural differences in older adults can result in efficient access to lexico-semantic representations across the lifespan.

What Language Disorders Reveal About the Mechanisms of Morphological Processing

We addressed an understudied topic in the literature of language disorders, that is, processing of derivational morphology, a domain which requires integration of semantic and syntactic knowledge. Current psycholinguistic literature suggests that word processing involves morpheme recognition, which occurs immediately upon encountering a complex word. Subsequent processes take place in order to interpret the combination of stem and affix. We investigated the abilities of individuals with agrammatic (PPA-G) and logopenic (PPA-L) variants of primary progressive aphasia (PPA) and individuals with stroke-induced agrammatic aphasia (StrAg) to process pseudowords which violate either the syntactic (word class) rules (*reheavy) or the semantic compatibility (argument structure specifications of the base form) rules (*reswim). To this end, we quantified aspects of word knowledge and explored how the distinct deficits of the populations under investigation affect their performance. Thirty brain-damaged individuals and 10 healthy controls participated in a lexical decision task. We hypothesized that the two agrammatic groups (PPA-G and StrAg) would have difficulties detecting syntactic violations, while no difficulties were expected for PPA-L. Accuracy and Reaction Time (RT) patterns indicated: the PPA-L group made fewer errors but yielded slower RTs compared to the two agrammatic groups which did not differ from one another. Accuracy rates suggest that individuals with PPA-L distinguish *reheavy from *reswim, reflecting access to and differential processing of syntactic vs. semantic violations. In contrast, the two agrammatic groups do not distinguish between *reheavy and *reswim. The lack of difference stems from a particularly impaired performance in detecting syntactic violations, as they were equally unsuccessful at detecting *reheavy and *reswim. Reduced grammatical abilities assessed through language measures are a significant predictor for this performance, suggesting that the "hardware" to process syntactic information is impaired. Therefore, they can only judge violations semantically where both *reheavy and *reswim fail to pass as semantically ill-formed. This finding further suggests that impaired grammatical knowledge can affect word level processing as well. Results are in line with the psycholinguistic literature which postulates the existence of various stages in accessing complex pseudowords, highlighting the contribution of syntactic/grammatical knowledge. Further, it points to the worth of studying impaired language performance for informing normal language processes.

Functional Role of the N400 and P600 in Language-Related ERP Studies with Respect to Semantic Anomalies: An Overview

In this study, the language-related ERP studies relevant to the functional role of the N400 and P600 in semantically anomalous sentences and the underlying reasons which may affect their functions were reviewed. Since their discovery, the N400 and P600 have been the most important language-related ERP components. The N400 has been mostly elicited as a result of processing sentences with lexical and semantic anomalies, but later on, in many studies instead of the expected lexical-semantic N400 effect, semantic anomalies elicited a P600 effect called semantic P600. However, the functional interpretation of these two ERP components has constantly been a matter of debate. Perhaps most notably, it is proposed that it is not just the N400 which is related to semantic anomalies but the P600 can also be reflected as a result of these kinds of anomalies. Reviewing the literature for explaining the functions of the two ERP components, the N400 and the P600, during the processing of semantic anomalies revealed that still there is a need for more research on language processing in order to make the researchers capable of describing the underlying factors influencing them, especially more focused investigations of the functional-anatomical and neurocomputational models may provide a clearer understanding of them. Moreover, any practical theory or model of the N400 and the P600 in language comprehension needs to consider the apparent inconsistencies in the elicitation pattern of the N400 and the P600 in order to successfully capture the full data spectrum.