Non-linear processing of a linear speech stream: The influence of morphological structure on the recognition of spoken Arabic words

Constraining current neuroanatomical models of reading: the view from Arabic

There is a growing interest in imaging understudied orthographies to unravel their neuronal correlates and their implications for existing computational and neuroanatomical models. Here, we review current brain mapping literature about Arabic words. We first offer a succinct description of some unique linguistic features of Arabic that challenge current cognitive models of reading. We then appraise the existing functional neuroimaging studies that investigated written Arabic word processing. Our review revealed that (1) Arabic is still understudied, (2) the most investigated features concerned the effects of vowelling and diglossia in Arabic reading, (3) findings were not always discussed in the light of existing reading models such as the dual route cascaded, the triangle, and the connectionist dual process models, and (4) current evidence is unreliable when it comes to the exact neuronal pathways that sustain Arabic word processing. Overall, despite the fact that Arabic has some unique linguistic features that challenge and ultimately enrich current reading models, the existing functional neuroimaging literature falls short of offering a reliable evidence about brain networks of Arabic reading. We conclude by highlighting the need for more systematic studies of the linguistic features of Arabic to build theoretical and neuroanatomical models that are concurrently specific and general.

The influence of language-specific properties on the role of consonants and vowels in a statistical learning task of an artificial language: A cross-linguistic comparison

The contribution of consonants and vowels in spoken word processing has been widely investigated, and studies have found a phenomenon of a Consonantal bias (C-bias), indicating that consonants carry more weight than vowels. However, across languages, various patterns have been documented, including that of no preference or a reverse pattern of Vowel bias. A central question is how the manifestation of the C-bias is modulated by language-specific factors. This question can be addressed by cross-linguistic studies. Comparing native Hebrew and native English speakers, this study examines the relative importance of transitional probabilities between non-adjacent consonants as opposed to vowels during auditory statistical learning (SL) of an artificial language. Hebrew is interesting because its complex Semitic morphological structure has been found to play a central role in lexical access, allowing us to examine whether morphological properties can modulate the C-bias in early phases of speech perception, namely, word segmentation. As predicted, we found a significant interaction between language and consonant/vowel manipulation, with a higher performance in the consonantal condition than in the vowel condition for Hebrew speakers, namely, C-bias, and no consonant/vowel asymmetry among English speakers. We suggest that the observed interaction is morphologically anchored, indicating that phonological and morphological processes interact during early phases of auditory word perception.

Swift Prosodic Modulation of Lexical Access: Brain Potentials From Three North Germanic Language Varieties

PURPOSE

According to most models of spoken word recognition, listeners probabilistically activate a set of lexical candidates, which is incrementally updated as the speech signal unfolds. Speech carries segmental (speech sound) as well as suprasegmental (prosodic) information. The role of the latter in spoken word recognition is less clear. We investigated how suprasegments (tone and voice quality) in three North Germanic language varieties affected lexical access by scrutinizing temporally fine-grained neurophysiological effects of lexical uncertainty and information gain.

METHOD

Three event-related potential (ERP) studies were reanalyzed. In all varieties investigated, suprasegments are associated with specific word endings. Swedish has two lexical "word accents" realized as pitch falls with different timings across dialects. In Danish, the distinction is in voice quality. We combined pronunciation lexica and frequency lists to calculate estimates of lexical uncertainty about an unfolding word and information gain upon hearing a suprasegmental cue and the segment upon which it manifests. We used single-trial mixed-effects regression models run every 4 ms.

RESULTS

Only lexical uncertainty showed solid results: a frontal effect at 150-400 ms after suprasegmental cue onset and a later posterior effect after 200 ms. While a model including only segmental information mostly performed better, it was outperformed by the suprasegmental model at 200-330 ms at frontal sites.

CONCLUSIONS

The study points to suprasegmental cues contributing to lexical access over and beyond segments after around 200 ms in the North Germanic varieties investigated. Furthermore, the findings indicate that a previously reported "pre-activation negativity" predominantly reflects forward-looking processing.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25016486.

Auditory Word Comprehension Is Less Incremental in Isolated Words

Partial speech input is often understood to trigger rapid and automatic activation of successively higher-level representations of words, from sound to meaning. Here we show evidence from magnetoencephalography that this type of incremental processing is limited when words are heard in isolation as compared to continuous speech. This suggests a less unified and automatic word recognition process than is often assumed. We present evidence from isolated words that neural effects of phoneme probability, quantified by phoneme surprisal, are significantly stronger than (statistically null) effects of phoneme-by-phoneme lexical uncertainty, quantified by cohort entropy. In contrast, we find robust effects of both cohort entropy and phoneme surprisal during perception of connected speech, with a significant interaction between the contexts. This dissociation rules out models of word recognition in which phoneme surprisal and cohort entropy are common indicators of a uniform process, even though these closely related information-theoretic measures both arise from the probability distribution of wordforms consistent with the input. We propose that phoneme surprisal effects reflect automatic access of a lower level of representation of the auditory input (e.g., wordforms) while the occurrence of cohort entropy effects is task sensitive, driven by a competition process or a higher-level representation that is engaged late (or not at all) during the processing of single words.

Neural dynamics of phoneme sequences reveal position-invariant code for content and order

Speech consists of a continuously-varying acoustic signal. Yet human listeners experience it as sequences of discrete speech sounds, which are used to recognise discrete words. To examine how the human brain appropriately sequences the speech signal, we recorded two-hour magnetoencephalograms from 21 participants listening to short narratives. Our analyses show that the brain continuously encodes the three most recently heard speech sounds in parallel, and maintains this information long past its dissipation from the sensory input. Each speech sound representation evolves over time, jointly encoding both its phonetic features and the amount of time elapsed since onset. As a result, this dynamic neural pattern encodes both the relative order and phonetic content of the speech sequence. These representations are active earlier when phonemes are more predictable, and are sustained longer when lexical identity is uncertain. Our results show how phonetic sequences in natural speech are represented at the level of populations of neurons, providing insight into what intermediary representations exist between the sensory input and sub-lexical units. The flexibility in the dynamics of these representations paves the way for further understanding of how such sequences may be used to interface with higher order structure such as lexical identity.

Parallel processing in speech perception with local and global representations of linguistic context

Speech processing is highly incremental. It is widely accepted that human listeners continuously use the linguistic context to anticipate upcoming concepts, words, and phonemes. However, previous evidence supports two seemingly contradictory models of how a predictive context is integrated with the bottom-up sensory input: Classic psycholinguistic paradigms suggest a two-stage process, in which acoustic input initially leads to local, context-independent representations, which are then quickly integrated with contextual constraints. This contrasts with the view that the brain constructs a single coherent, unified interpretation of the input, which fully integrates available information across representational hierarchies, and thus uses contextual constraints to modulate even the earliest sensory representations. To distinguish these hypotheses, we tested magnetoencephalography responses to continuous narrative speech for signatures of local and unified predictive models. Results provide evidence that listeners employ both types of models in parallel. Two local context models uniquely predict some part of early neural responses, one based on sublexical phoneme sequences, and one based on the phonemes in the current word alone; at the same time, even early responses to phonemes also reflect a unified model that incorporates sentence-level constraints to predict upcoming phonemes. Neural source localization places the anatomical origins of the different predictive models in nonidentical parts of the superior temporal lobes bilaterally, with the right hemisphere showing a relative preference for more local models. These results suggest that speech processing recruits both local and unified predictive models in parallel, reconciling previous disparate findings. Parallel models might make the perceptual system more robust, facilitate processing of unexpected inputs, and serve a function in language acquisition.

Parafoveal processing of orthographic, morphological, and semantic information during reading Arabic: A boundary paradigm investigation

Evidence shows that skilled readers extract information about upcoming words in the parafovea. Using the boundary paradigm, we investigated native Arabic readers' processing of orthographic, morphological, and semantic information available parafoveally. Target words were embedded in frame sentences, and prior to readers fixating them, one of the following previews were made available: (a) Identity preview; (b) Preview that shared the pattern morpheme with the target; (c) Preview that shared the root morpheme with the target; (d) Preview that was a synonym with the target word; (e) Preview with two of the root letters were transposed thus creating a new root, while preserving all letter identities of the target; (f) Preview with two of the root letters were transposed thus creating a pronounceable pseudo root, while also preserving all letter identities of the target; and (g) Previews that was unrelated to the target word and shared no information with it. The results showed that identity, root-preserving, and synonymous preview conditions yielded preview benefit. On the other hand, no benefit was obtained from the pattern-preserving previews, and significant disruption to processing was obtained from the previews that contained transposed root letters, particularly when this letter transposition created a new real root. The results thus reflect Arabic readers' dependance on morphological and semantic information, and suggest that these levels of representation are accessed as early as orthographic information. Implications for theory- and model-building, and the need to accommodate early morphological and semantic processing activities in more comprehensive models are further discussed.

Adaptation to mis-pronounced speech: evidence for a prefrontal-cortex repair mechanism

Speech is a complex and ambiguous acoustic signal that varies significantly within and across speakers. Despite the processing challenge that such variability poses, humans adapt to systematic variations in pronunciation rapidly. The goal of this study is to uncover the neurobiological bases of the attunement process that enables such fluent comprehension. Twenty-four native English participants listened to words spoken by a “canonical” American speaker and two non-canonical speakers, and performed a word-picture matching task, while magnetoencephalography was recorded. Non-canonical speech was created by including systematic phonological substitutions within the word (e.g. [s] → [sh]). Activity in the auditory cortex (superior temporal gyrus) was greater in response to substituted phonemes, and, critically, this was not attenuated by exposure. By contrast, prefrontal regions showed an interaction between the presence of a substitution and the amount of exposure: activity decreased for canonical speech over time, whereas responses to non-canonical speech remained consistently elevated. Grainger causality analyses further revealed that prefrontal responses serve to modulate activity in auditory regions, suggesting the recruitment of top-down processing to decode non-canonical pronunciations. In sum, our results suggest that the behavioural deficit in processing mispronounced phonemes may be due to a disruption to the typical exchange of information between the prefrontal and auditory cortices as observed for canonical speech.

Predictive Processing during a Naturalistic Statistical Learning Task in ASD

Children's sensitivity to regularities within the linguistic stream, such as the likelihood that syllables co-occur, is foundational to speech segmentation and language acquisition. Yet, little is known about the neurocognitive mechanisms underlying speech segmentation in typical development and in neurodevelopmental disorders that impact language acquisition such as autism spectrum disorder (ASD). Here, we investigate the neural signals of statistical learning in 15 human participants (children ages 8-12) with a clinical diagnosis of ASD and 14 age-matched and gender-matched typically developing peers. We tracked the evoked neural responses to syllable sequences in a naturalistic statistical learning corpus using magnetoencephalography (MEG) in the left primary auditory cortex, posterior superior temporal gyrus (pSTG), and inferior frontal gyrus (IFG), across three repetitions of the passage. In typically developing children, we observed a neural index of learning in all three regions of interest (ROIs), measured by the change in evoked response amplitude as a function of syllable surprisal across passage repetitions. As surprisal increased, the amplitude of the neural response increased; this sensitivity emerged after repeated exposure to the corpus. Children with ASD did not show this pattern of learning in all three regions. We discuss two possible hypotheses related to children's sensitivity to bottom-up sensory deficits and difficulty with top-down incremental processing.

Magnetoencephalography and Language

This article provides an overview of research that uses magnetoencephalography to understand the brain basis of human language. The cognitive processes and brain networks that have been implicated in written and spoken language comprehension and production are discussed in relation to different methodologies: we review event-related brain responses, research on the coupling of neural oscillations to speech, oscillatory coupling between brain regions (eg, auditory-motor coupling), and neural decoding approaches in naturalistic language comprehension.

How the brain composes morphemes into meaning

Morphemes (e.g. [tune], [-ful], [-ly]) are the basic blocks with which complex meaning is built. Here, I explore the critical role that morpho-syntactic rules play in forming the meaning of morphologically complex words, from two primary standpoints: (i) how semantically rich stem morphemes (e.g. explode, bake, post) combine with syntactic operators (e.g. -ion, -er, -age) to output a semantically predictable result; (ii) how this process can be understood in terms of mathematical operations, easily allowing the brain to generate representations of novel morphemes and comprehend novel words. With these ideas in mind, I offer a model of morphological processing that incorporates semantic and morpho-syntactic operations in service to meaning composition, and discuss how such a model could be implemented in the human brain. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Transposed letter priming effects and allographic variation in Arabic: Insights from lexical decision and the same-different task

Reading is resilient to distortion of letter order within a word. This is evidenced in the “transposed-letter (TL) priming effect,” the finding that a prime generated by transposing adjacent letters in a word (e.g., jugde) facilitates recognition of the base word (e.g., JUDGE), more than a “substituted-letter” control prime in which the transposed letters are replaced by unrelated letters (e.g., junpe -JUDGE). The TL priming effect is well documented for European languages that are written using the Roman alphabet. Unlike these languages, Arabic has a unique position-dependent allography whereby some letters change shape according to their position within a word. We investigate the TL priming effect using a lexical decision (Experiment 1) and a same–different match task with Arabic words (Experiment 2) and nonwords (Experiment 3). No TL priming effects were found in Experiment 1, suggesting that the lexical-decision task engages lexical access processes that are sensitive to the Semitic nonlinear morphological structure. Experiments 2 and 3 revealed a robust TL priming effect overall. Nonallographic TL primes produced significantly larger facilitation than allographic TL primes, indicating that Arabic readers use allographic variation to resolve the uncertainty in letter order during the early stages of orthographic processing. The implication of these results for current letter position coding models is discussed.

Transposed-letter (TL) priming effects are popularly interpreted as reflecting noisy perception of letter order. The Arabic writing system has a unique allographic feature whereby a letter’s shape and letter spacing depend on its position within a word. We show that TL priming effects for Arabic words and nonwords are modulated by allography in the same–different task. It is important to consider which unique property of the language/writing system produces the cross-language variation, and how it does so.

Rapid Transformation from Auditory to Linguistic Representations of Continuous Speech

During speech perception, a central task of the auditory cortex is to analyze complex acoustic patterns to allow detection of the words that encode a linguistic message [1]. It is generally thought that this process includes at least one intermediate, phonetic, level of representations [2-6], localized bilaterally in the superior temporal lobe [7-9]. Phonetic representations reflect a transition from acoustic to linguistic information, classifying acoustic patterns into linguistically meaningful units, which can serve as input to mechanisms that access abstract word representations [10, 11]. While recent research has identified neural signals arising from successful recognition of individual words in continuous speech [12-15], no explicit neurophysiological signal has been found demonstrating the transition from acoustic and/or phonetic to symbolic, lexical representations. Here, we report a response reflecting the incremental integration of phonetic information for word identification, dominantly localized to the left temporal lobe. The short response latency, approximately 114 ms relative to phoneme onset, suggests that phonetic information is used for lexical processing as soon as it becomes available. Responses also tracked word boundaries, confirming previous reports of immediate lexical segmentation [16, 17]. These new results were further investigated using a cocktail-party paradigm [18, 19] in which participants listened to a mix of two talkers, attending to one and ignoring the other. Analysis indicates neural lexical processing of only the attended, but not the unattended, speech stream. Thus, while responses to acoustic features reflect attention through selective amplification of attended speech, responses consistent with a lexical processing model reveal categorically selective processing.

The Form of Morphemes: MEG Evidence From Masked Priming of Two Hebrew Templates

Studies of lexical access have benefited from comparisons between languages like English, which shows concatenative morphology, and Semitic languages showing non-concatenative morphology of roots and patterns. Morphological decomposition in Semitic has previously been probed using masked priming, originally developed to investigate concatenative morphology. However, studies conducted on Semitic languages have often targeted Semitic-specific questions, such as whether the root and the verbal template prime lexical access. The overall consequence of these studies for our understanding of lexical access remains unclear. In two experiments on Hebrew using MEG, we demonstrate that a verbal form which is orthographically and phonologically indistinguishable from non-verbal forms is primed by other verbs in the same template but not by similar nouns and adjectives. These results suggest that masked priming taps into more than just visual forms but reflects morphological content, even if this content is abstract, showing no distinct orthographic or phonological marking.

The role of the root in auditory word recognition of Hebrew

Evidence from visual word recognition has shown that the root morpheme plays a particularly important role in recognition of nouns in templatic languages [e.g., Velan & Frost, 2009 (Hebrew), Perea, abu Mallouh, & Carreiras, 2010 (Arabic)]. Letter transposition studies in masked priming have proved a useful tool for investigating letter flexibility in the visual domain. Due to the linear nature of the auditory signal, such manipulation is not possible for spoken words. In this study, we use a novel application of the phonemic restoration paradigm to explore the role of morphology in auditory word recognition. In two separate experiments, we show that in auditory word recognition the root plays an important role in Hebrew noun recognition, with words with masked root sounds being especially difficult to recover. This study provides additional evidence in favor of the privileged role of the root in Semitic lexical access and its function in morphological decomposition.

In Spoken Word Recognition, the Future Predicts the Past

Speech is an inherently noisy and ambiguous signal. To fluently derive meaning, a listener must integrate contextual information to guide interpretations of the sensory input. Although many studies have demonstrated the influence of prior context on speech perception, the neural mechanisms supporting the integration of subsequent context remain unknown. Using MEG to record from human auditory cortex, we analyzed responses to spoken words with a varyingly ambiguous onset phoneme, the identity of which is later disambiguated at the lexical uniqueness point. Fifty participants (both male and female) were recruited across two MEG experiments. Our findings suggest that primary auditory cortex is sensitive to phonological ambiguity very early during processing at just 50 ms after onset. Subphonemic detail is preserved in auditory cortex over long timescales and re-evoked at subsequent phoneme positions. Commitments to phonological categories occur in parallel, resolving on the shorter timescale of ∼450 ms. These findings provide evidence that future input determines the perception of earlier speech sounds by maintaining sensory features until they can be integrated with top-down lexical information.SIGNIFICANCE STATEMENT The perception of a speech sound is determined by its surrounding context in the form of words, sentences, and other speech sounds. Often, such contextual information becomes available later than the sensory input. The present study is the first to unveil how the brain uses this subsequent information to aid speech comprehension. Concretely, we found that the auditory system actively maintains the acoustic signal in auditory cortex while concurrently making guesses about the identity of the words being said. Such a processing strategy allows the content of the message to be accessed quickly while also permitting reanalysis of the acoustic signal to minimize parsing mistakes.

Morphological representations are extrapolated from morpho-syntactic rules

The field of psycho- and neuro-linguistics has long-debated the decompositional model of visual word processing: Are written words processed via the visual forms of stem and affix morphemes, or as complex wholes? Although many have now settled upon a decompositional view, it is unclear what heuristic the brain uses to generate these visual morpheme-forms in the first place. Here we conduct a magneto-encephalography study to test two hypotheses for how this may be done: i) the brain encodes representations of the morphemes that follow the morpho-syntactic rules governing constituents: A stem morpheme will be represented if the word obeys the grammatical behaviour associated with its suffix; ii) the brain only encodes stem morphemes that occur with multiple suffixes or as words in isolation. Our results indicate that words with morpho-syntactic wellformedness as stem-suffix combinations are decomposed by the system, thus supporting the former hypothesis. This suggests that knowledge of morpho-syntactic rules can be used to form morphological representations of written words, in absence of independent experience with all of their constituent morphemes. Possible mechanisms supporting this computation are discussed.

Electrophysiology Reveals the Neural Dynamics of Naturalistic Auditory Language Processing: Event-Related Potentials Reflect Continuous Model Updates

The recent trend away from ANOVA-based analyses places experimental investigations into the neurobiology of cognition in more naturalistic and ecologically valid designs within reach. Using mixed-effects models for epoch-based regression, we demonstrate the feasibility of examining event-related potentials (ERPs), and in particular the N400, to study the neural dynamics of human auditory language processing in a naturalistic setting. Despite the large variability between trials during naturalistic stimulation, we replicated previous findings from the literature: the effects of frequency, animacy, and word order and find previously unexplored interaction effects. This suggests a new perspective on ERPs, namely, as a continuous modulation reflecting continuous stimulation instead of a series of discrete and essentially sequential processes locked to discrete events.

Bilingual Language Switching in the Laboratory versus in the Wild: The Spatiotemporal Dynamics of Adaptive Language Control

For a bilingual human, every utterance requires a choice about which language to use. This choice is commonly regarded as part of general executive control, engaging prefrontal and anterior cingulate cortices similarly to many types of effortful task switching. However, although language control within artificial switching paradigms has been heavily studied, the neurobiology of natural switching within socially cued situations has not been characterized. Additionally, although theoretical models address how language control mechanisms adapt to the distinct demands of different interactional contexts, these predictions have not been empirically tested. We used MEG (RRID: NIFINV:nlx_inv_090918) to investigate language switching in multiple contexts ranging from completely artificial to the comprehension of a fully natural bilingual conversation recorded “in the wild.” Our results showed less anterior cingulate and prefrontal cortex involvement for more natural switching. In production, voluntary switching did not engage the prefrontal cortex or elicit behavioral switch costs. In comprehension, while laboratory switches recruited executive control areas, fully natural switching within a conversation only engaged auditory cortices. Multivariate pattern analyses revealed that, in production, interlocutor identity was represented in a sustained fashion throughout the different stages of language planning until speech onset. In comprehension, however, a biphasic pattern was observed: interlocutor identity was first represented at the presentation of the interlocutor and then again at the presentation of the auditory word. In all, our findings underscore the importance of ecologically valid experimental paradigms and offer the first neurophysiological characterization of language control in a range of situations simulating real life to various degrees.

SIGNIFICANCE STATEMENT Bilingualism is an inherently social phenomenon, interactional context fully determining language choice. This research addresses the neural mechanisms underlying multilingual individuals' ability to successfully adapt to varying conversational contexts both while speaking and listening. Our results showed that interactional context critically determines language control networks' engagement: switching under external constraints heavily recruited prefrontal control regions, whereas natural, voluntary switching did not. These findings challenge conclusions derived from artificial switching paradigms, which suggested that language switching is intrinsically effortful. Further, our results predict that the so-called bilingual advantage should be limited to individuals who need to control their languages according to external cues and thus would not occur by virtue of an experience in which switching is fully free.

Linguistic Constraints on Statistical Word Segmentation: The Role of Consonants in Arabic and English

Statistical learning is often taken to lie at the heart of many cognitive tasks, including the acquisition of language. One particular task in which probabilistic models have achieved considerable success is the segmentation of speech into words. However, these models have mostly been tested against English data, and as a result little is known about how a statistical learning mechanism copes with input regularities that arise from the structural properties of different languages. This study focuses on statistical word segmentation in Arabic, a Semitic language in which words are built around consonantal roots. We hypothesize that segmentation in such languages is facilitated by tracking consonant distributions independently from intervening vowels. Previous studies have shown that human learners can track consonant probabilities across intervening vowels in artificial languages, but it is unknown to what extent this ability would be beneficial in the segmentation of natural language. We assessed the performance of a Bayesian segmentation model on English and Arabic, comparing consonant-only representations with full representations. In addition, we examined to what extent structurally different proto-lexicons reflect adult language. The results suggest that for a child learning a Semitic language, separating consonants from vowels is beneficial for segmentation. These findings indicate that probabilistic models require appropriate linguistic representations in order to effectively meet the challenges of language acquisition.