Predictions in speech comprehension: fMRI evidence on the meter–semantic interface

Integrated sentence-level speech perception evokes strengthened language networks and facilitates early speech development

Natural poetic speeches (i.e., proverbs, nursery rhymes, and commercial ads) with strong prosodic regularities are easily memorized by children and the harmonious acoustic patterns are suggested to facilitate their integrated sentence processing. Do children have specific neural pathways for perceiving such poetic utterances, and does their speech development benefit from it? We recorded the task-induced hemodynamic changes of 94 children aged 2 to 12 years using functional near-infrared spectroscopy (fNIRS) while they listened to poetic and non-poetic natural sentences. Seventy-three adult as controls were recruited to investigate the developmental specificity of children group. The results indicated that poetic sentences perceiving is a highly integrated process featured by a lower brain workload in both groups. However, an early activated large-scale network was induced only in the child group, coordinated by hubs for connectivity diversity. Additionally, poetic speeches evoked activation in the phonological encoding regions in the children's group rather than adult controls which decreases with children's ages. The neural responses to poetic speeches were positively linked to children's speech communication performance, especially the fluency and semantic aspects. These results reveal children's neural sensitivity to integrated speech perception which facilitate early speech development by strengthening more sophisticated language networks and the perception-production circuit.

Spectrotemporal cues and attention jointly modulate fMRI network topology for sentence and melody perception

Speech and music are two fundamental modes of human communication. Lateralisation of key processes underlying their perception has been related both to the distinct sensitivity to low-level spectrotemporal acoustic features and to top-down attention. However, the interplay between bottom-up and top-down processes needs to be clarified. In the present study, we investigated the contribution of acoustics and attention to melodies or sentences to lateralisation in fMRI functional network topology. We used sung speech stimuli selectively filtered in temporal or spectral modulation domains with crossed and balanced verbal and melodic content. Perception of speech decreased with degradation of temporal information, whereas perception of melodies decreased with spectral degradation. Applying graph theoretical metrics on fMRI connectivity matrices, we found that local clustering, reflecting functional specialisation, linearly increased when spectral or temporal cues crucial for the task goal were incrementally degraded. These effects occurred in a bilateral fronto-temporo-parietal network for processing temporally degraded sentences and in right auditory regions for processing spectrally degraded melodies. In contrast, global topology remained stable across conditions. These findings suggest that lateralisation for speech and music partially depends on an interplay of acoustic cues and task goals under increased attentional demands.

Predictive Processing in Poetic Language: Event-Related Potentials Data on Rhythmic Omissions in Metered Speech

Predictions during language comprehension are currently discussed from many points of view. One area where predictive processing may play a particular role concerns poetic language that is regularized by meter and rhyme, thus allowing strong predictions regarding the timing and stress of individual syllables. While there is growing evidence that these prosodic regularities influence language processing, less is known about the potential influence of prosodic preferences (binary, strong-weak patterns) on neurophysiological processes. To this end, the present electroencephalogram (EEG) study examined whether the predictability of strong and weak syllables within metered speech would differ as a function of meter (trochee vs. iamb). Strong, i.e., accented positions within a foot should be more predictable than weak, i.e., unaccented positions. Our focus was on disyllabic pseudowords that solely differed between trochaic and iambic structure, with trochees providing the preferred foot in German. Methodologically, we focused on the omission Mismatch Negativity (oMMN) that is elicited when an anticipated auditory stimulus is omitted. The resulting electrophysiological brain response is particularly interesting because its elicitation does not depend on a physical stimulus. Omissions in deviant position of a passive oddball paradigm occurred at either first- or second-syllable position of the aforementioned pseudowords, resulting in a 2-by-2 design with the factors foot type and omission position. Analyses focused on the mean oMMN amplitude and latency differences across the four conditions. The result pattern was characterized by an interaction of the effects of foot type and omission position for both amplitudes and latencies. In first position, omissions resulted in larger and earlier oMMNs for trochees than for iambs. In second position, omissions resulted in larger oMMNs for iambs than for trochees, but the oMMN latency did not differ. The results suggest that omissions, particularly in initial position, are modulated by a trochaic preference in German. The preferred strong-weak pattern may have strengthened the prosodic prediction, especially for matching, trochaic stimuli, such that the violation of this prediction led to an earlier and stronger prediction error. Altogether, predictive processing seems to play a particular role in metered speech, especially if the meter is based on the preferred foot type.

Linguistic syncopation: Meter-syntax alignment affects sentence comprehension and sensorimotor synchronization

The hierarchical organization of speech rhythm into meter putatively confers cognitive affordances for perception, memory, and motor coordination. Meter also aligns with phrasal structure in systematic ways. In this paper, we show that this alignment affects the robustness of syntactic comprehension and discuss possible underlying mechanisms. In two experiments, we manipulated meter-syntax alignment while sentences with relative clause structures were either read as text (experiment 1, n = 40) or listened to as speech (experiment 2, n = 40). In experiment 2, we also measured the stability with which participants could tap in time with the metrical accents in the sentences they were comprehending. In addition to making more mistakes, sensorimotor synchronization was disrupted when syntactic cues clashed with the metrical context. We suggest that this reflects a tight coordination of top-down linguistic knowledge with the sensorimotor system to optimize comprehension.

Neuro-Clinical Signatures of Language Impairments: A Theoretical Framework for Function-to-structure Mapping in Clinics

In the past decades, neuroscientists and clinicians have collected a considerable amount of data and drastically increased our knowledge about the mapping of language in the brain. The emerging picture from the accumulated knowledge is that there are complex and combinatorial relationships between language functions and anatomical brain regions. Understanding the underlying principles of this complex mapping is of paramount importance for the identification of the brain signature of language and Neuro-Clinical signatures that explain language impairments and predict language recovery after stroke. We review recent attempts to addresses this question of language-brain mapping. We introduce the different concepts of mapping (from diffeomorphic one-to-one mapping to many-to-many mapping). We build those different forms of mapping to derive a theoretical framework where the current principles of brain architectures including redundancy, degeneracy, pluri-potentiality and bow-tie network are described.

Adaptive Plasticity Under Adverse Listening Conditions is Disrupted in Developmental Dyslexia

OBJECTIVE

Acoustic distortions to the speech signal impair spoken language recognition, but healthy listeners exhibit adaptive plasticity consistent with rapid adjustments in how the distorted speech input maps to speech representations, perhaps through engagement of supervised error-driven learning. This puts adaptive plasticity in speech perception in an interesting position with regard to developmental dyslexia inasmuch as dyslexia impacts speech processing and may involve dysfunction in neurobiological systems hypothesized to be involved in adaptive plasticity.

METHOD

Here, we examined typical young adult listeners (N = 17), and those with dyslexia (N = 16), as they reported the identity of native-language monosyllabic spoken words to which signal processing had been applied to create a systematic acoustic distortion. During training, all participants experienced incremental signal distortion increases to mildly distorted speech along with orthographic and auditory feedback indicating word identity following response across a brief, 250-trial training block. During pretest and posttest phases, no feedback was provided to participants.

RESULTS

Word recognition across severely distorted speech was poor at pretest and equivalent across groups. Training led to improved word recognition for the most severely distorted speech at posttest, with evidence that adaptive plasticity generalized to support recognition of new tokens not previously experienced under distortion. However, training-related recognition gains for listeners with dyslexia were significantly less robust than for control listeners.

CONCLUSIONS

Less efficient adaptive plasticity to speech distortions may impact the ability of individuals with dyslexia to deal with variability arising from sources like acoustic noise and foreign-accented speech.

Attentional modulation of neural entrainment to sound streams in children with and without ADHD

To extract meaningful information from complex auditory scenes like a noisy playground, rock concert, or classroom, children can direct attention to different sound streams. One means of accomplishing this might be to align neural activity with the temporal structure of a target stream, such as a specific talker or melody. However, this may be more difficult for children with ADHD, who can struggle with accurately perceiving and producing temporal intervals. In this EEG study, we found that school-aged children's attention to one of two temporally-interleaved isochronous tone 'melodies' was linked to an increase in phase-locking at the melody's rate, and a shift in neural phase that aligned the neural responses with the attended tone stream. Children's attention task performance and neural phase alignment with the attended melody were linked to performance on temporal production tasks, suggesting that children with more robust control over motor timing were better able to direct attention to the time points associated with the target melody. Finally, we found that although children with ADHD performed less accurately on the tonal attention task than typically developing children, they showed the same degree of attentional modulation of phase locking and neural phase shifts, suggesting that children with ADHD may have difficulty with attentional engagement rather than attentional selection.

Word Recall is Affected by Surrounding Metrical Context

It has been claimed that English has a metrical structure, or rhythm, in which stressed and unstressed syllables alternate. In previous research regular, alternating patterns have been shown to facilitate online language comprehension. Expanding these findings to downstream processing would lead to the prediction that metrical regularity enhances memory. Research from the memory literature, however, indicates that regular patterns are less salient and therefore less well remembered, and also that strings of similar sounds are harder to remember. This work suggests that, like lists of words with similar sounds, lists of words with similar metrical patterns are less accurately remembered than comparable metrically irregular patterns. The present study tests these conflicting predictions by examining the effects of metrical regularity in a recall task. We find that words are better recalled when they do not match their metrical context, suggesting that a regular metrical structure may not be beneficial in all contexts.

Is there a prediction network? Meta-analytic evidence for a cortical-subcortical network likely subserving prediction

Predictive coding is an increasingly influential and ambitious concept in neuroscience viewing the brain as a 'hypothesis testing machine' that constantly strives to minimize prediction error, the gap between its predictions and the actual sensory input. Despite the invaluable contribution of this framework to the formulation of brain function, its neuroanatomical foundations have not been fully defined. To address this gap, we conducted activation likelihood estimation (ALE) meta-analysis of 39 neuroimaging studies of three functional domains (action perception, language and music) inherently involving prediction. The ALE analysis revealed a widely distributed brain network encompassing regions within the inferior and middle frontal gyri, anterior insula, premotor cortex, pre-supplementary motor area, temporoparietal junction, striatum, thalamus/subthalamus and the cerebellum. This network is proposed to subserve domain-general prediction and its relevance to motor control, attention, implicit learning and social cognition is discussed in light of the predictive coding scheme. Better understanding of the presented network may help advance treatments of neuropsychiatric conditions related to aberrant prediction processing and promote cognitive enhancement in healthy individuals.

Event-Related Potential Evidence of Implicit Metric Structure during Silent Reading

Under the Implicit Prosody Hypothesis, readers generate prosodic structures during silent reading that can direct their real-time interpretations of the text. In the current study, we investigated the processing of implicit meter by recording event-related potentials (ERPs) while participants read a series of 160 rhyming couplets, where the rhyme target was always a stress-alternating noun–verb homograph (e.g., permit, which is pronounced PERmit as a noun and perMIT as a verb). The target had a strong–weak or weak–strong stress pattern, which was either consistent or inconsistent with the stress expectation generated by the couplet. Inconsistent strong–weak targets elicited negativities between 80–155 ms and 325–375 ms relative to consistent strong–weak targets; inconsistent weak–strong targets elicited a positivity between 365–435 ms relative to consistent weak–strong targets. These results are largely consistent with effects of metric violations during listening, demonstrating that implicit prosodic representations are similar to explicit prosodic representations.

Neural mechanisms for coping with acoustically reduced speech

In spoken language, reductions of word forms occur regularly and need to be accommodated by the listener. Intriguingly, this accommodation is usually achieved without any apparent effort. The neural bases of this cognitive skill are not yet fully understood. We here presented participants with reduced words that were either preceded by a related or an unrelated visual prime and compared electric brain responses to reduced words with those to their full counterparts. In time-domain, we found a positivity between 400 and 600 ms differing between reduced and full forms. A later positivity distinguished primed and unprimed words and was modulated by reduction. In frequency-domain, alpha suppression was stronger for reduced than for full words. The time- and frequency-domain reduction effects converge towards the view that reduced words draw on attention and memory mechanisms. Our data demonstrate the importance of interactive processing of bottom-up and top-down information for the comprehension of reduced words.

The Temporal Prediction of Stress in Speech and Its Relation to Musical Beat Perception

While rhythmic expectancies are thought to be at the base of beat perception in music, the extent to which stress patterns in speech are similarly represented and predicted during on-line language comprehension is debated. The temporal prediction of stress may be advantageous to speech processing, as stress patterns aid segmentation and mark new information in utterances. However, while linguistic stress patterns may be organized into hierarchical metrical structures similarly to musical meter, they do not typically present the same degree of periodicity. We review the theoretical background for the idea that stress patterns are predicted and address the following questions: First, what is the evidence that listeners can predict the temporal location of stress based on preceding rhythm? If they can, is it thanks to neural entrainment mechanisms similar to those utilized for musical beat perception? And lastly, what linguistic factors other than rhythm may account for the prediction of stress in natural speech? We conclude that while expectancies based on the periodic presentation of stresses are at play in some of the current literature, other processes are likely to affect the prediction of stress in more naturalistic, less isochronous speech. Specifically, aspects of prosody other than amplitude changes (e.g., intonation) as well as lexical, syntactic and information structural constraints on the realization of stress may all contribute to the probabilistic expectation of stress in speech.

Right Lateral Cerebellum Represents Linguistic Predictability

Mounting evidence indicates that posterolateral portions of the cerebellum (right Crus I/II) contribute to language processing, but the nature of this role remains unclear. Based on a well-supported theory of cerebellar motor function, which ascribes to the cerebellum a role in short-term prediction through internal modeling, we hypothesize that right cerebellar Crus I/II supports prediction of upcoming sentence content. We tested this hypothesis using event-related fMRI in male and female human subjects by manipulating the predictability of written sentences. Our design controlled for motor planning and execution, as well as for linguistic features and working memory load; it also allowed separation of the prediction interval from the presentation of the final sentence item. In addition, three further fMRI tasks captured semantic, phonological, and orthographic processing to shed light on the nature of the information processed. As hypothesized, activity in right posterolateral cerebellum correlated with the predictability of the upcoming target word. This cerebellar region also responded to prediction error during the outcome of the trial. Further, this region was engaged in phonological, but not semantic or orthographic, processing. This is the first imaging study to demonstrate a right cerebellar contribution in language comprehension independently from motor, cognitive, and linguistic confounds. These results complement our work using other methodologies showing cerebellar engagement in linguistic prediction and suggest that internal modeling of phonological representations aids language production and comprehension.

SIGNIFICANCE STATEMENT The cerebellum is traditionally seen as a motor structure that allows for smooth movement by predicting upcoming signals. However, the cerebellum is also consistently implicated in nonmotor functions such as language and working memory. Using fMRI, we identify a cerebellar area that is active when words are predicted and when these predictions are violated. This area is active in a separate task that requires phonological processing, but not in tasks that require semantic or visuospatial processing. Our results support the idea of prediction as a unifying cerebellar function in motor and nonmotor domains. We provide new insights by linking the cerebellar role in prediction to its role in verbal working memory, suggesting that these predictions involve phonological processing.

Where Is the Beat? The Neural Correlates of Lexical Stress and Rhythmical Well-formedness in Auditory Story Comprehension

While listening to continuous speech, humans process beat information to correctly identify word boundaries. The beats of language are stress patterns that are created by combining lexical (word-specific) stress patterns and the rhythm of a specific language. Sometimes, the lexical stress pattern needs to be altered to obey the rhythm of the language. This study investigated the interplay of lexical stress patterns and rhythmical well-formedness in natural speech with fMRI. Previous electrophysiological studies on cases in which a regular lexical stress pattern may be altered to obtain rhythmical well-formedness showed that even subtle rhythmic deviations are detected by the brain if attention is directed toward prosody. Here, we present a new approach to this phenomenon by having participants listen to contextually rich stories in the absence of a task targeting the manipulation. For the interaction of lexical stress and rhythmical well-formedness, we found one suprathreshold cluster localized between the cerebellum and the brain stem. For the main effect of lexical stress, we found higher BOLD responses to the retained lexical stress pattern in the bilateral SMA, bilateral postcentral gyrus, bilateral middle fontal gyrus, bilateral inferior and right superior parietal lobule, and right precuneus. These results support the view that lexical stress is processed as part of a sensorimotor network of speech comprehension. Moreover, our results connect beat processing in language to domain-independent timing perception.

An fMRI investigation of the relationship between future imagination and cognitive flexibility

While future imagination is largely considered to be a cognitive process grounded in default mode network activity, studies have shown that future imagination recruits regions in both default mode and frontoparietal control networks. In addition, it has recently been shown that the ability to imagine the future is associated with cognitive flexibility, and that tasks requiring cognitive flexibility result in increased coupling of the default mode network with frontoparietal control and salience networks. In the current study, we investigated the neural correlates underlying the association between cognitive flexibility and future imagination in two ways. First, we experimentally varied the degree of cognitive flexibility required during future imagination by manipulating the disparateness of episodic details contributing to imagined events. To this end, participants generated episodic details (persons, locations, objects) within three social spheres; during fMRI scanning they were presented with sets of three episodic details all taken from the same social sphere (Congruent condition) or different social spheres (Incongruent condition) and required to imagine a future event involving the three details. We predicted that, relative to the Congruent condition, future simulation in the Incongruent condition would be associated with increased activity in regions of the default mode, frontoparietal and salience networks. Second, we hypothesized that individual differences in cognitive flexibility, as measured by performance on the Alternate Uses Task, would correspond to individual differences in the brain regions recruited during future imagination. A task partial least squares (PLS) analysis showed that the Incongruent condition resulted in an increase in activity in regions in salience networks (e.g. the insula) but, contrary to our prediction, reduced activity in many regions of the default mode network (including the hippocampus). A subsequent functional connectivity (within-subject seed PLS) analysis showed that the insula exhibited increased coupling with default mode regions during the Incongruent condition. Finally, a behavioral PLS analysis showed that individual differences in cognitive flexibility were associated with differences in activity in a number of regions from frontoparietal, salience and default-mode networks during both future imagination conditions, further highlighting that the cognitive flexibility underlying future imagination is grounded in the complex interaction of regions in these networks.

Cerebellar BOLD signal during the acquisition of a new lexicon predicts its early consolidation

Cerebellar contributions to language are presently poorly understood, but it has been argued that the cerebellar role in motor learning can be extended to learning in cognitive and linguistic domains. Here, we used fMRI to investigate whether the cerebellum is recruited in mapping novel words onto existing semantic concepts. On separate days, participants performed a Basque vocabulary learning task and a control English synonym task in the MRI scanner. Learning-related BOLD activity was found in left inferior frontal gyrus, bilateral insula, pre-SMA, left superior parietal cortex, right caudate, the right cerebellar vermis and right cerebellar Crus II. The extent to which the cerebellar regions, but not the cerebral areas, were recruited during learning correlated positively with participants' off-line improvement in performance after the learning task. These data provide evidence for a cerebellar role in lexical learning, and suggest that the right cerebellum may contribute toward consolidation of lexico-semantic associations in the language network.

Processing Metrical Information in Silent Reading: An ERP Study

Listeners are sensitive to the metric structure of words, i.e., an alternating pattern of stressed and unstressed syllables, in auditory speech processing: Event-related potentials recorded as participants listen to a sequence of words with a consistent metrical pattern, e.g., a series of trochaic words, suggest that participants register words metrically incongruent with the preceding sequence. Here we examine whether the processing of individual words in silent reading is similarly impacted by rhythmic properties of the surrounding context. We recorded participants’ EEG as they read lists of either three trochaic or iambic disyllabic words followed by a target word that was either congruent or incongruent with the preceding metric pattern. Event-Related Potentials (ERPs) to targets were modulated by an interaction between metrical structure (iambic vs. trochaic) and congruence: for iambs, more positive ERPs were observed in the incongruent than congruent condition 250–400 ms and 400–600 ms post-stimulus, whereas no reliable impact of congruence was found for trochees. We suggest that when iambs are in an incongruent context, i.e., preceded by trochees, the context contains the metrical structure that is more typical in participants’ native language which facilitates processing relative to when they are presented in a congruent context, containing the less typical, i.e., iambic, metrical structure. The results provide evidence that comprehenders are sensitive to the prosodic properties of the context even in silent reading, such that this sensitivity impacts lexico-semantic processing of individual words.

Adaptive plasticity in speech perception: Effects of external information and internal predictions

When listeners encounter speech under adverse listening conditions, adaptive adjustments in perception can improve comprehension over time. In some cases, these adaptive changes require the presence of external information that disambiguates the distorted speech signals, whereas in other cases mere exposure is sufficient. Both external (e.g., written feedback) and internal (e.g., prior word knowledge) sources of information can be used to generate predictions about the correct mapping of a distorted speech signal. We hypothesize that these predictions provide a basis for determining the discrepancy between the expected and actual speech signal that can be used to guide adaptive changes in perception. This study provides the first empirical investigation that manipulates external and internal factors through (a) the availability of explicit external disambiguating information via the presence or absence of postresponse orthographic information paired with a repetition of the degraded stimulus, and (b) the accuracy of internally generated predictions; an acoustic distortion is introduced either abruptly or incrementally. The results demonstrate that the impact of external information on adaptive plasticity is contingent upon whether the intelligibility of the stimuli permits accurate internally generated predictions during exposure. External information sources enhance adaptive plasticity only when input signals are severely degraded and cannot reliably access internal predictions. This is consistent with a computational framework for adaptive plasticity in which error-driven supervised learning relies on the ability to compute sensory prediction error signals from both internal and external sources of information. (PsycINFO Database Record

Beat Perception and Sociability: Evidence from Williams Syndrome

Beat perception in music has been proposed to be a human universal that may have its origins in adaptive processes involving temporal entrainment such as social communication and interaction. We examined beat perception skills in individuals with Williams syndrome (WS), a genetic, neurodevelopmental disorder. Musical interest and hypersociability are two prominent aspects of the WS phenotype although actual musical and social skills are variable. On a group level, beat and meter perception skills were poorer in WS than in age-matched peers though there was significant individual variability. Cognitive ability, sound processing style, and musical training predicted beat and meter perception performance in WS. Moreover, we found significant relationships between beat and meter perception and adaptive communication and socialization skills in WS. Results have implications for understanding the role of predictive timing in both music and social interactions in the general population, and suggest music as a promising avenue for addressing social communication difficulties in WS.

An fMRI study investigating effects of conceptually related sentences on the perception of degraded speech

Prior research has shown that the perception of degraded speech is influenced by within sentence meaning and recruits one or more components of a frontal-temporal-parietal network. The goal of the current study is to examine whether the overall conceptual meaning of a sentence, made up of one set of words, influences the perception of a second acoustically degraded sentence, made up of a different set of words. Using functional magnetic resonance imaging (fMRI), we presented an acoustically clear sentence followed by an acoustically degraded sentence and manipulated the semantic relationship between them: Related in meaning (but consisting of different content words), Unrelated in meaning, or Same. Results showed that listeners' word recognition accuracy for the acoustically degraded sentences was significantly higher when the target sentence was preceded by a conceptually related compared to a conceptually unrelated sentence. Sensitivity to conceptual relationships was associated with enhanced activity in middle and inferior frontal, temporal, and parietal areas. In addition, the left middle frontal gyrus (LMFG), left inferior frontal gyrus (LIFG), and left middle temporal gyrus (LMTG) showed activity that correlated with individual performance on the Related condition. The superior temporal gyrus (STG) showed increased activation in the Same condition suggesting that it is sensitive to perceptual similarity rather than the integration of meaning between the sentence pairs. A fronto-temporo-parietal network appears to consolidate information sources across multiple levels of language (acoustic, lexical, syntactic, semantic) to build, and ultimately integrate conceptual information across sentences and facilitate the perception of a degraded speech signal. However, the nature of the sources of information that are available differentially recruit specific regions and modulate their activity within this network. Implications of these findings for the functional architecture of the network are considered.

Auditory perceptual objects as generative models: Setting the stage for communication by sound

Communication by sounds requires that the communication channels (i.e. speech/speakers and other sound sources) had been established. This allows to separate concurrently active sound sources, to track their identity, to assess the type of message arriving from them, and to decide whether and when to react (e.g., reply to the message). We propose that these functions rely on a common generative model of the auditory environment. This model predicts upcoming sounds on the basis of representations describing temporal/sequential regularities. Predictions help to identify the continuation of the previously discovered sound sources to detect the emergence of new sources as well as changes in the behavior of the known ones. It produces auditory event representations which provide a full sensory description of the sounds, including their relation to the auditory context and the current goals of the organism. Event representations can be consciously perceived and serve as objects in various cognitive operations.

fMRI reveals lateralized pattern of brain activity modulated by the metrics of stimuli during auditory rhyme processing

Our fMRI study investigates auditory rhyme processing in spoken language to further elucidate the topic of functional lateralization of language processing. During scanning, 14 subjects listened to four different types of versed word strings and subsequently performed either a rhyme or a meter detection task. Our results show lateralization to auditory-related temporal regions in the right hemisphere irrespective of task. As for the left hemisphere we report responses in the supramarginal gyrus as well as in the opercular part of the inferior frontal gyrus modulated by the presence of regular meter and rhyme. The interaction of rhyme and meter was associated with increased involvement of the superior temporal sulcus and the putamen of the right hemisphere. Overall, these findings support the notion of right-hemispheric specialization for suprasegmental analyses during processing of spoken sentences and provide neuroimaging evidence for the influence of metrics on auditory rhyme processing.

How information structure influences the processing of rhythmic irregularities: ERP evidence from German phrases

This study explores the influence of focus and givenness on the cognitive processing of rhythmic irregularities occurring in natural speech. Previous ERP studies showed that even subtle rhythmic deviations are detected by the brain if attention is directed towards the rhythmic structure. By using question-answer pairs, it was investigated whether subtle rhythmic irregularities in form of stress clashes (two adjacent stressed syllables) and stress lapses (two adjacent unstressed syllables) are still perceived when presented in post-focus position in an answer sentence and attention is directed away from them, towards the meaning of the element in narrow focus position by the preceding wh-question. Moreover, by visually presenting the lexical-semantic input of the deviating structure in the question, the influence of rhythmical and lexical properties in these two forms of rhythmic deviations are disentangled. While words in the present stress clash condition do not deviate from lexical stress, stress lapses contain deviations from metrical and lexical stress. The data reveal an early negativity effect for stress clashes but not for stress lapses, supporting the assumption that they are processed differently. The absence of a negative component for stress lapses indicates that the metrical deviation alone is not salient enough to be registered in non-focus position. Moreover, the lack of a late positive component suggests that subtle rhythmic deviations are less perceivable and hence more acceptable when presented in non-focus position. Thus, these results show that attentional shift induced by information structure influences the degree of the processing of rhythm.

Left dominance for language perception starts in the extrastriate cortex: An ERP and sLORETA study

While it is well known that the left hemisphere is more efficient than the right in most tasks involving perception of speech stimuli, the neurophysiological pathways leading to these lateralised performance differences are as yet rather unclear. In particular, the question whether language lateralisation depends on semantic processing or is already evident in early perceptual stimulus processing has not been answered unequivocally. In the present study, we therefore recorded event-related potentials (ERPs) during tachistoscopic presentation of horizontally or vertically presented verbal stimuli in the left (LVF) and the right visual field (RVF). Participants were asked to indicate, whether the presented stimulus was a word or a non-word. On the behavioural level, participants showed stronger hemispheric asymmetries for horizontal, than for vertical stimulus presentation. In addition, ERP asymmetries were also modulated by stimulus presentation format, as the electrode by visual field interactions for P1 and N1 were stronger after vertical, than after horizontal stimulus presentation. Moreover, sLORETA revealed that ERP left-right asymmetries were mainly driven by the extrastriate cortex and reading-associated areas in the parietal cortex. Taken together, the present study shows electrophysiological support for the assumption that language lateralisation during speech perception arises from a left dominance for the processing of early perceptual stimulus aspects.

Four central questions about prediction in language processing

The notion that prediction is a fundamental principle of human information processing has been en vogue over recent years. The investigation of language processing may be particularly illuminating for testing this claim. Linguists traditionally have argued prediction plays only a minor role during language understanding because of the vast possibilities available to the language user as each word is encountered. In the present review I consider four central questions of anticipatory language processing: Why (i.e. what is the function of prediction in language processing)? What (i.e. what are the cues used to predict up-coming linguistic information and what type of representations are predicted)? How (what mechanisms are involved in predictive language processing and what is the role of possible mediating factors such as working memory)? When (i.e. do individuals always predict up-coming input during language processing)? I propose that prediction occurs via a set of diverse PACS (production-, association-, combinatorial-, and simulation-based prediction) mechanisms which are minimally required for a comprehensive account of predictive language processing. Models of anticipatory language processing must be revised to take multiple mechanisms, mediating factors, and situational context into account. Finally, I conjecture that the evidence considered here is consistent with the notion that prediction is an important aspect but not a fundamental principle of language processing. This article is part of a Special Issue entitled SI: Prediction and Attention.

The relevance of rhythmical alternation in language processing: an ERP study on English compounds

This study investigates the influence of rhythmic expectancies on language processing. It is assumed that language rhythm involves an alternation of strong and weak beats within a linguistic domain. Hence, in some contexts rhythmically induced stress shifts occur in order to comply with the Rhythm Rule. In English, this rule operates to prevent clashes of stressed adjacent syllables or lapses of adjacent unstressed syllables. While previous studies investigated effects on speech production and perception, this study focuses on brain responses to structures either obeying or deviating from this rule. Event-related potentials show that rhythmic regularity is relevant for language processing: rhythmic deviations evoked different ERP components reflecting the deviance from rhythmic expectancies. An N400 effect found for shifted items reflects higher costs in lexical processing due to stress deviation. The overall results disentangle lexical and rhythmical influences on language processing and complement the findings of previous studies on rhythmical processing.

Speech perception under adverse conditions: insights from behavioral, computational, and neuroscience research

Adult speech perception reflects the long-term regularities of the native language, but it is also flexible such that it accommodates and adapts to adverse listening conditions and short-term deviations from native-language norms. The purpose of this article is to examine how the broader neuroscience literature can inform and advance research efforts in understanding the neural basis of flexibility and adaptive plasticity in speech perception. Specifically, we highlight the potential role of learning algorithms that rely on prediction error signals and discuss specific neural structures that are likely to contribute to such learning. To this end, we review behavioral studies, computational accounts, and neuroimaging findings related to adaptive plasticity in speech perception. Already, a few studies have alluded to a potential role of these mechanisms in adaptive plasticity in speech perception. Furthermore, we consider research topics in neuroscience that offer insight into how perception can be adaptively tuned to short-term deviations while balancing the need to maintain stability in the perception of learned long-term regularities. Consideration of the application and limitations of these algorithms in characterizing flexible speech perception under adverse conditions promises to inform theoretical models of speech.

Decoding humor experiences from brain activity of people viewing comedy movies

Humans naturally have a sense of humor. Experiencing humor not only encourages social interactions, but also produces positive physiological effects on the human body, such as lowering blood pressure. Recent neuro-imaging studies have shown evidence for distinct mental state changes at work in people experiencing humor. However, the temporal characteristics of these changes remain elusive. In this paper, we objectively measured humor-related mental states from single-trial functional magnetic resonance imaging (fMRI) data obtained while subjects viewed comedy TV programs. Measured fMRI data were labeled on the basis of the lag before or after the viewer’s perception of humor (humor onset) determined by the viewer-reported humor experiences during the fMRI scans. We trained multiple binary classifiers, or decoders, to distinguish between fMRI data obtained at each lag from ones obtained during a neutral state in which subjects were not experiencing humor. As a result, in the right dorsolateral prefrontal cortex and the right temporal area, the decoders showed significant classification accuracies even at two seconds ahead of the humor onsets. Furthermore, given a time series of fMRI data obtained during movie viewing, we found that the decoders with significant performance were also able to predict the upcoming humor events on a volume-by-volume basis. Taking into account the hemodynamic delay, our results suggest that the upcoming humor events are encoded in specific brain areas up to about five seconds before the awareness of experiencing humor. Our results provide evidence that there exists a mental state lasting for a few seconds before actual humor perception, as if a viewer is expecting the future humorous events.