Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening

Brain network dynamics predict moments of surprise across contexts

We experience surprise when reality conflicts with our expectations. When we encounter such expectation violations in psychological tasks and daily life, are we experiencing completely different forms of surprise? Or is surprise a fundamental psychological process with shared neural bases across contexts? To address this question, we identified a brain network model, the surprise edge-fluctuation-based predictive model (EFPM), whose regional interaction dynamics measured with functional magnetic resonance imaging (fMRI) predicted surprise in an adaptive learning task. The same model generalized to predict surprise as a separate group of individuals watched suspenseful basketball games and as a third group watched videos violating psychological expectations. The surprise EFPM also uniquely predicts surprise, capturing expectation violations better than models built from other brain networks, fMRI measures and behavioural metrics. These results suggest that shared neurocognitive processes underlie surprise across contexts and that distinct experiences can be translated into the common space of brain dynamics.

The unique contribution of uncertainty reduction during naturalistic language comprehension

Language comprehension is an incremental process with prediction. Delineating various mental states during such a process is critical to understanding the relationship between human cognition and the properties of language. Entropy reduction, which indicates the dynamic decrease of uncertainty as language input unfolds, has been recognized as effective in predicting neural responses during comprehension. According to the entropy reduction hypothesis (Hale, 2006), entropy reduction is related to the processing difficulty of a word, the effect of which may overlap with other well-documented information-theoretical metrics such as surprisal or next-word entropy. However, the processing difficulty was often confused with the information conveyed by a word, especially lacking neural differentiation. We propose that entropy reduction represents the cognitive neural process of information gain that can be dissociated from processing difficulty. This study characterized various information-theoretical metrics using GPT-2 and identified the unique effects of entropy reduction in predicting fMRI time series acquired during language comprehension. In addition to the effects of surprisal and entropy, entropy reduction was associated with activations in the left inferior frontal gyrus, bilateral ventromedial prefrontal cortex, insula, thalamus, basal ganglia, and middle cingulate cortex. The reduction of uncertainty, rather than its fluctuation, proved to be an effective factor in modeling neural responses. The neural substrates underlying the reduction in uncertainty might imply the brain's desire for information regardless of processing difficulty.

Brain and grammar: revealing electrophysiological basic structures with competing statistical models

Acoustic, lexical, and syntactic information are simultaneously processed in the brain requiring complex strategies to distinguish their electrophysiological activity. Capitalizing on previous works that factor out acoustic information, we could concentrate on the lexical and syntactic contribution to language processing by testing competing statistical models. We exploited electroencephalographic recordings and compared different surprisal models selectively involving lexical information, part of speech, or syntactic structures in various combinations. Electroencephalographic responses were recorded in 32 participants during listening to affirmative active declarative sentences. We compared the activation corresponding to basic syntactic structures, such as noun phrases vs. verb phrases. Lexical and syntactic processing activates different frequency bands, partially different time windows, and different networks. Moreover, surprisal models based on part of speech inventory only do not explain well the electrophysiological data, while those including syntactic information do. By disentangling acoustic, lexical, and syntactic information, we demonstrated differential brain sensitivity to syntactic information. These results confirm and extend previous measures obtained with intracranial recordings, supporting our hypothesis that syntactic structures are crucial in neural language processing. This study provides a detailed understanding of how the brain processes syntactic information, highlighting the importance of syntactic surprisal in shaping neural responses during language comprehension.

A survey of brain functional network extraction methods using fMRI data

Functional network (FN) analyses play a pivotal role in uncovering insights into brain function and understanding the pathophysiology of various brain disorders. This paper focuses on classical and advanced methods for deriving brain FNs from functional magnetic resonance imaging (fMRI) data. We systematically review their foundational principles, advantages, shortcomings, and interrelations, encompassing both static and dynamic FN extraction approaches. In the context of static FN extraction, we present hypothesis-driven methods such as region of interest (ROI)-based approaches as well as data-driven methods including matrix decomposition, clustering, and deep learning. For dynamic FN extraction, both window-based and windowless methods are surveyed with respect to the estimation of time-varying FN and the subsequent computation of FN states. We also discuss the scope of application of the various methods and avenues for future improvements.

Bridging the Gap between Psychophysiological and Audiological Factors in the Assessment of Tinnitus: An EEG Investigation in the Beta Band

BACKGROUND

Despite substantial progress in investigating its psychophysical complexity, tinnitus remains a scientific and clinical enigma. The present study, through an ecological and multidisciplinary approach, aims to identify associations between electroencephalographic (EEG) and psycho-audiological variables.

METHODS

EEG beta activity, often related to stress and anxiety, was acquired from 12 tinnitus patients (TIN group) and 7 controls (CONT group) during an audio cognitive task and at rest. We also investigated psychological (SCL-90-R; STAI-Y; BFI-10) and audiological (THI; TQ12-I; Hyperacusis) variables using non-parametric statistics to assess differences and relationships between and within groups.

RESULTS

In the TIN group, frontal beta activity positively correlated with hyperacusis, parietal activity, and trait anxiety; the latter is also associated with depression in CONT. Significant differences in paranoid ideation and openness were found between groups.

CONCLUSIONS

The connection between anxiety trait, beta activity in the fronto-parietal cortices and hyperacusis provides insights into brain functioning in tinnitus patients, offering quantitative descriptions for clinicians and new multidisciplinary treatment hypotheses.

The organization of the semantic network as reflected by the neural correlates of six semantic dimensions

Multiple sensory-motor and non-sensory-motor dimensions have been proposed for semantic representation, but it remains unclear how the semantic system is organized along them in the human brain. Using naturalistic fMRI data and large-scale semantic ratings, we investigated the overlaps and dissociations between the neural correlates of six semantic dimensions: vision, motor, socialness, emotion, space, and time. Our findings revealed a more complex semantic atlas than what is predicted by current neurobiological models of semantic representation. Brain regions that are selectively sensitive to specific semantic dimensions were found both within and outside the brain networks assumed to represent multimodal general and/or abstract semantics. Overlaps between the neural correlates of different semantic dimensions were mainly found inside the default mode network, concentrated in the left anterior superior temporal gyrus and angular gyrus, which have been proposed as two connector hubs that bridge the multimodal experiential semantic system and the language-supported semantic system.

Will you read how I will read? Naturalistic fMRI predictors of emergent reading

Despite reading being an essential and almost universal skill in the developed world, reading proficiency varies substantially from person to person. To study why, the fMRI field is beginning to turn from single-word or nonword reading tasks to naturalistic stimuli like connected text and listening to stories. To study reading development in children just beginning to read, listening to stories is an appropriate paradigm because speech perception and phonological processing are important for, and are predictors of, reading proficiency. Our study examined the relationship between behavioral reading-related skills and the neural response to listening to stories in the fMRI environment. Functional MRI were gathered in a 3T TIM-Trio scanner. During the fMRI scan, children aged approximately 7 years listened to professionally narrated common short stories and answered comprehension questions following the narration. Analyses of the data used inter-subject correlation (ISC), and representational similarity analysis (RSA). Our primary finding is that ISC reveals areas of increased synchrony in both high- and low-performing emergent readers previously implicated in reading ability/disability. Of particular interest are that several previously identified brain regions (medial temporal gyrus (MTG), inferior frontal gyrus (IFG), inferior temporal gyrus (ITG)) were found to "synchronize" across higher reading ability participants, while lower reading ability participants had idiosyncratic activation patterns in these regions. Additionally, two regions (superior frontal gyrus (SFG) and another portion of ITG) were recruited by all participants, but their specific timecourse of activation depended on reading performance. These analyses support the idea that different brain regions involved in reading follow different developmental trajectories that correlate with reading proficiency on a spectrum rather than the usual dichotomy of poor readers versus strong readers.

Listening Effort in Tinnitus: A Pilot Study Employing a Light EEG Headset and Skin Conductance Assessment during the Listening to a Continuous Speech Stimulus under Different SNR Conditions

Background noise elicits listening effort. What else is tinnitus if not an endogenous background noise? From such reasoning, we hypothesized the occurrence of increased listening effort in tinnitus patients during listening tasks. Such a hypothesis was tested by investigating some indices of listening effort through electroencephalographic and skin conductance, particularly parietal and frontal alpha and electrodermal activity (EDA). Furthermore, tinnitus distress questionnaires (THI and TQ12-I) were employed. Parietal alpha values were positively correlated to TQ12-I scores, and both were negatively correlated to EDA; Pre-stimulus frontal alpha correlated with the THI score in our pilot study; finally, results showed a general trend of increased frontal alpha activity in the tinnitus group in comparison to the control group. Parietal alpha during the listening to stimuli, positively correlated to the TQ12-I, appears to reflect a higher listening effort in tinnitus patients and the perception of tinnitus symptoms. The negative correlation between both listening effort (parietal alpha) and tinnitus symptoms perception (TQ12-I scores) with EDA levels could be explained by a less responsive sympathetic nervous system to prepare the body to expend increased energy during the "fight or flight" response, due to pauperization of energy from tinnitus perception.

Explaining neural activity in human listeners with deep learning via natural language processing of narrative text

Deep learning (DL) approaches may also inform the analysis of human brain activity. Here, a state-of-art DL tool for natural language processing, the Generative Pre-trained Transformer version 2 (GPT-2), is shown to generate meaningful neural encodings in functional MRI during narrative listening. Linguistic features of word unpredictability (surprisal) and contextual importance (saliency) were derived from the GPT-2 applied to the text of a 12-min narrative. Segments of variable duration (from 15 to 90 s) defined the context for the next word, resulting in different sets of neural predictors for functional MRI signals recorded in 27 healthy listeners of the narrative. GPT-2 surprisal, estimating word prediction errors from the artificial network, significantly explained the neural data in superior and middle temporal gyri (bilaterally), in anterior and posterior cingulate cortices, and in the left prefrontal cortex. GPT-2 saliency, weighing the importance of context words, significantly explained the neural data for longer segments in left superior and middle temporal gyri. These results add novel support to the use of DL tools in the search for neural encodings in functional MRI. A DL language model like the GPT-2 may feature useful data about neural processes subserving language comprehension in humans, including next-word context-related prediction.

Auditory cortex hypoperfusion: a metabolic hallmark in Beta Thalassemia

Background

Sensorineural hearing loss in beta-thalassemia is common and it is generally associated with iron chelation therapy. However, data are scarce, especially on adult populations, and a possible involvement of the central auditory areas has not been investigated yet. We performed a multicenter cross-sectional audiological and single-center 3Tesla brain perfusion MRI study enrolling 77 transfusion-dependent/non transfusion-dependent adult patients and 56 healthy controls. Pure tone audiometry, demographics, clinical/laboratory and cognitive functioning data were recorded.

Results

Half of patients (52%) presented with high-frequency hearing deficit, with overt hypoacusia (Pure Tone Average (PTA) > 25 dB) in 35%, irrespective of iron chelation or clinical phenotype. Bilateral voxel clusters of significant relative hypoperfusion were found in the auditory cortex of beta-thalassemia patients, regardless of clinical phenotype. In controls and transfusion-dependent (but not in non-transfusion-dependent) patients, the relative auditory cortex perfusion values increased linearly with age (p < 0.04). Relative auditory cortex perfusion values showed a significant U-shaped correlation with PTA values among hearing loss patients, and a linear correlation with the full scale intelligence quotient (right side p = 0.01, left side p = 0.02) with its domain related to communication skills (right side p = 0.04, left side p = 0.07) in controls but not in beta-thalassemia patients. Audiometric test results did not correlate to cognitive test scores in any subgroup.

Conclusions

In conclusion, primary auditory cortex perfusion changes are a metabolic hallmark of adult beta-thalassemia, thus suggesting complex remodeling of the hearing function, that occurs regardless of chelation therapy and before clinically manifest hearing loss. The cognitive impact of perfusion changes is intriguing but requires further investigations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13023-021-01969-0.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.