Tracking competition and cognitive control during language comprehension with multi-voxel pattern analysis

Effects of Competition on Left Prefrontal and Temporal Cortex During Conceptual Comparison of Brand-Name Product Pictures: Analysis of fNIRS Using Tensor Decomposition

BACKGROUND/OBJECTIVES

Recent theories of the neurocognitive architecture of semantic memory have included a distinction between semantic control in the left inferior frontal gyrus (LIFG) and semantic representation in the left anterior temporal lobe (LATL). Support for this distinction has been found both in tasks in which high semantic selection demands have been instantiated and in tasks in which previous presentations of semantic information that compete with target information have been instantiated.

METHODS

In the current study, these manipulations were combined in a novel manner into a single task in which brand-name product pictures were used. Functional near-infrared spectroscopy (fNIRS) was used to measure hemodynamic activity and tensor decomposition, in addition to grand averaging, was used to analyze the fNIRS output.

RESULTS

Both analytic methods converged on the same set of findings. That is, in line with past studies, greater activity in the LIFG was observed in the competitive condition than in a repeated condition. However, unlike past studies, greater activity in the competitive condition was also observed in both the left and right anterior temporal lobes (ATLs).

CONCLUSIONS

While it was possible that the novel combination of high selection and competition into a single task unlocked a semantic selection mechanism in the bilateral ATL, a number of other post-hoc explanations were offered for this unusual finding, including a re-interpretation of the high-selection task as an ad hoc categorization task. Finally, the convergence of the tensor decomposition and grand averaging approaches on the same set of findings supported tensor decomposition as a viable approach to the analysis of fNIRS data.

Brain representations of lexical ambiguity: Disentangling homonymy, polysemy, and their meanings

In human languages, it is a common phenomenon for a single word to have multiple meanings. This study used fMRI to investigate how the brain processed different types of lexical ambiguity, and how it differentiated the meanings of ambiguous words. We focused on homonyms and polysemy that differed in the relatedness among multiple meanings. Participants (N = 35) performed a prime-target semantic relatedness task, where a specific meaning of an ambiguous word was primed. Results showed that homonyms elicited greater activation in bilateral dorsal prefrontal and posterior parietal cortices than polysemous words, suggesting that these regions may be more engaged in cognitive control when the meanings of ambiguous words are unrelated. Multivariate pattern analysis further revealed that meanings of homonyms with different syntactic categories were represented differently in the frontal and temporal cortices. The findings highlighted the importance of semantic relations and grammatical factors in the brain's representation of lexical ambiguities.

Cognitive control mechanisms in language processing: are there both within- and across-task conflict adaptation effects?

Cognitive conflict is regarded as a crucial factor in triggering subsequent adjustments in cognitive control. Recent studies have suggested that the implementation of control following conflict detection might be domain-general in that conflict experienced in the language domain recruits control processes that deal with conflict experienced in non-linguistic domains. During language comprehension, humans often have to recover from conflicting interpretations as quickly and accurately as possible. In this study, we investigate how people adapt to conflict experienced during processing semantically ambiguous sentences. Experiments 1 to 3 investigated whether such semantic conflict produces the congruency sequence effect (CSE) within a subsequent manual Stroop task and whether Stroop conflict leads to adjustments in semantic processing. Experiments 4 to 6 investigated whether semantic conflict results in conflict adaptation in subsequent sentence processing. Although processing conflict was consistently experienced during sentence reading and in the Stroop task, we did not observe any within-task or cross-task adaptation effects. Specifically, there were no cross-task CSEs from the linguistic task to the Stroop task and vice versa (experiments 1-3)-speaking against the assumption of domain-general control mechanisms. Moreover, experiencing conflict within a semantically ambiguous sentence did not ease the processing of a subsequent ambiguous sentence (experiments 4-6). Implications of these findings will be discussed.

Causal Contributions of the Domain-General (Multiple Demand) and the Language-Selective Brain Networks to Perceptual and Semantic Challenges in Speech Comprehension

Listening to spoken language engages domain-general multiple demand (MD; frontoparietal) regions of the human brain, in addition to domain-selective (frontotemporal) language regions, particularly when comprehension is challenging. However, there is limited evidence that the MD network makes a functional contribution to core aspects of understanding language. In a behavioural study of volunteers (n = 19) with chronic brain lesions, but without aphasia, we assessed the causal role of these networks in perceiving, comprehending, and adapting to spoken sentences made more challenging by acoustic-degradation or lexico-semantic ambiguity. We measured perception of and adaptation to acoustically degraded (noise-vocoded) sentences with a word report task before and after training. Participants with greater damage to MD but not language regions required more vocoder channels to achieve 50% word report, indicating impaired perception. Perception improved following training, reflecting adaptation to acoustic degradation, but adaptation was unrelated to lesion location or extent. Comprehension of spoken sentences with semantically ambiguous words was measured with a sentence coherence judgement task. Accuracy was high and unaffected by lesion location or extent. Adaptation to semantic ambiguity was measured in a subsequent word association task, which showed that availability of lower-frequency meanings of ambiguous words increased following their comprehension (word-meaning priming). Word-meaning priming was reduced for participants with greater damage to language but not MD regions. Language and MD networks make dissociable contributions to challenging speech comprehension: Using recent experience to update word meaning preferences depends on language-selective regions, whereas the domain-general MD network plays a causal role in reporting words from degraded speech.

Language Tasks and the Network Control Role of the Left Inferior Frontal Gyrus

Recent work has combined cognitive neuroscience and control theory to make predictions about cognitive control functions. Here, we test a link between whole-brain theories of semantics and the role of the left inferior frontal gyrus (LIFG) in controlled language performance using network control theory (NCT), a branch of systems engineering. Specifically, we examined whether two properties of node controllability, boundary and modal controllability, were linked to semantic selection and retrieval on sentence completion and verb generation tasks. We tested whether the controllability of the left IFG moderated language selection and retrieval costs and the effects of continuous θ burst stimulation (cTBS), an inhibitory form of transcranial magnetic stimulation (TMS) on behavior in 41 human subjects (25 active, 16 sham). We predicted that boundary controllability, a measure of the theoretical ability of a node to integrate and segregate brain networks, would be linked to word selection in the contextually-rich sentence completion task. In contrast, we expected that modal controllability, a measure of the theoretical ability of a node to drive the brain into specifically hard-to-reach states, would be linked to retrieval on the low-context verb generation task. Boundary controllability was linked to selection and to the ability of TMS to reduce response latencies on the sentence completion task. In contrast, modal controllability was not linked to performance on the tasks or TMS effects. Overall, our results suggest a link between the network integrating role of the LIFG and selection and the overall semantic demands of sentence completion.

Influence of homophone processing during auditory language comprehension on executive control processes: A dual-task paradigm

In the present preregistered study, we evaluated the possibility of a shared cognitive mechanism during verbal and non-verbal tasks and therefore the implication of domain-general cognitive control during language comprehension. We hypothesized that a behavioral cost will be observed during a dual-task including both verbal and non-verbal difficult processing. Specifically, to test this claim, we designed a dual-task paradigm involving: an auditory language comprehension task (sentence comprehension) and a non-verbal Flanker task (including congruent and incongruent trials). We manipulated sentence ambiguity and evaluated if the ambiguity effect modified behavioral performances in the non-verbal Flanker task. Under the assumption that ambiguous sentences induce a more difficult process than unambiguous sentences, we expected non-verbal flanker task performances to be impaired only when a simultaneous difficult language processing is performed. This would be specifically reflected by a performance cost during incongruent Flanker items only during ambiguous sentence presentation. Conversely, we observed a facilitatory effect for the incongruent Flanker items during ambiguous sentence suggesting better non-verbal inhibitory performances when an ambiguous sentence was simultaneously processed. Exploratory data analysis suggests that this effect is not only related to a more difficult language processing but also to the previous (n-1) Flanker item. Indeed, results showed that incongruent n-1 Flanker items led to a facilitation of the incongruent synchronized Flanker items only when ambiguous sentences were conjointly presented. This result, even if it needs to be corroborated in future studies, suggests that the recruitment of executive control mechanisms facilitates subsequent executive control implication during difficult language processing. The present study suggests a common executive control mechanism during difficult verbal and non-verbal tasks.

A Critical Review of Network-Based and Distributional Approaches to Semantic Memory Structure and Processes

Some of the earliest work on understanding how concepts are organized in memory used a network-based approach, where words or concepts are represented as nodes, and relationships between words are represented by links between nodes. Over the past two decades, advances in network science and graph theoretical methods have led to the development of computational semantic networks. This review provides a modern perspective on how computational semantic networks have proven to be useful tools to investigate the structure of semantic memory as well as search and retrieval processes within semantic memory, to ultimately model performance in a wide variety of cognitive tasks. Regarding representation, the review focuses on the distinctions and similarities between network-based (based on behavioral norms) approaches and more recent distributional (based on natural language corpora) semantic models, and the potential overlap between the two approaches. Capturing the type of relation between concepts appears to be particularly important in this modeling endeavor. Regarding processes, the review focuses on random walk models and the degree to which retrieval processes demand attention in pursuit of given task goals, which dovetails with the type of relation retrieved during tasks. Ultimately, this review provides a critical assessment of how the network perspective can be reconciled with distributional and machine-learning-based perspectives to meaning representation, and describes how cognitive network science provides a useful conceptual toolkit to probe both the structure and retrieval processes within semantic memory.

Semantic Knowledge of Famous People and Places Is Represented in Hippocampus and Distinct Cortical Networks

Studies have found that anterior temporal lobe (ATL) is critical for detailed knowledge of object categories, suggesting that it has an important role in semantic memory. However, in addition to information about entities, such as people and objects, semantic memory also encompasses information about places. We tested predictions stemming from the PMAT model, which proposes there are distinct systems that support different kinds of semantic knowledge: an anterior temporal (AT) network, which represents information about entities; and a posterior medial (PM) network, which represents information about places. We used representational similarity analysis to test for activation of semantic features when human participants viewed pictures of famous people and places, while controlling for visual similarity. We used machine learning techniques to quantify the semantic similarity of items based on encyclopedic knowledge in the Wikipedia page for each item and found that these similarity models accurately predict human similarity judgments. We found that regions within the AT network, including ATL and inferior frontal gyrus, represented detailed semantic knowledge of people. In contrast, semantic knowledge of places was represented within PM network areas, including precuneus, posterior cingulate cortex, angular gyrus, and parahippocampal cortex. Finally, we found that hippocampus, which has been proposed to serve as an interface between the AT and PM networks, represented fine-grained semantic similarity for both individual people and places. Our results provide evidence that semantic knowledge of people and places is represented separately in AT and PM areas, whereas hippocampus represents semantic knowledge of both categories.SIGNIFICANCE STATEMENT Humans acquire detailed semantic knowledge about people (e.g., their occupation and personality) and places (e.g., their cultural or historical significance). While research has demonstrated that brain regions preferentially respond to pictures of people and places, less is known about whether these regions preferentially represent semantic knowledge about specific people and places. We used machine learning techniques to develop a model of semantic similarity based on information available from Wikipedia, validating the model against similarity ratings from human participants. Using our computational model, we found that semantic knowledge about people and places is represented in distinct anterior temporal and posterior medial brain networks, respectively. We further found that hippocampus, an important memory center, represented semantic knowledge for both types of stimuli.

Barking up the right tree: Univariate and multivariate fMRI analyses of homonym comprehension

Homonyms are a critical test case for investigating how the brain resolves ambiguity in language and, more generally, how context influences semantic processing. Previous neuroimaging studies have associated processing of homonyms with greater engagement of regions involved in executive control of semantic processing. However, the precise role of these areas and the involvement of semantic representational regions in homonym comprehension remain elusive. We addressed this by combining univariate and multivariate fMRI analyses of homonym processing. We tested whether multi-voxel activation patterns could discriminate between presentations of the same homonym in different contexts (e.g., bark following tree vs. bark following dog). The ventral anterior temporal lobe, implicated in semantic representation but not previously in homonym comprehension, showed this meaning-specific coding, despite not showing increased mean activation for homonyms. Within inferior frontal gyrus (IFG), a key site for semantic control, there was a dissociation between pars orbitalis, which also showed meaning-specific coding, and pars triangularis, which discriminated more generally between semantically related and unrelated word pairs. IFG effects were goal-dependent, only occurring when the task required semantic decisions, in line with a top-down control function. Finally, posterior middle temporal cortex showed a hybrid pattern of responses, supporting the idea that it acts as an interface between semantic representations and the control system. The study provides new evidence for context-dependent coding in the semantic system and clarifies the role of control regions in processing ambiguity. It also highlights the importance of combining univariate and multivariate neuroimaging data to fully elucidate the role of a brain region in semantic cognition.

The Neural Time Course of Semantic Ambiguity Resolution in Speech Comprehension

Semantically ambiguous words challenge speech comprehension, particularly when listeners must select a less frequent (subordinate) meaning at disambiguation. Using combined magnetoencephalography (MEG) and EEG, we measured neural responses associated with distinct cognitive operations during semantic ambiguity resolution in spoken sentences: (i) initial activation and selection of meanings in response to an ambiguous word and (ii) sentence reinterpretation in response to subsequent disambiguation to a subordinate meaning. Ambiguous words elicited an increased neural response approximately 400-800 msec after their acoustic offset compared with unambiguous control words in left frontotemporal MEG sensors, corresponding to sources in bilateral frontotemporal brain regions. This response may reflect increased demands on processes by which multiple alternative meanings are activated and maintained until later selection. Disambiguating words heard after an ambiguous word were associated with marginally increased neural activity over bilateral temporal MEG sensors and a central cluster of EEG electrodes, which localized to similar bilateral frontal and left temporal regions. This later neural response may reflect effortful semantic integration or elicitation of prediction errors that guide reinterpretation of previously selected word meanings. Across participants, the amplitude of the ambiguity response showed a marginal positive correlation with comprehension scores, suggesting that sentence comprehension benefits from additional processing around the time of an ambiguous word. Better comprehenders may have increased availability of subordinate meanings, perhaps due to higher quality lexical representations and reflected in a positive correlation between vocabulary size and comprehension success.

The relationship between proceduralization and cognitive control

In this paper we discuss the relationship between the proceduralization process and cognitive control. Our discussion is based on theoretical models and clinical practices, with a focus on child language impairment. We argue that the same skill may require more or less cognitive control depending on the differences in contextual conditions.

Mental Fatigue and Functional Near-Infrared Spectroscopy (fNIRS) - Based Assessment of Cognitive Performance After Mild Traumatic Brain Injury

Pathological mental fatigue after mild traumatic brain injury (TBI-MF) is characterized by pronounced mental fatigue after cognitive activity. The neurological origin is unknown, and we aimed in the present study to investigate how prolonged mental activity affects cognitive performance and its neural correlates in individuals with TBI-MF. We recruited individuals with TBI-MF (n = 20) at least 5 months after injury, and age-matched healthy controls (n = 20). We used functional near-infrared spectroscopy (fNIRS) to assess hemodynamic changes in the frontal cortex. The self-assessed mental energy level was measured with a visual analog scale (VAS) before and after the experimental procedure. A battery of six neuropsychological tests including Stroop–Simon, Symbol Search, Digit Span, Parallel Serial Mental Operation (PaSMO), Sustained Attention and Working Memory test, and Digit Symbol Coding (DSC) were used. The sequence was repeated once after an 8 min sustained-attention test. The test procedure lasted 2½ h. The experimental procedure resulted in a decrease in mental energy in the TBI-MF group, compared to controls (interaction, p < 0.001, η_p² = 0.331). The TBI-MF group performed at a similar level on both DSC tests, whereas the controls improved their performance in the second session (interaction, p < 0.01, η_p² = 0.268). During the Stroop–Simon test, the fNIRS event-related response showed no time effect. However, the TBI-MF group exhibited lower oxygenated hemoglobin (oxy-Hb) concentrations in the frontal polar area (FPA), ventrolateral motor cortex, and dorsolateral prefrontal cortex (DLPFC) from the beginning of the test session. A Stroop and Group interaction was found in the left ventrolateral prefrontal cortex showing that the TBI-MF group did have the same oxy-Hb concentration for both congruent and incongruent trials, whereas the controls had more oxy-Hb in the incongruent trial compared to the congruent trial (interaction, p < 0.01, η_p² = 0.227). In sum these results indicate that individuals with TBI-MF have a reduced ability to recruit the frontal cortex, which is correlated with self-reported mental fatigue. This may result both in deterioration of cognitive function and the experience of a mental fatigue after extended mental activity.

Balancing Prediction and Sensory Input in Speech Comprehension: The Spatiotemporal Dynamics of Word Recognition in Context

Spoken word recognition in context is remarkably fast and accurate, with recognition times of ∼200 ms, typically well before the end of the word. The neurocomputational mechanisms underlying these contextual effects are still poorly understood. This study combines source-localized electroencephalographic and magnetoencephalographic (EMEG) measures of real-time brain activity with multivariate representational similarity analysis to determine directly the timing and computational content of the processes evoked as spoken words are heard in context, and to evaluate the respective roles of bottom-up and predictive processing mechanisms in the integration of sensory and contextual constraints. Male and female human participants heard simple (modifier-noun) English phrases that varied in the degree of semantic constraint that the modifier (W1) exerted on the noun (W2), as in pairs, such as "yellow banana." We used gating tasks to generate estimates of the probabilistic predictions generated by these constraints as well as measures of their interaction with the bottom-up perceptual input for W2. Representation similarity analysis models of these measures were tested against electroencephalographic and magnetoencephalographic brain data across a bilateral fronto-temporo-parietal language network. Consistent with probabilistic predictive processing accounts, we found early activation of semantic constraints in frontal cortex (LBA45) as W1 was heard. The effects of these constraints (at 100 ms after W2 onset in left middle temporal gyrus and at 140 ms in left Heschl's gyrus) were only detectable, however, after the initial phonemes of W2 had been heard. Within an overall predictive processing framework, bottom-up sensory inputs are still required to achieve early and robust spoken word recognition in context.SIGNIFICANCE STATEMENT Human listeners recognize spoken words in natural speech contexts with remarkable speed and accuracy, often identifying a word well before all of it has been heard. In this study, we investigate the brain systems that support this important capacity, using neuroimaging techniques that can track real-time brain activity during speech comprehension. This makes it possible to locate the brain areas that generate predictions about upcoming words and to show how these expectations are integrated with the evidence provided by the speech being heard. We use the timing and localization of these effects to provide the most specific account to date of how the brain achieves an optimal balance between prediction and sensory input in the interpretation of spoken language.

Evidence of semantic processing impairments in behavioural variant frontotemporal dementia and Parkinson's disease

PURPOSE OF REVIEW

Category-specific impairments caused by brain damage can provide important insights into how semantic concepts are organized in the brain. Recent research has demonstrated that disease to sensory and motor cortices can impair perceptual feature knowledge important to the representation of semantic concepts. This evidence supports the grounded cognition theory of semantics, the view that lexical knowledge is partially grounded in perceptual experience and that sensory and motor regions support semantic representations. Less well understood, however, is how heteromodal semantic hubs work to integrate and process semantic information.

RECENT FINDINGS

Although the majority of semantic research to date has focused on how sensory cortical areas are important for the representation of semantic features, new research explores how semantic memory is affected by neurodegeneration in regions important for semantic processing. Here, we review studies that demonstrate impairments to abstract noun knowledge in behavioural variant frontotemporal degeneration (bvFTD) and to action verb knowledge in Parkinson's disease, and discuss how these deficits relate to disease of the semantic selection network.

SUMMARY

Findings demonstrate that semantic selection processes are supported by the left inferior frontal gyrus (LIFG) and basal ganglia, and that disease to these regions in bvFTD and Parkinson's disease can lead to categorical impairments for abstract nouns and action verbs, respectively.

Not single brain areas but a network is involved in language: Applications in presurgical planning

OBJECTIVES

Language is an important human function, and is a determinant of the quality of life. In conditions such as brain lesions, disruption of the language function may occur, and lesion resection is a solution for that. Presurgical planning to determine the language-related brain areas would enhance the chances of language preservation after the operation; however, availability of a normative language template is essential.

PATIENTS AND METHODS

In this study, using data from 60 young individuals who were meticulously checked for mental and physical health, and using fMRI and robust imaging and data analysis methods, functional brain maps for the language production, perception and semantic were produced.

RESULTS

The obtained templates showed that the language function should be considered as the product of the collaboration of a network of brain regions, instead of considering only few brain areas to be involved in that.

CONCLUSION

This study has important clinical applications, and extends our knowledge on the neuroanatomy of the language function.

Striking a chord with healthy aging: memory system cooperation is related to preserved configural response learning in older adults

Evidence from tasks that primarily tap either hippocampal- or striatal-based memory systems suggests that although these systems often compete for control of behavior, aging is associated with greater cooperation between them. This may stem from altered prefrontal cortex function. Here, we use a configural response task designed to engage both memory systems to test how age affects their interaction with cortical regions including the prefrontal cortex. We found that although older and younger adults learned just as well, older adults showed greater initial activation in cortical networks associated with visuospatial-action mapping and resolving conflict for competing memory representations. Older adults also showed greater functional coupling of the striatum with the left inferior frontal gyrus, in parallel with similar hippocampal coupling to ventral visual regions as young adults. Overall, our results support the proposal that aging is associated with more cooperative memory systems, but we did not find that greater cooperation is associated with less interaction between the prefrontal cortex and core memory system structures during learning.