Language and cognition interaction neural mechanisms

Memristive neural network circuit implementation of associative learning with overshadowing and blocking

In the field of second language acquisition, overshadowing and blocking by cue competition effects in classical conditioning affect the learning and expression of human cognitive associations. In this work, a memristive neural network circuit based on neurobiological mechanisms is proposed, which consists of synapse module, neuron module, and control module. In particular, the designed network introduces an inhibitory interneuron to divide memristive synapses into excitatory and inhibitory memristive synapses, so as to mimic synaptic plasticity better. In addition, the proposed circuit can implement six functions of second language acquisition conditioning, including learning, overshadowing, blocking, recovery from overshadowing, recovery from blocking, and long-term effect of overshadowing over time leading to blocking. Overshadowing, which denotes that the more salient stimulus overshadows the learning of the less salient stimulus when two stimuli differ in salience, reduces the associative strength acquired by the less salient stimulus. Blocking, which indicates that pretraining on one stimulus blocks learning about a second stimulus, inhibits the associative strength acquired by a second stimulus. Finally, the correctness and effectiveness of implementing functions mentioned above are verified by the simulation results in PSPICE. Through further research, the proposed circuit is applied to bionic devices such as social robots or educational robots, which can address language and cognitive disorders via assisted learning and training.

Data-driven machine learning models for decoding speech categorization from evoked brain responses

Objective.Categorical perception (CP) of audio is critical to understand how the human brain perceives speech sounds despite widespread variability in acoustic properties. Here, we investigated the spatiotemporal characteristics of auditory neural activity that reflects CP for speech (i.e. differentiates phonetic prototypes from ambiguous speech sounds).Approach.We recorded 64-channel electroencephalograms as listeners rapidly classified vowel sounds along an acoustic-phonetic continuum. We used support vector machine classifiers and stability selection to determine when and where in the brain CP was best decoded across space and time via source-level analysis of the event-related potentials.Main results. We found that early (120 ms) whole-brain data decoded speech categories (i.e. prototypical vs. ambiguous tokens) with 95.16% accuracy (area under the curve 95.14%;F1-score 95.00%). Separate analyses on left hemisphere (LH) and right hemisphere (RH) responses showed that LH decoding was more accurate and earlier than RH (89.03% vs. 86.45% accuracy; 140 ms vs. 200 ms). Stability (feature) selection identified 13 regions of interest (ROIs) out of 68 brain regions [including auditory cortex, supramarginal gyrus, and inferior frontal gyrus (IFG)] that showed categorical representation during stimulus encoding (0-260 ms). In contrast, 15 ROIs (including fronto-parietal regions, IFG, motor cortex) were necessary to describe later decision stages (later 300-800 ms) of categorization but these areas were highly associated with the strength of listeners' categorical hearing (i.e. slope of behavioral identification functions).Significance.Our data-driven multivariate models demonstrate that abstract categories emerge surprisingly early (∼120 ms) in the time course of speech processing and are dominated by engagement of a relatively compact fronto-temporal-parietal brain network.

Data-driven machine learning models for decoding speech categorization from evoked brain responses

Objective.Categorical perception (CP) of audio is critical to understand how the human brain perceives speech sounds despite widespread variability in acoustic properties. Here, we investigated the spatiotemporal characteristics of auditory neural activity that reflects CP for speech (i.e. differentiates phonetic prototypes from ambiguous speech sounds).Approach.We recorded 64-channel electroencephalograms as listeners rapidly classified vowel sounds along an acoustic-phonetic continuum. We used support vector machine classifiers and stability selection to determine when and where in the brain CP was best decoded across space and time via source-level analysis of the event-related potentials.Main results. We found that early (120 ms) whole-brain data decoded speech categories (i.e. prototypical vs. ambiguous tokens) with 95.16% accuracy (area under the curve 95.14%;F1-score 95.00%). Separate analyses on left hemisphere (LH) and right hemisphere (RH) responses showed that LH decoding was more accurate and earlier than RH (89.03% vs. 86.45% accuracy; 140 ms vs. 200 ms). Stability (feature) selection identified 13 regions of interest (ROIs) out of 68 brain regions [including auditory cortex, supramarginal gyrus, and inferior frontal gyrus (IFG)] that showed categorical representation during stimulus encoding (0-260 ms). In contrast, 15 ROIs (including fronto-parietal regions, IFG, motor cortex) were necessary to describe later decision stages (later 300-800 ms) of categorization but these areas were highly associated with the strength of listeners' categorical hearing (i.e. slope of behavioral identification functions).Significance.Our data-driven multivariate models demonstrate that abstract categories emerge surprisingly early (∼120 ms) in the time course of speech processing and are dominated by engagement of a relatively compact fronto-temporal-parietal brain network.

Chinese-English bilinguals show linguistic-perceptual links in the brain associating short spoken phrases with corresponding real-world natural action sounds by semantic category

Higher cognitive functions such as linguistic comprehension must ultimately relate to perceptual systems in the brain, though how and why this forms remains unclear. Different brain networks that mediate perception when hearing real-world natural sounds has recently been proposed to respect a taxonomic model of acoustic-semantic categories. Using functional magnetic resonance imaging (fMRI) with Chinese/English bilingual listeners, the present study explored whether reception of short spoken phrases, in both Chinese (Mandarin) and English, describing corresponding sound-producing events would engage overlapping brain regions at a semantic category level. The results revealed a double-dissociation of cortical regions that were preferential for representing knowledge of human versus environmental action events, whether conveyed through natural sounds or the corresponding spoken phrases depicted by either language. These findings of cortical hubs exhibiting linguistic-perceptual knowledge links at a semantic category level should help to advance neurocomputational models of the neurodevelopment of language systems.

English spoken word segmentation activates the prefrontal cortex and temporo-parietal junction in Chinese ESL learners: A functional near-infrared spectroscopy (fNIRS) study

A direct measure of spoken lexical processing based on neuroimaging technology would provide us useful information to understand the neural mechanisms underlying speech or auditory language processing. The neural mechanisms of spoken word segmentation for English as a second language (ESL) learners remain elusive. The present study, using functional near-infrared spectroscopy (fNIRS), addresses this issue by measuring hemodynamic responses in the temporo-parietal junction (TPJ) and the prefrontal cortex (PFC) in a word-spotting task, designed with two task conditions (easy vs. difficult). Thirty participants, divided into a high listening proficiency group (HLG) and a low listening proficiency group (LLG), were tested. Results revealed significantly less TPJ activation in the HLG than in the LLG. Further analyses supported this result by showing that activation in the TPJ was in a negative correlation with listening proficiency. This association appears to be related to the more efficient use of processing resources in a bottom-up fashion for accurate and efficient sensory representations in high proficient language learners. In contrast, cortical activation in the PFC increased with listening proficiency and was stronger in the difficult task condition than in the easy task condition, implying that recruitment of top-down cognitive control functions might play a role in word segmentation. Our results suggest that the combination of the functions mediated via bottom-up sensory input processing (demonstrated in the TPJ activation) and top-down cognitive processing (demonstrated in the PFC activation) are crucial for ESL listeners' spoken word segmentation.

Hearing and orally mimicking different acoustic-semantic categories of natural sound engage distinct left hemisphere cortical regions

Oral mimicry is thought to represent an essential process for the neurodevelopment of spoken language systems in infants, the evolution of language in hominins, and a process that could possibly aid recovery in stroke patients. Using functional magnetic resonance imaging (fMRI), we previously reported a divergence of auditory cortical pathways mediating perception of specific categories of natural sounds. However, it remained unclear if or how this fundamental sensory organization by the brain might relate to motor output, such as sound mimicry. Here, using fMRI, we revealed a dissociation of activated brain regions preferential for hearing with the intent to imitate and the oral mimicry of animal action sounds versus animal vocalizations as distinct acoustic-semantic categories. This functional dissociation may reflect components of a rudimentary cortical architecture that links systems for processing acoustic-semantic universals of natural sound with motor-related systems mediating oral mimicry at a category level. The observation of different brain regions involved in different aspects of oral mimicry may inform targeted therapies for rehabilitation of functional abilities after stroke.

Physics of mind: Experimental confirmations of theoretical predictions

What is common among Newtonian mechanics, statistical physics, thermodynamics, quantum physics, the theory of relativity, astrophysics and the theory of superstrings? All these areas of physics have in common a methodology, which is discussed in the first few lines of the review. Is a physics of the mind possible? Is it possible to describe how a mind adapts in real time to changes in the physical world through a theory based on a few basic laws? From perception and elementary cognition to emotions and abstract ideas allowing high-level cognition and executive functioning, at nearly all levels of study, the mind shows variability and uncertainties. Is it possible to turn psychology and neuroscience into so-called "hard" sciences? This review discusses several established first principles for the description of mind and their mathematical formulations. A mathematical model of mind is derived from these principles. This model includes mechanisms of instincts, emotions, behavior, cognition, concepts, language, intuitions, and imagination. We clarify fundamental notions such as the opposition between the conscious and the unconscious, the knowledge instinct and aesthetic emotions, as well as humans' universal abilities for symbols and meaning. In particular, the review discusses in length evolutionary and cognitive functions of aesthetic emotions and musical emotions. Several theoretical predictions are derived from the model, some of which have been experimentally confirmed. These empirical results are summarized and we introduce new theoretical developments. Several unsolved theoretical problems are proposed, as well as new experimental challenges for future research.

Functional Preference for Object Sounds and Voices in the Brain of Early Blind and Sighted Individuals

Sounds activate occipital regions in early blind individuals. However, how different sound categories map onto specific regions of the occipital cortex remains a matter of debate. We used fMRI to characterize brain responses of early blind and sighted individuals to familiar object sounds, human voices, and their respective low-level control sounds. In addition, sighted participants were tested while viewing pictures of faces, objects, and phase-scrambled control pictures. In both early blind and sighted, a double dissociation was evidenced in bilateral auditory cortices between responses to voices and object sounds: Voices elicited categorical responses in bilateral superior temporal sulci, whereas object sounds elicited categorical responses along the lateral fissure bilaterally, including the primary auditory cortex and planum temporale. Outside the auditory regions, object sounds also elicited categorical responses in the left lateral and in the ventral occipitotemporal regions in both groups. These regions also showed response preference for images of objects in the sighted group, thus suggesting a functional specialization that is independent of sensory input and visual experience. Between-group comparisons revealed that, only in the blind group, categorical responses to object sounds extended more posteriorly into the occipital cortex. Functional connectivity analyses evidenced a selective increase in the functional coupling between these reorganized regions and regions of the ventral occipitotemporal cortex in the blind group. In contrast, vocal sounds did not elicit preferential responses in the occipital cortex in either group. Nevertheless, enhanced voice-selective connectivity between the left temporal voice area and the right fusiform gyrus were found in the blind group. Altogether, these findings suggest that, in the absence of developmental vision, separate auditory categories are not equipotent in driving selective auditory recruitment of occipitotemporal regions and highlight the presence of domain-selective constraints on the expression of cross-modal plasticity.

Auditory object perception: A neurobiological model and prospective review

Interaction with the world is a multisensory experience, but most of what is known about the neural correlates of perception comes from studying vision. Auditory inputs enter cortex with its own set of unique qualities, and leads to use in oral communication, speech, music, and the understanding of emotional and intentional states of others, all of which are central to the human experience. To better understand how the auditory system develops, recovers after injury, and how it may have transitioned in its functions over the course of hominin evolution, advances are needed in models of how the human brain is organized to process real-world natural sounds and "auditory objects". This review presents a simple fundamental neurobiological model of hearing perception at a category level that incorporates principles of bottom-up signal processing together with top-down constraints of grounded cognition theories of knowledge representation. Though mostly derived from human neuroimaging literature, this theoretical framework highlights rudimentary principles of real-world sound processing that may apply to most if not all mammalian species with hearing and acoustic communication abilities. The model encompasses three basic categories of sound-source: (1) action sounds (non-vocalizations) produced by 'living things', with human (conspecific) and non-human animal sources representing two subcategories; (2) action sounds produced by 'non-living things', including environmental sources and human-made machinery; and (3) vocalizations ('living things'), with human versus non-human animals as two subcategories therein. The model is presented in the context of cognitive architectures relating to multisensory, sensory-motor, and spoken language organizations. The models' predictive values are further discussed in the context of anthropological theories of oral communication evolution and the neurodevelopment of spoken language proto-networks in infants/toddlers. These phylogenetic and ontogenetic frameworks both entail cortical network maturations that are proposed to at least in part be organized around a number of universal acoustic-semantic signal attributes of natural sounds, which are addressed herein.

Neural reuse of action perception circuits for language, concepts and communication

Neurocognitive and neurolinguistics theories make explicit statements relating specialized cognitive and linguistic processes to specific brain loci. These linking hypotheses are in need of neurobiological justification and explanation. Recent mathematical models of human language mechanisms constrained by fundamental neuroscience principles and established knowledge about comparative neuroanatomy offer explanations for where, when and how language is processed in the human brain. In these models, network structure and connectivity along with action- and perception-induced correlation of neuronal activity co-determine neurocognitive mechanisms. Language learning leads to the formation of action perception circuits (APCs) with specific distributions across cortical areas. Cognitive and linguistic processes such as speech production, comprehension, verbal working memory and prediction are modelled by activity dynamics in these APCs, and combinatorial and communicative-interactive knowledge is organized in the dynamics within, and connections between APCs. The network models and, in particular, the concept of distributionally-specific circuits, can account for some previously not well understood facts about the cortical 'hubs' for semantic processing and the motor system's role in language understanding and speech sound recognition. A review of experimental data evaluates predictions of the APC model and alternative theories, also providing detailed discussion of some seemingly contradictory findings. Throughout, recent disputes about the role of mirror neurons and grounded cognition in language and communication are assessed critically.

Divergent Human Cortical Regions for Processing Distinct Acoustic-Semantic Categories of Natural Sounds: Animal Action Sounds vs. Vocalizations

A major gap in our understanding of natural sound processing is knowledge of where or how in a cortical hierarchy differential processing leads to categorical perception at a semantic level. Here, using functional magnetic resonance imaging (fMRI) we sought to determine if and where cortical pathways in humans might diverge for processing action sounds vs. vocalizations as distinct acoustic-semantic categories of real-world sound when matched for duration and intensity. This was tested by using relatively less semantically complex natural sounds produced by non-conspecific animals rather than humans. Our results revealed a striking double-dissociation of activated networks bilaterally. This included a previously well described pathway preferential for processing vocalization signals directed laterally from functionally defined primary auditory cortices to the anterior superior temporal gyri, and a less well-described pathway preferential for processing animal action sounds directed medially to the posterior insulae. We additionally found that some of these regions and associated cortical networks showed parametric sensitivity to high-order quantifiable acoustic signal attributes and/or to perceptual features of the natural stimuli, such as the degree of perceived recognition or intentional understanding. Overall, these results supported a neurobiological theoretical framework for how the mammalian brain may be fundamentally organized to process acoustically and acoustic-semantically distinct categories of ethologically valid, real-world sounds.

Physics of the Mind

Is it possible to turn psychology into “hard science”? Physics of the mind follows the fundamental methodology of physics in all areas where physics have been developed. What is common among Newtonian mechanics, statistical physics, quantum physics, thermodynamics, theory of relativity, astrophysics… and a theory of superstrings? The common among all areas of physics is a methodology of physics discussed in the first few lines of the paper. Is physics of the mind possible? Is it possible to describe the mind based on the few first principles as physics does? The mind with its variabilities and uncertainties, the mind from perception and elementary cognition to emotions and abstract ideas, to high cognition. Is it possible to turn psychology and neuroscience into “hard” sciences? The paper discusses established first principles of the mind, their mathematical formulations, and a mathematical model of the mind derived from these first principles, mechanisms of concepts, emotions, instincts, behavior, language, cognition, intuitions, conscious and unconscious, abilities for symbols, functions of the beautiful and musical emotions in cognition and evolution. Some of the theoretical predictions have been experimentally confirmed. This research won national and international awards. In addition to summarizing existing results the paper describes new development theoretical and experimental. The paper discusses unsolved theoretical problems as well as experimental challenges for future research.

Aesthetic Chills: Knowledge-Acquisition, Meaning-Making, and Aesthetic Emotions

This article addresses the relation between aesthetic emotions, knowledge-acquisition, and meaning-making. We briefly review theoretical foundations and present experimental data related to aesthetic chills. These results suggest that aesthetic chills are inhibited by exposing the subject to an incoherent prime prior to the chill-eliciting stimulation and that a meaningful prime makes the aesthetic experience more pleasurable than a neutral or an incoherent one. Aesthetic chills induced by narrative structures seem to be related to the pinnacle of the story, to have a significant calming effect and subjects describe a strong empathy for the characters. We discuss the relation between meaning-making and aesthetic emotions at the psychological, physiological, narratological, and mathematical levels and propose a series of hypotheses to be tested in future research.

Internally directed cognition and mindfulness: an integrative perspective derived from predictive and reactive control systems theory

In the present paper, we will apply the predictive and reactive control systems (PARCS) theory as a framework that integrates competing theories of neural substrates of awareness by describing the "default mode network" (DMN) and anterior insula (AI) as parts of two different behavioral and homeostatic control systems. The DMN, a network that becomes active at rest when there is no external stimulation or task to perform, has been implicated in self-reflective awareness and prospection. By contrast, the AI is associated with awareness and task-related attention. This has led to competing theories stressing the role of the DMN in self-awareness vs. the role of interoceptive and emotional information integration in the AI in awareness of the emotional moment. In PARCS, the respective functions of the DMN and AI in a specific control system explains their association with different qualities of awareness, and how mental states can shift from one state (e.g., prospective self-reflection) to the other (e.g., awareness of the emotional moment) depending on the relative dominance of control systems. These shifts between reactive and predictive control are part of processes that enable the intake of novel information, integration of this novel information within existing knowledge structures, and the creation of a continuous personal context in which novel information can be integrated and understood. As such, PARCS can explain key characteristics of mental states, such as their temporal and spatial focus (e.g., a focus on the here and now vs. the future; a first person vs. a third person perspective). PARCS further relates mental states to brain states and functions, such as activation of the DMN or hemispheric asymmetry in frontal cortical functions. Together, PARCS deepens the understanding of a broad range of mental states, including mindfulness, mind wandering, rumination, autobiographical memory, imagery, and the experience of self.

Brain. Conscious and unconscious mechanisms of cognition, emotions, and language

Conscious and unconscious brain mechanisms, including cognition, emotions and language are considered in this review. The fundamental mechanisms of cognition include interactions between bottom-up and top-down signals. The modeling of these interactions since the 1960s is briefly reviewed, analyzing the ubiquitous difficulty: incomputable combinatorial complexity (CC). Fundamental reasons for CC are related to the Gödel's difficulties of logic, a most fundamental mathematical result of the 20th century. Many scientists still "believed" in logic because, as the review discusses, logic is related to consciousness; non-logical processes in the brain are unconscious. CC difficulty is overcome in the brain by processes "from vague-unconscious to crisp-conscious" (representations, plans, models, concepts). These processes are modeled by dynamic logic, evolving from vague and unconscious representations toward crisp and conscious thoughts. We discuss experimental proofs and relate dynamic logic to simulators of the perceptual symbol system. "From vague to crisp" explains interactions between cognition and language. Language is mostly conscious, whereas cognition is only rarely so; this clarifies much about the mind that might seem mysterious. All of the above involve emotions of a special kind, aesthetic emotions related to knowledge and to cognitive dissonances. Cognition-language-emotional mechanisms operate throughout the hierarchy of the mind and create all higher mental abilities. The review discusses cognitive functions of the beautiful, sublime, music.