The label-feedback effect is influenced by target category in visual search

The label-feedback hypothesis states that language can modulate visual processing. In particular, hearing or reading aloud target names (labels) speeds up performance in visual search tasks by facilitating target detection and such advantage is often measured against a condition where the target name is shown visually (i.e. via the same modality as the search task). The current study conceptually complements and expands previous investigations. The effect of a multimodal label presentation (i.e., an audio+visual, AV, priming label) in a visual search task is compared to that of a multimodal (i.e. white noise+visual, NV, label) and two unimodal (i.e. audio, A, label or visual, V, label) control conditions. The name of a category (i.e. a label at the superordinate level) is used as a cue, instead of the more commonly used target name (a basic level label), with targets belonging to one of three categories: garments, improper weapons, and proper weapons. These categories vary for their structure, improper weapons being an ad hoc category (i.e. context-dependent), unlike proper weapons and garments. The preregistered analysis shows an overall facilitation of visual search performance in the AV condition compared to the NV condition, confirming that the label-feedback effect may not be explained away by the effects of multimodal stimulation only and that it extends to superordinate labels. Moreover, exploratory analyses show that such facilitation is driven by the garments and proper weapons categories, rather than improper weapons. Thus, the superordinate label-feedback effect is modulated by the structural properties of a category. These findings are consistent with the idea that the AV condition prompts an "up-regulation" of the label, a requirement for enhancing the label's beneficial effects, but not when the label refers to an ad hoc category. They also highlight the peculiar status of the category of improper weapons and set it apart from that of proper weapons.

Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data

We release a database of cloze probability values, predictability ratings, and computational estimates for a sample of 205 English sentences (1726 words), aligned with previously released word-by-word reading time data (both self-paced reading and eye-movement records; Frank et al., Behavior Research Methods, 45(4), 1182-1190. 2013) and EEG responses (Frank et al., Brain and Language, 140, 1-11. 2015). Our analyses show that predictability ratings are the best predictors of the EEG signal (N400, P600, LAN) self-paced reading times, and eye movement patterns, when spillover effects are taken into account. The computational estimates are particularly effective at explaining variance in the eye-tracking data without spillover. Cloze probability estimates have decent overall psychometric accuracy and are the best predictors of early fixation patterns (first fixation duration). Our results indicate that the choice of the best measurement of word predictability in context critically depends on the processing index being considered.

Processing of visual hapaxes in picture naming task: An event-related potential study

Object recognition and visual categorization are typically swift and seemingly effortless tasks that involve numerous underlying processes. In our investigation, we utilized a picture naming task to explore the processing of rarely encountered objects (visual hapaxes) in comparison to common objects. Our aim was to determine the stage at which these rare objects are classified as unnamable. Contrary to our expectations and in contrast to some prior research on event-related potentials (ERPs) with novel and atypical objects, no differences between conditions were observed in the late time windows corresponding to the P300 or N400 components. However, distinctive patterns between hapaxes and common objects surfaced in three early time windows, corresponding to the posterior N1 and P2 waves, as well as a widespread N2 wave. According to the ERP data, the differentiation between hapaxes and common objects occurs within the first 380 ms of the processing line, involving only limited and indirect top-down influence.

Invariant representations in abstract concept grounding - the physical world in grounded cognition

Grounded cognition states that mental representations of concepts consist of experiential aspects. For example, the concept "cup" consists of the sensorimotor experiences from interactions with cups. Typical modalities in which concepts are grounded are: The sensorimotor system (including interoception), emotion, action, language, and social aspects. Here, we argue that this list should be expanded to include physical invariants (unchanging features of physical motion; e.g., gravity, momentum, friction). Research on physical reasoning consistently demonstrates that physical invariants are represented as fundamentally as other grounding substrates, and therefore should qualify. We assess several theories of concept representation (simulation, conceptual metaphor, conceptual spaces, predictive processing) and their positions on physical invariants. We find that the classic grounded cognition theories, simulation and conceptual metaphor theory, have not considered physical invariants, while conceptual spaces and predictive processing have. We conclude that physical invariants should be included into grounded cognition theories, and that the core mechanisms of simulation and conceptual metaphor theory are well suited to do this. Furthermore, conceptual spaces and predictive processing are very promising and should also be integrated with grounded cognition in the future.

Consensus-building conversation leads to neural alignment

Conversation is a primary means of social influence, but its effects on brain activity remain unknown. Previous work on conversation and social influence has emphasized public compliance, largely setting private beliefs aside. Here, we show that consensus-building conversation aligns future brain activity within groups, with alignment persisting through novel experiences participants did not discuss. Participants watched ambiguous movie clips during fMRI scanning, then conversed in groups with the goal of coming to a consensus about each clip's narrative. After conversation, participants' brains were scanned while viewing the clips again, along with novel clips from the same movies. Groups that reached consensus showed greater similarity of brain activity after conversation. Participants perceived as having high social status spoke more and signaled disbelief in others, and their groups had unequal turn-taking and lower neural alignment. By contrast, participants with central positions in their real-world social networks encouraged others to speak, facilitating greater group neural alignment. Socially central participants were also more likely to become neurally aligned to others in their groups.

Semantic and Syntactic Predictions in Reading Aloud: Are Good Predictors Good Statistical Learners?

Recent research suggests that becoming a fluent reader may partially rely on a domain-general statistical learning (SL) mechanism that allows a person to automatically extract predictable patterns from the sensory input. The goal of the present study was to investigate a potential link between SL and the ability to make linguistic predictions. All previous studies investigated quite general levels of reading ability rather than the dynamic process of making linguistic predictions. We thus used a recently developed predictive reading task, which consisted of having participants read aloud words that were preceded by either semantically or syntactically predictive contexts. To measure the componential nature of SL, we used a visual and an auditory SL task (VSL, ASL) and the classic serial reaction time task (SRT). General reading ability was assessed with a reading speed/comprehension test. The study was conducted online on a sample of 120 participants to make it possible to explore interindividual differences. The results showed only weak and sometimes even negative correlations between the various SL measures. ASL correlated positively and predicted general reading ability but neither semantic nor syntactic prediction effects. Similarly, one of the SRT measures was significantly associated with reading level and reading speed but not with linguistic prediction effects. In sum, there is little evidence that domain-general SL is a good predictor of people's ability to make domain-specific linguistic predictions. In contrast, SL shows a weak but significant association with general reading ability.

Natural language syntax complies with the free-energy principle

Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design-such as "minimal search" criteria from theoretical syntax-adhere to the FEP. This affords a greater degree of explanatory power to the FEP-with respect to higher language functions-and offers linguistics a grounding in first principles with respect to computability. While we mostly focus on building new principled conceptual relations between syntax and the FEP, we also show through a sample of preliminary examples how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel-Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing-Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference.

Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities

Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).

Limited but specific engagement of the mature language network during linguistic statistical learning

Statistical learning (SL) is the ability to detect and learn regularities from input and is foundational to language acquisition. Despite the dominant role of SL as a theoretical construct for language development, there is a lack of direct evidence supporting the shared neural substrates underlying language processing and SL. It is also not clear whether the similarities, if any, are related to linguistic processing, or statistical regularities in general. The current study tests whether the brain regions involved in natural language processing are similarly recruited during auditory, linguistic SL. Twenty-two adults performed an auditory linguistic SL task, an auditory nonlinguistic SL task, and a passive story listening task as their neural activation was monitored. Within the language network, the left posterior temporal gyrus showed sensitivity to embedded speech regularities during auditory, linguistic SL, but not auditory, nonlinguistic SL. Using a multivoxel pattern similarity analysis, we uncovered similarities between the neural representation of auditory, linguistic SL, and language processing within the left posterior temporal gyrus. No other brain regions showed similarities between linguistic SL and language comprehension, suggesting that a shared neurocomputational process for auditory SL and natural language processing within the left posterior temporal gyrus is specific to linguistic stimuli.

Alexithymia modulates emotion concept activation during facial expression processing

Alexithymia is characterized by difficulties in emotional information processing. However, the underlying reasons for emotional processing deficits in alexithymia are not fully understood. The present study aimed to investigate the mechanism underlying emotional deficits in alexithymia. Using the Toronto Alexithymia Scale-20, we recruited college students with high alexithymia (n = 24) or low alexithymia (n = 24) in this study. Participants judged the emotional consistency of facial expressions and contextual sentences while recording their event-related potentials. Behaviorally, the high alexithymia group showed longer response times versus the low alexithymia group in processing facial expressions. The event-related potential results showed that the high alexithymia group had more negative-going N400 amplitudes compared with the low alexithymia group in the incongruent condition. More negative N400 amplitudes are also associated with slower responses to facial expressions. Furthermore, machine learning analyses based on N400 amplitudes could distinguish the high alexithymia group from the low alexithymia group in the incongruent condition. Overall, these findings suggest worse facial emotion perception for the high alexithymia group, potentially due to difficulty in spontaneously activating emotion concepts. Our findings have important implications for the affective science and clinical intervention of alexithymia-related affective disorders.

Compositional diversity in visual concept learning

Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate "alien figures" with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfigure existing parts in new ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a variety of compositional behavior when classifying and generating visual objects.

Designs on consciousness: literature and predictive processing

Predictive processing is a recent approach in cognitive science that describes the brain as an engine of probabilistic hierarchical inference. Initially proposed as a general theory of brain function, predictive processing has recently been expanding to account for questions of consciousness in philosophy and neuroscience. In my previous work (Kukkonen 2020 Probability designs: literature and predictive processing. New York, NY: Oxford University Press), I have shown how predictive processing can also be used to model our engagement with literary texts. In this article, I use my account of our engagement with literature in predictive processing terms, as well as recent work on predictive processing and consciousness, to explore how literature can shed light on various aspects of conscious experience, including qualia, counterfactual depth in conscious experience and sense of self. In the final section, I propose a number of theoretical questions that could be addressed by drawing on literature as a source of hypotheses and stimuli for possible experimental designs. The upshot is a picture where literature is not just a source of illustrative examples about conscious experience, but a designer environment through which we can explore and rethink consciousness. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Review of Binaural Processing With Asymmetrical Hearing Outcomes in Patients With Bilateral Cochlear Implants

Bilateral cochlear implants (BiCIs) result in several benefits, including improvements in speech understanding in noise and sound source localization. However, the benefit bilateral implants provide among recipients varies considerably across individuals. Here we consider one of the reasons for this variability: difference in hearing function between the two ears, that is, interaural asymmetry. Thus far, investigations of interaural asymmetry have been highly specialized within various research areas. The goal of this review is to integrate these studies in one place, motivating future research in the area of interaural asymmetry. We first consider bottom-up processing, where binaural cues are represented using excitation-inhibition of signals from the left ear and right ear, varying with the location of the sound in space, and represented by the lateral superior olive in the auditory brainstem. We then consider top-down processing via predictive coding, which assumes that perception stems from expectations based on context and prior sensory experience, represented by cascading series of cortical circuits. An internal, perceptual model is maintained and updated in light of incoming sensory input. Together, we hope that this amalgamation of physiological, behavioral, and modeling studies will help bridge gaps in the field of binaural hearing and promote a clearer understanding of the implications of interaural asymmetry for future research on optimal patient interventions.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Linguistic profile automated characterisation in pluripotential clinical high-risk mental state (CHARMS) conditions: methodology of a multicentre observational study

INTRODUCTION

Language is usually considered the social vehicle of thought in intersubjective communications. However, the relationship between language and high-order cognition seems to evade this canonical and unidirectional description (ie, the notion of language as a simple means of thought communication). In recent years, clinical high at-risk mental state (CHARMS) criteria (evolved from the Ultra-High-Risk paradigm) and the introduction of the Clinical Staging system have been proposed to address the dynamicity of early psychopathology. At the same time, natural language processing (NLP) techniques have greatly evolved and have been successfully applied to investigate different neuropsychiatric conditions. The combination of at-risk mental state paradigm, clinical staging system and automated NLP methods, the latter applied on spoken language transcripts, could represent a useful and convenient approach to the problem of early psychopathological distress within a transdiagnostic risk paradigm.

METHODS AND ANALYSIS

Help-seeking young people presenting psychological distress (CHARMS+/- and Clinical Stage 1a or 1b; target sample size for both groups n=90) will be assessed through several psychometric tools and multiple speech analyses during an observational period of 1-year, in the context of an Italian multicentric study. Subjects will be enrolled in different contexts: Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health (DINOGMI), Section of Psychiatry, University of Genoa-IRCCS Ospedale Policlinico San Martino, Genoa, Italy; Mental Health Department-territorial mental services (ASL 3-Genoa), Genoa, Italy; and Mental Health Department-territorial mental services (AUSL-Piacenza), Piacenza, Italy. The conversion rate to full-blown psychopathology (CS 2) will be evaluated over 2 years of clinical observation, to further confirm the predictive and discriminative value of CHARMS criteria and to verify the possibility of enriching them with several linguistic features, derived from a fine-grained automated linguistic analysis of speech.

ETHICS AND DISSEMINATION

The methodology described in this study adheres to ethical principles as formulated in the Declaration of Helsinki and is compatible with International Conference on Harmonization (ICH)-good clinical practice. The research protocol was reviewed and approved by two different ethics committees (CER Liguria approval code: 591/2020-id.10993; Comitato Etico dell'Area Vasta Emilia Nord approval code: 2022/0071963). Participants will provide their written informed consent prior to study enrolment and parental consent will be needed in the case of participants aged less than 18 years old. Experimental results will be carefully shared through publication in peer-reviewed journals, to ensure proper data reproducibility.

TRIAL REGISTRATION NUMBER

DOI:10.17605/OSF.IO/BQZTN.

Neural response and representation: Facial expressions in scenes

Previous studies have shown that the brain generates expectations based on scenes, which affects facial expression recognition. However, although facial expressions are known to interact with perception, the mechanism underlying this interaction remains poorly understood. Here, we used frequency labeling and decoding techniques to reveal the effects of scene-based expectation on the amplitude and representational strength of neural activity. We also reduced the relative reliability between expectation and sensory input by blurring facial expressions to further investigate the effects of this relative reliability on the pattern of neural activation and representation. Participants viewed emotional changes in unblurred or blurred facial expressions, which flickered at a rate of 6 Hz within a scene. We found that facial expressions that were congruent with the emotional significance of the scene elicited a larger steady-state visual evoked potential amplitude than did facial expressions that were incongruent with the emotional significance of a scene, in both unblurred and blurred conditions. We also found that expected facial expression representations were stronger than unexpected representations during the unblurred condition. In the blurred condition, unexpected representations were stronger than expected representations. Taken together, these results suggested that facial expression processing in the visual cortex is modulated by top-down signals. The relative reliability of expectation and sensory input moderated the influence of a scene on facial expression representation. Furthermore, our study showed that neural activation amplitudes did not correspond to representational strength.

Influence of language on perception and concept formation in a brain-constrained deep neural network model

A neurobiologically constrained model of semantic learning in the human brain was used to simulate the acquisition of concrete and abstract concepts, either with or without verbal labels. Concept acquisition and semantic learning were simulated using Hebbian learning mechanisms. We measured the network's category learning performance, defined as the extent to which it successfully (i) grouped partly overlapping perceptual instances into a single (abstract or concrete) conceptual representation, while (ii) still distinguishing representations for distinct concepts. Co-presence of linguistic labels with perceptual instances of a given concept generally improved the network's learning of categories, with a significantly larger beneficial effect for abstract than concrete concepts. These results offer a neurobiological explanation for causal effects of language structure on concept formation and on perceptuo-motor processing of instances of these concepts: supplying a verbal label during concept acquisition improves the cortical mechanisms by which experiences with objects and actions along with the learning of words lead to the formation of neuronal ensembles for specific concepts and meanings. Furthermore, the present results make a novel prediction, namely, that such 'Whorfian' effects should be modulated by the concreteness/abstractness of the semantic categories being acquired, with language labels supporting the learning of abstract concepts more than that of concrete ones. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Perceptual Learning of Noise-Vocoded Speech Under Divided Attention

Speech perception performance for degraded speech can improve with practice or exposure. Such perceptual learning is thought to be reliant on attention and theoretical accounts like the predictive coding framework suggest a key role for attention in supporting learning. However, it is unclear whether speech perceptual learning requires undivided attention. We evaluated the role of divided attention in speech perceptual learning in two online experiments (N = 336). Experiment 1 tested the reliance of perceptual learning on undivided attention. Participants completed a speech recognition task where they repeated forty noise-vocoded sentences in a between-group design. Participants performed the speech task alone or concurrently with a domain-general visual task (dual task) at one of three difficulty levels. We observed perceptual learning under divided attention for all four groups, moderated by dual-task difficulty. Listeners in easy and intermediate visual conditions improved as much as the single-task group. Those who completed the most challenging visual task showed faster learning and achieved similar ending performance compared to the single-task group. Experiment 2 tested whether learning relies on domain-specific or domain-general processes. Participants completed a single speech task or performed this task together with a dual task aiming to recruit domain-specific (lexical or phonological), or domain-general (visual) processes. All secondary task conditions produced patterns and amount of learning comparable to the single speech task. Our results demonstrate that the impact of divided attention on perceptual learning is not strictly dependent on domain-general or domain-specific processes and speech perceptual learning persists under divided attention.

The language marker hypothesis

According to the language marker hypothesis language has provided homo sapiens with a rich symbolic system that plays a central role in interpreting signals delivered by our sensory apparatus, in shaping action goals, and in creating a powerful tool for reasoning and inferencing. This view provides an important correction on embodied accounts of language that reduce language to action, perception, emotion and mental simulation. The presence of a language system has, however, also important consequences for perception, action, emotion, and memory. Language stamps signals from perception, action, and emotional systems with rich cognitive markers that transform the role of these signals in the overall cognitive architecture of the human mind. This view does not deny that language is implemented by means of universal principles of neural organization. However, language creates the possibility to generate rich internal models of the world that are shaped and made accessible by the characteristics of a language system. This makes us less dependent on direct action-perception couplings and might even sometimes go at the expense of the veridicality of perception. In cognitive (neuro)science the pendulum has swung from language as the key to understand the organization of the human mind to the perspective that it is a byproduct of perception and action. It is time that it partly swings back again.

Machine Learning as a Model for Cultural Learning: Teaching an Algorithm What it Means to be Fat

Public culture is a powerful source of cognitive socialization; for example, media language is full of meanings about body weight. Yet it remains unclear how individuals process meanings in public culture. We suggest that schema learning is a core mechanism by which public culture becomes personal culture. We propose that a burgeoning approach in computational text analysis - neural word embeddings - can be interpreted as a formal model for cultural learning. Embeddings allow us to empirically model schema learning and activation from natural language data. We illustrate our approach by extracting four lower-order schemas from news articles: the gender, moral, health, and class meanings of body weight. Using these lower-order schemas we quantify how words about body weight "fill in the blanks" about gender, morality, health, and class. Our findings reinforce ongoing concerns that machine-learning models (e.g., of natural language) can encode and reproduce harmful human biases.

Exploring the role of expectancy in older US participants' response to an accelerated resolution therapy intervention for prolonged grief disorder

Accelerated resolution therapy (ART) is a psychotherapy for the treatment of prolonged grief disorder (PGD) defined as severe, enduring longing for the lost person. Currently, ART lacks examination of intrapersonal processes, like expectancy, as behavioural mechanisms for action. Therefore, the purpose of this paper was to present the findings on participants' treatment expectations of ART for PGD and then discuss potential hypotheses for future testing. This study was a primary qualitative descriptive analysis of prospectively collected interview data (collected 2017-2019) accrued as part of a randomised, wait-list controlled clinical trial in bereaved hospice family caregivers in the United States. The sample included 29 former informal caregivers who were at least 1-year post death of their care recipient. They were primarily female, older (67.4 ± 7.1 years), and a little over half (n = 18) had been married to their care recipient. Thematic analysis resulted in three distinct themes with six sub-themes: The role of knowledge in expectations (sub-themes uncertainty, prior knowledge); The role of personality in expectations (sub-themes openness, positive affect); and Expecting a process (sub-themes cognitive processes, affective processes) which described the interaction of person and process in shaping expectations of our intervention. An across theme analysis of the specificity of the participants' expectations uncovered that knowledge and personality inform expectations of ART and that individuals who verbalise a process for recovery tend to be very specific in their expectations. Three hypotheses for testing are put forward and implications for practice, research and policy discussed.

The Effects of Semantic and Syntactic Prediction on Reading Aloud

Semantic and syntactic prediction effects were investigated in a word naming task using semantic or syntactic contexts that varied between three and six words. Participants were asked to read the contexts silently and name a target word, which was indicated by a color change. Semantic contexts were composed of lists of semantically associated words without any syntactic information. Syntactic contexts were composed of semantically neutral sentences, in which the grammatical category but not the lexical identity of the final word was highly predictable. When the presentation time of the context words was long (1,200 ms), both semantically and syntactically related contexts facilitated reading aloud latencies of target words and syntactically related contexts produced larger priming effects than semantically related contexts in two out of three analyses. When the presentation time was short (200 ms), however, syntactic context effects disappeared, while semantic context effects remained significant. Across the three experiments, longer contexts produced faster response latencies, but longer contexts did not produce larger priming effects. The results are discussed in the context of the extant literature on semantic and syntactic priming and more recent evidence, suggesting that syntactic information constrains single word recognition.

Word frequency effects found in free recall are rather due to Bayesian surprise

The inconsistent relation between word frequency and free recall performance (sometimes a positive one, sometimes a negative one, and sometimes no relation) and the non-monotonic relation found between the two cannot all be explained by current theories. We propose a theoretical framework that can explain all extant results. Based on an ecological psychology analysis of the free recall situation in terms of environmental and informational resources available to the participants, we propose that because participants' cognitive system has been shaped by their native language, free recall performance is best understood as the end result of relational properties that preexist the experimental situation and of the way the words from the experimental list interact with those. In addition to this, we borrow from predictive coding theory the idea that the brain constantly predicts "what is coming next" so that it is mainly prediction errors that will propagate information forward. Our ecological psychology analysis indicates there will be "prediction errors" because the word frequency distribution in an experimental word list is inevitably different from the particular Zipf's law distribution of the words in the language that shaped participants' brains. We further propose the particular distributional discrepancies inherent to a given word list will trigger, as a function of the words that are included in the list, their order, and of the words that are absent from the list, a surprisal signal in the brain, something that is isomorphic to the concept of Bayesian surprise. The precise moment when Bayesian surprise is triggered will determine to what word of the list that Bayesian surprise will be associated with, and the word the Bayesian surprise will be associated with will benefit from it and become more memorable as a direct function of the magnitude of the surprisal. Two experiments are presented that show a proxy of Bayesian surprise explains the free recall performance and that no effect of word frequency is found above and beyond the effect of that proxy variable. We then discuss how our view can account for all data extant in the literature on the effect of word frequency on free recall.

Extended active inference: Constructing predictive cognition beyond skulls

Cognitive niche construction is the process whereby organisms create and maintain cause-effect models of their niche as guides for fitness influencing behavior. Extended mind theory claims that cognitive processes extend beyond the brain to include predictable states of the world. Active inference and predictive processing in cognitive science assume that organisms embody predictive (i.e., generative) models of the world optimized by standard cognitive functions (e.g., perception, action, learning). This paper presents an active inference formulation that views cognitive niche construction as a cognitive function aimed at optimizing organisms' generative models. We call that process of optimization extended active inference.

Concepts for Which We Need Others More: The Case of Abstract Concepts

The capability to form and use concepts is a core component of human cognition. Although all concepts are grounded in sensorimotor processes, more abstract concepts (e.g., “truth”) collect more heterogeneous and perceptually dissimilar exemplars; thus, linguistic interaction and social interaction are particularly crucial for their acquisition and use. Because of their indeterminacy, abstract concepts generate more uncertainty than more concrete concepts; hence, they induce people to monitor their inner knowledge longer and then to consult others to ask for information and negotiate the concept’s meaning. I propose that people need others more for abstract concepts than for concrete concepts: Other people are essential to acquire, process, and use abstract concepts. Conceiving abstract concepts in these terms requires the employment of novel, interactive methods to investigate how people represent them during their use.

Semantic Expectation Effects on Object Detection: Using Figure Assignment to Elucidate Mechanisms

Recent evidence suggesting that object detection is improved following valid rather than invalid labels implies that semantics influence object detection. It is not clear, however, whether the results index object detection or feature detection. Further, because control conditions were absent and labels and objects were repeated multiple times, the mechanisms are unknown. We assessed object detection via figure assignment, whereby objects are segmented from backgrounds. Masked bipartite displays depicting a portion of a mono-oriented object (a familiar configuration) on one side of a central border were shown once only for 90 or 100 ms. Familiar configuration is a figural prior. Accurate detection was indexed by reports of an object on the familiar configuration side of the border. Compared to control experiments without labels, valid labels improved accuracy and reduced response times (RTs) more for upright than inverted objects (Studies 1 and 2). Invalid labels denoting different superordinate-level objects (DSC; Study 1) or same superordinate-level objects (SSC; Study 2) reduced accuracy for upright displays only. Orientation dependency indicates that effects are mediated by activated object representations rather than features which are invariant over orientation. Following invalid SSC labels (Study 2), accurate detection RTs were longer than control for both orientations, implicating conflict between semantic representations that had to be resolved before object detection. These results demonstrate that object detection is not just affected by semantics, it entails semantics.

Shared computational principles for language processing in humans and deep language models

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

Deep language models have revolutionized natural language processing. The paper discovers three computational principles shared between deep language models and the human brain, which can transform our understanding of the neural basis of language.

Schema-Centred Unity and Process-Centred Pluralism of the Predictive Mind

Proponents of the predictive processing (PP) framework often claim that one of the framework’s significant virtues is its unificatory power. What is supposedly unified are predictive processes in the mind, and these are explained in virtue of a common prediction error-minimisation (PEM) schema. In this paper, I argue against the claim that PP currently converges towards a unified explanation of cognitive processes. Although the notion of PEM systematically relates a set of posits such as ‘efficiency’ and ‘hierarchical coding’ into a unified conceptual schema, neither the frameworks’ algorithmic specifications nor its hypotheses about their implementations in the brain are clearly unified. I propose a novel way to understand the fruitfulness of the research program in light of a set of research heuristics that are partly shared with those common to Bayesian reverse engineering. An interesting consequence of this proposal is that pluralism is at least as important as unification to promote the positive development of the predictive mind.

A systematic review on resting state functional connectivity in patients with neurodegenerative disease and hallucinations

•

Our review produced ten resting functional magnetic resonance imaging studies.

•

Most of the included studies focused on synucleinopathies and visual hallucinations.

•

Disrupted Default Mode and attentional networks can be involved in hallucinations.

•

The intrusion of predictive processes can be responsible for hallucination phenomena.

•

Studies on hallucinations in tauopathies and different sensory modality are needed.

Hallucinations are a complex and multidimensional phenomenon which can differ based on the involved pathology, typology and sensory modality. Hallucinations are common in patients with neurodegenerative diseases. Recent sparse evidence from resting state functional magnetic resonance imaging (rs-fMRI) studies has identified altered functional connectivity in those patients within several brain networks, such as the default mode, attentional and sensory ones, without, however, providing an organized picture of the mechanisms involved. This systematic review, following PRISMA guidelines, aims at critically analyzing the current literature on the brain networks associated with the phenomenon of hallucinations in patients with neurodegenerative diseases. Ten rs-fMRI studies fulfilled our selection criteria. All these studies focused on synucleinopathies, and most of them focused on visual hallucinations and were characterized by a heterogeneous methodology. Thus, instead of offering a definite picture of the mechanisms underlying hallucinations in neurodegeneration, this systematic review encourages further research especially concerning tauopathies. Notwithstanding, the findings overall suggest a disruption in the top-down (associated with memory intrusion and difficulty of inhibition) and in the bottom-up processes (associated with the sensory areas involved in the hallucinations). Further investigations are needed in order to disentangle the brain mechanisms involved in hallucinations and to overcome possible limitations characterizing the current literature.

Modelling concrete and abstract concepts using brain-constrained deep neural networks

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically 'ground' concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their 'shared neurons', thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Bayes, predictive processing, and the cognitive architecture of motor control

Despite their popularity, relatively scant attention has been paid to the upshot of Bayesian and predictive processing models of cognition for views of overall cognitive architecture. Many of these models are hierarchical; they posit generative models at multiple distinct "levels," whose job is to predict the consequences of sensory input at lower levels. I articulate one possible position that could be implied by these models, namely, that there is a continuous hierarchy of perception, cognition, and action control comprising levels of generative models. I argue that this view is not entailed by a general Bayesian/predictive processing outlook. Bayesian approaches are compatible with distinct formats of mental representation. Focusing on Bayesian approaches to motor control, I argue that the junctures between different types of mental representation are places where the transitivity of hierarchical prediction may be broken, and I consider the upshot of this conclusion for broader discussions of cognitive architecture.

Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension

Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.

Predictive processing models and affective neuroscience

The neural bases of affective experience remain elusive. Early neuroscience models of affect searched for specific brain regions that uniquely carried out the computations that underlie dimensions of valence and arousal. However, a growing body of work has failed to identify these circuits. Research turned to multivariate analyses, but these strategies, too, have made limited progress. Predictive processing models offer exciting new directions to address this problem. Here, we use predictive processing models as a lens to critique prevailing functional neuroimaging research practices in affective neuroscience. Our review highlights how much work relies on rigid assumptions that are inconsistent with a predictive processing approach. We outline the central aspects of a predictive processing model and draw out their implications for research in affective and cognitive neuroscience. Predictive models motivate a reformulation of "reverse inference" in cognitive neuroscience, and placing a greater emphasis on external validity in experimental design.

Semantic Predictability Facilitates Comprehension of Degraded Speech in a Graded Manner

Previous studies have shown that at moderate levels of spectral degradation, semantic predictability facilitates language comprehension. It is argued that when speech is degraded, listeners have narrowed expectations about the sentence endings; i.e., semantic prediction may be limited to only most highly predictable sentence completions. The main objectives of this study were to (i) examine whether listeners form narrowed expectations or whether they form predictions across a wide range of probable sentence endings, (ii) assess whether the facilitatory effect of semantic predictability is modulated by perceptual adaptation to degraded speech, and (iii) use and establish a sensitive metric for the measurement of language comprehension. For this, we created 360 German Subject-Verb-Object sentences that varied in semantic predictability of a sentence-final target word in a graded manner (high, medium, and low) and levels of spectral degradation (1, 4, 6, and 8 channels noise-vocoding). These sentences were presented auditorily to two groups: One group (n =48) performed a listening task in an unpredictable channel context in which the degraded speech levels were randomized, while the other group (n =50) performed the task in a predictable channel context in which the degraded speech levels were blocked. The results showed that at 4 channels noise-vocoding, response accuracy was higher in high-predictability sentences than in the medium-predictability sentences, which in turn was higher than in the low-predictability sentences. This suggests that, in contrast to the narrowed expectations view, comprehension of moderately degraded speech, ranging from low- to high- including medium-predictability sentences, is facilitated in a graded manner; listeners probabilistically preactivate upcoming words from a wide range of semantic space, not limiting only to highly probable sentence endings. Additionally, in both channel contexts, we did not observe learning effects; i.e., response accuracy did not increase over the course of experiment, and response accuracy was higher in the predictable than in the unpredictable channel context. We speculate from these observations that when there is no trial-by-trial variation of the levels of speech degradation, listeners adapt to speech quality at a long timescale; however, when there is a trial-by-trial variation of the high-level semantic feature (e.g., sentence predictability), listeners do not adapt to low-level perceptual property (e.g., speech quality) at a short timescale.

Unification by Fiat: Arrested Development of Predictive Processing

Predictive processing (PP) has been repeatedly presented as a unificatory account of perception, action, and cognition. In this paper, we argue that this is premature: As a unifying theory, PP fails to deliver general, simple, homogeneous, and systematic explanations. By examining its current trajectory of development, we conclude that PP remains only loosely connected both to its computational framework and to its hypothetical biological underpinnings, which makes its fundamentals unclear. Instead of offering explanations that refer to the same set of principles, we observe systematic equivocations in PP‐based models, or outright contradictions with its avowed principles. To make matters worse, PP‐based models are seldom empirically validated, and they are frequently offered as mere just‐so stories. The large number of PP‐based models is thus not evidence of theoretical progress in unifying perception, action, and cognition. On the contrary, we maintain that the gap between theory and its biological and computational bases contributes to the arrested development of PP as a unificatory theory. Thus, we urge the defenders of PP to focus on its critical problems instead of offering mere re‐descriptions of known phenomena, and to validate their models against possible alternative explanations that stem from different theoretical assumptions. Otherwise, PP will ultimately fail as a unified theory of cognition.

What Am I Drinking? Vision Modulates the Perceived Flavor of Drinks, but No Evidence of Flavor Altering Color Perception in a Mixed Reality Paradigm

It is well established that vision, and in particular color, may modulate our experience of flavor. Such cross-modal correspondences have been argued to be bilateral, in the sense that one modality can modulate the other and vice versa. However, the amount of literature assessing how vision modulates flavor is remarkably larger than that directly assessing how flavor might modulate vision. This is more exaggerated in the context of cross-modal contrasts (when the expectancy in one modality contrasts the experience through another modality). Here, using an embodied mixed reality setup in which participants saw a liquid while ingesting a contrasting one, we assessed both how vision might modulate basic dimensions of flavor perception and how the flavor of the ingested liquid might alter the perceived color of the seen drink. We replicated findings showing the modulation of flavor perception by vision but found no evidence of flavor modulating color perception. These results are discussed in regard to recent accounts of multisensory integration in the context of visual modulations of flavor and bilateral cross-modulations. Our findings might be important as a step in understanding bilateral visual and flavor cross-modulations (or the lack of them) and might inform developments using embodied mixed reality technologies.

At the Neural Intersection Between Language and Emotion

What role does language play in emotion? Behavioral research shows that emotion words such as "anger" and "fear" alter emotion experience, but questions still remain about mechanism. Here, we review the neuroscience literature to examine whether neural processes associated with semantics are also involved in emotion. Our review suggests that brain regions involved in the semantic processing of words: (i) are engaged during experiences of emotion, (ii) coordinate with brain regions involved in affect to create emotions, (iii) hold representational content for emotion, and (iv) may be necessary for constructing emotional experience. We relate these findings with respect to four theoretical relationships between language and emotion, which we refer to as "non-interactive," "interactive," "constitutive," and "deterministic." We conclude that findings are most consistent with the interactive and constitutive views with initial evidence suggestive of a constitutive view, in particular. We close with several future directions that may help test hypotheses of the constitutive view.

Enacting Media. An Embodied Account of Enculturation Between Neuromediality and New Cognitive Media Theory

This paper argues that the still-emerging paradigm of situated cognition requires a more systematic perspective on media to capture the enculturation of the human mind. By virtue of being media, cultural artifacts present central experiential models of the world for our embodied minds to latch onto. The paper identifies references to external media within embodied, extended, enactive, and predictive approaches to cognition, which remain underdeveloped in terms of the profound impact that media have on our mind. To grasp this impact, I propose an enactive account of media that is based on expansive habits as media-structured, embodied ways of bringing forth meaning and new domains of values. We apply such habits, for instance, when seeing a picture or perceiving a movie. They become established through a process of reciprocal adaptation between media artifacts and organisms and define the range of viable actions within such a media ecology. Within an artifactual habit, we then become attuned to a specific media work (e.g., a TV series, a picture, a text, or even a city) that engages us. Both the plurality of habits and the dynamical adjustments within a habit require a more flexible neural architecture than is addressed by classical cognitive neuroscience. To detail how neural and media processes interlock, I will introduce the concept of neuromediality and discuss radical predictive processing accounts that could contribute to the externalization of the mind by treating media themselves as generative models of the world. After a short primer on general media theory, I discuss media examples in three domains: pictures and moving images; digital media; architecture and the built environment. This discussion demonstrates the need for a new cognitive media theory based on enactive artifactual habits-one that will help us gain perspective on the continuous re-mediation of our mind.

The Influences of Category Learning on Perceptual Reconstructions

We explore different ways in which the human visual system can adapt for perceiving and categorizing the environment. There are various accounts of supervised (categorical) and unsupervised perceptual learning, and different perspectives on the functional relationship between perception and categorization. We suggest that common experimental designs are insufficient to differentiate between hypothesized perceptual learning mechanisms and reveal their possible interplay. We propose a relatively underutilized way of studying potential categorical effects on perception, and we test the predictions of different perceptual learning models using a two-dimensional, interleaved categorization-plus-reconstruction task. We find evidence that the human visual system adapts its encodings to the feature structure of the environment, uses categorical expectations for robust reconstruction, allocates encoding resources with respect to categorization utility, and adapts to prevent miscategorizations.

Predictive processing, cognitive control, and tonality stability of music: An fMRI study of chromatic harmony

The present study aimed at identifying the brain regions which preferentially responded to music with medium degrees of key stability. There were three types of auditory stimuli. Diatonic music based strictly on major and minor scales has the highest key stability, whereas atonal music has the lowest key stability. Between these two extremes, chromatic music is characterized by sophisticated uses of out-of-key notes, which challenge the internal model of musical pitch and lead to higher precision-weighted prediction error compared to diatonic and atonal music. The brain activity of 29 adults with excellent relative pitch was measured with functional magnetic resonance imaging while they listened to diatonic music, chromatic music, and atonal random note sequences. Several frontoparietal regions showed significantly greater response to chromatic music than to diatonic music and atonal sequences, including the pre-supplementary motor area (extending into the dorsal anterior cingulate cortex), dorsolateral prefrontal cortex, rostrolateral prefrontal cortex, intraparietal sulcus, and precuneus. We suggest that these frontoparietal regions may support working memory processes, hierarchical sequencing, and conflict resolution of remotely related harmonic elements during the predictive processing of chromatic music. This finding suggested a possible correlation between precision-weighted prediction error and the frontoparietal regions implicated in cognitive control.

Early detection of language categories in face perception

Does language categorization influence face identification? The present study addressed this question by means of two experiments. First, to establish language categorization of faces, the memory confusion paradigm was used to create two language categories of faces, Spanish and English. Subsequently, participants underwent an oddball paradigm, in which faces that had been previously paired with one of the two languages (Spanish or English), were presented. We measured EEG perceptual differences (vMMN) between standard and two types of deviant faces: within-language category (faces sharing language with standards) or between-language category (faces paired with the other language). Participants were more likely to confuse faces within the language category than between categories, an index that faces were categorized by language. At the neural level, early vMMN were obtained for between-language category faces, but not for within-language category faces. At a later stage, however, larger vMMNs were obtained for those faces from the same language category. Our results showed that language is a relevant social cue that individuals used to categorize others and this categorization subsequently affects face perception.

Rhyme as resonance in poetry comprehension: An expert-novice study

Previous research has identified alliteration as a powerful device for investigating implicit memory effects. For example, alliterative phrases can provide retrieval cues that extend to a sublexical level and reactivate previous information that shares alliterative content (Lea et al., Psychological Science, 19[7], 709-716, 2008). But it is an open question if other surface forms might provide similar effects in line with these empirical findings, and in accord with writer intuitions. The present study examined whether rhyme produces analogous memory-reactivation effects, given the ubiquity of its use and endorsement of its power in a range of materials and experiences. We also examined whether the surface benefits attributed to rhyme might support anticipatory processes such as those traditionally examined with semantic content. In Experiment 1, participants exhibited faster recognition responses to previous poetic content as a function of rhyming cues. In Experiment 2, we recruited participants identified as experts on the study and use of rhyme, replicating the probe facilitations obtained in Experiment 1, but also revealing anticipations of imminent rhymes. The results are discussed in terms of implications for theories of memory-based text processing and of nonsemantic anticipatory processes during the reading of poetry, and perhaps for discourse experiences more generally.