Seeing for speaking: Semantic and lexical information provided by briefly presented, naturalistic action scenes

Asymmetries in encoding event roles: Evidence from language and cognition

It has long been hypothesized that the linguistic structure of events, including event participants and their relative prominence, draws on the non-linguistic nature of events and the roles that these events license. However, the precise relation between the prominence of event participants in language and cognition has not been tested experimentally in a systematic way. Here we address this gap. In four experiments, we investigate the relative prominence of (animate) Agents, Patients, Goals and Instruments in the linguistic encoding of complex events and the prominence of these event roles in cognition as measured by visual search and change blindness tasks. The relative prominence of these event roles was largely similar-though not identical-across linguistic and non-linguistic measures. Across linguistic and non-linguistic tasks, Patients were more salient than Goals, which were more salient than Instruments. (Animate) Agents were more salient than Patients in linguistic descriptions and visual search; however, this asymmetrical pattern did not emerge in change detection. Overall, our results reveal homologies between the linguistic and non-linguistic prominence of individual event participants, thereby lending support to the claim that the linguistic structure of events builds on underlying conceptual event representations. We discuss implications of these findings for linguistic theory and theories of event cognition.

Digital remote assessment of speech acoustics in cognitively unimpaired adults: feasibility, reliability and associations with amyloid pathology

BACKGROUND

Digital speech assessment has potential relevance in the earliest, preclinical stages of Alzheimer's disease (AD). We evaluated the feasibility, test-retest reliability, and association with AD-related amyloid-beta (Aβ) pathology of speech acoustics measured over multiple assessments in a remote setting.

METHODS

Fifty cognitively unimpaired adults (Age 68 ± 6.2 years, 58% female, 46% Aβ-positive) completed remote, tablet-based speech assessments (i.e., picture description, journal-prompt storytelling, verbal fluency tasks) for five days. The testing paradigm was repeated after 2-3 weeks. Acoustic speech features were automatically extracted from the voice recordings, and mean scores were calculated over the 5-day period. We assessed feasibility by adherence rates and usability ratings on the System Usability Scale (SUS) questionnaire. Test-retest reliability was examined with intraclass correlation coefficients (ICCs). We investigated the associations between acoustic features and Aβ-pathology, using linear regression models, adjusted for age, sex and education.

RESULTS

The speech assessment was feasible, indicated by 91.6% adherence and usability scores of 86.0 ± 9.9. High reliability (ICC ≥ 0.75) was found across averaged speech samples. Aβ-positive individuals displayed a higher pause-to-word ratio in picture description (B = -0.05, p = 0.040) and journal-prompt storytelling (B = -0.07, p = 0.032) than Aβ-negative individuals, although this effect lost significance after correction for multiple testing.

CONCLUSION

Our findings support the feasibility and reliability of multi-day remote assessment of speech acoustics in cognitively unimpaired individuals with and without Aβ-pathology, which lays the foundation for the use of speech biomarkers in the context of early AD.

Word frequency has similar effects in picture naming and gender decision: A failure to replicate Jescheniak and Levelt (1994)

Word frequency plays a key role in theories of lexical access, which assume that the word frequency effect (WFE, faster access to high-frequency than low-frequency words) occurs as a result of differences in the representation and processing of the words. In a seminal paper, Jescheniak and Levelt (1994) proposed that the WFE arises during the retrieval of word forms, rather than the retrieval of their syntactic representations (their lemmas) or articulatory commands. An important part of Jescheniak and Levelt's argument was that they found a stable WFE in a picture naming task, which requires complete lexical access, but not in a gender decision task, which only requires access to the words' lemmas and not their word forms. We report two attempts to replicate this pattern, one with new materials, and one with Jescheniak and Levelt's orginal pictures. In both studies we found a strong WFE when the pictures were shown for the first time, but much weaker effects on their second and third presentation. Importantly these patterns were seen in both the picture naming and the gender decision tasks, suggesting that either word frequency does not exclusively affect word form retrieval, or that the gender decision task does not exclusively tap lemma access.

People solve rebuses unwittingly-Both forward and backward: Empirical evidence for the mental effectiveness of the signifier

Introduction

Freud proposed that names of clinically salient objects or situations, such as for example a beetle (Käfer) in Mr. E's panic attack, refer through their phonological word form, and not through their meaning, to etiologically important events-here, "Que faire?" which summarizes the indecisiveness of Mr. E's mother concerning her marriage with Mr. E's father. Lacan formalized these ideas, attributing full-fledged mental effectiveness to the signifier, and summarized this as "the unconscious structured as a language". We tested one aspect of this theory, namely that there is an influence of the ambiguous phonological translation of the world upon our mental processing without us being aware of this influence.

Methods

For this, we used a rebus priming paradigm, including 14 French rebuses, composed of two images depicting common objects, such as paon /pã/ "peacock" and terre /tεr/ "earth," together forming the rebus panthère /pãtεr/ "panther." These images were followed by a target word semantically related to the rebus resolution, e.g., félin "feline," upon which the participants, unaware of the rebus principle, produced 6 written associations. A total of 1,458 participants were randomly assigned either to Experiment 1 in which they were shown the rebus images in either forward or in reverse order or to Experiment 2, in which they were shown only one of both rebus images, either the first or the last.

Results and discussion

The results show that the images induced inadvertent rebus priming in naïve participants. In other words, our results show that people solve rebuses unwittingly independent of stimulus order, thereby constituting empirical evidence for the mental effectiveness of the signifier.

The electrophysiological correlates of word pre-activation during associative word learning

Human beings continuously make use of learned associations to generate predictions about future occurrences in the environment. Such memory-related predictive processes provide a scaffold for learning in that mental representations of foreseeable events can be adjusted or strengthened based on a specific outcome. Learning the meaning of novel words through picture-word associations constitutes a prime example of associative learning because pictures preceding words can trigger word prediction through the pre-activation of the related mnemonic representations. In the present electroencephalography (EEG) study, we used event-related potentials (ERPs) to compare neural indices of word pre-activation between a word learning condition with maximal prediction likelihood and a non-learning control condition with low prediction. Results revealed that prediction-related N400 amplitudes in response to pictures decreased over time at central electrodes as a function of word learning, whereas late positive component (LPC) amplitudes increased. Notably, N400 but not LPC changes were also predictive of word learning performance, suggesting that the N400 component constitutes a sensitive marker of word pre-activation during associative word learning.

A voice without a mouth no more: The neurobiology of language and consciousness

Most research on the neurobiology of language ignores consciousness and vice versa. Here, language, with an emphasis on inner speech, is hypothesised to generate and sustain self-awareness, i.e., higher-order consciousness. Converging evidence supporting this hypothesis is reviewed. To account for these findings, a 'HOLISTIC' model of neurobiology of language, inner speech, and consciousness is proposed. It involves a 'core' set of inner speech production regions that initiate the experience of feeling and hearing words. These take on affective qualities, deriving from activation of associated sensory, motor, and emotional representations, involving a largely unconscious dynamic 'periphery', distributed throughout the whole brain. Responding to those words forms the basis for sustained network activity, involving 'default mode' activation and prefrontal and thalamic/brainstem selection of contextually relevant responses. Evidence for the model is reviewed, supporting neuroimaging meta-analyses conducted, and comparisons with other theories of consciousness made. The HOLISTIC model constitutes a more parsimonious and complete account of the 'neural correlates of consciousness' that has implications for a mechanistic account of mental health and wellbeing.

In which case is working memory for movements affected by verbal interference? Evidence from the verbal description of movement

Current perspectives on whether verbal interference affects working memory for movements have not yet reached a consensus. This study explored the causes of this controversy to reveal the relation between working memory for movements and the phonological loop. Experiment 1 explored whether the verbal description of movement moderated the effect of verbal interference (articulatory suppression) on working memory for movements. Verbal interference only affected working memory for easy-to-describe movements (lower accuracy). Experiment 2 excluded the interpretation of familiarity to the controversy and the effect of familiarity on the results of Experiment 1. Experiment 3 verified the results of Experiment 1 with another form of verbal interference, i.e., presenting irrelevant words visually. These three experiments suggest that the phonological loop is not recruited for processing working memory for movements in nature, but the two may interact through the verbal description prestored in the long-term memory. Thus, the current study provides a certain level of support for the separable movement-based subsystem hypothesis (Smyth, M. M., Pearson, N. A., & Pendleton, L. R. (1988). Movement and working memory: Patterns and positions in space. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology, 40(3), 497-514. doi:10.1080/02724988843000041).

Global Image Properties Predict Ratings of Affective Pictures

Affective pictures are widely used in studies of human emotions. The objects or scenes shown in affective pictures play a pivotal role in eliciting particular emotions. However, affective processing can also be mediated by low-level perceptual features, such as local brightness contrast, color or the spatial frequency profile. In the present study, we asked whether image properties that reflect global image structure and image composition affect the rating of affective pictures. We focused on 13 global image properties that were previously associated with the esthetic evaluation of visual stimuli, and determined their predictive power for the ratings of five affective picture datasets (IAPS, GAPED, NAPS, DIRTI, and OASIS). First, we used an SVM-RBF classifier to predict high and low ratings for valence and arousal, respectively, and achieved a classification accuracy of 58–76% in this binary decision task. Second, a multiple linear regression analysis revealed that the individual image properties account for between 6 and 20% of the variance in the subjective ratings for valence and arousal. The predictive power of the image properties varies for the different datasets and type of ratings. Ratings tend to share similar sets of predictors if they correlate positively with each other. In conclusion, we obtained evidence from non-linear and linear analyses that affective pictures evoke emotions not only by what they show, but they also differ by how they show it. Whether the human visual system actually uses these perceptive cues for emotional processing remains to be investigated.