Interaction between phonological and semantic representations: time matters

Patterns of performance on the animal fluency task in logopenic variant of primary progressive aphasia: A reflection of phonological and semantic skills

PURPOSE

This study aimed to characterize the quantitative (total number of correct words generated) and qualitative (psycholinguistic properties of correct words generated) performance patterns on the animal fluency task in individuals with the logopenic variant of primary progressive aphasia and to investigate the influence of phonological and semantic abilities to these patterns.

METHODS

Fifteen participants with lvPPA and twenty neurotypical adults completed the animal fluency task and an assessment battery to characterize their phonological and semantic abilities. We recorded the total number of correct words produced and their psycholinguistic properties. Group differences were analyzed using independent samples t-tests and analysis of covariance. Stepwise and multiple linear regression analyses were implemented to investigate the contribution of psycholinguistic properties on word generation as well as the role of phonological and semantic abilities on performance. We also investigated the mediating role of phonological and semantic abilities on the relationship between relevant psycholinguistic properties and word generation output.

RESULTS

Compared to neurotypical controls, participants with lvPPA produced fewer correct responses and more words with lower age of acquisition. The total number of correct words generated was predicted by the age of word acquisition, such that individuals who generated more responses, produced words acquired later in life. Phonology and semantics influenced the number of correct words generated and their frequency, age of acquisition, and semantic neighborhood density. Familiarity and arousal were driven by semantic abilities. Phonological abilities partially mediated the relationship between age of acquisition and word generation output.

CONCLUSIONS

This study provides valuable insights into the performance patterns of the animal fluency task in lvPPA. Individuals with lvPPA with more intact phonological and semantic abilities generated greater number of words with more complex psycholinguistic properties. Our findings contribute to the understanding of language processes underlying word retrieval in lvPPA.

Beyond the Visual Word Form Area - a cognitive characterization of the left ventral occipitotemporal cortex

The left ventral occipitotemporal cortex has been traditionally viewed as a pathway for visual object recognition including written letters and words. Its crucial role in reading was strengthened by the studies on the functionally localized "Visual Word Form Area" responsible for processing word-like information. However, in the past 20 years, empirical studies have challenged the assumptions of this brain region as processing exclusively visual or even orthographic stimuli. In this review, we aimed to present the development of understanding of the left ventral occipitotemporal cortex from the visually based letter area to the modality-independent symbolic language related region. We discuss theoretical and empirical research that includes orthographic, phonological, and semantic properties of language. Existing results showed that involvement of the left ventral occipitotemporal cortex is not limited to unimodal activity but also includes multimodal processes. The idea of the integrative nature of this region is supported by the broad functional and structural connectivity with language-related and attentional brain networks. We conclude that although the function of the area is not yet fully understood in human cognition, its role goes beyond visual word form processing. The left ventral occipitotemporal cortex seems to be crucial for combining higher-level language information with abstract forms that convey meaning independently of modality.

I'm not sure that curve means what you think it means: Toward a [more] realistic understanding of the role of eye-movement generation in the Visual World Paradigm

The Visual World Paradigm (VWP) is a powerful experimental paradigm for language research. Listeners respond to speech in a "visual world" containing potential referents of the speech. Fixations to these referents provides insight into the preliminary states of language processing as decisions unfold. The VWP has become the dominant paradigm in psycholinguistics and extended to every level of language, development, and disorders. Part of its impact is the impressive data visualizations which reveal the millisecond-by-millisecond time course of processing, and advances have been made in developing new analyses that precisely characterize this time course. All theoretical and statistical approaches make the tacit assumption that the time course of fixations is closely related to the underlying activation in the system. However, given the serial nature of fixations and their long refractory period, it is unclear how closely the observed dynamics of the fixation curves are actually coupled to the underlying dynamics of activation. I investigated this assumption with a series of simulations. Each simulation starts with a set of true underlying activation functions and generates simulated fixations using a simple stochastic sampling procedure that respects the sequential nature of fixations. I then analyzed the results to determine the conditions under which the observed fixations curves match the underlying functions, the reliability of the observed data, and the implications for Type I error and power. These simulations demonstrate that even under the simplest fixation-based models, observed fixation curves are systematically biased relative to the underlying activation functions, and they are substantially noisier, with important implications for reliability and power. I then present a potential generative model that may ultimately overcome many of these issues.

The pictures who shall not be named: Empirical support for benefits of preview in the Visual World Paradigm

A common critique of the Visual World Paradigm (VWP) in psycholinguistic studies is that what is designed as a measure of language processes is meaningfully altered by the visual context of the task. This is crucial, particularly in studies of spoken word recognition, where the displayed images are usually seen as just a part of the measure and are not of fundamental interest. Many variants of the VWP allow participants to sample the visual scene before a trial begins. However, this could bias their interpretations of the later speech or even lead to abnormal processing strategies (e.g., comparing the input to only preactivated working memory representations). Prior work has focused only on whether preview duration changes fixation patterns. However, preview could affect a number of processes, such as visual search, that would not challenge the interpretation of the VWP. The present study uses a series of targeted manipulations of the preview period to ask if preview alters looking behavior during a trial, and why. Results show that evidence of incremental processing and phonological competition seen in the VWP are not dependent on preview, and are not enhanced by manipulations that directly encourage phonological prenaming. Moreover, some forms of preview can eliminate nuisance variance deriving from object recognition and visual search demands in order to produce a more sensitive measure of linguistic processing. These results deepen our understanding of how the visual scene interacts with language processing to drive fixations patterns in the VWP, and reinforce the value of the VWP as a tool for measuring real-time language processing. Stimuli, data and analysis scripts are available at https://osf.io/b7q65/.

It's about time! Time as a parameter for lexical and syntactic processing: an eye-tracking-while-listening investigation

We examined the time-course of lexical activation, deactivation, and the syntactic operation of dependency linking during the online processing of object-relative sentence constructions using eye-tracking-while-listening. We explored how manipulating temporal aspects of the language input affects the tight lexical and syntactic temporal constraints found in sentence processing. The three temporal manipulations were (1) increasing the duration of the direct object noun, (2) adding the disfluency uh after the noun, and (3) replacing the disfluency with a silent pause. The findings from this experiment revealed that the disfluent and silence temporal manipulations enhanced the processing of subject and object noun phrases by modulating activation and deactivation. The manipulations also changed the time-course of dependency linking (increased reactivation of the direct object). The modulated activation dynamics of these lexical items are thought to play a role in mitigating interference and suggest that deactivation plays a beneficial role in complex sentence processing.

Lexical Recognition in Deaf Children Learning American Sign Language: Activation of Semantic and Phonological Features of Signs

Children learning language efficiently process single words, and activate semantic, phonological, and other features of words during recognition. We investigated lexical recognition in deaf children acquiring American Sign Language (ASL) to determine how perceiving language in the visual-spatial modality affects lexical recognition. Twenty native- or early-exposed signing deaf children (ages 4 to 8 years) participated in a visual world eye-tracking study. Children were presented with a single ASL sign, target picture, and three competitor pictures that varied in their phonological and semantic relationship to the target. Children shifted gaze to the target picture shortly after sign offset. Children showed robust evidence for activation of semantic but not phonological features of signs, however in their behavioral responses children were most susceptible to phonological competitors. Results demonstrate that single word recognition in ASL is largely parallel to spoken language recognition among children who are developing a mature lexicon.

Love thy neighbor: Facilitation and inhibition in the competition between parallel predictions

Ample evidence suggests that during word recognition and production, simultaneously activated lexical and sublexical representations interact, demonstrating varied patterns of facilitation and inhibition in various tasks and measures. A separate line of research has led to a growing consensus that prediction during sentence processing involves activating multiple possible predictions. However, very little is known about the nature of the interactions between parallel predictions. The current study employed a speeded cloze task to probe competition between simultaneously activated predictions. We focused on the modal response (the most probable completion for a sentence) and its strongest competitor (the second most probable completion). Examining production latencies of the modal response, the results showed an interaction between competitor strength and the semantic relatedness between the competitor and the modal: when the two were related, the stronger the competitor was, the more it facilitated production; however, when the two were unrelated, the stronger the competitor was, the more inhibition it caused. These results contrast with the pattern observed for the influences of near and distant semantic neighbors on word recognition and production. However, we show that when the different nature of the tasks is taken into consideration, these patterns of interaction between parallel predictions can be accounted for by the interactive activation and competition (IAC) model used to account for previous neighborhood effects (Chen & Mirman, 2012).

The Influence of Form- and Meaning-Based Predictions on Cortical Speech Processing Under Challenging Listening Conditions: A MEG Study

Under adverse listening conditions, prior linguistic knowledge about the form (i.e., phonology) and meaning (i.e., semantics) help us to predict what an interlocutor is about to say. Previous research has shown that accurate predictions of incoming speech increase speech intelligibility, and that semantic predictions enhance the perceptual clarity of degraded speech even when exact phonological predictions are possible. In addition, working memory (WM) is thought to have specific influence over anticipatory mechanisms by actively maintaining and updating the relevance of predicted vs. unpredicted speech inputs. However, the relative impact on speech processing of deviations from expectations related to form and meaning is incompletely understood. Here, we use MEG to investigate the cortical temporal processing of deviations from the expected form and meaning of final words during sentence processing. Our overall aim was to observe how deviations from the expected form and meaning modulate cortical speech processing under adverse listening conditions and investigate the degree to which this is associated with WM capacity. Results indicated that different types of deviations are processed differently in the auditory N400 and Mismatch Negativity (MMN) components. In particular, MMN was sensitive to the type of deviation (form or meaning) whereas the N400 was sensitive to the magnitude of the deviation rather than its type. WM capacity was associated with the ability to process phonological incoming information and semantic integration.

The sampling precision of research in five major areas of psychology

After obtaining a sample of published, peer-reviewed articles from journals with high and low impact factors in social, cognitive, neuro-, developmental, and clinical psychology, we used a priori equations recently derived by Trafimow (Educational and Psychological Measurement, 77, 831-854, 2017; Trafimow & MacDonald in Educational and Psychological Measurement, 77, 204-219, 2017) to compute the articles' median levels of precision. Our findings indicate that developmental research performs best with respect to precision, whereas cognitive research performs the worst; however, none of the psychology subfields excelled. In addition, we found important differences in precision between journals in the upper versus lower echelons with respect to impact factors in cognitive, neuro-, and clinical psychology, whereas the difference was dramatically attenuated for social and developmental psychology. Implications are discussed.

Unfolding meaning in context: The dynamics of conceptual similarity

How are relationships between concepts affected by the interplay between short-term contextual constraints and long-term conceptual knowledge? Across two studies we investigate the consequence of changes in visual context for the dynamics of conceptual processing. Participants' eye movements were tracked as they viewed a visual depiction of e.g. a canary in a birdcage (Experiment 1), or a canary and three unrelated objects, each in its own quadrant (Experiment 2). In both studies participants heard either a semantically and contextually similar "robin" (a bird; similar size), an equally semantically similar but not contextually similar "stork" (a bird; bigger than a canary, incompatible with the birdcage), or unrelated "tent". The changing patterns of fixations across time indicated first, that the visual context strongly influenced the eye movements such that, in the context of a birdcage, early on (by word offset) hearing "robin" engendered more looks to the canary than hearing "stork" or "tent" (which engendered the same number of looks), unlike in the context of unrelated objects (in which case "robin" and "stork" engendered equivalent looks to the canary, and more than did "tent"). Second, within the 500 ms post-word-offset eye movements in both experiments converged onto a common pattern (more looks to the canary after "robin" than after "stork", and for both more than after "tent"). We interpret these findings as indicative of the dynamics of activation within semantic memory accessed via pictures and via words, and reflecting the complex interaction between systems representing context-independent and context-dependent conceptual knowledge driven by predictive processing.

Impaired Lexical Selection and Fluency in Post-Stroke Aphasia

BACKGROUND

Deficits in fluent language production are a hallmark of aphasia and may arise from impairments at different levels in the language system. It has been proposed that difficulty resolving lexical competition contributes to fluency deficits.

AIMS

The present study tested this hypothesis in a novel way: by examining whether narrative speech production fluency is associated with difficulty resolving lexical competition in spoken word recognition as measured by sensitivity to phonological neighborhood density.

METHODS & PROCEDURES

Nineteen participants with aphasia and 15 neurologically intact older adults identified spoken words that varied in phonological neighborhood density and were presented in moderate noise.

OUTCOMES & RESULTS

Neurologically intact participants exhibited the standard inhibitory effect of phonological neighborhood density on response times: slower recognition of spoken words from denser neighborhoods. Among participants with aphasia, the inhibitory effect of phonological neighborhood density (less accurate recognition of spoken words from denser neighborhoods) was smaller for participants with greater fluency. The neighborhood effect was larger for participants with greater receptive vocabulary knowledge, indicating that the fluency effect was not a result of general lexical deficits.

CONCLUSIONS

These results are consistent with the hypothesis that impaired lexical selection is a contributing factor in fluency deficits in post-stroke aphasia.

The ventrolateral prefrontal cortex facilitates processing of sentential context to locate referents

Left ventrolateral prefrontal cortex (VLPFC) has been implicated in both integration and conflict resolution in sentence comprehension. Most evidence in favor of the integration account comes from processing ambiguous or anomalous sentences, which also poses a demand for conflict resolution. In two eye-tracking experiments we studied the role of VLPFC in integration when demands for conflict resolution were minimal. Two closely-matched groups of individuals with chronic post-stroke aphasia were tested: the Anterior group had damage to left VLPFC, whereas the Posterior group had left temporo-parietal damage. In Experiment 1 a semantic cue (e.g., "She will eat the apple") uniquely marked the target (apple) among three distractors that were incompatible with the verb. In Experiment 2 phonological cues (e.g., "She will see an eagle."/"She will see a bear.") uniquely marked the target among three distractors whose onsets were incompatible with the cue (e.g., all consonants when the target started with a vowel). In both experiments, control conditions had a similar format, but contained no semantic or phonological contextual information useful for target integration (e.g., the verb "see", and the determiner "the"). All individuals in the Anterior group were slower in using both types of contextual information to locate the target than were individuals in the Posterior group. These results suggest a role for VLPFC in integration beyond conflict resolution. We discuss a framework that accommodates both integration and conflict resolution.

Semantic Relations Cause Interference in Spoken Language Comprehension When Using Repeated Definite References, Not Pronouns

The choice and processing of referential expressions depend on the referents' status within the discourse, such that pronouns are generally preferred over full repetitive references when the referent is salient. Here we report two visual-world experiments showing that: (1) in spoken language comprehension, this preference is reflected in delayed fixations to referents mentioned after repeated definite references compared with after pronouns; (2) repeated references are processed differently than new references; (3) long-term semantic memory representations affect the processing of pronouns and repeated names differently. Overall, these results support the role of semantic discourse representation in referential processing and reveal important details about how pronouns and full repeated references are processed in the context of these representations. The results suggest the need for modifications to current theoretical accounts of reference processing such as Discourse Prominence Theory and the Informational Load Hypothesis.

The neural basis of inhibitory effects of semantic and phonological neighbors in spoken word production

Theories of word production and word recognition generally agree that multiple word candidates are activated during processing. The facilitative and inhibitory effects of these "lexical neighbors" have been studied extensively using behavioral methods and have spurred theoretical development in psycholinguistics, but relatively little is known about the neural basis of these effects and how lesions may affect them. This study used voxel-wise lesion overlap subtraction to examine semantic and phonological neighbor effects in spoken word production following left hemisphere stroke. Increased inhibitory effects of near semantic neighbors were associated with inferior frontal lobe lesions, suggesting impaired selection among strongly activated semantically related candidates. Increased inhibitory effects of phonological neighbors were associated with posterior superior temporal and inferior parietal lobe lesions. In combination with previous studies, these results suggest that such lesions cause phonological-to-lexical feedback to more strongly activate phonologically related lexical candidates. The comparison of semantic and phonological neighbor effects and how they are affected by left hemisphere lesions provides new insights into the cognitive dynamics and neural basis of phonological, semantic, and cognitive control processes in spoken word production.

Effects of phonological and semantic deficits on facilitative and inhibitory consequences of item repetition in spoken word comprehension

Repeating a word can have both facilitative and inhibitory effects on subsequent processing. The present study investigated these dynamics by examining the facilitative and inhibitory consequences of different kinds of item repetition in two individuals with aphasia and a group of neurologically intact control participants. The two individuals with aphasia were matched on overall aphasia severity, but had deficits at different levels of processing: one with a phonological deficit and spared semantic processing, the other with a semantic deficit and spared phonological processing. Participants completed a spoken word-to-picture matching task in which they had to pick which of four object images matched the spoken word. The trials were grouped into pairs such that exactly two objects from the first trial in a pair were present on screen during the second trial in the pair. When the second trial's target was the same as the first trial's target, compared to control participants, both participants with aphasia exhibited equally larger repetition priming effects. When the second trial's target was one of the new items, the participant with a phonological deficit exhibited a significantly more negative effect (i.e., second trial response slower than first trial response) than the control participants and the participant with a semantic deficit. Simulations of a computational model confirmed that this pattern of results could arise from (1) normal residual activation being functionally more significant when overall lexical processing is slower and (2) residual phonological activation of the previous trial's target having a particularly strong inhibitory effect specifically when phonological processing is impaired because the task was phonologically-driven (the spoken input specified the target). These results provide new insights into perseveration errors and lexical access deficits in aphasia.