Automatic Scoring of Semantic Fluency

Neuroanatomical Signature of the Transition from Normal Cognition to MCI in Parkinson's Disease

The progression of Parkinson's disease (PD) is often accompanied by cognitive decline. We had previously developed a brain age estimation program utilizing structural MRI data of 949 healthy individuals from publicly available sources. Structural MRI data of 244 PD patients who were cognitively normal at baseline was acquired from the Parkinson Progression Markers Initiative (PPMI). 192 of these showed stable normal cognitive function from baseline out to 5 years (PD-SNC), and the remaining 52 had unstable normal cognition and developed mild cognitive impairment within 5 years (PD-UNC). 105 healthy controls were also included in the analysis as a reference. First, we examined if there were any baseline differences in regional brain structure between PD-UNC and PD-SNC cohorts utilizing the three most widely used atrophy estimation pipelines, i.e., voxel-based morphometry (VBM), deformation-based morphometry and cortical thickness analyses. We then investigated if accelerated brain age estimation with our multivariate regressive machine learning algorithm was different across these groups (HC, PD-SNC, and PD-UNC). As per the VBM analysis, PD-UNC patients demonstrated a noticeable increase in GM volume in the posterior and anterior lobes of the cerebellum, sub-lobar, extra-nuclear, thalamus, and pulvinar regions when compared to PD-SNC at baseline. PD-UNC patients were observed to have significantly older brain age compared to both PD-SNC patients (p=0.009) and healthy controls (p<0.009). The increase in GM volume in the PD-UNC group could potentially indicate an inflammatory or neuronal hypertrophy response, which could serve as a biomarker for future cognitive decline among this population.

Predicting the occurrence of mild cognitive impairment in Parkinson's disease using structural MRI data

Introduction

Mild cognitive impairment (MCI) is a common symptom observed in individuals with Parkinson's disease (PD) and a main risk factor for progressing to dementia. Our objective was to identify early anatomical brain changes that precede the transition from healthy cognition to MCI in PD.

Methods

Structural T1-weighted magnetic resonance imaging data of PD patients with healthy cognition at baseline were downloaded from the Parkinson's Progression Markers Initiative database. Patients were divided into two groups based on the annual cognitive assessments over a 5-year time span: (i) PD patients with unstable healthy cognition who developed MCI over a 5-year follow-up (PD-UHC, n = 52), and (ii) PD patients who maintained stable healthy cognitive function over the same period (PD-SHC, n = 52). These 52 PD-SHC were selected among 192 PD-SHC patients using propensity score matching method to have similar demographic and clinical characteristics with PD-UHC at baseline. Seventy-five percent of these were used to train a support vector machine (SVM) algorithm to distinguish between the PD-UHC and PD-SHC groups, and tested on the remaining 25% of individuals. Shapley Additive Explanations (SHAP) feature analysis was utilized to identify the most informative brain regions in SVM classifier.

Results

The average accuracy of classifying PD-UHC vs. PD-SHC was 80.76%, with 82.05% sensitivity and 79.48% specificity using 10-fold cross-validation. The performance was similar in the hold-out test sets with all accuracy, sensitivity, and specificity at 76.92%. SHAP analysis showed that the most influential brain regions in the prediction model were located in the frontal, occipital, and cerebellar regions as well as midbrain.

Discussion

Our machine learning-based analysis yielded promising results in identifying PD individuals who are at risk of cognitive decline from the earliest disease stage and revealed the brain regions which may be linked to the prospective cognitive decline in PD before clinical symptoms emerge.

Semantic verbal fluency in native speakers of Turkish: a systematic review of category use, scoring metrics and normative data in healthy individuals

INTRODUCTION

Semantic verbal fluency (SVF) is a widely used measure of frontal executive function and access to semantic memory. SVF scoring metrics include the number of unique words generated, perseverations, intrusions, semantic cluster size and switching between clusters, and scores vary depending on the language the test is administered in. In this paper, we review the existing normative data for Turkish, the main metrics used for scoring SVF data in Turkish, and the most frequently used categories.

METHOD

We conducted a systematic review of peer-reviewed papers using Medline, EMBASE, PsycInfo, Web of Science, and two Turkish databases, TR-Dizin and Yok-Tez. Included papers contained data on the SVF performance of healthy adult native speakers of Turkish, and reported the categories used. Versions of the SVF that required participants to alternate categories were excluded. We extracted and tabulated demographics, descriptions of groups, metrics used, categories used, and sources of normative data. Studies were assessed for level of detail in reporting findings.

RESULTS

1400 studies were retrieved. After deduplication, abstract, full text screening, and merging of theses with their published versions, 121 studies were included. 114 studies used the semantic category "animal", followed by first names (N = 14, 12%). All studies reported word count. More complex measures were rare (perseverations: N = 12, 10%, clustering and switching: N = 5, 4%). Four of seven normative studies reported only word count, two also measured perseverations, and one reported category violations and perseverations. Two normative studies were published in English.

CONCLUSIONS

There is a lack of normative Turkish SVF data with more complex metrics, such as clustering and switching, and a lack of normative data published in English. Given the size of the Turkish diaspora, normative SVF data should include monolingual and bilingual speakers. Limitations include a restriction to key English and Turkish databases.

Systematic administration and analysis of verbal fluency tasks: Preliminary evidence for reliable exploration of processes underlying task performance

Verbal fluency (VF) tasks are typically scored by the number of acceptable words generated within an allotted time (i.e., total score). However, total scores do not provide insight into verbal and executive processes underlying VF task performance. Further analyses have been implemented to increase the analytical power of VF tasks, but systematic scoring guidelines are needed. We generated instructions for administration, scoring, and analyses of total scores, errors, temporal parameters, clustering, and switching with strong inter-rater reliability. To investigate the reliability of the proposed analysis, we modeled the performance of Finnish-speaking older adults (N = 50) in phonemic (/k/, /a/, and /p/) and semantic (animals) categories. Our results are in line with previous studies: We observed a higher performance on semantic than phonemic fluency (p ≤ 0.001, d = 0.91) and significant effects for education (p ≤ 0.001, d = 1.11) and gender (p ≤ 0.001, d = -1.11), but not for age (p = 0.10, d = 0.48). Most errors were repetitions. Performance declined over the allotted time frame as measured in 15-s segments (all ps < 0.001 with medium to large effect sizes). Task congruent clustering and switching were productive strategies (all ps < 0.001 with large effect sizes), and participants generated task discrepant clusters in both phonemic (p = 0.004, d = 0.69) and semantic tasks (p = 0.66, d = 0.18). The results substantiate the proposed method, providing evidence that these guidelines are a reliable starting point for VF task performance analyses in various clinical populations investigating VF task performance in depth.

Towards a temporospatial framework for measurements of disorganization in speech using semantic vectors

Incoherent speech in schizophrenia has long been described as the mind making "leaps" of large distances between thoughts and ideas. Such a view seems intuitive, and for almost two decades, attempts to operationalize these conceptual "leaps" in spoken word meanings have used language-based embedding spaces. An embedding space represents meaning of words as numerical vectors where a greater proximity between word vectors represents more shared meaning. However, there are limitations with word vector-based operationalizations of coherence which can limit their appeal and utility in clinical practice. First, the use of esoteric word embeddings can be conceptually hard to grasp, and this is complicated by several different operationalizations of incoherent speech. This problem can be overcome by a better visualization of methods. Second, temporal information from the act of speaking has been largely neglected since models have been built using written text, yet speech is spoken in real time. This issue can be resolved by leveraging time stamped transcripts of speech. Third, contextual information - namely the situation of where something is spoken - has often only been inferred and never explicitly modeled. Addressing this situational issue opens up new possibilities for models with increased temporal resolution and contextual relevance. In this paper, direct visualizations of semantic distances are used to enable the inspection of examples of incoherent speech. Some common operationalizations of incoherence are illustrated, and suggestions are made for how temporal and spatial contextual information can be integrated in future implementations of measures of incoherence.

Cognitive Impairment and Self-Reported Dementia in UK Retired Professional Soccer Players: A Cross Sectional Comparative Study

BACKGROUND

Previous studies based on death certificates have found professional soccer players were more likely to die with neurodegenerative diseases, including dementia. Therefore, this study aimed to investigate whether retired professional male soccer players would perform worse on cognitive tests and be more likely to self-report dementia diagnosis than general population control men.

METHODS

A cross-sectional comparative study was conducted between August 2020 and October 2021 in the United Kingdom (UK). Professional soccer players were recruited through different soccer clubs in England, and general population control men were recruited from the East Midlands in the UK. We obtained self-reported postal questionnaire data on dementia and other neurodegenerative diseases, comorbidities and risk factors from 468 soccer players and 619 general population controls. Of these, 326 soccer players and 395 general population controls underwent telephone assessment for cognitive function.

RESULTS

Retired soccer players were approximately twice as likely to score below established dementia screening cut-off scores on the Hopkins Verbal Learning Test (OR 2.06, 95%CI 1.11-3.83) and Verbal Fluency (OR 1.78, 95% CI 1.18-2.68), but not the Test Your Memory, modified Telephone Interview for Cognitive Status, and Instrumental Activities of Daily Living. Analyses were adjusted for age, education, hearing loss, body mass index, stroke, circulatory problems in the legs and concussion. While retired soccer players were younger, had fewer cardiovascular diseases and other morbidities and reported healthier lifestyles, 2.8% of retired soccer players reported medically diagnosed dementia and other neurodegenerative disease compared to 0.9% of controls (OR = 3.46, 95% CI 1.25-9.63) after adjustment for age and possible confounders.

CONCLUSIONS

UK male retired soccer players had a higher risk of performing below established cut-off scores of dementia screening tests and were more likely to self-report medically diagnosed dementia and neurodegenerative diseases, despite having better overall physical health and fewer dementia risk factors. Further study is needed to determine specific soccer-related risk factors.

The Accuracy of Speech and Linguistic Analysis in Early Diagnostics of Neurocognitive Disorders in a Memory Clinic Setting

OBJECTIVE

To investigate whether automatic analysis of the Semantic Verbal Fluency test (SVF) is reliable and can extract additional information that is of value for identifying neurocognitive disorders. In addition, the associations between the automatically derived speech and linguistic features and other cognitive domains were explored.

METHOD

We included 135 participants from the memory clinic of the Maastricht University Medical Center+ (with Subjective Cognitive Decline [SCD; N = 69] and Mild Cognitive Impairment [MCI]/dementia [N = 66]). The SVF task (one minute, category animals) was recorded and processed via a mobile application, and speech and linguistic features were automatically extracted. The diagnostic performance of the automatically derived features was investigated by training machine learning classifiers to differentiate SCD and MCI/dementia participants.

RESULTS

The intraclass correlation for interrater reliability between the clinical total score (golden standard) and automatically derived total word count was 0.84. The full model including the total word count and the automatically derived speech and linguistic features had an Area Under the Curve (AUC) of 0.85 for differentiating between people with SCD and MCI/dementia. The model with total word count only and the model with total word count corrected for age showed an AUC of 0.75 and 0.81, respectively. Semantic switching correlated moderately with memory as well as executive functioning.

CONCLUSION

The one-minute SVF task with automatically derived speech and linguistic features was as reliable as the manual scoring and differentiated well between SCD and MCI/dementia. This can be considered as a valuable addition in the screening of neurocognitive disorders and in clinical practice.

The role of structured exercise interventions on cognitive function in older individuals with stable Chronic Obstructive Pulmonary Disease: A scoping review

A decline in cognitive performance has been associated with disease severity, exacerbations rate, presence of comorbidities, and low activity level in people with chronic obstructive pulmonary disease (COPD). Participation in exercise programs appears to have neuroprotective effects and to improve cognitive performance in older people. The present work undertook a scoping review of the effects of exercise-based interventions on cognitive function in older individuals with stable COPD.

METHODS

The methodological framework for scoping review was used and electronic searches of five databases performed. Original research and observational studies published between January 2010 and December 2021, administering exercise-based interventions and cognitive function evaluation, were included.

RESULTS

Of 13 full-text manuscripts assessed for eligibility, five were allocated to analysis. Three studies administered exercise training within pulmonary outpatient rehabilitation program (PR), and one inpatient PR. The fifth study conducted a structured training intervention in which either aerobic or a combination with resistance exercises were included. Twelve cognitive function screening tools were used in the five studies included in the analysis. Results extracted were based on 245 COPD (33% female) with moderate to very-severe airflow limitation. Interventions ranged from 12 to 36 sessions. Studies reported statistically significant improvements after intervention in different cognitive function domains, such as global cognition, immediate and delayed recall ability, cognitive flexibility, verbal fluency, attention, abstract reasoning, praxis ability.

CONCLUSIONS

Exercise-based interventions improve several areas of cognitive function in patients with stable COPD. However, the magnitude of gain varies among studies, and this is possibly due to the heterogeneity of tests used. Future research is needed to validate the optimal battery of screening tests, and to support the definition of guidelines for cognitive function evaluation in COPD.

Semantic and phonetic similarity of verbal fluency responses in early-stage psychosis

Linguistic abnormalities can emerge early in the course of psychotic illness. Computational tools that quantify similarity of responses in standardized language-based tasks such as the verbal fluency test could efficiently characterize the nature and functional correlates of these disturbances. Participants with early-stage psychosis (n=20) and demographically matched controls without a psychiatric diagnosis (n=20) performed category and letter verbal fluency. Semantic similarity was measured via predicted context co-occurrence in a large text corpus using Word2Vec. Phonetic similarity was measured via edit distance using the VFClust tool. Responses were designated as clusters (related items) or switches (transitions to less related items) using similarity-based thresholds. Results revealed that participants with early-stage psychosis compared to controls had lower fluency scores, lower cluster-related semantic similarity, and fewer switches; mean cluster size and phonetic similarity did not differ by group. Lower fluency semantic similarity was correlated with greater speech disorganization (Communication Disturbances Index), although more strongly in controls, and correlated with poorer social functioning (Global Functioning: Social), primarily in the psychosis group. Findings suggest that search for semantically related words may be impaired soon after psychosis onset. Future work is warranted to investigate the impact of language disturbances on social functioning over the course of psychotic illness.

Computational linguistic analysis applied to a semantic fluency task: A replication among first-episode psychosis patients with and without derailment and tangentiality

Automated tools do not yet exist to measure formal thought disorder, including derailment and tangentiality, both of which can be subjectively rated using the Scale for the Assessment of Positive Symptoms after a clinical research interview. CoVec, a new automated tool, measures the semantic similarity among words averaged in a five- and ten-word window (Coherence-5 and Coherence-10, respectively). One prior report demonstrated that this tool was able to differentiate between patients with those types of thought disorder and patients without them (and controls). Here, we attempted a replication of the initial findings using data from a different sample of patients hospitalized for initial evaluation of first-episode psychosis. Participants were administered a semantic fluency task and the animal lists were analyzed with CoVec. In this study, we partially replicated the prior findings, showing that first-episode patients with derailment had significantly lower Coherence-5 and Coherence-10 compared with patients without derailment. Further research is warranted on this and other highly reliable and objective methods of detecting formal thought disorder through simple assessments such as semantic fluency tasks.

Differences between episodic and semantic memory in predicting observation-based activities of daily living in mild cognitive impairment and Alzheimer's disease

Individuals with mild cognitive impairment (MCI) can often progress into Alzheimer's Disease (AD). Research suggests that decline in episodic memory and semantic memory, as well as functional abilities, can be sensitive in predicting disease progression. This study aimed to (a) investigate episodic and semantic memory performance differences between AD and MCI, (b) determine if memory performance predicts observation-based activities of daily living (ADLs), and (c) explore whether semantic memory mediates the relationship between episodic memory and ADLs. Fifty-eight AD, 53 MCI, and 72 healthy control participants were administered the Rey-O, California Verbal Learning Test, Animal Fluency Test, Boston Naming Test, and Direct Assessment of Functional Status (DAFS). The results revealed, first, that AD participants performed significantly lower than the MCI participants across semantic memory and episodic memory tasks, with the exception of the Boston Naming Test. Second, hierarchical-stepwise regression analyses found that semantic memory significantly predicted DAFS orientation, communication, and financial skills in AD, but episodic memory predicted shopping skills. Furthermore, semantic memory significantly predicted DAFS transportation skills in AD and MCI. Third, within the overall sample, semantic memory mediated the relationship between episodic memory and ADLs. Taken together, the findings suggest decline in semantic memory (as measured by confrontational naming and category fluency) and episodic memory (as measured by list and complex visual design learning and recall) may lead to decline in different and specific aspects of functional abilities in AD and MCI.

SNAFU: The Semantic Network and Fluency Utility

The verbal fluency task—listing words from a category or words that begin with a specific letter—is a common experimental paradigm that is used to diagnose memory impairments and to understand how we store and retrieve knowledge. Data from the verbal fluency task are analyzed in many different ways, often requiring manual coding that is time intensive and error-prone. Researchers have also used fluency data from groups or individuals to estimate semantic networks—latent representations of semantic memory that describe the relations between concepts—that further our understanding of how knowledge is encoded. However computational methods used to estimate networks are not standardized and can be difficult to implement, which has hindered widespread adoption. We present SNAFU: the Semantic Network and Fluency Utility, a tool for estimating networks from fluency data and automatizing traditional fluency analyses, including counting cluster switches and cluster sizes, intrusions, perseverations, and word frequencies. In this manuscript, we provide a primer on using the tool, illustrate its application by creating a semantic network for foods, and validate the tool by comparing results to trained human coders using multiple datasets.