Psychometric Properties of Cognitive Assessment in Amyotrophic Lateral Sclerosis: A Systematic Review

What is the extent of reliability and validity evidence for screening tools for cognitive and behavioral change in people with ALS? A systematic review

OBJECTIVE

This systematic review provides an updated summary of the existing literature on the validity of screening tools for cognitive and behavioral impairment in people with Amyotrophic Lateral Sclerosis (pwALS), and also focuses on their reliability.

METHOD

The following cognitive and behavioral screening tools were assessed in this review: the Edinburgh Cognitive and Behavioral ALS Screen (ECAS); the ALS Cognitive Behavioral Screen (ALS-CBS), the Mini Addenbrooke's Cognitive Examination (Mini-ACE), the Beaumont Behavioral Interview (BBI); the MND Behavior Scale (MiND-B); and the ALS-FTD Questionnaire (ALS-FTD-Q). A search, using Medline, PsychINFO and Embase (21/09/2023), generated 37 results after exclusion criteria were applied. Evidence of internal consistency, item-total correlations, inter-rater reliability, clinical validity, convergent validity, and structural validity were extracted and assessed and risk of bias was evaluated.

RESULTS

The cognitive component of the ECAS was the tool with most evidence of reliability and validity for the assessment of cognitive impairment in ALS. It is well-suited to accommodate physical symptoms of ALS. For behavioral assessment, the BBI or ALS-FTD-Q had the most evidence of reliability and validity. The BBI is more thorough, but the ALS-FTD-Q is briefer.

CONCLUSIONS

There is good but limited evidence for the reliability and validity of cognitive and behavioral screens. Further evidence of clinical and convergent validity would increase confidence in their clinical and research use.

Ecological validity of performance-based cognitive screeners in amyotrophic lateral sclerosis: preliminary evidence

BACKGROUND

This study aimed at preliminarily assessing, in a cohort of non-demented amyotrophic lateral sclerosis (ALS) patients, the ecological validity, and more specifically the veridicality, of the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) and the ALS Cognitive Behavioral Screen (ALS-CBS™), by relating their scores to caregiver-report ratings of cognitive changes.

METHODS

N = 147 patient-caregiver dyads were recruited. Patients were administered the ECAS and ALS-CBS™, whilst caregiver the Caregiver Behavioral Questionnaire (CBQ) and Beaumont Behavioural Inventory (BBI). An Ecological Cognitive Functioning Index (ECFI) was derived from those items of the CBQ and BBI that tap on executive and language changes. Ecological validity was assessed via both correlational and predictive analyses net of caregiver-rated behavioural changes (as assessed by the ECAS-Carer Interview).

RESULTS

The ECFI was associated with the total scores on both the ECAS (p = .014) and ALS-CBS™ (p = .017). When looking at ECAS and ALS-CBS™ subscales, those assessing verbal fluency were selectively associated with the ECFI. The ECFI was higher in patients performing defectively on the ECAS (p = .004) and on the ALS-CBS™ (p = .027).

DISCUSSION

This study suggests that both the ECAS and the ALS-CBS™ represent a valid estimate of non-demented ALS patients' cognitive status in the real world, also highlighting the clinical relevance of cognitive changes reported by caregivers.

Accessing and Receiving Speech-Language Pathology Services at the Multidisciplinary Amyotrophic Lateral Sclerosis Clinic: An Exploratory Qualitative Study of Patient Experiences and Needs

PURPOSE

This study sought to explore how patients with amyotrophic lateral sclerosis (ALS) presenting with coexisting bulbar and cognitive impairments and their caregivers experienced the speech-language pathologist (SLP) services provided in multidisciplinary ALS clinics in Canada and identified their perceived needs for bulbar symptom management.

METHOD

This qualitative study was informed by interpretive description. Seven interviews were conducted with patients with severe bulbar dysfunction or severe bulbar and cognitive dysfunction due to ALS or ALS-frontotemporal dementia, respectively, and/or their caregivers. Purposive sampling was used to recruit individuals with severe bulbar or bulbar and cognitive disease. Thematic analysis was used to analyze interview data.

RESULTS

Patients and caregivers reported difficulties with accessing and receiving SLP services at the multidisciplinary ALS clinic. These difficulties were further exacerbated in those with severe cognitive disease. Participants expressed a need for more specific (i.e., disease and service-related) information and personalized care to address their changing needs and preferences. Engaging caregivers earlier in SLP appointments was perceived as vital to support care planning and provide in-time caregiver education.

CONCLUSIONS

This study highlighted the challenges experienced by patients and caregivers in accessing and receiving SLP services. There is a pressing need for a more person-centered approach to ALS care and a continuing need for education of SLPs on care provision in cases of complex multisymptom diseases within a multidisciplinary ALS clinic.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24069222.

Clinimetrics of the cognitive section of the Italian ALS Cognitive Behavioral Screen (ALS-CBS™)

BACKGROUND

The present study aimed at (1) providing further validity and reliability evidence for the Italian version of the cognitive section of the ALS Cognitive Behavioral Screen (ALS-CBS™) and (2) testing its diagnostics within an Italian ALS cohort, as well as at (3) exploring its capability to discriminate patients from healthy controls (HCs).

METHODS

N = 293 non-demented ALS patients were administered the cognitive sections of the ALS-CBS™ and Edinburgh Cognitive and Behavioural ALS Screen (ECAS). N = 96 HCs demographically matched with N = 96 patients were also administered the cognitive section of the ALS-CBS™. In patients, factorial and construct validity, internal reliability, and diagnostics against a defective score on the cognitive section of the ECAS were tested. Case-control discrimination was assessed via a logistic regression.

RESULTS

ALS-CBS™ cognitive subscales were underpinned by a simple, unidimensional structure, internally reliable (McDonald's ω = 0.74), and mostly related with ECAS executive and fluency scores (r_s = 0.54-0.71). Both raw and age- and education-adjusted scores on the cognitive section of the ALS-CBS™ accurately detected ECAS-defined cognitive impairment (AUC = 0.80 and .88, respectively), yielding optimal error-based, information-based and unitary diagnostics. A cut-off of < 15.374 was identified on adjusted scores. The test was able to discriminate patients from HCs (p < 0.001).

DISCUSSION

The cognitive section of the Italian ALS-CBS™ is a valid, reliable, and diagnostically sound ALS-specific screener for detecting frontotemporal, executive-/attentive-based cognitive inefficiency in non-demented ALS patients, being also able to discriminate them from normotypical individuals.

Norwegian version of the Edinburgh cognitive and behavioural ALS screen: Construct validity, internal consistency, inter-rater, and test-retest reliability

BACKGROUND

Research collaboration highlight a need for validated tests in other languages than English. Translation and culture adjustments may threaten essential features of the original instrument.

OBJECTIVE

To assess the internal consistency, inter-rater and test-retest reliability, and construct validity of the Norwegian version of the Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis (ALS) Screen (ECAS-N).

METHODS

Performance of 71 subjects with ALS, 85 healthy controls (HC) and 6 controls with Alzheimer's disease (AD) were assessed with the ECAS-N. Test-retest interval was four months. Internal consistency was evaluated using Cronbach's alpha; reliability was assessed using intraclass correlation coefficient (ICC), Cohen's kappa, and Bland Altman plot. Five hypothesis, including the Montreal Cognitive Assessment (MoCA) screen, was evaluated for construct validity.

RESULTS

ECAS-N total score produced a Cronbach's alpha of 0.65, had excellent inter-rater reliability (ICC = 0.99) and acceptable test-retest reliability (ICC = 0.73). Construct validity analysis suggested valid use of the ECAS-N to distinguish people with ALS-specific cognitive impairment from HC (p = 0.001) and those with AD (p = 0.002). The MoCA and ECAS-N were moderately correlated (r = 0.53).

CONCLUSION

The ECAS-N has potential to be used by different testers in clinical practice and research to screen patients with ALS who speak Norwegian and for documenting cognitive impairment over time.

Feasibility and diagnostics of the Frontal Assessment Battery (FAB) in amyotrophic lateral sclerosis

BACKGROUND

The present study aimed at evaluating the diagnostic properties of the Frontal Assessment Battery (FAB) in non-demented ALS patients by addressing the Edinburgh Cognitive Behavioural ALS Screen (ECAS) as the gold standard, as well as by examining the association between its administrability and scores with motor-functional measures.

MATERIALS

N = 348 consecutive patients were administered the ECAS and FAB. Disease severity (ALSFRS-R), duration, progression rate (ΔFS), and stages (via King's and Milano-Torino systems) were considered. Administrability rates and prevalence of below-cut-off FAB scores were compared across clinical stages; regression models allowed to test whether, net of the ECAS-Total, motor features predicted the probability of the FAB not being administrable and of a defective FAB score. Intrinsic and post-test diagnostics were explored against a combined defective ECAS-Executive and ECAS-Fluency scores.

RESULTS

85.3% of patients managed to complete the FAB. FAB administrability rates decreased with advanced clinical stages, whereas the prevalence of below-cut-off FAB scores did not. The probability of the FAB not being administrable was predicted only by lower ALSFRS-R-bulbar and ALSFRS-R-upper-limb scores; no motor features, but the ECAS-Total, predicted a below-cut-off performance on the FAB. Raw and adjusted FAB scores showed high accuracy (AUC = .85 and .81, respectively) and good intrinsic and post-test properties.

DISCUSSION

The FAB is featured by optimal diagnostics for detecting executive deficits in ALS, provided that it can be administered according to its original, standardized procedure, and thus that patients have sufficiently spared motor abilities to complete the test.

A clinical bulbar assessment scale (CBAS) for amyotrophic lateral sclerosis

INTRODUCTION/AIMS

Comprehensive and valid bulbar assessment scales for use within amyotrophic lateral sclerosis (ALS) clinics are critically needed. The aims of this study are to develop the Clinical Bulbar Assessment Scale (CBAS) and complete preliminary validation.

METHODS

The authors selected CBAS items from among the literature and expert opinion, and content validity ratio (CVR) was calculated. Following consent, the CBAS was administered to a pilot sample of English-speaking adults with El Escorial defined ALS (N = 54) from a multidisciplinary clinic, characterizing speech, swallowing, and extrabulbar features. Criterion validity was assessed by correlating CBAS scores with commonly used ALS scales, and internal consistency reliability was obtained.

RESULTS

Expert raters reported strong agreement for the CBAS items (CVR = 1.00; 100% agreement). CBAS scores yielded a moderate, significant, negative correlation with ALS Functional Rating Scale-Revised (ALSFRS-R) total scores (r = -0.652, p < .001), and a strong, significant, negative correlation with ALSFRS-R bulbar subscale scores (r = -0.795, p < .001). There was a strong, significant, positive correlation with Center for Neurologic Studies Bulbar Function Scale (CNS-BFS) scores (r = 0.819, p < .001). CBAS scores were significantly higher for bulbar onset (mean = 38.9% of total possible points, SD = 22.6) than spinal onset (mean = 18.7%, SD = 15.8; p = .004). Internal consistency reliability (Cronbach's alpha) values were: (a) total CBAS, α = 0.889; (b) Speech subscale, α = 0.903; and (c) Swallowing subscale, α = 0.801.

DISCUSSION

The CBAS represents a novel means of standardized bulbar data collection using measures of speech, swallowing, respiratory, and cognitive-linguistic skills. Preliminary evidence suggests the CBAS is a valid, reliable scale for clinical assessment of bulbar dysfunction.

Validity and diagnostics of the Reading the Mind in the Eyes Test (RMET) in non-demented amyotrophic lateral sclerosis (ALS) patients

Background

The aim of this study was to explore the construct validity and diagnostic properties of the Reading the Mind in the Eyes Test (RMET) in non-demented patients with amyotrophic lateral sclerosis (ALS).

Materials

A total of 61 consecutive patients and 50 healthy controls (HCs) were administered the 36-item RMET. Additionally, patients underwent a comprehensive assessment of social cognition via the Story-Based Empathy Task (SET), which encompasses three subtests targeting Causal Inference, Emotion Attribution (SET-EA), and Intention Attribution (SET-IA), as well as global cognitive [the Edinburgh Cognitive and Behavioral ALS Screen (ECAS)] and behavioral screening [the Frontal Behavioral Inventory (FBI); the Dimensional Apathy Scale (DAS); the Beck Depression Inventory (BDI); and the State and Trait Anxiety Inventory-Y]. The construct validity of the RMET was tested by regressing it within a stepwise model that encompassed as predictors the abovementioned cognitive and behavioral measures, covarying for demographic and motor confounders. Receiver-operating characteristics (ROC) analyses allowed exploring intrinsic and post-test properties of the RMET both in discriminating patients from HCs and in identifying patients with a defective SET-EA performance.

Results

The RMET was solely predicted by the SET-EA (p = 0.003) and SET-IA (p = 0.005). RMET scores showed high accuracy both in discriminating patients from HCs (AUC = 0.81) and in identifying patients with a defective SET-EA score (AUC = 0.82), with adequate-to-optimal both intrinsic and post-test properties.

Discussion

The RMET is a convergently and divergently valid measure of affective social cognition in non-demented ALS patients, also featuring optimal intrinsic and post-test diagnostic properties in both case-control and case-finding scenarios.

Translation, cultural adaptation, and validation of a screening test for cognitive and behavioural changes in amyotrophic lateral sclerosis

PURPOSE

To describe challenges of translating and culturally adapting the Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis (ALS) Screen into Norwegian (ECAS-N), evaluate its content validity; provide age- and education-balanced norms for verbal fluency and cut-off values for abnormal performance.

MATERIALS AND METHODS

Translation to Norwegian and back-translation to English complied with standard methods. Patients and ALS experts evaluated the relevance, comprehensiveness and comprehensibility of the translated ECAS (ECAS-N). Content validity indexes at the item level (I-CVI), scale level (S-CVI) and inter-rater agreement were calculated. Performance of controls determined norms for written and spoken verbal fluency (n = 559) and cut-off scores for abnormal performance (n = 85).

RESULTS

High levels of content validity was achieved for all items of the ECAS-N, I-CVI, S-CVI and inter-rater agreement was 87.5%. Age- and education-balanced norms for written and spoken verbal fluency were produced. Cut-off scores of abnormal performance were slightly lower than the original ECAS.

CONCLUSIONS

The ECAS-N holds promise for detecting cognitive and behavioural impairment in Norwegian patients with ALS. Cut-off scores are situational and could slightly vary between different cultures. The ECAS-N can be used in international research, but researchers should be aware of the differences between the tests applied in the studies.Implications for rehabilitationThe ECAS-N, a Norwegian version of the English-language ECAS was produced for clinical use in Norway.Rigorous translation methods were used, and expert-judged modifications were made for Norwegian-specific culture and language nuances.Determination of ECAS-N cut-off scores for abnormal cognition was guided by a biostatistician, are slightly lower than the original ECAS, and are based on results from an objectively calculated number of healthy Norwegians.A nuanced picture of cognitive impairment in ALS drawn from a well-developed scale as the ECAS-N can contribute to a management process better tailored to patients'- and carers' individual needs.

Diagnostics and clinical usability of the Montreal Cognitive Assessment (MoCA) in amyotrophic lateral sclerosis

Background

The present study aimed at (1) assessing the diagnostic properties of the Montreal Cognitive Assessment (MoCA) in non-demented ALS patients and at (2) exploring the MoCA administrability according to motor-functional status.

Materials

N = 348 patients were administered the MoCA and Edinburgh Cognitive and Behavioural ALS Screen (ECAS). Administrability rates and prevalence of defective MoCA scores were compared across King’s and Milano-Torino clinical stages. Regression models were run to test whether the non-administrability of the MoCA and a defective score on it were predicted, net of the ECAS-Total, by disease duration, ALS Functional Rating Scale-Revised (ALSFRS-R) and progression rate, computed as (48: ALSFRS-R)/disease duration. Intrinsic and post-test diagnostics were tested against a below-cut-off ECAS-total score.

Results

The 79.9% of patients successfully underwent the MoCA, whose administrability rates decreased with advanced clinical stages, at variance with its defective score prevalence. The probability of the FAB not being administrable was predicted only by lower ALSFRS-R-bulbar and-upper-limb scores; no motor features, but the ECAS-Total, predicted a defective MoCA performance. The MoCA showed high accuracy (AUC = 0.82) and good intrinsic and post-test properties—being slightly more specific than sensitive.

Discussion

In non-demented ALS patients, the MoCA is featured by optimal diagnostics as a screener for cognitive impairment, especially for ruling-out its occurrence, as long as patients are in the early stages of the disease and have sufficiently spared bulbar and upper-limb functions.

The diagnostic value of the Italian version of the Edinburgh Cognitive and Behavioral ALS Screen (ECAS)

Objectives: The diagnostic capabilities of specific cognitive screeners are clinically crucial for an early detection of frontotemporal features in amyotrophic lateral sclerosis (ALS) patients. This study aimed at providing diagnostic properties of the cognitive section of the Edinburgh Cognitive and Behavioral ALS Screen (ECAS) in Italian ALS patients.Methods: Eighty-nine consecutive Italian ALS patients were classified according to Strong et al. (2017) criteria with a comprehensive neuropsychological assessment. The Italian version of ECAS was also administered, and its accuracy, sensitivity (SE), specificity (SP), negative and positive predictive values (PPV; NPV) and likelihood ratios (LR+; LR-) were computed against clinical diagnoses.Results: The ECAS and its subscales yielded moderate-to-high accuracy (AUC = .7-.87). High SP was overall found (81.8%-100%). The most sensitive measures were ALS-specific and Executive (73.9-78.3%) subscales, while the least were the ALS-non-specific subscales (8.7-17.4%). ECAS measures showed good PPVs (60%-100%) and NPVs (75.9%-91.5%). Acceptable LRs were found (LR+: 6.97-4.3; LR-: .29-.91), with total and ALS-specific measures yielding the best estimates.Conclusions: This is the first study demonstrating the diagnostic value of the ECAS against a thorough neuropsychological assessment in Italy. ECAS-total and -ALS-specific scores are diagnostically sound as to both intrinsic and post-test features, whereas ALS-non-specific measures perform slightly worse.

Psychometrics and diagnostics of Italian cognitive screening tests: a systematic review

BACKGROUND

Cognitive screening tests (CSTs) are crucial to neuropsychological diagnostics, and thus need to be featured by robust psychometric and diagnostic properties. However, CSTs happen not to meet desirable statistical standards, negatively affecting their level of recommendations and applicability. This study aimed at (a) providing an up-to-date compendium of available CSTs in Italy, (b) report their psychometric and diagnostic properties, and (c) address related limitations.

METHODS

This review was implemented by consulting Preferred Reporting Items for Systematic Reviews and Meta-Analyses and pre-registered on the International Prospective Register of Systematic Reviews. Standardization and usability studies focusing on norms, validity, reliability, or sensitivity/specificity (and derived metrics) in adults were considered for eligibility. Quality assessment was performed by means of an ad hoc checklist collecting information on sampling, psychometrics/diagnostics, norming, and feasibility.

RESULTS

Sixty studies were included out of an initial N = 683. Identified CSTs (N = 40) were classified into general, domain-, and disease-specific (N = 17, 7, and 16, respectively), the latter being less statistically robust than remaining categories. Validity and reliability evidence was provided for 29 and 26 CSTs, respectively, sensitivity/specificity for 20 and norms for 33. Prevalence- and post-test-based diagnostic metrics were seldomly represented; factorial structures, ceiling/floor effects, and acceptability rarely investigated; content, face, and ecological validity never assessed.

DISCUSSION

Although available Italian CSTs overall met basic psychometric/diagnostic requirements, their statistical profile often proved to be poor on several properties that are desirable for clinical applications, with a few exceptions among general and domain-specific ones.

Cognitive Profiles of Amyotrophic Lateral Sclerosis Differ in Resting-State Functional Connectivity: An fMRI Study

Background

Half of all amyotrophic lateral sclerosis-frontotemporal spectrum disorder (ALS-FTSD) patients are classified as cognitively impaired, of which 10% have frontotemporal dementia (FTD), and an additional 40% suffer from a frontotemporal syndrome not severe enough to be described as dementia (cognitively impaired/ALSci). As changes in cerebral function measured by resting-state magnet resonance imaging (rs-fMRI) are known in ALS, we investigated whether group differences in resting-state functional connectivity (RSFC) networks could be observed between ALS patients with different cognitive profiles against healthy controls (HC). Furthermore, we correlated cognition and motor functioning with network connectivity.

Methods

Healthy controls, 69, and 97 ALS patients underwent functional MRI scanning and cognitive assessment. The ALS patients were categorized as non-impaired (ALSni; n = 68), cognitively impaired (ALSci; n = 21), and ALS-FTD (n = 8). Group differences in connectivity of the default mode network (DMN), motor network (MN), and ventral attention network (VAN) were investigated using a full-factorial model; correlations between global cognitive performance, shifting, and motor symptom severity were established using Pearson’s correlation.

Results

At a liberal alpha level of uncorrected p < 0.005 and a cluster size exceeding 20 voxels, we found widespread decreases in functional connectivity in all three networks when comparing ALS patients to HC. Similar patterns of hypoconnectivity in the bilateral motor cortices and frontotemporal emerged when comparing the ALSci and ALS-FTD patients to those not cognitively impaired. Hyperconnectivity in the DMN temporal gyrus correlated with worse global cognition; moreover, hyperconnectivity in the VAN thalamus, insula, and putamen correlated with worse shifting ability. Better-preserved motor function correlated with higher MN connectivity. Only the motor-related effects prevailed at a more conservative significance level of p_FDR< 0.001.

Conclusion

Resting-state functional connectivity differs between cognitive profiles of ALS and is directly associated with clinical presentation, specifically with motor function, and cognitive shifting.