iCAMS: Assessing the Reliability of a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) Tablet Application

Mobile Health App (DIGICOG-MS) for Self-Assessment of Cognitive Impairment in People With Multiple Sclerosis: Instrument Validation and Usability Study

BACKGROUND

Mobile health (mHealth) apps have proven useful for people with multiple sclerosis (MS). Thus, easy-to-use digital solutions are now strongly required to assess and monitor cognitive impairment, one of the most disturbing symptoms in MS that is experienced by almost 43% to 70% of people with MS. Therefore, we developed DIGICOG-MS (Digital assessment of Cognitive Impairment in Multiple Sclerosis), a smartphone- and tablet-based mHealth app to self-assess cognitive impairment in MS.

OBJECTIVE

This study aimed to test the validity and usability of the novel mHealth app with a sample of people with MS.

METHODS

DIGICOG-MS includes 4 digital tests assumed to evaluate the most affected cognitive domains in MS (visuospatial memory [VSM], verbal memory [VM], semantic fluency [SF], and information processing speed [IPS]) and inspired by traditional paper-based tests that assess the same cognitive functions (10/36 Spatial Recall Test, Rey Auditory Verbal Learning Test, Word List Generation, Symbol Digit Modalities Test). Participants were asked to complete both digital and traditional assessments in 2 separate sessions. Convergent validity was analyzed using the Pearson correlation coefficient to determine the strength of the associations between digital and traditional tests. To test the app's reliability, the agreement between 2 repeated measurements was assessed using intraclass correlation coefficients (ICCs). Usability of DIGICOG-MS was evaluated using the System Usability Scale (SUS) and mHealth App Usability Questionnaire (MAUQ) administered at the conclusion of the digital session.

RESULTS

The final sample consisted of 92 people with MS (60 women) followed as outpatients at the Italian Multiple Sclerosis Society (AISM) Rehabilitation Service of Genoa (Italy). They had a mean age of 51.38 (SD 11.36) years, education duration of 13.07 (SD 2.74) years, disease duration of 12.91 (SD 9.51) years, and a disability level (Expanded Disability Status Scale) of 3.58 (SD 1.75). Relapsing-remitting MS was most common (68/92, 74%), followed by secondary progressive (15/92, 16%) and primary progressive (9/92, 10%) courses. Pearson correlation analyses indicated significantly strong correlations for VSM, VM, SF, and IPS (all P<.001), with r values ranging from 0.58 to 0.78 for all cognitive domains. Test-retest reliability of the mHealth app was excellent (ICCs>0.90) for VM and IPS and good for VSM and SF (ICCs>0.80). Moreover, the SUS score averaged 84.5 (SD 13.34), and the mean total MAUQ score was 104.02 (SD 17.69), suggesting that DIGICOG-MS was highly usable and well appreciated.

CONCLUSIONS

The DIGICOG-MS tests were strongly correlated with traditional paper-based evaluations. Furthermore, people with MS positively evaluated DIGICOG-MS, finding it highly usable. Since cognitive impairment poses major limitations for people with MS, these findings open new paths to deploy digital cognitive tests for MS and further support the use of a novel mHealth app for cognitive self-assessment by people with MS in clinical practice.

Oculomotor analysis to assess brain health: preliminary findings from a longitudinal study of multiple sclerosis using novel tablet-based eye-tracking software

A growing body of evidence supports the link between eye movement anomalies and brain health. Indeed, the oculomotor system is composed of a diverse network of cortical and subcortical structures and circuits that are susceptible to a variety of degenerative processes. Here we show preliminary findings from the baseline measurements of an ongoing longitudinal cohort study in MS participants, designed to determine if disease and cognitive status can be estimated and tracked with high accuracy based on eye movement parameters alone. Using a novel gaze-tracking technology that can reliably and accurately track eye movements with good precision without the need for infrared cameras, using only an iPad Pro embedded camera, we show in this cross-sectional study that several eye movement parameters significantly correlated with clinical outcome measures of interest. Eye movement parameters were extracted from fixation, pro-saccade, anti-saccade, and smooth pursuit visual tasks, whereas the clinical outcome measures were the scores of several disease assessment tools and standard cognitive tests such as the Expanded Disability Status Scale (EDSS), Brief International Cognitive Assessment for MS (BICAMS), the Multiple Sclerosis Functional Composite (MSFC) and the Symbol Digit Modalities Test (SDMT). Furthermore, partial least squares regression analyses show that a small set of oculomotor parameters can explain up to 84% of the variance of the clinical outcome measures. Taken together, these findings not only replicate previously known associations between eye movement parameters and clinical scores, this time using a novel mobile-based technology, but also the notion that interrogating the oculomotor system with a novel eye-tracking technology can inform us of disease severity, as well as the cognitive status of MS participants.

Validation of an iPad version of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS)

BACKGROUND

The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is the most widely used screening tool for cognitive impairment in Multiple Sclerosis (MS). However, the administration and scoring procedures of the paper version are time consuming and prone to errors. Aim of our study was to develop a tablet version of BICAMS (iBICAMS), and to assess its reliability compared to the paper version.

METHODS

We administered both BICAMS and iBICAMS to 139 MS patients in two different sessions. We compared scores on both versions using a paired t-test. We used a repeated measures ANOVA to test the impact of rater, order of administration and test-retest time on test-retest performances. We used the Intraclass Correlation Coefficient (ICC) to assess the reliability between BICAMS and iBICAMS.

RESULTS

All three sub-tests of the BICAMS (SDMT, CVLT-II and BVMT-R) were different between the paper and the tablet versions. Order of administration influenced test-retest performances at the SDMT (p<0.001), CVLT- II (p<0.001) and BVMT-R (p<0.001). Intraclass coefficient correlation (ICC) revealed a high level of agreement between the paper BICAMS and the iPad version for all three tests: SDMT (0.92), CVLT-II (0.83) and BVMT-R (0.82).

CONCLUSIONS

We found a high reliability between BICAMS and iBICAMS. Considering the inherent advantages of automated scoring, digital storage of data, standardized timing, the iBICAMS could become a standard in clinical practice.

Older Adults' Likelihood of Participating in Home-Based Cognitive Assessment

INTRODUCTION

Timely detection of cognitive impairment among older adults has shown to lead to better health and financial outcomes but is hampered by psychological, financial, and physical barriers to participation in cognitive assessment. Home-based cognitive assessment (HBCA) could help overcome some of these barriers. This study aimed to examine older adults' likelihood of participation in HBCA and identify factors predicting this likelihood.

MATERIALS AND METHODS

A cross-sectional online survey distributed through Amazon Mechanical Turk, was used to collect data from adults aged 50 years or older residing in the USA The survey was designed to gauge attitudes toward technology and cognitive assessment and to measure psychological variables including subjective cognitive decline (SCD), depression, and anxiety. Information on income and geographic location (rural vs. suburban and urban) was also collected. Univariate and hierarchical regression analyses were conducted to examine the contributions of these variables to a composite measure of likelihood of participation in HBCA.

RESULTS

Complete data were obtained on n = 483 (age 50-79). Approximately, two-thirds of respondents described themselves as likely or very likely to participate in HBCA. In univariate analyses, younger age, higher income, higher technology assessment acceptance scores, and higher SCD burden were associated with higher likelihood of participation. Hierarchical regression revealed significant stepwise increments in explained variance: demographic variables 4.1%, technology acceptance 25.2%, assessment acceptance 15.4%, and SCD burden 1.6%. The contribution of SCD was moderated by sex and found for women but not for men.

DISCUSSION/CONCLUSION

A large proportion of adults aged 50 and above described themselves likely to participate in HBCA. Middle-aged, technology-savvy, higher income adults expressed the most positive attitudes toward this type of service. Of interest is that HBCA may be particularly acceptable among older women with SCD, a group known to be at risk of cognitive decline. Our findings support the expansion of cognitive assessment services to the home setting.

Remote administration of BICAMS measures and the Trail-Making Test to assess cognitive impairment in multiple sclerosis

Reliable remote cognitive testing could provide a safer assessment of cognitive impairment in multiple sclerosis (MS) during the COVID-19 pandemic and thereafter. Here we aimed to investigate the reliability and feasibility of administering Brief International Cognitive Assessment for MS (BICAMS) and the Trail-Making Test (TMT) to people with MS online. Between-group differences on BICAMS and the TMT were examined in a sample of 68 participants. Group 1 (N = 34) was tested in-person pre-pandemic. Group 2 was tested remotely. Within-group differences for in-person and virtual administrations were examined for Group 1. No significant differences between virtual and in-person administrations of the CVLT-II and SDMT were detected. BVMT-R scores were significantly higher for virtual administrations (M = 20.59, SD = 6.65) compared to in-person administrations (M = 16.35, SD = 6.05), possibly indicating inter-rater differences. Strong positive correlations were found for in-person and virtual scores within Group 1 on the CVLT-II (r = .84), SDMT (r = .85), TMT-A (r = .88), TMT-B (r = .76) and BVMT-R (r = .72). No significant differences between in-person and remote administrations of CVLT-II and SDMT in people living with MS were detected. Recommendations for future studies employing the TMT and BVMT-R online are provided.

Smartphone-based symbol-digit modalities test reliably captures brain damage in multiple sclerosis

As the burden of neurodegenerative diseases increases, time-limited clinic encounters do not allow quantification of complex neurological functions. Patient-collected digital biomarkers may remedy this, if they provide reliable information. However, psychometric properties of digital tools remain largely un-assessed. We developed a smartphone adaptation of the cognitive test, the Symbol-Digit Modalities Test (SDMT) by randomizing the test’s symbol-number codes and testing sequences. The smartphone SDMT showed comparable psychometric properties in 154 multiple sclerosis (MS) patients and 39 healthy volunteers (HV). E.g., smartphone SDMT achieved slightly higher correlations with cognitive subscores of neurological examinations and with brain injury measured by MRI (R² = 0.75, Rho = 0.83, p < 0.0001) than traditional SDMT. Mathematical adjustment for motoric disability of the dominant hand, measured by another smartphone test, compensates for the disadvantage of touch-based test. Averaging granular home measurements of the digital biomarker also increases accuracy of identifying true neurological decline.