1
|
Podda J, Tacchino A, Ponzio M, Di Antonio F, Susini A, Pedullà L, Battaglia MA, Brichetto G. Mobile Health App (DIGICOG-MS) for Self-Assessment of Cognitive Impairment in People With Multiple Sclerosis: Instrument Validation and Usability Study. JMIR Form Res 2024; 8:e56074. [PMID: 38900535 PMCID: PMC11224705 DOI: 10.2196/56074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Revised: 05/06/2024] [Accepted: 05/08/2024] [Indexed: 06/21/2024] Open
Abstract
BACKGROUND Mobile health (mHealth) apps have proven useful for people with multiple sclerosis (MS). Thus, easy-to-use digital solutions are now strongly required to assess and monitor cognitive impairment, one of the most disturbing symptoms in MS that is experienced by almost 43% to 70% of people with MS. Therefore, we developed DIGICOG-MS (Digital assessment of Cognitive Impairment in Multiple Sclerosis), a smartphone- and tablet-based mHealth app to self-assess cognitive impairment in MS. OBJECTIVE This study aimed to test the validity and usability of the novel mHealth app with a sample of people with MS. METHODS DIGICOG-MS includes 4 digital tests assumed to evaluate the most affected cognitive domains in MS (visuospatial memory [VSM], verbal memory [VM], semantic fluency [SF], and information processing speed [IPS]) and inspired by traditional paper-based tests that assess the same cognitive functions (10/36 Spatial Recall Test, Rey Auditory Verbal Learning Test, Word List Generation, Symbol Digit Modalities Test). Participants were asked to complete both digital and traditional assessments in 2 separate sessions. Convergent validity was analyzed using the Pearson correlation coefficient to determine the strength of the associations between digital and traditional tests. To test the app's reliability, the agreement between 2 repeated measurements was assessed using intraclass correlation coefficients (ICCs). Usability of DIGICOG-MS was evaluated using the System Usability Scale (SUS) and mHealth App Usability Questionnaire (MAUQ) administered at the conclusion of the digital session. RESULTS The final sample consisted of 92 people with MS (60 women) followed as outpatients at the Italian Multiple Sclerosis Society (AISM) Rehabilitation Service of Genoa (Italy). They had a mean age of 51.38 (SD 11.36) years, education duration of 13.07 (SD 2.74) years, disease duration of 12.91 (SD 9.51) years, and a disability level (Expanded Disability Status Scale) of 3.58 (SD 1.75). Relapsing-remitting MS was most common (68/92, 74%), followed by secondary progressive (15/92, 16%) and primary progressive (9/92, 10%) courses. Pearson correlation analyses indicated significantly strong correlations for VSM, VM, SF, and IPS (all P<.001), with r values ranging from 0.58 to 0.78 for all cognitive domains. Test-retest reliability of the mHealth app was excellent (ICCs>0.90) for VM and IPS and good for VSM and SF (ICCs>0.80). Moreover, the SUS score averaged 84.5 (SD 13.34), and the mean total MAUQ score was 104.02 (SD 17.69), suggesting that DIGICOG-MS was highly usable and well appreciated. CONCLUSIONS The DIGICOG-MS tests were strongly correlated with traditional paper-based evaluations. Furthermore, people with MS positively evaluated DIGICOG-MS, finding it highly usable. Since cognitive impairment poses major limitations for people with MS, these findings open new paths to deploy digital cognitive tests for MS and further support the use of a novel mHealth app for cognitive self-assessment by people with MS in clinical practice.
Collapse
Affiliation(s)
- Jessica Podda
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Andrea Tacchino
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Michela Ponzio
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Federica Di Antonio
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Alessia Susini
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Ludovico Pedullà
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| | - Mario Alberto Battaglia
- Department of Physiopathology, Experimental Medicine and Public Health, University of Siena, Siena, Italy
| | - Giampaolo Brichetto
- Scientific Research Area, Italian Multiple Sclerosis Foundation, Genoa, Italy
| |
Collapse
|
2
|
de Villers-Sidani É, Voss P, Bastien N, Cisneros-Franco JM, Hussein S, Mayo NE, Koch NA, Drouin-Picaro A, Blanchette F, Guitton D, Giacomini PS. Oculomotor analysis to assess brain health: preliminary findings from a longitudinal study of multiple sclerosis using novel tablet-based eye-tracking software. Front Neurol 2023; 14:1243594. [PMID: 37745656 PMCID: PMC10516298 DOI: 10.3389/fneur.2023.1243594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 08/07/2023] [Indexed: 09/26/2023] Open
Abstract
A growing body of evidence supports the link between eye movement anomalies and brain health. Indeed, the oculomotor system is composed of a diverse network of cortical and subcortical structures and circuits that are susceptible to a variety of degenerative processes. Here we show preliminary findings from the baseline measurements of an ongoing longitudinal cohort study in MS participants, designed to determine if disease and cognitive status can be estimated and tracked with high accuracy based on eye movement parameters alone. Using a novel gaze-tracking technology that can reliably and accurately track eye movements with good precision without the need for infrared cameras, using only an iPad Pro embedded camera, we show in this cross-sectional study that several eye movement parameters significantly correlated with clinical outcome measures of interest. Eye movement parameters were extracted from fixation, pro-saccade, anti-saccade, and smooth pursuit visual tasks, whereas the clinical outcome measures were the scores of several disease assessment tools and standard cognitive tests such as the Expanded Disability Status Scale (EDSS), Brief International Cognitive Assessment for MS (BICAMS), the Multiple Sclerosis Functional Composite (MSFC) and the Symbol Digit Modalities Test (SDMT). Furthermore, partial least squares regression analyses show that a small set of oculomotor parameters can explain up to 84% of the variance of the clinical outcome measures. Taken together, these findings not only replicate previously known associations between eye movement parameters and clinical scores, this time using a novel mobile-based technology, but also the notion that interrogating the oculomotor system with a novel eye-tracking technology can inform us of disease severity, as well as the cognitive status of MS participants.
Collapse
Affiliation(s)
- Étienne de Villers-Sidani
- Innodem Neurosciences, Montreal, QC, Canada
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Patrice Voss
- Innodem Neurosciences, Montreal, QC, Canada
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | | | - J. Miguel Cisneros-Franco
- Innodem Neurosciences, Montreal, QC, Canada
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | | | - Nancy E. Mayo
- Faculty of Medicine, School of Physical and Occupational Therapy, McGill University, Montreal, QC, Canada
| | - Nils A. Koch
- Innodem Neurosciences, Montreal, QC, Canada
- Integrated Program in Neuroscience, McGill University, Montreal, QC, Canada
| | | | | | - Daniel Guitton
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Paul S. Giacomini
- Innodem Neurosciences, Montreal, QC, Canada
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| |
Collapse
|
3
|
Costabile T, Signoriello E, Lauro F, Altieri M, Ziello AR, D'Ambrosio A, Bisecco A, Maniscalco G, Bonavita S, Gallo A, Brescia Morra V, Lus G, Saccà F, Russo CV. Validation of an iPad version of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler Relat Disord 2023; 74:104723. [PMID: 37086633 DOI: 10.1016/j.msard.2023.104723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 04/02/2023] [Accepted: 04/16/2023] [Indexed: 04/24/2023]
Abstract
BACKGROUND The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is the most widely used screening tool for cognitive impairment in Multiple Sclerosis (MS). However, the administration and scoring procedures of the paper version are time consuming and prone to errors. Aim of our study was to develop a tablet version of BICAMS (iBICAMS), and to assess its reliability compared to the paper version. METHODS We administered both BICAMS and iBICAMS to 139 MS patients in two different sessions. We compared scores on both versions using a paired t-test. We used a repeated measures ANOVA to test the impact of rater, order of administration and test-retest time on test-retest performances. We used the Intraclass Correlation Coefficient (ICC) to assess the reliability between BICAMS and iBICAMS. RESULTS All three sub-tests of the BICAMS (SDMT, CVLT-II and BVMT-R) were different between the paper and the tablet versions. Order of administration influenced test-retest performances at the SDMT (p<0.001), CVLT- II (p<0.001) and BVMT-R (p<0.001). Intraclass coefficient correlation (ICC) revealed a high level of agreement between the paper BICAMS and the iPad version for all three tests: SDMT (0.92), CVLT-II (0.83) and BVMT-R (0.82). CONCLUSIONS We found a high reliability between BICAMS and iBICAMS. Considering the inherent advantages of automated scoring, digital storage of data, standardized timing, the iBICAMS could become a standard in clinical practice.
Collapse
Affiliation(s)
- Teresa Costabile
- Multiple Sclerosis Center, II Division of Neurology, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Elisabetta Signoriello
- Multiple Sclerosis Center, II Division of Neurology, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Francesca Lauro
- Department of Neurosciences, Reproductive and Odontostomatological Sciences, University Federico II, Naples, Italy
| | - Manuela Altieri
- Department of Psychology, University of Campania "Luigi Vanvitelli", Caserta, Italy
| | | | - Alessandro D'Ambrosio
- Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Alvino Bisecco
- Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Naples, Italy
| | | | - Simona Bonavita
- Multiple Sclerosis Center, II Division of Neurology, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Antonio Gallo
- Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Vincenzo Brescia Morra
- Department of Neurosciences, Reproductive and Odontostomatological Sciences, University Federico II, Naples, Italy
| | - Giacomo Lus
- Multiple Sclerosis Center, II Division of Neurology, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Francesco Saccà
- Department of Neurosciences, Reproductive and Odontostomatological Sciences, University Federico II, Naples, Italy.
| | - Cinzia Valeria Russo
- Department of Neurosciences, Reproductive and Odontostomatological Sciences, University Federico II, Naples, Italy
| |
Collapse
|
4
|
Splonskowski M, Jacova C. Older Adults' Likelihood of Participating in Home-Based Cognitive Assessment. Dement Geriatr Cogn Disord 2022; 51:32-41. [PMID: 35193138 DOI: 10.1159/000521903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Accepted: 01/05/2022] [Indexed: 11/19/2022] Open
Abstract
INTRODUCTION Timely detection of cognitive impairment among older adults has shown to lead to better health and financial outcomes but is hampered by psychological, financial, and physical barriers to participation in cognitive assessment. Home-based cognitive assessment (HBCA) could help overcome some of these barriers. This study aimed to examine older adults' likelihood of participation in HBCA and identify factors predicting this likelihood. MATERIALS AND METHODS A cross-sectional online survey distributed through Amazon Mechanical Turk, was used to collect data from adults aged 50 years or older residing in the USA The survey was designed to gauge attitudes toward technology and cognitive assessment and to measure psychological variables including subjective cognitive decline (SCD), depression, and anxiety. Information on income and geographic location (rural vs. suburban and urban) was also collected. Univariate and hierarchical regression analyses were conducted to examine the contributions of these variables to a composite measure of likelihood of participation in HBCA. RESULTS Complete data were obtained on n = 483 (age 50-79). Approximately, two-thirds of respondents described themselves as likely or very likely to participate in HBCA. In univariate analyses, younger age, higher income, higher technology assessment acceptance scores, and higher SCD burden were associated with higher likelihood of participation. Hierarchical regression revealed significant stepwise increments in explained variance: demographic variables 4.1%, technology acceptance 25.2%, assessment acceptance 15.4%, and SCD burden 1.6%. The contribution of SCD was moderated by sex and found for women but not for men. DISCUSSION/CONCLUSION A large proportion of adults aged 50 and above described themselves likely to participate in HBCA. Middle-aged, technology-savvy, higher income adults expressed the most positive attitudes toward this type of service. Of interest is that HBCA may be particularly acceptable among older women with SCD, a group known to be at risk of cognitive decline. Our findings support the expansion of cognitive assessment services to the home setting.
Collapse
Affiliation(s)
| | - Claudia Jacova
- School of Graduate Psychology, Pacific University, Hillsboro, Oregon, USA
| |
Collapse
|
5
|
Rogers F, Bane E, Dwyer CP, Alvarez-Iglesias A, Joyce RA, Hynes SM. Remote administration of BICAMS measures and the Trail-Making Test to assess cognitive impairment in multiple sclerosis. Neuropsychol Rehabil 2022; 33:903-926. [PMID: 35297748 DOI: 10.1080/09602011.2022.2052324] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Reliable remote cognitive testing could provide a safer assessment of cognitive impairment in multiple sclerosis (MS) during the COVID-19 pandemic and thereafter. Here we aimed to investigate the reliability and feasibility of administering Brief International Cognitive Assessment for MS (BICAMS) and the Trail-Making Test (TMT) to people with MS online. Between-group differences on BICAMS and the TMT were examined in a sample of 68 participants. Group 1 (N = 34) was tested in-person pre-pandemic. Group 2 was tested remotely. Within-group differences for in-person and virtual administrations were examined for Group 1. No significant differences between virtual and in-person administrations of the CVLT-II and SDMT were detected. BVMT-R scores were significantly higher for virtual administrations (M = 20.59, SD = 6.65) compared to in-person administrations (M = 16.35, SD = 6.05), possibly indicating inter-rater differences. Strong positive correlations were found for in-person and virtual scores within Group 1 on the CVLT-II (r = .84), SDMT (r = .85), TMT-A (r = .88), TMT-B (r = .76) and BVMT-R (r = .72). No significant differences between in-person and remote administrations of CVLT-II and SDMT in people living with MS were detected. Recommendations for future studies employing the TMT and BVMT-R online are provided.
Collapse
Affiliation(s)
| | - Eimear Bane
- Student Counselling and Wellbeing, University of Limerick, Limerick, Ireland
| | - Christopher P Dwyer
- School of Social Science, Technological University of the Shannon IE, Athlone, Ireland
| | - Alberto Alvarez-Iglesias
- Health Research Board Clinical Research Facility, National University of Ireland Galway, Galway, Ireland
| | - Robert A Joyce
- Discipline of Occupational Therapy, School of Health Sciences, National University of Ireland Galway, Galway, Ireland
| | - Sinéad M Hynes
- Discipline of Occupational Therapy, School of Health Sciences, National University of Ireland Galway, Galway, Ireland
| |
Collapse
|
6
|
Smartphone-based symbol-digit modalities test reliably captures brain damage in multiple sclerosis. NPJ Digit Med 2021; 4:36. [PMID: 33627777 PMCID: PMC7904910 DOI: 10.1038/s41746-021-00401-y] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Accepted: 01/06/2021] [Indexed: 02/07/2023] Open
Abstract
As the burden of neurodegenerative diseases increases, time-limited clinic encounters do not allow quantification of complex neurological functions. Patient-collected digital biomarkers may remedy this, if they provide reliable information. However, psychometric properties of digital tools remain largely un-assessed. We developed a smartphone adaptation of the cognitive test, the Symbol-Digit Modalities Test (SDMT) by randomizing the test’s symbol-number codes and testing sequences. The smartphone SDMT showed comparable psychometric properties in 154 multiple sclerosis (MS) patients and 39 healthy volunteers (HV). E.g., smartphone SDMT achieved slightly higher correlations with cognitive subscores of neurological examinations and with brain injury measured by MRI (R2 = 0.75, Rho = 0.83, p < 0.0001) than traditional SDMT. Mathematical adjustment for motoric disability of the dominant hand, measured by another smartphone test, compensates for the disadvantage of touch-based test. Averaging granular home measurements of the digital biomarker also increases accuracy of identifying true neurological decline.
Collapse
|