Developing and Validating a Big-Store Multiple Errands Test

Comparing the Oxford Digital Multiple Errands Test (OxMET) to a real-life version: Convergence, feasibility, and acceptability

We aimed to assess the convergence, feasibility, and acceptability of the Oxford Digital Multiple Errands Test (OxMET) and the in-person Multiple Errands Test-Home version (MET-Home). Participants completed OxMET, MET-Home, Montreal Cognitive Assessment (MoCA), and questionnaires on activities of daily living, depression, technology usage, mobility, and disability. Forty-eight stroke survivors (mean age 69.61, 41.67% female, and average 16.5 months post-stroke) and 50 controls (mean age 71.46, 56.00% female) took part. No performance differences were found for healthy and stroke participants for MET-Home, and only found below p = .05 for OxMET but not below the corrected p = .006. Convergent validity was found between MET-Home and OxMET metrics (most r ≥ .30, p < .006). MET-Home accuracy was related to age (B = -.04, p = .03), sex (B = -.98, p = .03), disability (B = -0.63, p = .04), and MoCA (B = .26, p < .001), whereas OxMET accuracy was predicted by MoCA score (B = .40, p < .001). Feedback indicated that the OxMET was easy and fun and more acceptable than the MET-Home. The MET-Home was more stressful and interesting. The MET tasks demonstrated good convergent validity, with the OxMET digital administration providing a more feasible, inclusive, and acceptable assessment, especially to people with mobility restrictions and more severe stroke.

Immersive Virtual Reality-Based Methods for Assessing Executive Functioning: Systematic Review

BACKGROUND

Neuropsychological assessments traditionally include tests of executive functioning (EF) because of its critical role in daily activities and link to mental disorders. Established traditional EF assessments, although robust, lack ecological validity and are limited to single cognitive processes. These methods, which are suitable for clinical populations, are less informative regarding EF in healthy individuals. With these limitations in mind, immersive virtual reality (VR)-based assessments of EF have garnered interest because of their potential to increase test sensitivity, ecological validity, and neuropsychological assessment accessibility.

OBJECTIVE

This systematic review aims to explore the literature on immersive VR assessments of EF focusing on (1) EF components being assessed, (2) how these assessments are validated, and (3) strategies for monitoring potential adverse (cybersickness) and beneficial (immersion) effects.

METHODS

EBSCOhost, Scopus, and Web of Science were searched in July 2022 using keywords that reflected the main themes of VR, neuropsychological tests, and EF. Articles had to be peer-reviewed manuscripts written in English and published after 2013 that detailed empirical, clinical, or proof-of-concept studies in which a virtual environment using a head-mounted display was used to assess EF in an adult population. A tabular synthesis method was used in which validation details from each study, including comparative assessments and scores, were systematically organized in a table. The results were summed and qualitatively analyzed to provide a comprehensive overview of the findings.

RESULTS

The search retrieved 555 unique articles, of which 19 (3.4%) met the inclusion criteria. The reviewed studies encompassed EF and associated higher-order cognitive functions such as inhibitory control, cognitive flexibility, working memory, planning, and attention. VR assessments commonly underwent validation against gold-standard traditional tasks. However, discrepancies were observed, with some studies lacking reported a priori planned correlations, omitting detailed descriptions of the EF constructs evaluated using the VR paradigms, and frequently reporting incomplete results. Notably, only 4 of the 19 (21%) studies evaluated cybersickness, and 5 of the 19 (26%) studies included user experience assessments.

CONCLUSIONS

Although it acknowledges the potential of VR paradigms for assessing EF, the evidence has limitations. The methodological and psychometric properties of the included studies were inconsistently addressed, raising concerns about their validity and reliability. Infrequent monitoring of adverse effects such as cybersickness and considerable variability in sample sizes may limit interpretation and hinder psychometric evaluation. Several recommendations are proposed to improve the theory and practice of immersive VR assessments of EF. Future studies should explore the integration of biosensors with VR systems and the capabilities of VR in the context of spatial navigation assessments. Despite considerable promise, the systematic and validated implementation of VR assessments is essential for ensuring their practical utility in real-world applications.

Further investigations into performance variance on the Multiple Errands Test

INTRODUCTION

The Multiple Errands Test (MET) is a complex, performance-based assessment that is useful for characterising the impact of impairments of executive function on everyday activities. However, performance variance amongst those without neurological pathology, and the impact of non-cognitive factors on this, requires further investigation.

METHODS

This was a cross-sectional analytic study, conducted with a convenience sample of 40 neurologically intact community-dwelling Australian adults. Participants completed a hospital or shopping centre version of the MET, where their Performance Efficiency, Task Completions and Rule Breaks were recorded. Non-cognitive factors of interest were demographic (age, sex and education), psychological (measured with the Hospital Anxiety and Depression Scale and self-ratings of test anxiety) and assessment-related (assessment site, self-reported site familiarity and observed strategy use). MET performance was analysed using descriptive statistics. A series of standard multiple and binary logistic regression analyses examined the relationships between MET performance and non-cognitive factors.

RESULTS

Most participants (n = 35, 87.5%) completed at least 10 of the 12 prescribed tasks and broke an average of four rules (SD = 2.36). They achieved an average performance efficiency rating of 0.75/1 (SD = 0.15), suggesting variability in the extent to which participants made non-essential location stops and/or failed to complete tasks whilst at an essential location. The assessment site and participant site familiarity had a statistically significant (p < 0.05) impact on Performance Efficiency and Task Completion scores, and psychological factors had a statistically significant (p < 0.05) relationship with Rule Breaks.

CONCLUSION

Findings suggest that the impact of factors other than cognition should be considered when interpreting MET performance. The assessment site and participant site familiarity may contribute to significant variability in Performance Efficiency scores. Clinicians should also be aware of the potential impact of these assessment-related factors on Task Completions and psychological distress on Rule Breaks.

The Multiple Errands Test: A Guide for Site-Specific Version Development

Background. The complex and real-world nature of the Multiple Errands Test (MET) makes it a valuable and increasingly popular assessment of cognitive function. However, these same qualities make its local implementation challenging. Purpose. To produce an evidence-based guide for site-specific adaptation of the MET. Method. The CAN-IMPLEMENT© knowledge translation framework informed a structured approach to the creation of a guide to site-specific version development, informed by twenty-two published approaches to MET adaptation. Applicability of the guide was supported by a two-phase revision process, in which a site-specific hospital and community version produced from its recommendations were administered with forty-two neurologically intact participants and stakeholder feedback obtained. Findings. We offer an outline of core components which maintain the integrity of the MET, and adaptable peripheries which may be modified when required by the local setting. Implications. The proposed guide provides a systematic yet flexible guide for site-specific MET development.

Predictive validity of the Oxford digital multiple errands test (OxMET) for functional outcomes after stroke

The Oxford Digital Multiple Errands Test (OxMET) is a brief computer-tablet based cognitive screen, intended as an ecologically valid assessment of executive dysfunction. We examined aspects of predictive validity in relation to functional outcomes. Participants (≤ 2 months post-stroke) were recruited from an English-speaking stroke rehabilitation in-patient setting. Participants completed OxMET. The Barthel Index, Therapy Outcome Measure (TOMS), and modified Rankin Scale (mRS) were collected from medical notes. Participants were followed up after 6-months and completed the Nottingham Extended Activities of Daily Living (NEADL) scale. 117 participants were recruited (M = 26.18 days post-stroke (SD = 25.16), mean 74.44yrs (SD  = 12.88), median NIHSS 8.32 (IQR = 5-11)). Sixty-six completed a follow-up (M = 73.94yrs (SD  = 12.68), median NIHSS 8 (IQR = 4-11)). Significant associations were found between TOMS and mRS. At 6-month follow up, we found a moderate predictive relationship between the OxMET accuracy and NEADL (R2 = .29, p < .001), and we did not find this prediction with MoCA taken at 6-months. The subacute OxMET associated with measures of functionality and disability in a rehabilitation context, and in activities of daily living. The OxMET is an assessment of executive function with good predictive validity on clinically relevant functional outcome measures that may be more predictive than other cognitive tests.

Daily Activities: The Impact of COPD and Cognitive Dysfunction

OBJECTIVE

Chronic obstructive pulmonary disease (COPD) is characterized by airflow limitation; however, pulmonary function does not fully account for patients' functional difficulties. The primary aim of the study was to determine the association between several domains of cognition and daily activity among those with COPD.

METHOD

Eighty-nine former smokers completed a neuropsychological battery including measures across multiple domains of cognition, pulmonary function measures, and daily activity questionnaires. Using a cross-sectional design, we compared daily activity between former smokers with and without COPD using two measures (St. George's Respiratory Questionnaire [SGRQ] Activity Subscale and Lawton Instrumental Activities of Daily Living [IADL] Scale) and examined the association between cognition and daily activity among those with COPD.

RESULTS

As expected, former smokers with COPD reported more difficulty than those without COPD on both activity measures (SGRQ Activity Subscale p < .001; Lawton IADL Scale p = .040). Among former smokers with COPD, poorer delayed recall was associated with more difficulty with daily activities (SGRQ Activity Subscale) (p = .038) while adjusting for severity of airflow limitation, exercise tolerance, oxygen use, dyspnea, and symptoms of anxiety and depression.

CONCLUSION

The findings suggest that cognition is associated with daily activity in patients with COPD. Future research should examine whether cognitive interventions may help to maximize patients' engagement in daily activities.

Factorial Structure of the EOCL-1 Scale to Assess Executive Functions

The process of assessing executive functions through behavioral observation scales is still under theoretical and empirical construction. This article reports on the analysis of the factorial structure of the EOCL-1 scale that assesses executive functions, as proposed by the theory developed by Luria, which has not been previously considered in this type of evaluation. In this scale, the executive functions taken into account are error correction, internal behavioral and cognition regulatory language, limbic system conscious regulation, decision making, future consideration of consequences of actions, goal-directed behavior, inhibitory control of automatic responses, creation of new behavioral repertoires, and cognitive–behavioral activity verification. A variety of validity and reliability analyses were carried out, with the following results: (a) an adequate internal consistency level of executive functions between α = 0.70 and α = 0.83, (b) significant convergent validity with a scale that assesses frontal deficits between r = −0.07 and r = 0.28, and (c) the scale’s construct validity that proposes a model with an executive central factor comparative fit index (CFI) = 0.93, root mean square error of approximation (RMSEA) = 0.04 (LO.04 and HI.04), standardized root mean square residual (SRMR) = 0.04, and x²₍₃₁₂₎ = 789.29, p = 0.001. The findings are discussed based on previous literature reports and in terms of the benefits of using a scale to assess the proposed executive functions.

Self-reported strategy generation and implementation in the multiple errands test: A qualitative description

The Multiple Errands Test (MET) is a naturalistic assessment of executive function. Strategy use during the MET can provide useful information for the development of a cognitive profile and intervention plan in patients with brain injury. However, while observed external strategy use in the MET is well-documented, information about internal strategy use and reference data with healthy controls is limited. Contextual influences on strategy selection in this real-world assessment are also not well understood. This qualitative descriptive study explored the internal and external strategies used during MET performance by cognitively intact adults. Strategies were categorized as planning, checking, and problem solving. When planning, participants simplified and familiarized themselves with MET requirements before developing an action plan. They checked their performance by asking for help and using cues in the physical environment. When problems arose, these were solved through self-talk, comparing alternatives, applying context and modifying their plan. Results highlighted that individuals employ both visible and hidden strategies during the MET. This suggests that reflective discussions with patients following cognitive task engagement may be important, to uncover and understand strategy use, both to inform analysis of performance and guide strategy training.

The Oxford digital multiple errands test (OxMET): Validation of a simplified computer tablet based multiple errands test

Impairments in executive functioning are common following Acquired Brain Injury, though there are few screening tools which present a time efficient and ecologically valid approach to assessing the consequences of executive impairments. We present the Oxford Digital Multiple Errands Test (OxMET), a novel and simplified computer-tablet version of a Multiple Errands Test. We recruited 124 neurologically healthy controls and 105 stroke survivors to complete the OxMET task. Normative data and internal consistency were established from the healthy control data. Convergent and divergent validation was assessed in a mixed subset of 158 participants who completed the OxMET and OCS-Plus. Test-retest reliability was examined across a mixed subset of 39 participants. Finally, we investigated the known-group discriminability of the OxMET. The OxMET demonstrated very high internal consistency, and stable group level test-retest performance as well as good convergent and divergent validity. The OxMET demonstrated high sensitivity and good specificity in overall differentiation of stroke survivors from controls. The Oxford Digital Multiple Errands Test is a brief, easy to administer tool, designed to quickly screen for potential consequences of executive impairments in a virtual environment shopping task on a computer tablet. Initial normative data and validation within a chronic stroke cohort is presented.