Clinical utility of the Multiple Errands Test in schizophrenia: A preliminary assessment

Conceptualization of the term "ecological validity" in neuropsychological research on executive function assessment: a systematic review and call to action

OBJECTIVE

"Ecological validity" (EV) is classically defined as test's ability to predict real-world functioning, either alone or together with test's similarity to real-world tasks. In neuropsychological literature on assessment of executive functions (EF), EV is conceptualized inconsistently, leading to misconceptions about the utility of tests. The goal of this systematic review was to examine how EV is conceptualized in studies of EF tests described as ecologically valid.

METHOD

MEDLINE and PsychINFO Databases were searched. PRISMA guidelines were observed. After applying inclusion and exclusion criteria, this search yielded 90 articles. Deductive content analysis was employed to determine how the term EV was used.

RESULTS

About 1/3 of the studies conceptualized EV as the test's ability to predict functional outcomes, 1/3 as both the ability to predict functional outcome and similarity to real-world tasks, and 1/3 were either unclear about the meaning of the term or relied on notions unrelated to classical definitions (e.g., similarity to real-world tasks alone, association with other tests, or the ability to discriminate between populations).

CONCLUSIONS

Conceptualizations of the term EV in literature on EF assessment vary grossly, subsuming the notions of criterion, construct, and face validity, as well as sensitivity/specificity. Such inconsistency makes it difficult to interpret clinical utility of tests that are described as ecologically valid. We call on the field to require that, at minimum, the term EV be clearly defined in all publications, or replaced with more concrete terminology (e.g., criterion validity).

Effects of multitask training on cognition and motor control in people with schizophrenia spectrum disorders

Schizophrenia spectrum disorder (SSD) is a disabling mental illness that causes considerable deficits in motor and cognitive functions. The purpose of this study was to examine the effects of combining traditional multitask training (TMT) and video games--a new form of multitask training (video game multitask training VGMT)--on cognition and motor control performance in people with SSD. This was a quasi-experimental, pretest-posttest design study. A total of 25 patients participated in this study voluntarily (13 males and 12 females, average age = 59.61 years, SD– 11.46 years). All participants underwent two stages of training. The first stage involved TMT, and the second stage involved VGMT. Each training stage was 12 weeks long, with sessions twice a week that lasted for 40 minutes. Cognition, upper extremity motor and postural control performance, and functional mobility and subjective balance confidence were measured at three times: before and after the first-stage training and after the second-stage training. The results revealed that TMT and the combination of TMT and VGMT improved SSD patient’s cognition, upper extremity motor control, functional mobility and postural control performance. The subjective confidence of balance during the performance of daily activities was also mildly improved. Training with multitasks in the form of video games tended to further improve the outcome measures. Patients with SSD could benefit from regular participation in various forms of multitasking activities. Whether video games training are better than TMT in improving the functional ability of people with SSD needs further investigation.

Study protocol registration: Clinicaltrials.gov, ID: NCT04629898. Registered brief title: Level of Immersion of Virtual Reality and Cognition and Motor Performance in Patients of Schizophrenia Spectrum Disorder.

Executive Dysfunctions in Schizophrenia: A Critical Review of Traditional, Ecological, and Virtual Reality Assessments

In recent years, interest has grown in measuring executive function in schizophrenia with ecological and virtual reality (VR) tools. However, there is a lack of critical analysis comparing those tools with traditional ones. This paper aims to characterize executive dysfunction in schizophrenia by comparing ecological and virtual reality assessments with traditional tools, and to describe the neurobiological and psychopathological correlates. The analysis revealed that ecological and VR tests have higher levels of verisimilitude and similar levels of veridicality compared to traditional tools. Both negative symptoms and disorganization correlate significantly with executive dysfunction as measured by traditional tools, but their relationships with measures based on ecological and VR methods are still unclear. Although there is much research on brain correlates of executive impairments in schizophrenia with traditional tools, it is uncertain if these results will be confirmed with the use of ecological and VR tools. In the diagnosis of executive dysfunction, it is important to use a variety of neuropsychological methods—especially those with confirmed ecological validity—to properly recognize the underlying characteristics of the observed deficits and to implement effective forms of therapy.

The night out task and scoring application: an ill-structured, open-ended clinic-based test representing cognitive capacities used in everyday situations

OBJECTIVE

The night out task (NOT) was developed as a naturalistic, open-ended, multitasking measure that requires individuals to complete eight subtasks comparable to those encountered during real-world functioning (e.g., pack travel bag, prepare tea). We examined psychometric properties and administration feasibility of this direct observation measure within a clinic-like setting using a tablet-based coding application.

METHOD

A sample of 148 community-dwelling older adults (82% cognitively healthy; 18% mild cognitive impairment) and 57 younger adults completed the NOT along with other neurocognitive tests and questionnaires.

RESULTS

Inter-rater reliability across NOT primary (i.e., time, accuracy, efficiency) and process-related (e.g., error-types, self-corrections) variables was mostly excellent. NOT primary measures showed expected patterns of convergent and discriminant validity with measures of cognition, demographics, and well-being. External validity was established by the NOT ability to distinguish between age and diagnostic (cognitively healthy vs. mild cognitive impairment) groups. Demonstrating incremental validity, the NOT primary variables (execution time in particular) were predictive of self-reported functional abilities and completion quality of in-home everyday tasks over and earlier variables such as demographics, cognition, and mobility.

CONCLUSIONS

These findings suggest that the NOT and its app interface, which allows for continuous logging of observations, are a feasible in-clinic measure to assess cognitive capacities important for real-world functioning. With further validation, the NOT may allow for earlier detection of functional difficulties. Understanding errors and strategies used during NOT performance could also have implications for individualized interventions.

The validity, reliability and clinical utility of a performance-based executive function assessment in people with mild to moderate dementia

Objective: Performance-based evaluation of executive function by using real-world daily living activities is an important area of study. This approach has been used extensively in evaluating patients after stroke or traumatic brain injury and patients with schizophrenia. Most important is the fact that until now, there has been no validated performance-based evaluation of executive function in people with dementia.Methods: To address that knowledge gap, this study recruited 80 patients diagnosed with dementia and 80 demographically matched healthy controls. The participants were administered tests for evaluating their performance-based executive function (Chinese Multiple Errands Test), their instrumental activities of daily living (Lawton Instrumental Activities of Daily Living Scale, Chinese Version), and their functional disability (Chinese Version of the Disability Assessment for Dementia), along with a cognitive screening test (Montreal Cognitive Assessment, Hong Kong Version) and a neuropsychological test of executive function (Trail-making Test).Results: The Chinese Multiple Errands Test demonstrated excellent inter-rater reliability, test-retest reliability and high internal consistency. Results revealed that the healthy controls out-performed the dementia patients in the performance-based executive function and cognitive screening, but not in the instrumental activities of daily living tests. Additionally, the performance efficiency scores of the older adults with dementia on the Chinese Multiple Errands Test correlated significantly with their performance results on the neuropsychological test of executive function and on the tests of functional disability and cognitive function.Conclusion: Our results indicated that the Chinese Multiple Errands Test is a reliable and valid instrument for assessing executive function in Chinese older people with dementia.

Developing and Validating a Big-Store Multiple Errands Test

The Multiple Errands Test (MET) is an ecologically valid assessment that characterizes how executive dysfunction manifests in everyday activities. Due to the naturalistic nature of this assessment, clinicians and researchers have had to develop site-specific versions resulting in numerous published versions and making it difficult to establish standard psychometric properties. The aim of this study was to develop a standardized, community version of the MET designed to be used in large department stores meeting set criteria that would not require site specific modifications. This paper reports on the development, content validity, feasibility, and inter-rater reliability of a Big-Store MET, and the performance of healthy participants on this test. Items were selected to match previously published versions in relation to quantity and complexity. Content validity was established by having experts (n = 4) on the MET review the proposed Big-Store version and evaluate the task consistency with previously published versions. To assess feasibility of administration, and inter-rater reliability, a convenience sample of 14 community dwelling adults, self-reporting as healthy, were assessed by two trained raters. We found the Big-Store MET to be feasible to deliver (completed within 30 min, scores show variability, acceptable to participants in community environment) and inter-rater reliability to be very high (ICCs = 0.92-0.99) with the exception of frequency of strategy use. This study introduces the Big-Store MET to the literature, establishes its preliminary validity and reliability thus laying the foundation for a standardized, community-based version of the MET.

The efficacy of computer-based cognitive training for executive dysfunction in schizophrenia

The purpose of this study was to assess the effect of computer-based cognitive training on executive dysfunction in patients with schizophrenia. Sixty-five patients with schizophrenia were randomly assigned to a training group (n = 33) or a non-training group (n = 32), and compared in terms of executive performance to a healthy control group (n = 33). Executive function was assessed using the Trail Making Test, the Stroop Color and Word Test, and the Wisconsin Card Sorting Test (computer version). Cognitive training was performed using RehaCom software over a course of 16 individual sessions. Primary outcomes were training (performance at three different timepoints) and neuropsychological components (flexibility and cognitive inhibition, high executive processing, and processing speed). In both clinical groups, all aspects of executive function were found to be deficient. In the patient training group, the use of computer-based training alongside pharmacological treatment was more effective in terms of cognitive improvement than pharmacological treatment alone. However, there was no significant effect of cognitive training on processing speed. Cognitive training in schizophrenia patients was effective at improving several aspects of executive function, but did not improve processing speed.

A direct examination of the cognitive underpinnings of multitasking abilities: A first study examining schizophrenia

Many real world activities are complex and require multitasking abilities. However, the nature of these abilities remains poorly understood, and in particular in schizophrenia. The aim of the present study was to provide a better understanding of such abilities with the help of a newly developed computerized tool, the Computerized Meeting Preparation Task (CMPT). Fifty-seven individuals with schizophrenia and 39 healthy controls completed the CMPT and an extensive cognitive battery. Patients were also evaluated with a series of clinical measures. During the CMPT, participants are asked to prepare a room for a meeting while, at the same time, dealing with interruptions, solving problems, and remembering prospective memory instructions. The CMPT was found to significantly differentiate patients and healthy controls for several variables. Results also showed that multitasking abilities were related to a large array of cognitive functions and, in particular, to those associated to executive functioning. These relations were not explained by the presence of a general cognitive impairment. Finally, a double dissociation between multitasking abilities and performance on standard cognitive tests was observed. Altogether, these results underline the importance of evaluating multitasking abilities in schizophrenia as it allows detecting cognitive difficulties that cannot be identified by standard cognitive tests.

Development of a Contextualized Version of the Multiple Errands Test for People with Substance Dependence

BACKGROUND

Substance dependence is associated with deficits in several areas of executive functioning, such as planning and multitasking. Traditional tests of executive functioning have been criticized for their low ecological validity. Shallice and Burgess (1991) developed the Multiple Errands Test (MET), where participants are required to perform multiple tasks in a real-world setting. In this study, we aimed to adapt the MET for use in populations with substance dependence.

OBJECTIVES

This study aimed to: (1) examine the applicability of a contextualized version of the Multiple Errands Test, namely, the Multiple Errands Test - Contextualized Version (MET-CV); (2) study the testing context reliability and convergent validity of MET-CV scores; (3) compare the performance of people with substance dependence to a control group.

METHODS

Data were collected from 60 participants with substance dependence and 30 healthy controls. Both groups performed a neuropsychological assessment, which consisted of the MET-CV and traditional neuropsychological tests (Letter Number Sequencing, Zoo Map Test, Revised Strategy Application Test, Information Sampling Test, Stockings of Cambridge).

RESULTS

The MET-CV has adequate testing context reliability and moderate convergent validity relative to traditional planning measures. People with substance dependence (PWSD) showed significant deficits in executive function tests compared to healthy controls, most notably on the MET-CV rather than on traditional neuropsychological tests.

CONCLUSIONS

The MET-CV can be feasibly applied in the context of therapeutic communities treatment in PWSD. The MET-CV showed adequate reliability and validity, and detected planning deficits in PWSD. (JINS, 2018, 24, 347-359).