Evaluation of a new serious game based multitasking assessment tool for cognition and activities of daily living: Comparison with a real cooking task

Applying linguistic demand analysis to functional multitasking assessments

PURPOSE

The linguistic demand of task instructions in cognitive assessments may mask or even invalidate cognitive testing results for individuals with language differences or impairments. The purpose of this study was to apply an analysis of linguistic demand to a sample of functional multitasking assessments.

METHOD

We sampled the task instructions and materials from seven functional multitasking assessments that are based on everyday activities. We calculated indices of linguistic demand.

RESULTS

The Density Index is based on total number of words, total number of sentences, total number of different words, total number of syllables, and complex (not-simple) sentences. The Breakfast Task and the Frisch Cooking Task consistently ranked lowest in linguistic demand for both instructions and materials based on the indices reviewed. These tasks are most likely to be appropriate for individuals with language impairment.

CONCLUSIONS

Clinicians have a responsibility to fairly evaluate cognitive functions of individuals with impaired language functioning. The analysis of linguistic demand provides a useful way to evaluate task instructions and materials in a systematic way, so that individuals with language impairments or language differences might be more appropriately evaluated.

Systematic review of memory assessment in virtual reality: evaluating convergent and divergent validity with traditional neuropsychological measures

Introduction

The evaluation of memory is a crucial aspect of both cognitive research and clinical applications, as it offers valuable insights into an individual's cognitive wellbeing and performance. Conventional neuropsychological assessments represent the established method for assessing different aspects of memory. Recent technological advancements, specifically in the field of virtual reality (VR), have introduced novel methods for evaluating memory.

Objective

This systematic review aims to examine the current state of memory assessment using VR technologies, assessing the degree of convergence and divergence between VR-based memory assessments and conventional neuropsychological tests.

Method

A systematic review of the literature was conducted searching PubMed, PsycINFO, Web of Science databases, leading to the incorporation of 24 studies. Studies were grouped according to the examined memory domain (episodic, prospective, spatial domain). Convergence and divergence validity were examined for each, and information on software and hardware features was collected.

Results

This review demonstrates a notable alignment between VR-based memory assessments and conventional neuropsychological tests. Moreover, VR tasks have shown to exhibit associations with executive functions and overall cognitive performance. The inclusion of various ecological contexts, such as residential environments, commercial establishments, and simulated scenarios, serves to augment the ecological validity of memory evaluations conducted in VR.

Discussion

The findings indicate that VR assessments demonstrate a functional perspective by effectively capturing the dynamic relationship between memory, executive functions, and overall cognitive performance. Nevertheless, it is imperative to acknowledge and tackle certain constraints that may hinder the widespread adoption and utilization of VR tasks. These limitations encompass factors such as restricted accessibility to VR tasks and the presence of heterogeneity in VR hardware and software. The dynamic and ever-changing nature of VR technology presents a range of potential avenues for future investigation and utilization in the domain of memory evaluation.

A Serious Game for Cognitive Stimulation of Older People With Mild Cognitive Impairment: Design and Pilot Usability Study

Background

Cognitive stimulation of older people helps prevent, and even treat, age-related diseases, such as mild cognitive impairment. Playing games reduces the probability of experiencing this pathology, which is related to the loss of the ability to carry out some instrumental activities of daily living.

Objective

This work describes the design and development of a serious game for the cognitive stimulation of older people, with exercises related to the daily life task of shopping. A pilot study for its preliminary usability validation is also presented.

Methods

The designed serious game includes 4 exercises consisting of shopping in a hypermarket, ordering products, making payments, and organizing the purchase, thus dealing with the most frequent cognitive problems of older people associated with episodic declarative memory, naming, calculation, and organization, respectively.

Results

A total of 19 older people participated in the pilot study for the usability validation of the serious game. They indicated that they like the aesthetic and interesting topic of the game. They reported that it provides a high level of entertainment and could be useful in daily life for mental stimulation. The participants found the serious game to be intuitive, but the ease of use and readability of the instructions could be improved.

Conclusions

This study suggests that the innovative serious game developed could be accepted by older people for their cognitive stimulation to prevent or treat mild cognitive impairment, although a long-term intervention study should be performed as future work. Its ecological validity design, with everyday tasks, adaptable levels of difficulty, and motivational mechanisms, is a differentiating factor compared to similar serious games.

The limitations and challenges in the assessment of executive dysfunction associated with real-world functioning: The opportunity of serious games

Nowadays, there is a broad range of methods for detecting and evaluating executive dysfunction ranging from clinical interview to neuropsychological evaluation. Nevertheless, a critical issue of these assessments is the lack of correspondence of the neuropsychological test's results with real-world functioning. This paper proposes serious games as a new framework to improve the neuropsychological assessment of real-world functioning. We briefly discuss the contribution and limitations of current methods of evaluation of executive dysfunction (paper-and-pencil tests, naturalistic observation methods, and Information and Communications Technologies) to inform on daily life functioning. Then, we analyze what are the limitations of these methods to predict real-world performance: (1) A lack of appropriate instruments to investigate the complexity of real-world functioning, (2) the vast majority of neuropsychological tests assess well-structured tasks, and (3) measurement of behaviors are based on simplistic data collection and statistical analysis. This work shows how serious games offer an opportunity to develop more efficient tools to detect executive dysfunction in everyday life contexts. Serious games provide meaningful narrative stories and virtual or real environments that immerse the user in natural and social environments with social interactions. In those highly interactive game environments, the player needs to adapt his/her behavioral performance to novel and ill-structured tasks which are suited for collecting user interaction evidence. Serious games offer a novel opportunity to develop better tools to improve diagnosis of the executive dysfunction in everyday life contexts. However, more research is still needed to implement serious games in everyday clinical practice.

Virtual reality technology in neuropsychological testing: A systematic review

Neuropsychological testing aims to measure individuals' cognitive abilities (e.g. memory, attention), analysing their performance on specific behavioural tasks. Most neuropsychological tests are administered in the so-called 'paper-and-pencil' modality or via computerised protocols. The adequacy of these procedures has been recently questioned, with more specific concerns about their ecological validity, i.e. the relation between test scores observed in the laboratory setting and the actual everyday cognitive functioning. In developing more ecological tasks, researchers started to implement virtual reality (VR) technology as an administration technique focused on exposing individuals to simulated but realistic stimuli and environments, maintaining at the same time a controlled laboratory setting and collecting advanced measures of cognitive functioning. This systematic review aims to present how VR procedures for neuropsychological testing have been implemented in the last years. We initially explain the rationale for supporting VR as an advanced assessment tool, but we also discuss the challenges and risks that can limit the widespread implementation of this technology. Then, we systematised the large body of studies adopting VR for neuropsychological testing, describing the VR tools' distribution amongst different cognitive functions through a PRISMA-guided systematic review. The systematic review highlighted that only very few instruments are ready for clinical use, reporting psychometric proprieties (e.g. validity) and providing normative data. Most of the tools still need to be standardised on large cohorts of participants, having published only limited data on small samples up to now. Finally, we discussed the possible future directions of the VR neuropsychological test development linked to technological advances.

FISHERMAN: A Serious Game for Executive Function Assessment of Older Adults

Executive functions (EFs) are essential for daily living activities but decline with age. Convenient assessment and timely intervention have particular significance for older adults. However, the traditional laboratory tasks of EFs are typically monotonous and inconvenient. The current study aimed to develop an interesting and convenient supplementary tool to assess EFs for older adults. According to the theory of EFs, we developed a serious game, FISHERMAN, to assess EFs. The game includes three subgames, Cautious Fisherman, Agile Fisherman, and Wise Fisherman, targeting core components of inhibition, shifting, and working memory, respectively. The current study aims to verify the reliability and validity of the game. One hundred and eight healthy older adults participated in this study and were tested through the FISHERMAN game and a battery of cognitive tests. The results show that the FISHERMAN game has high internal consistency reliability and good construct validity as well as criterion-related validity, suggesting that the game design is valid and can be used in EFs assessment for older adults. Future studies are warranted to establish the norm of the FISHERMAN game in older adults and investigate whether the FISHERMAN game can be generalized to other populations.

A Review on Serious Games for Dementia Care in Ageing Societies

Dementia is a severe disease threatening ageing societies, which not only causes great harm to patients both physically and psychologically but also places a heavy burden on patients’ families. Medications have been used for the treatment of dementia but with little success. However, serious games, as a new form of dementia therapy, stand out from various therapeutic methods and pave the way for dementia treatment. In the field of serious games for dementia care (SGDC) in ageing societies, there exists abundant research related to this topic. While, a detailed review of the development route and a category framework for characteristics of dementia are still needed. Besides, due to the large number of games, it is difficult to select out effective ones. Yet, there is no unified and comprehensive assessment methods for SGDC. So a reliable assessment model is worth studying. In this paper, we review these existing research work on SGDC from two perspectives: (1) the development of SGDC; (2) the different symptoms in different dementia stages. We also propose a comprehensive and professional assessment model of the therapeutic effectiveness of SGDC to compensate for the simplicity of existing assessment methods. Finally, a discussion related to SGDC is presented.

A Serious Game-Derived Index for Detecting Children With Heterogeneous Developmental Disabilities: Randomized Controlled Trial

BACKGROUND

Developmental disabilities are a set of heterogeneous delays or difficulties in one or more areas of neuropsychological development. Considering that childhood is an essential stage of brain development and developmental delays lead to personal or social burdens, the early detection of childhood developmental disabilities is important. However, early screening for developmental disabilities has been a challenge because of the fear of positive results, expensive tests, differences in diagnosis depending on examiners' abilities, and difficulty in diagnosis arising from the need for long-term follow-up observation.

OBJECTIVE

This study aimed to assess the feasibility of using a serious game-derived index to identify heterogeneous developmental disabilities. This study also examines the correlation between the game-derived index and existing neuropsychological test results.

METHODS

The randomized controlled trial involved 48 children with either normal development or developmental disabilities. In this clinical trial, we used 19 features (6 from the Korean-Wechsler Preschool and Primary Scale of Intelligence, 8 from the Psychoeducational Profile Revised, 2 from the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition, and 3 from the Pediatric Evaluation of Disability Inventory) from neuropsychological tests and 9 (7 game scores, path accuracy, and completion rate) from the serious game, DoBrain. The following analysis was conducted based on participants' baseline information and neuropsychological test and game-derived index data for one week: (1) we compared the baseline information between the normal development and developmental disabilities groups; (2) then we measured the correlation between the game-derived index and the neuropsychological test scores for each group; and (3) we built a classifier based on the game-derived index with a Gaussian process method and then compared the area under the curve (AUC) with a model based on neuropsychological test results.

RESULTS

A total of 16 children (normal development=9; developmental disabilities=7) were analyzed after selection. Their developmental abilities were assessed before they started to play the serious games, and statistically significant differences were found in both groups. Specifically, the normal development group was more developed than the developmental disabilities group in terms of social function, gross motor function, full-scale IQ, and visual motor imitation, in that order. Similarly, the normal development group obtained a higher score on the game-derived index than the developmental disabilities group. In the correlation analysis between the game-derived index and the neuropsychological tests, the normal development group showed greater correlation with more variables than the developmental disabilities group. The game-derived index-based model had an AUC=0.9, a similar detection value as the neuropsychological test-based model's AUC=0.86.

CONCLUSIONS

A game-derived index based on serious games can detect children with heterogenous developmental disabilities. This suggests that serious games can be used as a potential screening tool for developmental disabilities.

TRIAL REGISTRATION

Clinical Research Information Service KCT0003247; https://cris.nih.go.kr/cris/en/search/search_result_st01 .jsp?seq=12365.

Interaction Modalities Used on Serious Games for Upper Limb Rehabilitation: A Systematic Review

This systematic review aims to analyze the state-of-the-art regarding interaction modalities used on serious games for upper limb rehabilitation. A systematic search was performed in IEEE Xplore and Web of Science databases. PRISMA and QualSyst protocols were used to filter and assess the articles. Articles must meet the following inclusion criteria: they must be written in English; be at least four pages in length; use or develop serious games; focus on upper limb rehabilitation; and be published between 2007 and 2017. Of 121 articles initially retrieved, 33 articles met the inclusion criteria. Three interaction modalities were found: vision systems (42.4%), complementary vision systems (30.3%), and no-vision systems (27.2%). Vision systems and no-vision systems obtained a similar mean QualSyst (86%) followed by complementary vision systems (85.7%). Almost half of the studies used vision systems as the interaction modality (42.4%) and used the Kinect sensor to collect the body movements (48.48%). The shoulder was the most treated body part in the studies (19%). A key limitation of vision systems and complementary vision systems is that their device performances might be affected by lighting conditions. A main limitation of the no-vision systems is that the range-of-motion in angles of the body movement might not be measured accurately. Due to a limited number of studies, fruitful areas for further research could be the following: serious games focused on finger rehabilitation and trauma injuries, game difficulty adaptation based on user's muscle strength and posture, and multisensor data fusion on interaction modalities.

The Virtual Shop: A new immersive virtual reality environment and scenario for the assessment of everyday memory

BACKGROUND

Assessing and predicting memory performance in everyday life is a common assignment for neuropsychologists. However, most traditional neuropsychological tasks are not conceived to capture everyday memory performance.

NEW METHOD

The Virtual Shop is a fully immersive task developed to assess memory in a more ecological way than traditional neuropsychological assessments. Two studies were undertaken to assess the feasibility of the Virtual Shop and to appraise its ecological and construct validity. In study 1, 20 younger and 19 older adults completed the Virtual Shop task to evaluate its level of difficulty and the way the participants interacted with the VR material. The construct validity was examined with the contrasted-group method, by comparing the performance of younger and older adults. In study 2, 35 individuals with subjective cognitive decline completed the Virtual Shop task. Performance was correlated with an existing questionnaire evaluating everyday memory in order to appraise its ecological validity. To add further support to its construct validity, performance was correlated with traditional episodic memory and executive tasks.

RESULTS

All participants successfully completed the Virtual Shop. The task had an appropriate level of difficulty that helped differentiate younger and older adults, supporting the feasibility and construct validity of the task.

COMPARISON WITH EXISTING METHOD(S)

The performance on the Virtual Shop was significantly and moderately correlated with the performance on the questionnaire and on the traditional memory and executive tasks.

CONCLUSIONS

Results support the feasibility and both the ecological and construct validity of the Virtual Shop.

Smart Aging Platform for Evaluating Cognitive Functions in Aging: A Comparison with the MoCA in a Normal Population

Background: Smart Aging is a Serious games (SGs) platform in a 3D virtual environment in which users perform a set of screening tests that address various cognitive skills. The tests are structured as 5 tasks of activities of daily life in a familiar environment. The main goal of the present study is to compare a cognitive evaluation made with Smart Aging with those of a classic standardized screening test, the Montreal Cognitive Assessment (MoCA).

Methods: One thousand one-hundred thirty-one healthy adults aged between 50 and 80 (M = 64.3 ± 8.3) were enrolled in the study. They received a cognitive evaluation with the MoCA and the Smart Aging platform. Participants were grouped according to their MoCA global and specific cognitive domain (i.e., memory, executive functions, working memory, visual spatial elaboration, language, and orientation) scores and we explored differences among these groups in the Smart Aging indices.

Results: One thousand eighty-six older adults (M = 64.0 ± 8.0) successfully completed the study and were stratified according to their MoCA score: Group 1 with MoCA < 27 (n = 360); Group 2 with 27 ≥ MoCA < 29 (n = 453); and Group 3 with MoCA ≥ 29 (n = 273). MoCA groups significantly differed in most of the Smart Aging indices considered, in particular as concerns accuracy (ps < 0.001) and time (ps < 0.001) for completing most of the platform tasks. Group 1 was outperformed by the other two Groups and was slower than them in these tasks, which were those supposed to assess memory and executive functions. In addition, significant differences across groups also emerged when considering the single cognitive domains of the MoCA and the corresponding performances in each Smart Aging task. In particular, this platform seems to be a good proxy for assessing memory, executive functions, working memory, and visual spatial processes.

Conclusion: These findings demonstrate the validity of Smart Aging for assessing cognitive functions in normal aging. Future studies will validate this platform also in the clinical aging populations.