Are 3D virtual environments better than 2D interfaces in serious games performance? An explorative study for the assessment of executive functions

Naturalistic assessments across the lifespan: Systematic review of inhibition measures in ecological settings

Inhibitory control is essential for our everyday lives. Despite this, it is commonly assessed using non-naturalistic assessments. In this systematic review, we argue for the importance of taking an ecological approach to assess cognition. The aims are to present the state-of-knowledge in naturalistic assessments of inhibitory control, focusing on their methodological characteristics, including psychometric properties and user experience. PubMed, PsycINFO and Web of Science were searched until September 2024. Studies were included if they used at least one naturalistic method of assessing inhibition. The included studies (N=64) were grouped into three methodological categories: gamification, virtual reality, and brief, repeated assessments in participants' usual environment in the form of ecological momentary assessments. Sample sizes spanned three orders of magnitude (N=12-22,098). We report considerable heterogeneity in the types of tasks used, and the psychometric details reported. Nonetheless, naturalistic tasks were generally comparable with standardised equivalents, although some tasks assessed mixed-domain constructs. Tasks were feasible and acceptable for participants, with generally high completion rates and engagement. Recommendations for future research are discussed.

Feasibility of virtual reality and machine learning to assess personality traits in an organizational environment

Introduction

Personality plays a crucial role in shaping an individual's interactions with the world. The Big Five personality traits are widely used frameworks that help describe people's psychological behaviours. These traits predict how individuals behave within an organizational setting.

Methods

In this article, we introduce a virtual reality (VR) strategy for relatively scoring an individual's personality to evaluate the feasibility of predicting personality traits from implicit measures captured from users interacting in VR simulations of different organizational situations. Specifically, eye-tracking and decision-making patterns were used to classify individuals according to their level in each of the Big Five dimensions using statistical machine learning (ML) methods. The virtual environment was designed using an evidence-centered design approach.

Results

The dimensions were assessed using NEO-FFI inventory. A random forest ML model provided 83% accuracy in predicting agreeableness. A k-nearest neighbour ML model provided 75%, 75%, and 77% accuracy in predicting openness, neuroticism, and conscientiousness, respectively. A support vector machine model provided 85% accuracy for predicting extraversion. These analyses indicated that the dimensions could be differentiated by eye-gaze patterns and behaviours during immersive VR.

Discussion

Eye-tracking measures contributed more significantly to this differentiation than the behavioural metrics. Currently, we have obtained promising results with our group of participants, but to ensure the robustness and generalizability of our findings, it is imperative to replicate the study with a considerably larger sample. This study demonstrates the potential of VR and ML to recognize personality traits.

An ecological assessment of decision-making under risk and ambiguity through the virtual serious game Kalliste Decision Task

Traditional methods for evaluating decision-making provide valuable insights yet may fall short in capturing the complexity of this cognitive capacity, often providing insufficient for the multifaceted nature of decisions. The Kalliste Decision Task (KDT) is introduced as a comprehensive, ecologically valid tool aimed at bridging this gap, offering a holistic perspective on decision-making. In our study, 81 participants completed KDT alongside established tasks and questionnaires, including the Mixed Gamble Task (MGT), Iowa Gambling Task (IGT), and Stimulating & Instrumental Risk Questionnaire (S&IRQ). They also completed the User Satisfaction Evaluation Questionnaire (USEQ). The results showed excellent usability, with high USEQ scores, highlighting the user-friendliness of KDT. Importantly, KDT outcomes showed significant correlations with classical decision-making variables, shedding light on participants' risk attitudes (S&IRQ), rule-based decision-making (MGT), and performance in ambiguous contexts (IGT). Moreover, hierarchical clustering analysis of KDT scores categorized participants into three distinct profiles, revealing significant differences between them on classical measures. The findings highlight KDT as a valuable tool for assessing decision-making, addressing limitations of traditional methods, and offering a comprehensive, ecologically valid approach that aligns with the complexity and heterogeneity of real-world decision-making, advancing research and providing insights for understanding and assessing decision-making across multiple domains.

Immersive Virtual Reality-Based Methods for Assessing Executive Functioning: Systematic Review

BACKGROUND

Neuropsychological assessments traditionally include tests of executive functioning (EF) because of its critical role in daily activities and link to mental disorders. Established traditional EF assessments, although robust, lack ecological validity and are limited to single cognitive processes. These methods, which are suitable for clinical populations, are less informative regarding EF in healthy individuals. With these limitations in mind, immersive virtual reality (VR)-based assessments of EF have garnered interest because of their potential to increase test sensitivity, ecological validity, and neuropsychological assessment accessibility.

OBJECTIVE

This systematic review aims to explore the literature on immersive VR assessments of EF focusing on (1) EF components being assessed, (2) how these assessments are validated, and (3) strategies for monitoring potential adverse (cybersickness) and beneficial (immersion) effects.

METHODS

EBSCOhost, Scopus, and Web of Science were searched in July 2022 using keywords that reflected the main themes of VR, neuropsychological tests, and EF. Articles had to be peer-reviewed manuscripts written in English and published after 2013 that detailed empirical, clinical, or proof-of-concept studies in which a virtual environment using a head-mounted display was used to assess EF in an adult population. A tabular synthesis method was used in which validation details from each study, including comparative assessments and scores, were systematically organized in a table. The results were summed and qualitatively analyzed to provide a comprehensive overview of the findings.

RESULTS

The search retrieved 555 unique articles, of which 19 (3.4%) met the inclusion criteria. The reviewed studies encompassed EF and associated higher-order cognitive functions such as inhibitory control, cognitive flexibility, working memory, planning, and attention. VR assessments commonly underwent validation against gold-standard traditional tasks. However, discrepancies were observed, with some studies lacking reported a priori planned correlations, omitting detailed descriptions of the EF constructs evaluated using the VR paradigms, and frequently reporting incomplete results. Notably, only 4 of the 19 (21%) studies evaluated cybersickness, and 5 of the 19 (26%) studies included user experience assessments.

CONCLUSIONS

Although it acknowledges the potential of VR paradigms for assessing EF, the evidence has limitations. The methodological and psychometric properties of the included studies were inconsistently addressed, raising concerns about their validity and reliability. Infrequent monitoring of adverse effects such as cybersickness and considerable variability in sample sizes may limit interpretation and hinder psychometric evaluation. Several recommendations are proposed to improve the theory and practice of immersive VR assessments of EF. Future studies should explore the integration of biosensors with VR systems and the capabilities of VR in the context of spatial navigation assessments. Despite considerable promise, the systematic and validated implementation of VR assessments is essential for ensuring their practical utility in real-world applications.

Interactive Laboratories for Science Education: A Subjective Study and Systematic Literature Review

In science education laboratory experimentation has a vital role for students’ learning enhancement. Keeping in view the importance of modern day technologies in teaching learning process, various interactive laboratories (ISLs) have been developed to assist students in hands-on experiments in science education. In this paper we describe the potential contributions of existing interactive science laboratories (ISLs) in the major subjects of science, i.e., chemistry, biology and physics. The existing ISLs include virtual labs and simulation software where users performed their experiments. Important problems and challenges in the existing ISLs are highlighted. The systematic literature review (SLR) methodology is used for article searching, selection, and quality assessments. For this study, 86 articles after final selection using SLR are selected and classified into different categories. Each article is selected after briefly studying its different information, including category of the article, key idea, evaluation criterion, and its strengths and weaknesses. A subjective study with field experts was also conducted to investigate one of our existing virtual lab about the practical implementation and to find out the key issues in its implementation and use. Then, considering the suggestions of the subjective study, some guidelines are proposed for the improvement of future ISLs.

Virtual reality and speech analysis for the assessment of impulsivity and decision-making: protocol for a comparison with neuropsychological tasks and self-administered questionnaires

INTRODUCTION

Impulsivity is present in a range of mental disorders and has been associated with suicide. Traditional measures of impulsivity have certain limitations, such as the lack of ecological validity. Virtual reality (VR) may overcome these issues. This study aims to validate the VR assessment tool 'Spheres & Shield Maze Task' and speech analysis by comparing them with traditional measures. We hypothesise that these innovative tools will be reliable and acceptable by patients, potentially improving the simultaneous assessment of impulsivity and decision-making.

METHODS AND ANALYSIS

This study will be carried out at the University Hospital Fundación Jiménez Díaz (Madrid, Spain). Our sample will consist of adults divided into three groups: psychiatric outpatients with a history of suicidal thoughts and/or behaviours, psychiatric outpatients without such a history and healthy volunteers. The target sample size was established at 300 participants (100 per group). Participants will complete the Barratt Impulsiveness Scale 11; the Urgency, Premeditation, Perseverance, Sensation Seeking, Positive Urgency, Impulsive Behaviour Scale; Iowa Gambling Task; Continuous Performance Test; Stop signal Task, and Go/no-go task, three questions of emotional affect, the Spheres & Shield Maze Task and two satisfaction surveys. During these tasks, participant speech will be recorded. Construct validity of the VR environment will be calculated. We will also explore the association between VR-assessed impulsivity and history of suicidal thoughts and/or behaviour, and the association between speech and impulsivity and decision-making.

ETHICS AND DISSEMINATION

This study was approved by the Ethics Committee of the University Hospital Fundación Jiménez Díaz (PIC128-21_FJD). Participants will be required to provide written informed consent. The findings will be presented in a series of manuscripts that will be submitted to peer-reviewed journals for publication.

TRIAL REGISTRATION NUMBER

NCT05109845; Pre-results.

Examining the Academic Trends in Neuropsychological Tests for Executive Functions Using Virtual Reality: Systematic Literature Review

Background

In neuropsychology, fully immersive virtual reality (VR) has been spotlighted as a promising tool. It is considered that VR not only overcomes the existing limitation of neuropsychological tests but is also appropriate for treating executive functions (EFs) within activities of daily living (ADL) due to its high ecological validity. While fully immersive VR offers new possibilities of neuropsychological tests, there are few studies that overview the intellectual landscape and academic trends in the research related to mainly targeted EFs with fully immersive VR.

Objective

The objective of this study is to get an overview of the research trends that use VR in neuropsychological tests and to analyze the research trends using fully immersive VR neuropsychological tests with experimental articles.

Methods

This review was carried out according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Articles were searched in three web databases using keywords related to VR, EFs, and cognitive abilities. The study was conducted in two steps, keyword analysis and in-depth systematic review. In the web database search from 2000 to 2019, 1167 articles were initially collected, of which 234 articles in the eligibility phase were used to conduct keyword analysis and a total of 47 articles were included for systematic review.

Results

In keyword analysis, the number of articles focused on dementia including the keywords “MCI,” “SCD,” and “dementia” were highlighted over the period, rather than other symptoms. In addition, we identified that the use of behavioral and physiological data in virtual environments (VEs) has dramatically increased in recent studies. In the systematic review, we focused on the purpose of study, assessment, treatment, and validation of usability and structure. We found that treatment studies and uncategorized studies including presence and cybersickness issues have emerged in the recent period. In addition, the target symptoms and range of participants were diversified.

Conclusions

There has been a continuously increasing interest in dealing with neuropsychology by using fully immersive VR. Target cognitive abilities have been diversified, as well as target symptoms. Moreover, the concept of embodied cognition was transplanted in this research area.

Combining Virtual Reality and Organizational Neuroscience for Leadership Assessment

In this article, we introduce three-dimensional Serious Games (3DSGs) under an evidence-centered design (ECD) framework and use an organizational neuroscience-based eye-tracking measure to capture implicit behavioral signals associated with leadership skills. While ECD is a well-established framework used in the design and development of assessments, it has rarely been utilized in organizational research. The study proposes a novel 3DSG combined with organizational neuroscience methods as a promising tool to assess and recognize leadership-related behavioral patterns that manifest during complex and realistic social situations. We offer a research protocol for assessing task- and relationship-oriented leadership skills that uses ECD, eye-tracking measures, and machine learning. Seamlessly embedding biological measures into 3DSGs enables objective assessment methods that are based on machine learning techniques to achieve high ecological validity. We conclude by describing a future research agenda for the combined use of 3DSGs and organizational neuroscience methods for leadership and human resources.

Do Older and Young Adults Learn to Integrate Geometry While Navigating in an Environment of a Serious Game?

We evaluated the outcomes of an intervention using a serious game designed to be played on iPads for improving spatial reorientation by training users to integrate geometry of the environment, instead of relying solely on featural cues. Using data logged online through a clinical study of using this game, the effect of training among 16 older adults (69.3 ± 6.4 years, 4 males), who played the game repeatedly (self-administered) over a period of 8 weeks, was investigated. The game contains a hexagonal room with 3 objects, textured walls, and grids on the floor, which are removed one by one as the participant played the game. In each level, the room also rotates such that the viewpoint of the user is different from that of the previous level. Participants cannot play a higher level unless they make no mistake during the trials of the lower test level. In addition to data of older adults available from that clinical trial, we recruited 16 young adults (27.3 ± 5.6 years, 4 males) to play the game for 5 sessions and compared their results with those of the older adults. We evaluated the error type made in each test level and the scores for each session among older adults. Further, we compared the frequency of each error type between young and older adults during the test levels that a landmark adjacent to the target was removed over the first 5 sessions. The results of older adults’ performance suggest they learned to make fewer mistakes over the sessions. Also, both young and older adults learned to integrate the geometrical cues rather than relying on the landmark cue adjacent to the target to find the target. Overall, the results indicate the designed hexagonal room game can enhance spatial cognition among all age groups of adults.

An Immersive Virtual Reality Game for Predicting Risk Taking through the Use of Implicit Measures

Risk taking (RT) measurement constitutes a challenge for researchers and practitioners and has been addressed from different perspectives. Personality traits and temperamental aspects such as sensation seeking and impulsivity influence the individual’s approach to RT, prompting risk-seeking or risk-aversion behaviors. Virtual reality has emerged as a suitable tool for RT measurement, since it enables the exposure of a person to realistic risks, allowing embodied interactions, the application of stealth assessment techniques and physiological real-time measurement. In this article, we present the assessment on decision making in risk environments (AEMIN) tool, as an enhanced version of the spheres and shield maze task, a previous tool developed by the authors. The main aim of this article is to study whether it is possible is to discriminate participants with high versus low scores in the measures of personality, sensation seeking and impulsivity, through their behaviors and physiological responses during playing AEMIN. Applying machine learning methods to the dataset we explored: (a) if through these data it is possible to discriminate between the two populations in each variable; and (b) which parameters better discriminate between the two populations in each variable. The results support the use of AEMIN as an ecological assessment tool to measure RT, since it brings to light behaviors that allow to classify the subjects into high/low risk-related psychological constructs. Regarding physiological measures, galvanic skin response seems to be less salient in prediction models.

The Spheres & Shield Maze Task: A Virtual Reality Serious Game for the Assessment of Risk Taking in Decision Making

Risk taking (RT) is an essential component in decision-making process that depicts the propensity to make risky decisions. RT assessment has traditionally focused on self-report questionnaires. These classical tools have shown clear distance from real-life responses. Behavioral tasks assess human behavior with more fidelity, but still show some limitations related to transferability. A way to overcome these constraints is to take advantage from virtual reality (VR), to recreate real-simulated situations that might arise from performance-based assessments, supporting RT research. This article presents results of a pilot study in which 41 individuals explored a gamified VR environment: the Spheres & Shield Maze Task (SSMT). By eliciting implicit behavioral measures, we found relationships between scores obtained in the SSMT and self-reported risk-related constructs, as engagement in risky behaviors and marijuana consumption. We conclude that decontextualized Virtual Reality Serious Games are appropriate to assess RT, since they could be used as a cross-disciplinary tool to assess individuals' capabilities under the stealth assessment paradigm.