Decoding Mental Workload in Virtual Environments: A fNIRS Study using an Immersive n-back Task

The Differential Effects of Multisensory Attentional Cues on Task Performance in VR Depending on the Level of Cognitive Load and Cognitive Capacity

As the utilization of VR is expanding across diverse fields, research on devising attentional cues that could optimize users' task performance in VR has become crucial. Since the cognitive load imposed by the context and the individual's cognitive capacity are representative factors that are known to determine task performance, we aimed to examine how the effects of multisensory attentional cues on task performance are modulated by the two factors. For this purpose, we designed a new experimental paradigm in which participants engaged in dual (N-back, visual search) tasks under different levels of cognitive load while an attentional cue (visual, tactile, or visuotactile) was presented to facilitate search performance. The results showed that multi-sensory attentional cues are generally more effective than uni-sensory cues in enhancing task performance, but the benefit of multi-sensory cues changes according to the level of cognitive load and the individual's cognitive capacity; the amount of benefit increases as the cognitive load is higher and the cognitive capacity is lower. The findings of this study provide practical implications for designing attentional cues to enhance VR task performance, considering both the complexity of the VR context and users' internal characteristics.

A Survey on Affective and Cognitive VR

In Virtual Reality (VR), users can be immersed in emotionally intense and cognitively engaging experiences. Yet, despite strong interest from scholars and a large amount of work associating VR and Affective and Cognitive States (ACS), there is a clear lack of structured and systematic form in which this research can be classified. We define "Affective and Cognitive VR" to relate to works which (1) induce ACS, (2) recognize ACS, or (3) exploit ACS by adapting virtual environments based on ACS measures. This survey clarifies the different models of ACS, presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an Immersive Virtual Environment (IVE) in a non-clinical context. Our article covers the main research lines in Affective and Cognitive VR. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.

In search of a naturalistic neuroimaging approach: Exploration of general feasibility through the case of VR-fMRI and application in the domain of episodic memory

Virtual Reality (VR) is an increasingly widespread tool for research as it allows the creation of experiments taking place in multimodal and daily-life-like environments, while keeping a strong experimental control. Adding neuroimaging to VR leads to a better understanding of the underlying brain networks activated during a naturalistic task, whether for research purposes or rehabilitation. The present paper focuses on the specific use of concurrent VR and fMRI and its technical challenges and feasibility, with a brief examination of the general existing solutions. Following the PRISMA guidelines, the review investigates the particular case of how VR-fMRI has explored episodic memory so far, with a comparison of object- and place-based episodic memory. This review confirms the involvement of cerebral regions well-known to be implicated in episodic memory and unravels other regions devoted to bodily and narrative aspects of the self, promoting new avenues of research in the domain of naturalistic episodic memory. Future studies should develop more immersive and interactive virtual neuroimaging features to increase ecological and embodied neurocognition aspects.

Unsupervised fNIRS feature extraction with CAE and ESN autoencoder for driver cognitive load classification

Objective. Understanding the cognitive load of drivers is crucial for road safety. Brain sensing has the potential to provide an objective measure of driver cognitive load. We aim to develop an advanced machine learning framework for classifying driver cognitive load using functional near-infrared spectroscopy (fNIRS).Approach. We conducted a study using fNIRS in a driving simulator with theN-back task used as a secondary task to impart structured cognitive load on drivers. To classify different driver cognitive load levels, we examined the application of convolutional autoencoder (CAE) and Echo State Network (ESN) autoencoder for extracting features from fNIRS.Main results. By using CAE, the accuracies for classifying two and four levels of driver cognitive load with the 30 s window were 73.25% and 47.21%, respectively. The proposed ESN autoencoder achieved state-of-art classification results for group-level models without window selection, with accuracies of 80.61% and 52.45% for classifying two and four levels of driver cognitive load.Significance. This work builds a foundation for using fNIRS to measure driver cognitive load in real-world applications. Also, the results suggest that the proposed ESN autoencoder can effectively extract temporal information from fNIRS data and can be useful for other fNIRS data classification tasks.

Will virtual rehabilitation replace clinicians: a contemporary debate about technological versus human obsolescence

This article is inspired by a pseudo Oxford-style debate, which was held in Tel Aviv University, Israel at the International Conference on Virtual Rehabilitation (ICVR) 2019, which is the official conference of the International Society for Virtual Rehabilitation. The debate, between two 2-person teams with a moderator, was organized by the ICVR Program committee to address the question "Will virtual rehabilitation replace clinicians?" It brought together five academics with technical, research, and/or clinical backgrounds-Gerry Fluet, Tal Krasovsky, Anat Lubetzky, Philippe Archambault, W. Geoffrey Wright-to debate the pros and cons of using virtual reality (VR) and related technologies to help assess, diagnose, treat, and track recovery, and more specifically investigate the likelihood that advanced technology will ultimately replace human clinicians. Both teams were assigned a side to defend, whether it represented their own viewpoint or not, and to take whatever positions necessary to make a persuasive argument and win the debate. In this paper we present a recapitulation of the arguments presented by both sides, and further include an in-depth consideration of the question. We attempt to judiciously lay out a number of arguments that fall along a spectrum from moderate to extreme; the most extreme and/or indefensible positions are presented for rhetorical and demonstrative purposes. Although there may not be a clear answer today, this paper raises questions which are related to the basic nature of the rehabilitation profession, and to the current and potential role of technology within it.

Tracking Changes in Frontal Lobe Hemodynamic Response in Individual Adults With Developmental Language Disorder Following HD tDCS Enhanced Phonological Working Memory Training: An fNIRS Feasibility Study

Background: Current research suggests a neurobiological marker of developmental language disorder (DLD) in adolescents and young adults may be an atypical neural profile coupled with behavioral performance that overlaps with that of normal controls. Although many imaging techniques are not suitable for the study of speech and language processing in DLD populations, fNIRS may be a viable option. In this study we asked if fNIRS can be used to identify atypical cortical activation patterns in individual adults with DLD and track potential changes in cortical activation patterns following a phonological working memory training protocol enhanced with anodal HD tDCS stimulation to the presupplementary motor area (preSMA).

Objective/Hypothesis: The purpose of this study was two-fold: (1) to determine if fNIRS can be used to identify atypical hemodynamic responses in individual young adults with DLD during active spoken word processing and, (2) to determine if fNIRS can detect changes in hemodynamic response in these same adults with DLD following anodal HD tDCS enhanced phonological working memory training.

Methods: Two adult subjects with DLD (female, age 25) completed a total of two sessions of fNIRs working memory task prior to and following one session of a non-word repetition task paired with anodal HD tDCS (1.0 mA tDCS; 20 min) to the preSMA. Standardized z-scores of behavioral measures (accuracy and reaction time) and changes in hemodynamic response during an n-back working memory task for the two participants with DLD was compared to that of a normative sample of 21 age- and gender- matched normal controls (ages 18 to 25) prior to and following phonological working memory training.

Results: Individual standardized z-scores for each participant with DLD indicated that prior to training, hemoglobin response in the prefrontal lobe for both participants was markedly different from each other and normal controls. Following training, standard scores showed that the hemodynamic response for both participants moved within normal limits for ROIs.

Conclusion: These findings highlight the feasibility of fNIRS to establish individual differences in the link between behavior and neural patterns in single subjects with DLD, as well as track individual differences in changes in brain activity following working memory training.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.