Attention Detection in Virtual Environments Using EEG Signals: A Scoping Review

Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

The role of attention in immersion: The two-competitor model

Currently, we face an exponentially increasing interest in immersion, especially sensory-driven immersion, mainly due to the rapid development of ideas and business models centered around a digital virtual universe as well as the increasing availability of affordable immersive technologies for education, communication, and entertainment. However, a clear definition of 'immersion', in terms of established neurocognitive concepts and measurable properties, remains elusive, slowing research on the human side of immersive interfaces. To address this problem, we propose a conceptual, taxonomic model of attention in immersion. We argue (a) modeling immersion theoretically as well as studying immersion experimentally requires a detailed characterization of the role of attention in immersion, even though (b) attention, while necessary, cannot be a sufficient condition for defining immersion. Our broader goal is to characterize immersion in terms that will be compatible with established psychophysiolgical measures that could then in principle be used for the assessment and eventually the optimization of an immersive experience. We start from the perspective that immersion requires the projection of attention to an induced reality, and build on accepted taxonomies of different modes of attention for the development of our two-competitor model. The two-competitor model allows for a quantitative implementation and has an easy graphical interpretation. It helps to highlight the important link between different modes of attention and affect in studying immersion.

A comprehensive dataset for home appliance control using ERP-based BCIs with the application of inter-subject transfer learning

Brain-computer interfaces (BCIs) have a potential to revolutionize human-computer interaction by enabling direct links between the brain and computer systems. Recent studies are increasingly focusing on practical applications of BCIs-e.g., home appliance control just by thoughts. One of the non-invasive BCIs using electroencephalography (EEG) capitalizes on event-related potentials (ERPs) in response to target stimuli and have shown promise in controlling home appliance. In this paper, we present a comprehensive dataset of online ERP-based BCIs for controlling various home appliances in diverse stimulus presentation environments. We collected online BCI data from a total of 84 subjects among whom 60 subjects controlled three types of appliances (TV: 30, door lock: 15, and electric light: 15) with 4 functions per appliance, 14 subjects controlled a Bluetooth speaker with 6 functions via an LCD monitor, and 10 subjects controlled air conditioner with 4 functions via augmented reality (AR). Using the dataset, we aimed to address the issue of inter-subject variability in ERPs by employing the transfer learning in two different approaches. The first approach, "within-paradigm transfer learning," aimed to generalize the model within the same paradigm of stimulus presentation. The second approach, "cross-paradigm transfer learning," involved extending the model from a 4-class LCD environment to different paradigms. The results demonstrated that transfer learning can effectively enhance the generalizability of BCIs based on ERP across different subjects and environments.

Effects of Virtual Reality Cognitive Training on Neuroplasticity: A Quasi-Randomized Clinical Trial in Patients with Stroke

Cognitive Rehabilitation (CR) is a therapeutic approach designed to improve cognitive functioning after a brain injury, including stroke. Two major categories of techniques, namely traditional and advanced (including virtual reality-VR), are widely used in CR for patients with various neurological disorders. More objective outcome measures are needed to better investigate cognitive recovery after a stroke. In the last ten years, the application of electroencephalography (EEG) as a non-invasive and portable neuroimaging method has been explored to extract the hallmarks of neuroplasticity induced by VR rehabilitation approaches, particularly within the chronic stroke population. The aim of this study is to investigate the neurophysiological effects of CR conducted in a virtual environment using the VRRS device. Thirty patients with moderate-to-severe ischemic stroke in the chronic phase (at least 6 months after the event), with a mean age of 58.13 (±8.33) for the experimental group and 57.33 (±11.06) for the control group, were enrolled. They were divided into two groups: an experimental group and a control group, receiving neurocognitive stimulation using VR and the same amount of conventional neurorehabilitation, respectively. To study neuroplasticity changes after the training, we focused on the power band spectra of theta, alpha, and beta EEG rhythms in both groups. We observed that when VR technology was employed to amplify the effects of treatments on cognitive recovery, significant EEG-related neural improvements were detected in the primary motor circuit in terms of power spectral density and time-frequency domains. Indeed, EEG analysis suggested that VR resulted in a significant increase in both the alpha band power in the occipital areas and the beta band power in the frontal areas, while no significant variations were observed in the theta band power. Our data suggest the potential effectiveness of a VR-based rehabilitation approach in promoting neuroplastic changes even in the chronic phase of ischemic stroke.

Narrative simulation of social experiences in naturalistic context - A neurocinematic approach

Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.

Attention in flux

Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.

A Novel Method Based on ERP and Brain Graph for the Simultaneous Assessment of Various Types of Attention

Assessment of attention is of great importance as one of human cognitive abilities. Although neuropsychological tests have been developed and used to evaluate the ability to pay attention, their validity and reliability have been reduced due to some limitations such as the presence of intervention factors, including human factors, limited range of languages, and cultural influences. Therefore, direct outputs of the brain system, represented by event-related potentials (ERPs), and the analysis of its function in cognitive activities have become very important as a complementary tool to assess various types of attention. This research tries to assess 4 types of attention including sustained, alternative, selective, and divided, using an integrated visual-auditory test and brain signals simultaneously. Thus, the electroencephalogram (EEG) data were recorded using 19 channels, and the integrated visual and auditory (IVA-AE) test was simultaneously performed on twenty-eight healthy volunteers including 22 male and 6 female subjects with the average age of 27 ± 5.3 years. Then ERPs related to auditory and visual stimuli with synchronous averaging technique were extracted. A topographic brain mapping (topo-map) was plotted for each frame of stimulation. Next, an optical flow method was implemented on different topo-maps to obtain motion vectors from one map to another. After obtaining the overall brain graph of an individual, some features were extracted and used as measures of local and global connectivity. The extracted features were consequently evaluated along with the parameters of the IVA test by support vector machine regression (SVM-R). The volume of attention was then quantified by combining the IVA parameters. Ultimately, estimation accuracy of each type of attention including focused attention (86.1%), sustained attention (83.4%), selective attention (80.9%), and divided attention (79.9%) was obtained. According to the present study, there is a significant relationship between response control and attention indicators of the IVA test as well as ERP brain signals.

Alternations in neural oscillation related to attention network reveal influence of indoor toluene on cognition at low concentration

Despite accumulative literature reporting negative impacts of high-concentration toluene, cognitive effects of toluene at low concentration are still unclear. Twenty-two healthy college students were exposed in a closed environmental chamber to investigate the influence of indoor toluene on cognitive performance and brain activity. During each toluene exposure condition (0 ppb, 17.5 ppb, 35 ppb, and 70 ppb), attention network test and electroencephalogram (EEG) recording were synchronously performed after 4-hour toluene exposure. Characteristic neural oscillation patterns in three attention networks were compared between four groups. The statistical analyses indicated that short-term exposure to toluene had no significant impact on behavioral performance of attention network. However, there was a significant increase in the power of theta and alpha band of executive network and orienting network in the whole brain, especially in frontal region when exposed to toluene. Besides, no significant difference was observed in alerting network. The alternations in neural oscillation demonstrated that more effort was required to accomplish the same tasks when exposed to toluene. The present study revealed that short-term exposure to toluene affected brain activity of attention network even at low concentration, which provided a theoretical basis for the development of safer evaluation methods and standards in the future.

Relationship between electroencephalographic data and comfort perception captured in a Virtual Reality design environment of an aircraft cabin

Successful aircraft cabin design depends on how the different stakeholders are involved since the first phases of product development. To predict passenger satisfaction prior to the manufacturing phase, human response was investigated in a Virtual Reality (VR) environment simulating a cabin aircraft. Subjective assessments of virtual designs have been collected via questionnaires, while the underlying neural mechanisms have been captured through electroencephalographic (EEG) data. In particular, we focused on the modulation of EEG alpha rhythm as a valuable marker of the brain’s internal state and investigated which changes in alpha power and connectivity can be related to a different visual comfort perception by comparing groups with higher and lower comfort rates. Results show that alpha-band power decreased in occipital regions during subjects’ immersion in the virtual cabin compared with the relaxation state, reflecting attention to the environment. Moreover, alpha-band power was modulated by comfort perception: lower comfort was associated with a lower alpha power compared to higher comfort. Further, alpha-band Granger connectivity shows top-down mechanisms in higher comfort participants, modulating attention and restoring partial relaxation. Present results contribute to understanding the role of alpha rhythm in visual comfort perception and demonstrate that VR and EEG represent promising tools to quantify human–environment interactions.

An Exploratory EEG Analysis on the Effects of Virtual Reality in People with Neuropathic Pain Following Spinal Cord Injury

Neuropathic pain in people with spinal cord injury is thought to be due to altered central neuronal activity. A novel therapeutic intervention using virtual reality (VR) head-mounted devices was investigated in this study for pain relief. Given the potential links to neuronal activity, the aim of the current study was to determine whether use of VR was associated with corresponding changes in electroencephalography (EEG) patterns linked to the presence of neuropathic pain. Using a within-subject, randomised cross-over pilot trial, we compared EEG activity for three conditions: no task eyes open state, 2D screen task and 3D VR task. We found an increase in delta activity in frontal regions for 3D VR with a decrease in theta activity. There was also a consistent decrease in relative alpha band (8–12 Hz) and an increase in low gamma (30–45 Hz) power during 2D screen and 3D VR corresponding, with reduced self-reported pain. Using the nonlinear and non-oscillatory method of extracting fractal dimensions, we found increases in brain complexity during 2D screen and 3D VR. We successfully classified the 3D VR condition from 2D screen and eyes opened no task conditions with an overall accuracy of 80.3%. The findings in this study have implications for using VR applications as a therapeutic intervention for neuropathic pain in people with spinal cord injury.