Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT

Test-Retest Reliability of Immersive Virtual Reality Measures of Perceptual-Motor Performance

The duration, accuracy, and consistency of responses to various types of stimuli are widely accepted as indirect indicators of the efficiency of brain information processing, but current clinical tests appear to lack sufficient sensitivity to detect subtle impairments. Immersive virtual reality (VR) offers a new means to acquire measures of perceptual-motor responses to moving visual stimuli that require rapid conflict resolution, but their test-retest reliability has not yet been demonstrated. Repeated measures. We analyzed data from 19 healthy young adults who performed a 40-trial VR test on three consecutive days. We focused on response time (RT) and perceptual latency (PL) for eye, neck, arm, and whole-body step displacements involved in executing a reaching/lunging movement in a right or left direction toward a peripherally located virtual target. Measures of RT and PL included a 40-trial mean, an intra-individual variability (IIV) value, and a rate correct score (RCS) that incorporated both response duration and accuracy. Most mean and IIV values for PL and RT demonstrated a positive distributional skew that was substantially reduced by natural logarithm transformation. While a learning effect was evident between sessions 1 and 2 for 7 of 8 mean PL and RT measures, 3-session intraclass correlation coefficient (ICC) values were moderate to excellent for 15 of 16 transformed PL and RT measures (range: .618 to .922). The composite RCS metric did not require transformation for either PL or RT, whose respective 3-session ICC values were .877 and .851. This moderate to excellent test-retest reliability for various VR measures of perceptual-motor function, combined with evidence of their validity from both past and future research, suggest that these measures can advance clinical detection of impaired brain processing and longitudinal assessments of potentially modifiable performance deficiencies.

Biophilic classroom environments on stress and cognitive performance: A randomized crossover study in virtual reality (VR)

The emerging Metaverse will likely increase time expenditure in indoor virtual environments, which could impact human health and well-being. The biophilia hypothesis suggests that humans have an innate tendency to seek connections with the natural world and there is increasing evidence that biophilic design such as the incorporation of green plants can yield health benefits. Recently, virtual reality (VR) has been used to regulate stress and improve overall wellness, particularly by incorporating natural settings. In this randomized crossover study, we designed five virtual classroom scenes with different biophilic elements and turbidity in VR and investigated whether the visual stimulations can affect the stress levels and cognitive functions of 30 young adults from a university in China. We measured their physiological indicators of stress reaction by wearable biomonitoring sensors (blood pressure (BP), heart rate (HR), heart rate variability (HRV), and skin conductance level (SCL)), conducted verbal cognitive tests on attention and creativity, and evaluated subjective/perceived (self-reported) stress levels and connection with nature. Albeit our results suggested no significant change in physiological stress reactions or cognitive functions induced by the biophilic and turbid interventions in VR, the addition of biophilic elements in the Metaverse could benefit students' health due to significantly decreased perceived stress levels and increased connections with nature.

The brain-derived neurotrophic factor Val66met polymorphism is associated with better attention and working memory performance and resilience to mild chronic stress

The val66met polymorphism of the brain-derived neurotrophic factor (BDNF) gene has been identified as a potential moderator for the relationship between chronic stress and executive functioning. However, whether the presence of the met allele increases cognitive vulnerability or resilience to stress has yet to be determined. Given the established effects of autonomic activity and psychological arousal on executive functioning, in the present study, 56 healthy university students completed self-report measures of chronic stress, positive arousal (vigour) and negative arousal (anxiety) and measured heart-rate variability to quantify autonomic activity. Participants then completed a cognitive test battery that measured attention, decision-making, visual learning and working memory. Regression analyses demonstrated that Val/met participants performed better on attention and working memory tasks than Val/val participants, but no differences were seen in decision-making and visual learning. Further, Val/met participants were protected from stress-related differences in attention seen in Val/val participants. Val66met was not associated with physiological or psychological arousal. This study demonstrates that val66met plays an important but selective role in cognitive performance.

Ecological validity in neurocognitive assessment: Systematized review, content analysis, and proposal of an instrument

OBJECTIVES

The main objectives of this study are to identify the dimensions of Ecological Validity (EV) within the definitions of this concept, understand how they are operationalized in neurocognitive tests, and propose a checklist for EV attributes in neurocognitive tests.

METHOD

A systematized review was combined with content analysis of the selected papers, using the inductive method. We analyzed 82 studies on the EV of neurocognitive tests, 19 literature reviews and 63 empirical studies. Based on this review, we identified the relevant criteria for evaluating EV.

RESULTS

EV is a multidimensional concept with two main dimensions: representativeness and generalization. Representativeness involves the subdimensions simplicity-complexity and artificial-natural and several criteria organized on a continuum from low EV to high EV. Generalization is dependent on representativeness and is influenced by different cognitive and non-cognitive factors. We propose six stages for operationalizing EV, from defining the objectives of the neurocognitive assessment to the methodology for scoring and interpreting the results.

CONCLUSION

This systematized review helps to operationalize the concept of EV by providing a tool for evaluating and improving EV while developing new tests. Further studies with a longitudinal design can compare the predictive value of tests with higher versus lower EV-checklist scores.Key pointsQuestion: Understand the definition of EV, its dimensions and subdimensions, how EV is operationalized in neurocognitive tests and propose a checklist for the EV attributes of neurocognitive tests.Findings: The primary findings were that representativeness and generalization are the main dimensions of EV. Representativeness involves several subdimensions, whereas generalization is dependent on representativeness and is influenced by cognitive and non-cognitive factors. We provided an EV-checklist organized into six parts.Importance: The EV-checklist can be used to guide the development of ecologically valid neurocognitive tests and/or assess the EV of existing ones.Next steps: Examine the predictive value of tests that have higher EV-checklist scores.

Automatic Detection of Cognitive Impairment with Virtual Reality

Cognitive impairment features in neuropsychiatric conditions and when undiagnosed can have a severe impact on the affected individual's safety and ability to perform daily tasks. Virtual Reality (VR) systems are increasingly being explored for the recognition, diagnosis and treatment of cognitive impairment. In this paper, we describe novel VR-derived measures of cognitive performance and show their correspondence with clinically-validated cognitive performance measures. We use an immersive VR environment called VStore where participants complete a simulated supermarket shopping task. People with psychosis (k=26) and non-patient controls (k=128) participated in the study, spanning ages 20-79 years. The individuals were split into two cohorts, a homogeneous non-patient cohort (k=99 non-patient participants) and a heterogeneous cohort (k=26 patients, k=29 non-patient participants). Participants' spatio-temporal behaviour in VStore is used to extract four features, namely, route optimality score, proportional distance score, execution error score, and hesitation score using the Traveling Salesman Problem and explore-exploit decision mathematics. These extracted features are mapped to seven validated cognitive performance scores, via linear regression models. The most statistically important feature is found to be the hesitation score. When combined with the remaining extracted features, the multiple linear regression model resulted in statistically significant results with R2 = 0.369, F-Stat = 7.158, p(F-Stat) = 0.000128.

Comparison of Traditional and Virtual Reality-Based Episodic Memory Performance in Clinical and Non-Clinical Cohorts

The California Verbal Learning Test, Second Edition (CVLT-II) and the Virtual Environment Grocery Store (VEGS) use list learning and recognition tasks to assess episodic memory. This study aims to: (1) Replicate prior construct validity results among a new sample of young adults and healthy older adults; (2) Extend this work to a clinical sample of older adults with a neurocognitive diagnosis; (3) Compare CVLT-II and VEGS performance among these groups; and (4) Validate the independence of CVLT and VEGS episodic memory performance measures from executive functioning performance measures. Typically developing young adults (n = 53) and older adults (n = 85), as well as older adults with a neurocognitive diagnosis (n = 18), were administered the CVLT-II, VEGS, and D-KEFS CWIT. Results found that (1) the relationship of the VEGS and CVLT-II measures was highly correlated on all variables, (2) compared to the CVLT-II, participants (particularly older adults) recalled fewer items on the VEGS, and (3) the CVLT-II and VEGS were generally independent of D-KEFS CWIT. It appeared that the VEGS may be more difficult than the CVLT-II, possibly reflecting the word length effect. Performance may have also been impacted by the presence of everyday distractors in the virtual environment.

Gender moderates the association between chronic academic stress with top-down and bottom-up attention

Research on the relationship between chronic stress and cognition is limited by a lack of concurrent measurement of state-anxiety, physiological arousal, and gender. For the first time, we assessed the impact of these factors on top-down/conscious (simple and choice reaction time) and bottom-up/reflexive (saccadic reaction time) measures of attention using CONVIRT virtual-reality cognitive tests. Participants (N = 163) completed measures of academic stress (effort-reward imbalance; ERI) and state-anxiety while heart-rate variability was recorded continuously throughout the experiment. Gender moderated the association between academic stress with the top-down measures (b = -0.002, t = -2.023, p = .045; b = -0.063, t = -3.080, p = .002) and higher academic stress was associated with poorer/slower reaction times only for male participants. For bottom-up attention, heart rate variability moderated the relationship between academic stress and saccadic reaction time (b = 0.092, t = 1.991, p = .048), and only female participants who were more stressed (i.e., ERI ≥ 1) and displayed stronger sympathetic dominance had slower reaction times. Our findings align with emerging evidence that chronic stress is related to hyperarousal in women and cognitive decrements in men. Our findings suggest that higher ERI and sympathetic dominance during cognitive testing was associated with poorer bottom-up attention in women, whereas for men, academic stress was related with poorer top-down attention irrespective of sympathovagal balance.

Virtual-reality-based social cognition and interaction training for patients with schizophrenia: A preliminary efficacy study

BACKGROUND

Social cognition and interaction training (SCIT) is a psychosocial intervention program for patients with psychosis, designed to improve their social functioning by improving social cognition. Although the feasibility and efficacy of SCIT have been verified, patients with schizophrenia tend to suffer from motivational deficits and low treatment adherence. It has been suggested that using virtual reality (VR) technology might be effective in addressing these issues. In this study, we aimed to develop a VR-based SCIT and compare its efficacy with that of traditional SCIT.

MATERIALS AND METHODS

We developed a novel VR-based social cognition and interaction training (VR-SCIT) that combines traditional SCIT (TR-SCIT) intervention with VR technology. Participants were randomly assigned in a 1:1:1 ratio to the VR-SCIT (n = 28), TR-SCIT (n = 30), or waiting-list groups (n = 29). All treatments were combined with treatment-as-usual. Assessments of social cognition (i.e., Chinese version of Face-Affective Identification Task, Chinese version of Social Cognition Screening Questionnaire) and social functioning (i.e., Chinese version of Personal and Social Performance Scale) were administered from baseline to post-intervention.

RESULTS

Patients receiving VR-SCIT and TR-SCIT showed a significantly greater improvement on the assessments of emotion perception (Cohen's d was 1.66, 0.55, and 0.10 for VR-SCIT, TR-SCIT, and Waiting-list, respectively), hostile attributional bias (Cohen's d was 0.48, 0.44, and 0.05 for VR-SCIT, TR-SCIT, and Waiting-list, respectively), metacognition (Cohen's d was 1.66, 0.76, and 0.06 for VR-SCIT, TR-SCIT, and waiting-list, respectively), and social functioning (Cohen's d was 1.09, 0.90, and 0.20 for VR-SCIT, TR-SCIT, and waiting-list, respectively) from baseline to post-intervention, compared to those in waiting-list group. Additionally, VR-SCIT showed an advantage over TR-SCIT in improving emotion perception and metacognition with higher treatment compliance.

CONCLUSION

These preliminary findings indicate that VR-SCIT is a feasible and promising method for improving social cognition and social functioning in patients with schizophrenia.

Using Medical Device Standards for Design and Risk Management of Immersive Virtual Reality for At-Home Therapy and Remote Patient Monitoring

Numerous virtual reality (VR) systems have received regulatory clearance as therapeutic medical devices for in-clinic and at-home use. These systems enable remote patient monitoring of clinician-prescribed rehabilitation exercises, although most of these systems are nonimmersive. With the expanding availability of affordable and easy-to-use head-mounted display (HMD)-based VR, there is growing interest in immersive VR therapies. However, HMD-based VR presents unique risks. Following standards for medical device development, the objective of this paper is to demonstrate a risk management process for a generic immersive VR system for remote patient monitoring of at-home therapy. Regulations, standards, and guidance documents applicable to therapeutic VR design are reviewed to provide necessary background. Generic requirements for an immersive VR system for home use and remote patient monitoring are identified using predicate analysis and specified for both patients and clinicians using user stories. To analyze risk, failure modes and effects analysis, adapted for medical device risk management, is performed on the generic user stories and a set of risk control measures is proposed. Many therapeutic applications of VR would be regulated as a medical device if they were to be commercially marketed. Understanding relevant standards for design and risk management early in the development process can help expedite the availability of innovative VR therapies that are safe and effective.

Brain-derived Neurotropic Factor val66met is a Strong Predictor of Decision Making and Attention Performance on the CONVIRT Virtual Reality Cognitive Battery

The val66met polymorphism of the brain-derived neurotrophic factor gene has been associated with changes in components of executive functioning such as decision making; however, this relationship remains unclear. Val66met-related changes in attention and visual processing speed may explain potential changes in decision making. Furthermore, chronic stress disrupts executive functions and alters autonomic activity. Because the relationship between val66met and cognition has not been investigated in the context of chronic stress or stress-related autonomic changes, in this study 55 healthy university students completed self-report measures of chronic stress and mental health. Participants then completed a virtual reality cognitive test battery (CONVIRT) measuring decision making, attention, and visual processing reaction times. To measure autonomic activity, saliva alpha amylase and heart rate variability (HRV) were assessed at baseline and after CONVIRT testing. Saliva samples were used to identify val66met genotype. Regression analyses demonstrated that val66met was the strongest predictor of decision making and attention, but not visual processing, where valine/methionine (Val/met) participants had faster reaction times than Val/val participants. Val/met participants also had higher perceived chronic stress and heightened increases in sympathetic activity, but not parasympathetic activity. Neither stress nor autonomic activity moderated the effect of val66met on decision making or attention. This study is the first to investigate the role of val66met in decision making, attention, and visual processing while taking into account chronic stress and autonomic activity. This multifactorial approach revealed that carriers of the Val/met genotype may have better decision making and attention than Val/val carriers.

Assessing the utility of a virtual-reality neuropsychological test battery, 'CONVIRT', in detecting alcohol-induced cognitive impairment

New technologies such as virtual reality (VR) and eye-tracking software have paved the way for more sophisticated and ecologically valid measures of cognitive function. Testing the sensitivity and reliability of such measurements in response to acute alcohol intoxication provides a first step in establishing how these measures may operate in relation to cognitive impairments observed post-concussion. Healthy young adults (N = 54, M = 20.65, SD = 2.06, 30 females) completed the CONVIRT test battery (manual simple and choice reaction-time and saccade reaction-time) at three breath alcohol concentration (BrAC) levels: 0.00%^T1, 0.05%^T2, 0.08%^T3. Participants consumed alcoholic beverages at 30-min intervals, with BrAC monitored at 15-min intervals using a breathalyser. All three CONVIRT measures were sensitive to changes in cognitive performance induced by alcohol at BrAC levels at or exceeding 0.05%. A composite measure was also sensitive to alcohol intoxication (Cohen's d = .85 at BrAC = 0.05%; d = 1.20 at BrAC = 0.08%). Strong test-retest reliability was observed (all r < .80), with no gender differences noted. CONVIRT measures were reliable and detected dose-dependent changes in alcohol-induced cognitive impairment. Potentially, the ecologically valid measures may assist in better quantifying the effects of conditions such as concussion, on cognitive performance.