Multiplayer Interaction Platform With Gaze Tracking for Individuals With Autism

A Novel Scenario-Based, Mixed-Reality Platform for Training Nontechnical Skills of Battlefield First Aid: Prospective Interventional Study

BACKGROUND

Although battlefield first aid (BFA) training shares many common features with civilian training, such as the need to address technical skills and nontechnical skills (NTSs), it is more highly scenario-dependent. Studies into extended reality show clear benefits in medical training; however, the training effects of extended reality on NTSs, including teamwork and decision-making in BFA, have not been fully proven.

OBJECTIVE

The current study aimed to create and test a scenario-based, mixed-reality platform suitable for training NTSs in BFA.

METHODS

First, using next-generation modeling technology and an animation synchronization system, a 10-person offensive battle drill was established. Decision-making training software addressing basic principles of tactical combat casualty care was constructed and integrated into the scenarios with Unreal Engine 4 (Epic Games). Large-space teamwork and virtual interaction systems that made sense in the proposed platform were developed. Unreal Engine 4 and software engineering technology were used to combine modules to establish a mixed-reality BFA training platform. A total of 20 Grade 4 medical students were recruited to accept BFA training with the platform. Pretraining and posttraining tests were carried out in 2 forms to evaluate the training effectiveness: one was knowledge acquisition regarding the NTS and the other was a real-world, scenario-based test. In addition, the students were asked to rate their agreement with a series of survey items on a 5-point Likert scale.

RESULTS

A battlefield geographic environment, tactical scenarios, scenario-based decision software, large-space teamwork, and virtual interaction system modules were successfully developed and combined to establish the mixed-reality training platform for BFA. The posttraining score of the students' knowledge acquisition was significantly higher than that of pretraining (t=-12.114; P≤.001). Furthermore, the NTS score and the total score that the students obtained in the real-world test were significantly higher than those before training (t=-17.756 and t=-21.354, respectively; P≤.001). However, there was no significant difference between the scores of technical skills that the students obtained before and after training. A posttraining survey revealed that the students found the platform helpful in improving NTSs for BFA, and they were confident in applying BFA skills after training. However, most trainees thought that the platform was not helpful for improving the technical skills of BFA, and 45% (9/20) of the trainees were not satisfied with the simulation effect.

CONCLUSIONS

A scenario-based, mixed-reality platform was constructed in this study. In this platform, interaction of the movement of multiple players in a large space and the interaction of decision-making by the trainees between the real world and the virtual world were accomplished. The platform could improve the NTSs of BFA. Future works, including improvement of the simulation effects and development of a training platform that could effectively improve both the technical skills and NTSs of BFA, will be carried out.

Virtual reality in the diagnostic and therapy for mental disorders: A systematic review

BACKGROUND

Virtual reality (VR) technologies are playing an increasingly important role in the diagnostics and treatment of mental disorders.

OBJECTIVE

To systematically review the current evidence regarding the use of VR in the diagnostics and treatment of mental disorders.

DATA SOURCE

Systematic literature searches via PubMed (last literature update: 9^th of May 2022) were conducted for the following areas of psychopathology: Specific phobias, panic disorder and agoraphobia, social anxiety disorder, generalized anxiety disorder, posttraumatic stress disorder (PTSD), obsessive-compulsive disorder, eating disorders, dementia disorders, attention-deficit/hyperactivity disorder, depression, autism spectrum disorder, schizophrenia spectrum disorders, and addiction disorders.

ELIGIBILITY CRITERIA

To be eligible, studies had to be published in English, to be peer-reviewed, to report original research data, to be VR-related, and to deal with one of the above-mentioned areas of psychopathology.

STUDY EVALUATION

For each study included, various study characteristics (including interventions and conditions, comparators, major outcomes and study designs) were retrieved and a risk of bias score was calculated based on predefined study quality criteria.

RESULTS

Across all areas of psychopathology, k = 9315 studies were inspected, of which k = 721 studies met the eligibility criteria. From these studies, 43.97% were considered assessment-related, 55.48% therapy-related, and 0.55% were mixed. The highest research activity was found for VR exposure therapy in anxiety disorders, PTSD and addiction disorders, where the most convincing evidence was found, as well as for cognitive trainings in dementia and social skill trainings in autism spectrum disorder.

CONCLUSION

While VR exposure therapy will likely find its way successively into regular patient care, there are also many other promising approaches, but most are not yet mature enough for clinical application.

REVIEW REGISTRATION

PROSPERO register CRD42020188436.

FUNDING

The review was funded by budgets from the University of Bonn. No third party funding was involved.

Measuring Social Modulation of Gaze in Autism Spectrum Condition With Virtual Reality Interviews

Gaze behavior in dyadic conversations can indicate active listening and attention. However, gaze behavior that is different from the engagement expected during neurotypical social interaction cues may be interpreted as uninterested or inattentive, which can be problematic in both personal and professional situations. Neurodivergent individuals, such as those with autism spectrum conditions, often exhibit social communication differences broadly including via gaze behavior. This project aims to support situational social gaze practice through a virtual reality (VR) mock job interview practice using the HTC Vive Pro Eye VR headset. We show how gaze behavior varies in the mock job interview between neurodivergent and neurotypical participants. We also investigate the social modulation of gaze behavior based on conversational role (speaking and listening). Our three main contributions are: (i) a system for fully-automatic analysis of social modulation of gaze behavior using a portable VR headset with a novel realistic mock job interview, (ii) a signal processing pipeline, which employs Kalman filtering and spatial-temporal density-based clustering techniques, that can improve the accuracy of the headset's built-in eye-tracker, and (iii) being the first to investigate social modulation of gaze behavior among neurotypical/divergent individuals in the realm of immersive VR.

Virtual and Augmented Reality in Social Skills Interventions for Individuals with Autism Spectrum Disorder: A Scoping Review

In the last decade, there has been an increase in publications on technology-based interventions for autism spectrum disorder (ASD). Virtual reality based assessments and intervention tools are promising and have shown to be acceptable amongst individuals with ASD. This scoping review reports on 49 studies utilizing virtual reality and augmented reality technology in social skills interventions for individuals with ASD. The included studies mostly targeted children and adolescents, but few targeted very young children or adults. Our findings show that the mode number of participants with ASD is low, and that female participants are underrepresented. Our review suggests that there is need for studies that apply virtual and augmented realty with more rigorous designs involving established and evidenced-based intervention strategies.

A Data-Driven Framework for Intention Prediction via Eye Movement With Applications to Assistive Systems

Fast and accurate human intention prediction can significantly advance the performance of assistive devices for patients with limited motor or communication abilities. Among available modalities, eye movement can be valuable for inferring the user's intention, as it can be tracked non-invasively. However, existing limited studies in this domain do not provide the level of accuracy required for the reliable operation of assistive systems. By taking a data-driven approach, this paper presents a new framework that utilizes the spatial and temporal patterns of eye movement along with deep learning to predict the user's intention. In the proposed framework, the spatial patterns of gaze are identified by clustering the gaze points based on their density over displayed images in order to find the regions of interest (ROIs). The temporal patterns of gaze are identified via hidden Markov models (HMMs) to find the transition sequence between ROIs. Transfer learning is utilized to identify the objects of interest in the displayed images. Finally, models are developed to predict the user's intention after completing the task as well as at early stages of the task. The proposed framework is evaluated in an experiment involving predicting intended daily-life activities. Results indicate that an average classification accuracy of 97.42% is achieved, which is considerably higher than existing gaze-based intention prediction studies.