The Effects of Virtual Reality, Augmented Reality, and Mixed Reality as Training Enhancement Methods: A Meta-Analysis

Using virtual environments to facilitate refugee integration in third countries

Virtual experiences (VEs) have significant potential to enrich emotional interactions, to encourage socialisation and improve communication. In education, VEs offer new approaches for delivering content. In this paper we consider the application of VEs for assisting refugees in Senegal to learn how to navigate the complexities of the UK health system; a substantial stumbling block for their integration into society and for their own health. Participants ( N = 122 ), refugees awaiting to be repatriated, were exposed to material presented via three different media text, 360° videos and virtual reality (VR) across a total of seven different modalities. The experiment investigated specific attributes of the media that would facilitate refugees' integration, such as knowledge received and retained, experience, usability and presence. The results show that interactive media, in particular VR, was significantly better across all tested attributes.

Using the DiCoT framework for integrated multimodal analysis in mixed-reality training environments

Simulation-based training (SBT) programs are commonly employed by organizations to train individuals and teams for effective workplace cognitive and psychomotor skills in a broad range of applications. Distributed cognition has become a popular cognitive framework for the design and evaluation of these SBT environments, with structured methodologies such as Distributed Cognition for Teamwork (DiCoT) used for analysis. However, the analysis and evaluations generated by such distributed cognition frameworks require extensive domain-knowledge and manual coding and interpretation, and the analysis is primarily qualitative. In this work, we propose and develop the application of multimodal learning analysis techniques to SBT scenarios. Using these analysis methods, we can use the rich multimodal data collected in SBT environments to generate more automated interpretations of trainee performance that supplement and extend traditional DiCoT analysis. To demonstrate the use of these methods, we present a case study of nurses training in a mixed-reality manikin-based (MRMB) training environment. We show how the combined analysis of the video, speech, and eye-tracking data collected as the nurses train in the MRMB environment supports and enhances traditional qualitative DiCoT analysis. By applying such quantitative data-driven analysis methods, we can better analyze trainee activities online in SBT and MRMB environments. With continued development, these analysis methods could be used to provide targeted feedback to learners, a detailed review of training performance to the instructors, and data-driven evidence for improving the environment to simulation designers.

Virtual reality-based training for mental health staff: a novel approach to increase empathy, compassion, and subjective understanding of service user experience

Background

Mental health service users report that staff empathy is key to developing positive therapeutic relationships but promoting empathy in staff training is challenging. Staff may struggle to maintain their compassion, particularly in challenging settings, and have limited clinical confidence when treating conditions of which they lack subjective understanding. Novel interventions are required to address these needs.

Main body of the text

Virtual reality-based simulation training has been shown to be an effective training modality for healthcare professionals; it has the potential to deliver crucial empathy-building learning for frontline mental health staff due to its capacity to increase staff understanding of service users’ experiences. Virtual reality and simulation technology take interactivity and experiential learning to a level beyond which we have seen in teaching and training before. Subjective understanding is elicited because this is a technology for enhanced experiential learning, which in turn fosters greater empathy and compassion. Increased empathy in the workforce is likely to yield significant benefits for service users. Greater empathy in nursing is linked with reduced restrictive practices and reduced conflict between staff and service users. Restrictive practices, including restraint and seclusion, are widely used in mental health settings within the UK, and are an aspect of mental health nursing that is at odds with the therapeutic role of nursing. Despite these innovative developments, there are challenges ahead. Many nurses feel that complete eradication of restrictive practices is impossible and that barriers include a limitation of resources, communication, management, and lack of education. There is a need to make simulation training economically viable so that it can be upscaled and widely available. Therefore, greater investment and resources are needed to bring this innovative training to the wider workforce to support staff and to realise the benefits for service users.

Short conclusion

Virtual reality-based training has great potential for mental health staff, which could have important consequences in terms of improved staff empathy and reductions in harmful restrictive practices. Further research and funding for such training is necessary so that it can be more widely available.

Simulation-Based VR Training for the Nuclear Sector—A Pilot Study

Simulation-based training has seen an increased use of extended reality, that is, augmented reality (AR), virtual reality (VR) and mixed reality (MR) displays. The health professions have been using VR for simulation-based training more extensively than others. This method can offer the possibility of immediate feedback, which promotes more accurate training to self-correct mistakes in environments that are otherwise risky or provide unsafe conditions. The nuclear industry has attempted to seize the same digital and educational transformation to train their personnel to handle dangerous scenarios. We ran a pilot study with the aim of evaluating the VR training scenario developed for the specific target group and the acceptance of the VR technology itself for this kind of training. We used the Kirkpatrick framework to evaluate the training and the VR-HAM acceptance model to evaluate the acceptance of VR. A VR scenario was developed to test specific technical skills of nuclear power plant personnel. The 13 participants showed results within the basic level of the Kirkpatrick framework and held both positive reactions and concerns, that is, they enjoyed the training with VR and expressed concerns regarding the stability of the VR technology. The participants also showed positive attitudes towards the perceived ease of use and usefulness of the VR-HAM and its various constructs. Even though the COVID-19 pandemic limited our testing, we could show valuable data and positive attitudes and perceived ease by the participants. Simulation-based VR training could be an important complement to traditional training methods, especially where safety is a priority, but we still need to provide solid evidence.

AR-AI Tools as a Response to High Employee Turnover and Shortages in Manufacturing during Regular, Pandemic, and War Times

The world faces the continuously increasing issue of a lack of skilled employees, staff migration, and turnover. It is strengthened by unexpected situations such as wars, pandemics, and other civilization crises. Solutions are sought and researched in various branches of industry and academia, including engineering, social sciences, management, and political and computer sciences. From the viewpoint of this paper, this is a side topic of Industry 4.0 and, more specifically, sustainability in working environments, and the issue is related to production employees who perform manual operations. Some of the tasks cannot be carried out under robotization or automation; therefore, novel human-work support tools are expected. This paper presents such highly demanded support tools related to augmented reality (AR) and artificial intelligence (AI). First, a panoramic literature review is given. Secondly, the authors explain the main objective of the presented contribution. Then the authors’ achievements are described—the R&D focus on such solutions and the introduction of the developed tools that are based on AR and AI. Benefits connected to the AR-AI technology applications are presented in terms of both time savings with the tool usage and job simplification, enabling inexperienced, unskilled, or less skilled employees to perform the work in the selected manual production processes.

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

In this paper, we address the challenge of estimating the 6DoF pose of objects in 2D equirectangular images. This solution allows the transition to the objects’ 3D model from their current pose. In particular, it finds application in the educational use of 360° videos, where it enhances the learning experience of students by making it more engaging and immersive due to the possible interaction with 3D virtual models. We developed a general approach usable for any object and shape. The only requirement is to have an accurate CAD model, even without textures of the item, whose pose must be estimated. The developed pipeline has two main steps: vehicle segmentation from the image background and estimation of the vehicle pose. To accomplish the first task, we used deep learning methods, while for the second, we developed a 360° camera simulator in Unity to generate synthetic equirectangular images used for comparison. We conducted our tests using a miniature truck model whose CAD was at our disposal. The developed algorithm was tested using a metrological analysis applied to real data. The results showed a mean difference of 1.5° with a standard deviation of 1° from the ground truth data for rotations, and 1.4 cm with a standard deviation of 1.5 cm for translations over a research range of ±20° and ±20 cm, respectively.

Digital Game-Based Support for Learning the Phlebotomy Procedure in the Biomedical Laboratory Scientist Education

Practice-based training in education is important, expensive, and resource-demanding. Digital games can provide complementary training opportunities for practicing procedural skills and increase the value of the limited laboratory training time in biomedical laboratory science (BLS) education. This paper presents how a serious game can be integrated in a BLS course and supplement traditional learning and teaching with accessible learning material for phlebotomy. To gather information on challenges relevant to integrating Digital Game-Based Learning (DGBL), a case was carried out using mixed methods. Through a semester-long study, following a longitudinal, interventional cohort study, data and information were obtained from teachers and students about the learning impact of the current application. The game motivated students to train more, and teachers were positive towards using it in education. The results provide increased insights into how DGBL can be integrated into education and give rise to a discussion of the current challenges of DGBL for practice-based learning.

Super Divya to the rescue! Exploring Nurse Mentor Supervisor perceptions on a digital tool to support learning and engagement for simulation educators in Bihar, India

BACKGROUND

Since 2014, the Government of Bihar and CARE India have implemented a nurse mentoring program that utilizes PRONTO International's simulation and team trainings to improve obstetric and neonatal care. Together they trained simulation educators known as Nurse Mentor Supervisors to conduct simulation trainings in rural health facilities across the state. Sustaining the knowledge and engagement of these simulation educators at a large-scale has proven difficult and resource intensive. To address this, the University of Utah with PRONTO International and with input from the University of California San Francisco, created an interactive, virtual education module based on a comic superhero named Super Divya to reinforce simulation educator concepts. This study examined the perceptions of Nurse Mentor Supervisors on Super Divya's accessibility, usefulness, and potential after implementation of Super Divya: Origin Story.

METHODS

We conducted qualitative interviews with 17 Nurse Mentor Supervisors in Bihar, India. In light of the COVID-19 pandemic, interviews were conducted virtually via Zoom™ using a semi-structured interview guide in Hindi and English. Participants were identified with strict inclusion criteria and convenience sampling methods. Interviews were analyzed using a framework analysis.

RESULTS

Nurse Mentor Supervisors found Super Divya to be engaging, innovative, relatable, and useful in teaching tips and tricks for simulation training. Supervisors thought the platform was largely accessible with some concerns around internet connectivity and devices. The majority reacted positively to the idea of distributing Super Divya to other simulation educators in the nurse mentoring program and had suggestions for additional clinical and simulation educator training topics.

CONCLUSIONS

This study demonstrates the potential of Super Divya to engage simulation educators in continuous education. At a time when virtual education is increasingly important and in-person training was halted by the COVID-19 pandemic, Super Divya engaged Supervisors in the nurse mentoring program. We have incorporated suggestions for improvement of Super Divya into future modules. Further research can help understand how knowledge from Super Divya can improve simulation facilitation skills and behaviors, and explore potential for reinforcing clinical skills via this platform.

ETHICAL APPROVAL

This study was approved by the institutional review board at the University of California San Francisco (IRB # 20-29902).

The usefulness of virtual, augmented, and mixed reality technologies in the diagnosis and treatment of attention deficit hyperactivity disorder in children: an overview of relevant studies

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental condition characterized by attention problems, excessive physical activity, and impulsivity. ADHD affects not only the patients but also their families. The development and use of technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR) for ADHD has increased over recent years. However, little is known about their potential usefulness. This overview aimed to clarify the current knowledge about the use of these three innovative technologies for the diagnosis and treatment of children with ADHD.

METHODS

This overview was conducted using the PubMed, Web of Science, and Scopus databases until January 24th, 2021. The following descriptive information was compiled from the identified studies: country, year of publication, sample size, study design, ADHD diagnosis methods, applied technology, hardware equipment, clinical target, and main findings.

RESULTS

The initial database searches yielded 409 articles, but 103 were removed as duplicates. Eventually, 30 eligible studies remained for analysis, the majority of which were case-control (n = 22, 73%). Regarding the applied technology/hardware equipment, VR (n = 27; 90%), head-mounted displays (n = 19, 63%), VR-based continuous performance tests (VR-CPT) (n = 21, 70%) were most frequently used. Most studies (n = 21, 70%) used the DSM criteria for the diagnosis of childhood ADHD. They primarily evaluated the utility of these technologies in assessing ADHD symptoms (n = 10, 33%) and improving the ADHD diagnostic process (n = 7, 23%).

CONCLUSION

This comprehensive overview evaluated the studies on the use of VR, AR, and MR technologies for children with ADHD. These technologies seem to be promising tools for improving the diagnosis and management of ADHD in this population.

Assessing internet and web services based webdom and virtual web-data-centric geographical study

At present, the traditional geographical study is somewhat shifted to virtual web-datacentric geographical study and it was started almost around last decade of the twentieth century concerning remotely sensed earth surface, digitisation, virtual data storing and their artificial intelligence based machine learning for spatio-temporal analysis, prediction and thematic mapping. Now in the age of webdom-a global digital kingdom of internet and web services, when internet services flooded everything relating to continuous up-gradation of computer-related science and technologies along with increasing number of internet users and GPS enabled smart phones, geographical data are generated by exaflood leading to bigdata. Many online data are now easily available to everyone from anywhere at any time. The objective is to assess the webdom and to understand the scope of contemporary virtual web-data-centric geographical study involving geographical bigdata, machine learning and WebGIS, when virtual world abundantly expands gradually during this period of webdom. Each data in bigdata contains geospatial attributes. Worldwide under the exponential growth and advancement of data science like other scientific disciplines geography is also predicting different phenomenon based on artificial intelligence and machine learning. The three main facets of virtual web-data-centric geographical study are geographical bigdata, geographical machine learning, and WebGIS. It is centrally concerned with digital data and web services. Here data are analysed through machine learning methods. Sometimes reliability of web-data is in question without field verification which is considered as the primary requisition of traditional geography.

Visual-haptic integration, action and embodiment in virtual reality

The current generation of virtual reality (VR) technologies has improved substantially from legacy systems, particularly in the resolution and latency of their visual display. The presentation of haptic cues remains challenging, however, because haptic systems do not generalise well over the range of stimuli (both tactile and proprioceptive) normally present when interacting with objects in the world. This study investigated whether veridical tactile and proprioceptive cues lead to more efficient interaction with a virtual environment. Interaction in the world results in spatial and temporal correlation of tactile, proprioceptive and visual cues. When cues in VR are similarly correlated, observers experience a sense of embodiment and agency of their avatars. We investigated whether sensorimotor performance was mediated by embodiment of the avatar hands. Participants performed a Fitts' tapping task in different conditions (VR with no haptics, active haptics, passive haptics, and on a real touchscreen). The active-haptic condition provided abstract tactile cues and the passive haptic condition provided veridical tactile and proprioceptive cues. An additional (hybrid haptics) condition simulated an ideal passive haptic system. Movement efficiency (throughput) and embodiment were higher for the passive than for the active and no-haptics conditions. However, components of embodiment (perceived agency and ownership) did not predict unique variance in throughput. Improved sensorimotor performance and ratings of presence and realism support the use of passive haptics in VR environments where objects are in known and stable locations, regardless of whether performance was mediated by the sense of embodiment.

An Augmented Reality Device for Remote Supervision of Ultrasound Examinations in International Exercise Science Projects: Usability Study

Background

Support for long-distance research and clinical collaborations is in high demand and has increased owing to COVID-19–related restrictions on travel and social contact. New digital approaches are required for remote scientific exchange.

Objective

This study aims to analyze the options of using an augmented reality device for remote supervision of exercise science examinations.

Methods

A mobile ultrasound examination of the diameter and intima-media thickness of the femoral and carotid arteries was remotely supervised using a head-mounted augmented reality device. All participants were provided with a link to a YouTube video of the technique in advance. In part 1, 8 international experts from the fields of engineering and sports science were remotely connected to the study setting. Internet connection speed was noted, and a structured interview was conducted. In part 2, 2 remote supervisors evaluated 8 physicians performing an examination on a healthy human subject. The results were recorded, and an evaluation was conducted using a 25-item questionnaire.

Results

In part 1, the remote experts were connected over a mean distance of 1587 km to the examination site. Overall transmission quality was good (mean upload speed: 28.7 Mbps, mean download speed: 97.3 Mbps, mean ping: 21.6 milliseconds). In the interview, participants indicated that the main potential benefits would be to the fields of education, movement analysis, and supervision. Challenges regarding internet connection stability and previous training with the devices used were reported. In part 2, physicians’ examinations showed good interrater correlation (interclass correlation coefficient: 0.84). Participants valued the experienced setting as highly positive.

Conclusions

The study showed the good feasibility of the chosen design and a highly positive attitude of all participants toward this digital approach. Head-mounted augmented reality devices are generally recommended for collaborative research projects with physical examination–based research questions.

Is Virtually Everything Possible? The Relevance of Ethics and Human Rights for Introducing Extended Reality in Forensic Psychiatry

Extended Reality (XR) systems, such as Virtual Reality (VR) and Augmented Reality (AR), provide a digital simulation either of a complete environment, or of particular objects within the real world. Today, XR is used in a wide variety of settings, including gaming, design, engineering, and the military. In addition, XR has been introduced into psychology, cognitive sciences and biomedicine for both basic research as well as diagnosing or treating neurological and psychiatric disorders. In the context of XR, the simulated 'reality' can be controlled and people may safely learn to cope with their feelings and behavior. XR also enables to simulate environments that cannot easily be accessed or created otherwise. Therefore, Extended Reality systems are thought to be a promising tool in the resocialization of criminal offenders, more specifically for purposes of risk assessment and treatment of forensic patients. Employing XR in forensic settings raises ethical and legal intricacies which are not raised in case of most other healthcare applications. Whereas a variety of normative issues of XR have been discussed in the context of medicine and consumer usage, the debate on XR in forensic settings is, as yet, straggling. By discussing two general arguments in favor of employing XR in criminal justice, and two arguments calling for caution in this regard, the present paper aims to broaden the current ethical and legal debate on XR applications to their use in the resocialization of criminal offenders, mainly focusing on forensic patients.

The Science of Workplace Instruction: Learning and Development Applied to Work

Learning is the engagement in mental processes resulting in the acquisition and retention of knowledge, skills, and/or affect over time and applied when needed. Building on this definition, we integrate the science of training and the science of learning to propose a new science of workplace instruction, linking the design of instructional events to instructional outcomes such as transfer and job performance through the mediating effects of learner events and learning outcomes. We propose three foundational elements: the learner, instructional principles, and training delivery (methods and media). Understanding and applying instructional principles are the primary methods for enhancing training effectiveness; thus, we detail 15 empirically supported principles. We then discuss the erroneous pursuit of aptitude-by-treatment interactions under the guise of learner styles and age-specific instruction. Finally, we offer suggestions for future research that draw on the foundation of instructional principles to optimize self-directed learning and learning in synthetic learning environments.

A Review of Extended Reality (XR) Technologies for Manufacturing Training

Recently, the use of extended reality (XR) systems has been on the rise, to tackle various domains such as training, education, safety, etc. With the recent advances in augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies and ease of availability of high-end, commercially available hardware, the manufacturing industry has seen a rise in the use of advanced XR technologies to train its workforce. While several research publications exist on applications of XR in manufacturing training, a comprehensive review of recent works and applications is lacking to present a clear progress in using such advance technologies. To this end, we present a review of the current state-of-the-art of use of XR technologies in training personnel in the field of manufacturing. First, we put forth the need of XR in manufacturing. We then present several key application domains where XR is being currently applied, notably in maintenance training and in performing assembly task. We also reviewed the applications of XR in other vocational domains and how they can be leveraged in the manufacturing industry. We finally present some current barriers to XR adoption in manufacturing training and highlight the current limitations that should be considered when looking to develop and apply practical applications of XR.

Extended Reality for the Clinical, Affective, and Social Neurosciences

Brain science research often involves the use of low-dimensional tools and stimuli that lack several of the potentially valuable features of everyday activities and interactions. Although this research has provided important information about cognitive, affective, and social processes for both clinical and nonclinical populations, there is growing interest in high-dimensional simulations that extend reality. These high-dimensional simulations involve dynamic stimuli presented serially or concurrently to permit the assessment and training of perceivers' integrative processes over time. Moreover, high-dimensional simulation platforms can contextually restrain interpretations of cues about a target's internal states. Extended reality environments extend assessment and training platforms that balance experimental control with emotionally engaging background narratives aimed at extending the affective experience and social interactions. Herein, we highlight the promise of extended reality platforms for greater ecological validity in the clinical, affective, and social neurosciences.

Transcending human frailties with technological enhancements and replacements: Transhumanist perspective in nursing and healthcare

As human beings age, they become weak, fragile, and feeble. It is a slowly progressing yet complex syndrome in which old age or some disabilities are not prerequisites; neither does loss of human parts lead to frailty among the physically fit older persons. This paper aims to describe the influences of transhumanist perspectives on human-technology enhancements and replacements in the transcendence of human frailties, including those of older persons, in which technology is projected to deliver solutions toward transcending these frailties. Through technologies including genetic screening and other technological manipulations, intelligent machines and augmented humans improve, maintain, and remedy human-linked susceptibilities. Furthermore, other technologies replace parts fabricated through inorganic-mechanical processes such as 3D-printing. Advancing technologies are reaching the summit of technological sophistication contributing to the transhumanist views of being human in a technological world. Technologies enhance the transcendence of human frailties as essential expressions of the symbiosis between human beings and technology in a transcendental world.

Digital Health Innovations, Tools, and Resources to Support Helping Babies Survive Programs

The Helping Babies Survive (HBS) initiative features a suite of evidence-based curricula and simulation-based training programs designed to provide health workers in low- and middle-income countries (LMICs) with the knowledge, skills, and competencies to prevent, recognize, and manage leading causes of newborn morbidity and mortality. Global scale-up of HBS initiatives has been rapid. As HBS initiatives rolled out across LMIC settings, numerous bottlenecks, gaps, and barriers to the effective, consistent dissemination and implementation of the programs, across both the pre- and in-service continuums, emerged. Within the first decade of expansive scale-up of HBS programs, mobile phone ownership and access to cellular networks have also concomitantly surged in LMICs. In this article, we describe a number of HBS digital health innovations and resources that have been developed from 2010 to 2020 to support education and training, data collection for monitoring and evaluation, clinical decision support, and quality improvement. Helping Babies Survive partners and stakeholders can potentially integrate the described digital tools with HBS dissemination and implementation efforts in a myriad of ways to support low-dose high-frequency skills practice, in-person refresher courses, continuing medical and nursing education, on-the-job training, or peer-to-peer learning, and strengthen data collection for key newborn care and quality improvement indicators and outcomes. Thoughtful integration of purpose-built digital health tools, innovations, and resources may assist HBS practitioners to more effectively disseminate and implement newborn care programs in LMICs, and facilitate progress toward the achievement of Sustainable Development Goal health goals, targets, and objectives.