A Review of Extended Reality (XR) Technologies for Manufacturing Training

Investigating Whether the Mass of a Tool Replica Influences Virtual Training Learning Outcomes

Virtual Reality (VR) has emerged as a promising solution to address the pressing concern of transferring know-how in the manufacturing industry. Making an immersive training experience often involves designing an instrumented replica of a tool whose use is to be learned through virtual training. The process of making a replica can alter its mass, making it different from that of the original tool. As far as we know, the influence of this difference on learning outcomes has never been evaluated. To investigate this subject, an immersive training experience was designed with pre and post-training phases under real conditions, dedicated to learning the use of a rotary tool. 80 participants took part in this study, split into three groups: a control group performing the virtual training using a replica with the same mass as the original tool ($\mathrm{m}=100\%$), a second group that used a replica with a lighter mass than the original tool ($\mathrm{m}= 50\%$) and a third group using a replica heavier than the original tool ($\mathrm{m}=150\%$). Despite variations in the mass of the replica used for training, this study revealed that the learning outcomes remained comparable across all groups, while also demonstrating significant enhancements in certain performance measures, including task completion time. Overall, these findings provide useful insights regarding the design of tool replicas for immersive training.

Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality-Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review

OBJECTIVE

The present scoping review aims to transform the diverse field of research on the effects of mixed reality-based training on performance in manual assembly tasks into comprehensive statements about industrial needs for and effects of mixed reality-based training.

BACKGROUND

Technologies such as augmented and virtual reality, referred to as mixed reality, are seen as promising media for training manual assembly tasks. Nevertheless, current literature shows partly contradictory results, which is due to the diversity of the hardware used, manual assembly tasks as well as methodological approaches to investigate the effects of mixed reality-based training.

METHOD

Following the methodological approach of a scoping review, we selected 24 articles according to predefined criteria and analyzed them concerning five key aspects: (1) the needs in the industry for mixed reality-based training, (2) the actual use and classification of mixed reality technologies, (3) defined measures for evaluating the outcomes of mixed reality-based training, (4) findings on objectively measured performance and subjective evaluations, as well as (5) identified research gaps.

RESULTS

Regarding the improvement of performance and effectiveness through mixed reality-based training, promising results were found particularly for augmented reality-based training, while virtual reality-based training is mostly-but not consistently-as good as traditional training.

APPLICATION

Mixed reality-based training is still not consistently better, but mostly at least as good as traditional training. However, depending on the use case and technology used, the training outcomes in terms of assembly performance and subjective evaluations show promising results of mixed reality-based training.

Assemble it like this! - Is AR- or VR-based training an effective alternative to video-based training in manual assembly?

AR- and VR-based training is increasingly being used in the industry to train workers safely and effectively for new tasks. In this study, we investigated and compared the effects of AR-, VR- and video-based training on short- and long-term objective performance measures and subjective evaluations in a manual assembly task. Our results showed that there was no difference between AR-, VR- and video-based training concerning the objective performance measures task completion time and error count. However, in the subjective evaluations VR-based training showed a significantly higher perceived task load and a lower usability rating than the AR- and video-based training regimes. An exploratory analysis additionally revealed partially better results for AR than for VR after adjusting the data for the age of the participants. Future research should further investigate the advantage of AR- and video-based methods over VR when the age and technology experience of participants are taken into account.

Investigating the effect of intelligent assistance systems on motivational work characteristics in assembly

Intelligent assistance systems (IAS) are designed to counteract rising cognitive demands caused by increasingly individualized manufacturing processes in assembly. How IAS affect work characteristics which are crucial for promoting work motivation of employees is yet unclear. Based on the cyber-physical systems transformation framework, the model of routine-biased technological change, and a comprehensive model of work design, we expected in- and decreases in motivational work characteristics (MWC) when working with IAS. Furthermore, we posited a buffering effect of the option of voluntary use on decreasing knowledge characteristics. Applying an online case study with experimental vignette methodology (EVM) allowed us to identify effects of the IAS on MWC before it is widely implemented. 203 German and British blue-collar workers evaluated an assembly workplace according to three experimental conditions (work without IAS, work with IAS, work with voluntary use of IAS). We identified enhanced feedback from the job and information processing in work with IAS in contrast to a traditional assembly workplace but found no restrictions (or elevations) in terms of other task (i.e., autonomy) or knowledge characteristics (i.e., job complexity, problem solving, specialization, skill variety). Thus, our results indicate that the IAS improves some motivational work characteristics of the assembly workplace, although it misses the primary goal of cognitive relief. Our study highlights the need for work design theories that specify the effect of IAS on motivational work characteristics and the potential benefit of IAS in assembly of the future.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10845-023-02086-4.

Metaverse in Healthcare Integrated with Explainable AI and Blockchain: Enabling Immersiveness, Ensuring Trust, and Providing Patient Data Security

Digitization and automation have always had an immense impact on healthcare. It embraces every new and advanced technology. Recently the world has witnessed the prominence of the metaverse which is an emerging technology in digital space. The metaverse has huge potential to provide a plethora of health services seamlessly to patients and medical professionals with an immersive experience. This paper proposes the amalgamation of artificial intelligence and blockchain in the metaverse to provide better, faster, and more secure healthcare facilities in digital space with a realistic experience. Our proposed architecture can be summarized as follows. It consists of three environments, namely the doctor's environment, the patient's environment, and the metaverse environment. The doctors and patients interact in a metaverse environment assisted by blockchain technology which ensures the safety, security, and privacy of data. The metaverse environment is the main part of our proposed architecture. The doctors, patients, and nurses enter this environment by registering on the blockchain and they are represented by avatars in the metaverse environment. All the consultation activities between the doctor and the patient will be recorded and the data, i.e., images, speech, text, videos, clinical data, etc., will be gathered, transferred, and stored on the blockchain. These data are used for disease prediction and diagnosis by explainable artificial intelligence (XAI) models. The GradCAM and LIME approaches of XAI provide logical reasoning for the prediction of diseases and ensure trust, explainability, interpretability, and transparency regarding the diagnosis and prediction of diseases. Blockchain technology provides data security for patients while enabling transparency, traceability, and immutability regarding their data. These features of blockchain ensure trust among the patients regarding their data. Consequently, this proposed architecture ensures transparency and trust regarding both the diagnosis of diseases and the data security of the patient. We also explored the building block technologies of the metaverse. Furthermore, we also investigated the advantages and challenges of a metaverse in healthcare.

Critical Analysis of the Evaluation Methods of Extended Reality (XR) Experiences for Construction Safety

The construction industry has high accident rates. The sector is exploring various tools to improve safety management, training, and awareness to achieve zero accidents. This work focuses on extended reality (XR), which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies. Several authors have developed training experiences for construction safety in XR environments with positive conclusions about their effectiveness. However, there is no standardization regarding the evaluation methods used in the sector, and many experiences do not use any method. This lack is critical, as whenever the aim is to evaluate the degree of awareness of security issues, the implementation of evaluation systems is indispensable to make known the methods used in the literature to evaluate the effectiveness of the experiences and represent support for future research. This research identifies developments in XR experiences and analyzes the validation methods through a systematic review using the PRISMA methodology. It identifies two evaluation methods, objective and subjective, which are each broken down into four categories. The results show the types of evaluation, safety-related purposes, and safety application objectives used by the database classification.

A Primer on the Factories of the Future

In a dynamic and rapidly changing world, customers’ often conflicting demands have continued to evolve, outstripping the ability of the traditional factory to address modern-day production challenges. To fix these challenges, several manufacturing paradigms have been proposed. Some of these have monikers such as the smart factory, intelligent factory, digital factory, and cloud-based factory. Due to a lack of consensus on general nomenclature, the term Factory of the Future (or Future Factory) has been used in this paper as a collective euphemism for these paradigms. The Factory of the Future constitutes a creative convergence of multiple technologies, techniques, and capabilities that represent a significant change in current production capabilities, models, and practices. Using the semi-narrative research methodology in concert with the snowballing approach, the authors reviewed the open literature to understand the organizing principles behind the most common smart manufacturing paradigms with a view to developing a creative reference that articulates their shared characteristics and features under a collective lingua franca, viz., Factory of the Future. Serving as a review article and a reference monograph, the paper details the meanings, characteristics, technological framework, and applications of the modern factory and its various connotations. Amongst other objectives, it characterizes the next-generation factory and provides an overview of reference architectures/models that guide their structured development and deployment. Three advanced communication technologies capable of advancing the goals of the Factory of the Future and rapidly scaling advancements in the field are discussed. It was established that next-generation factories would be data rich environments. The realization of their ultimate value would depend on the ability of stakeholders to develop the appropriate infrastructure to extract, store, and process data to support decision making and process optimization.

AR-AI Tools as a Response to High Employee Turnover and Shortages in Manufacturing during Regular, Pandemic, and War Times

The world faces the continuously increasing issue of a lack of skilled employees, staff migration, and turnover. It is strengthened by unexpected situations such as wars, pandemics, and other civilization crises. Solutions are sought and researched in various branches of industry and academia, including engineering, social sciences, management, and political and computer sciences. From the viewpoint of this paper, this is a side topic of Industry 4.0 and, more specifically, sustainability in working environments, and the issue is related to production employees who perform manual operations. Some of the tasks cannot be carried out under robotization or automation; therefore, novel human-work support tools are expected. This paper presents such highly demanded support tools related to augmented reality (AR) and artificial intelligence (AI). First, a panoramic literature review is given. Secondly, the authors explain the main objective of the presented contribution. Then the authors’ achievements are described—the R&D focus on such solutions and the introduction of the developed tools that are based on AR and AI. Benefits connected to the AR-AI technology applications are presented in terms of both time savings with the tool usage and job simplification, enabling inexperienced, unskilled, or less skilled employees to perform the work in the selected manual production processes.

Review on visualization technology in simulation training system for major natural disasters

Major natural disasters have occurred frequently in the last few years, resulting in increased loss of life and economic damage. Most emergency responders do not have first-hand experience with major natural disasters, and thus, there is an urgent need for pre-disaster training. Due to the scenes unreality of traditional emergency drills, the failure to appeal to the target audience and the novel coronavirus pandemic, people are forced to maintain safe social distancing. Therefore, it is difficult to carry out transregional or transnational emergency drills in many countries under the lockdown. There is an increasing demand for simulation training systems that use virtual reality, augmented reality, and mixed reality visualization technologies to simulate major natural disasters. The simulation training system related to natural disasters provides a new way for popular emergency avoidance science education and emergency rescue personnel to master work responsibilities and improve emergency response capabilities. However, to our knowledge, there is no overview of the simulation training system for major natural disasters. Hence, this paper uncovers the visualization techniques commonly used in simulation training systems, and compares, analyses and summarizes the architecture and functions of the existing simulation training systems for different emergency phases of common natural disasters. In addition, the limitations of the existing simulation training system in practical applications and future development directions are discussed to provide reference for relevant researchers to better understand the modern simulation training system.

Robust Assembly Assistance Using Informed Tree Search with Markov Chains

Manual work accounts for one of the largest workgroups in the European manufacturing sector, and improving the training capacity, quality, and speed brings significant competitive benefits to companies. In this context, this paper presents an informed tree search on top of a Markov chain that suggests possible next assembly steps as a key component of an innovative assembly training station for manual operations. The goal of the next step suggestions is to provide support to inexperienced workers or to assist experienced workers by providing choices for the next assembly step in an automated manner without the involvement of a human trainer on site. Data stemming from 179 experiment participants, 111 factory workers, and 68 students, were used to evaluate different prediction methods. From our analysis, Markov chains fail in new scenarios and, therefore, by using an informed tree search to predict the possible next assembly step in such situations, the prediction capability of the hybrid algorithm increases significantly while providing robust solutions to unseen scenarios. The proposed method proved to be the most efficient for next assembly step prediction among all the evaluated predictors and, thus, the most suitable method for an adaptive assembly support system such as for manual operations in industry.

Data-Centric Architecture for Self-Driving Laboratories with Autonomous Discovery of New Nanomaterials

Artificial intelligence (AI) approaches continue to spread in almost every research and technology branch. However, a simple adaptation of AI methods and algorithms successfully exploited in one area to another field may face unexpected problems. Accelerating the discovery of new functional materials in chemical self-driving laboratories has an essential dependence on previous experimenters' experience. Self-driving laboratories help automate and intellectualize processes involved in discovering nanomaterials with required parameters that are difficult to transfer to AI-driven systems straightforwardly. It is not easy to find a suitable design method for self-driving laboratory implementation. In this case, the most appropriate way to implement is by creating and customizing a specific adaptive digital-centric automated laboratory with a data fusion approach that can reproduce a real experimenter's behavior. This paper analyzes the workflow of autonomous experimentation in the self-driving laboratory and distinguishes the core structure of such a laboratory, including sensing technologies. We propose a novel data-centric research strategy and multilevel data flow architecture for self-driving laboratories with the autonomous discovery of new functional nanomaterials.

Controlling an Industrial Robot Using a Graphic Tablet in Offline and Online Mode

The article presents the possibility of using a graphics tablet to control an industrial robot. The paper presents elements of software development for offline and online control of a robot. The program for the graphic tablet and the operator interface was developed in C# language in Visual Studio environment, while the program controlling the industrial robot was developed in RAPID language in the RobotStudio environment. Thanks to the development of a digital twin of the real robotic workstation, tests were carried out on the correct functioning of the application in offline mode (without using the real robot). The obtained results were verified in online mode (on a real production station). The developed computer programmes have a modular structure, which makes it possible to easily adapt them to one’s needs. The application allows for changing the parameters of the robot and the parameters of the path drawing. Tests were carried out on the influence of the sampling frequency and the tool diameter on the quality of the reconstructed trajectory of the industrial robot. The results confirmed the correctness of the application. Thanks to the new method of robot programming, it is possible to quickly modify the path by the operator, without the knowledge of robot programming languages. Further research will focus on analyzing the influence of screen resolution and layout scale on the accuracy of trajectory generation.