Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis

Successful operation of a teleoperated robot depends on a well-designed control scheme to translate human motion into robot motion; however, a single control scheme may not be suitable for all users. On the other hand, individual personalization of control schemes may be infeasible for designers to produce. In this paper, we present a method by which users may be classified into groups with mutually compatible control scheme preferences. Users are asked to demonstrate freehand motions to control a simulated robot in a virtual reality environment. Hand pose data is captured and compared with other users using SLAM trajectory similarity analysis techniques. The resulting pairwise trajectory error metrics are used to cluster participants based on their control motions, without foreknowledge of the number or types of control scheme preferences that may exist. The clusters identified for two different robots shows that a small number of clusters form stably for each case, each with its own control scheme paradigm. Survey data from participants validates that the clusters identified through this method correspond to the participants' control scheme rationales, and also identify nuances in participant control scheme descriptions that may not be obvious to designers relying only on participant explanations of their preferences.

Survey on the Developments of Unmanned Marine Vehicles: Intelligence and Cooperation

With the recent development of artificial intelligence (AI) and information and communication technology, manned vehicles operated by humans used on the ground, air, and sea are evolving into unmanned vehicles (UVs) that operate without human intervention. In particular, unmanned marine vehicles (UMVs), including unmanned underwater vehicles (UUVs) and unmanned surface vehicles (USVs), have the potential to complete maritime tasks that are unachievable for manned vehicles, lower the risk of man power, raise the power required to carry out military missions, and reap huge economic benefits. The aim of this review is to identify past and current trends in UMV development and present insights into future UMV development. The review discusses the potential benefits of UMVs, including completing maritime tasks that are unachievable for manned vehicles, lowering the risk of human intervention, and increasing power for military missions and economic benefits. However, the development of UMVs has been relatively tardy compared to that of UVs used on the ground and in the air due to adverse environments for UMV operation. This review highlights the challenges in developing UMVs, particularly in adverse environments, and the need for continued advancements in communication and networking technologies, navigation and sound exploration technologies, and multivehicle mission planning technologies to improve UMV cooperation and intelligence. Furthermore, the review identifies the importance of incorporating AI and machine learning technologies in UMVs to enhance their autonomy and ability to perform complex tasks. Overall, this review provides insights into the current state and future directions for UMV development.

Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System

In mixed-reality (MR) telecollaboration, the local environment is remotely presented to a remote user wearing a virtual reality (VR) head-mounted display (HMD) via a video capture device. However, remote users frequently face challenges in naturally and actively manipulating their viewpoints. In this paper, we propose a telepresence system with viewpoint control, which involves a robotic arm equipped with a stereo camera in the local environment. This system enables remote users to actively and flexibly observe the local environment by moving their heads to manipulate the robotic arm. Additionally, to solve the problem of the limited field of view of the stereo camera and limited movement range of the robotic arm, we propose a 3D reconstruction method combined with a stereo video field-of-view enhancement technique to guide remote users to move within the movement range of the robotic arm and provide them with a larger range of local environment perception. Finally, a mixed-reality telecollaboration prototype was built, and two user studies were conducted to evaluate the overall system. User study A evaluated the interaction efficiency, system usability, workload, copresence, and user satisfaction of our system from the remote user's perspective, and the results showed that our system can effectively improve the interaction efficiency while achieving a better user experience than two traditional view-sharing techniques based on 360 video and based on the local user's first-person view. User study B evaluated our MR telecollaboration system prototype from both the remote-user side and the local-user side as a whole, providing directions and suggestions for the subsequent design and improvement of our mixed-reality telecollaboration system.

Facial Emotion Recognition from an Unmanned Flying Social Robot for Home Care of Dependent People

This work is part of an ongoing research project to develop an unmanned flying social robot to monitor dependants at home in order to detect the person’s state and bring the necessary assistance. In this sense, this paper focuses on the description of a virtual reality (VR) simulation platform for the monitoring process of an avatar in a virtual home by a rotatory-wing autonomous unmanned aerial vehicle (UAV). This platform is based on a distributed architecture composed of three modules communicated through the message queue telemetry transport (MQTT) protocol: the UAV Simulator implemented in MATLAB/Simulink, the VR Visualiser developed in Unity, and the new emotion recognition (ER) system developed in Python. Using a face detection algorithm and a convolutional neural network (CNN), the ER System is able to detect the person’s face in the image captured by the UAV’s on-board camera and classify the emotion among seven possible ones (surprise; fear; happiness; sadness; disgust; anger; or neutral expression). The experimental results demonstrate the correct integration of this new computer vision module within the VR platform, as well as the good performance of the designed CNN, with around 85% in the F1-score, a mean of the precision and recall of the model. The developed emotion detection system can be used in the future implementation of the assistance UAV that monitors dependent people in a real environment, since the methodology used is valid for images of real people.