LPWAN-Based Real-Time 2D SLAM and Object Localization for Teleoperation Robot Control

In this study, the teleoperation robot control on a mobile robot with 2D SLAM and object localization using LPWAN is proposed. The mobile robot is a technology gaining popularity due to flexibility and robustness in a variety of terrains. In search and rescue activities, the mobile robots can be used to perform some missions, assist and preserve human life. However, teleoperation control becomes a challenging problem for this implementation. The robust wireless communication not only allows the operator to stay away from dangerous area, but also increases the mobility of the mobile robot itself. Most of teleoperation mobile robots use Wi-Fi having high-bandwidth, yet short communication range. LoRa as LPWAN, on the other hand, has much longer range but low-bandwidth communication speed. Therefore, the combination of them complements each other’s weaknesses. The use of a two-LoRa configuration also enhances the teleoperation capabilities. All information from the mobile robot can be sent to the PC controller in relatively fast enough for real-time SLAM implementation. Furthermore, the mobile robot is also capable of real-time object detection, localization, and transmitting images. Another problem of LoRa communication is a timeout. We apply timeout recovery algorithms to handle this issue, resulting in more stable data. All data have been confirmed by real-time trials and the proposed method can approach the Wi-Fi performance with a low waiting time or delay.

Towards a balancing safety against performance approach in human–robot co-manipulation for door-closing emergencies

Telemanipulation in power stations commonly require robots first to open doors and then gain access to a new workspace. However, the opened doors can easily close by disturbances, interrupt the operations, and potentially lead to collision damages. Although existing telemanipulation is a highly efficient master–slave work pattern due to human-in-the-loop control, it is not trivial for a user to specify the optimal measures to guarantee safety. This paper investigates the safety-critical motion planning and control problem to balance robotic safety against manipulation performance during work emergencies. Based on a dynamic workspace released by door-closing, the interactions between the workspace and robot are analyzed using a partially observable Markov decision process, thereby making the balance mechanism executed as belief tree planning. To act the planning, apart from telemanipulation actions, we clarify other three safety-guaranteed actions: on guard, defense and escape for self-protection by estimating collision risk levels to trigger them. Besides, our experiments show that the proposed method is capable of determining multiple solutions for balancing robotic safety and work efficiency during telemanipulation tasks.

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

This article presents a collection of multimodal raw data captured from a manned all-terrain vehicle in the course of two realistic outdoor search and rescue (SAR) exercises for actual emergency responders conducted in Málaga (Spain) in 2018 and 2019: the UMA-SAR dataset. The sensor suite, applicable to unmanned ground vehicles (UGVs), consisted of overlapping visible light (RGB) and thermal infrared (TIR) forward-looking monocular cameras, a Velodyne HDL-32 three-dimensional (3D) lidar, as well as an inertial measurement unit (IMU) and two global positioning system (GPS) receivers as ground truth. Our mission was to collect a wide range of data from the SAR domain, including persons, vehicles, debris, and SAR activity on unstructured terrain. In particular, four data sequences were collected following closed-loop routes during the exercises, with a total path length of 5.2 km and a total time of 77 min. In addition, we provide three more sequences of the empty site for comparison purposes (an extra 4.9 km and 46 min). Furthermore, the data is offered both in human-readable format and as rosbag files, and two specific software tools are provided for extracting and adapting this dataset to the users’ preference. The review of previously published disaster robotics repositories indicates that this dataset can contribute to fill a gap regarding visual and thermal datasets and can serve as a research tool for cross-cutting areas such as multispectral image fusion, machine learning for scene understanding, person and object detection, and localization and mapping in unstructured environments. The full dataset is publicly available at: www.uma.es/robotics-and-mechatronics/sar-datasets .

Particle swarm optimization algorithm for the optimization of rescue task allocation with uncertain time constraints

This paper focuses on the problem of robot rescue task allocation, in which multiple robots and a global optimal algorithm are employed to plan the rescue task allocation. Accordingly, a modified particle swarm optimization (PSO) algorithm, referred to as task allocation PSO (TAPSO), is proposed. Candidate assignment solutions are represented as particles and evolved using an evolutionary process. The proposed TAPSO method is characterized by a flexible assignment decoding scheme to avoid the generation of unfeasible assignments. The maximum number of successful tasks (survivors) is considered as the fitness evaluation criterion under a scenario where the survivors’ survival time is uncertain. To improve the solution, a global best solution update strategy, which updates the global best solution depends on different phases so as to balance the exploration and exploitation, is proposed. TAPSO is tested on different scenarios and compared with other counterpart algorithms to verify its efficiency.

Technologies Enabling Situational Awareness During Disaster Response: A Systematic Review

Situational awareness (SA) is critical to mobilizing a rapid, efficient, and effective response to disasters. Limited by time and resources, response agencies must make decisions about rapidly evolving situations, which requires the collection, analysis, and sharing of actionable information across a complex landscape. Emerging technologies, if appropriately applied, can enhance SA and enable responders to make quicker, more accurate decisions. The aim of this systematic review is to identify technologies that can improve SA and assist decision-making across the United States Government and the domestic and international agencies they support during disaster response operations. A total of 1459 articles and 36 after-action reports were identified during literature searches. Following the removal of duplicates and application of inclusion/exclusion criteria, 302 articles and after-action reports were included in the review. Our findings suggest SA is constrained primarily due to unreliable and significantly delayed communications, time-intensive data analysis and visualization, and a lack of interoperable sensor networks and other capabilities providing data to shared platforms. Many of these challenges could be addressed by existing technologies. Bridging the divide between research and development efforts and the operational needs of response agencies should be prioritized.

3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.