Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Monocular camera and laser based semantic mapping system with temporal-spatial data association for indoor mobile robots

In the future, the goal of service robots is to operate in human-centric indoor environments, requiring close cooperation with humans. In order to enable the robot to perform various interactive tasks, it is necessary for robots to perceive and understand environments from a human perspective. Semantic map is an augmented representation of the environment, containing both geometric information and high-level qualitative features. It can help the robot to comprehensively understand the environment and bridge the gap in human-robot interaction. In this paper, we propose a unified semantic mapping system for indoor mobile robots. This system utilizes the techniques of scene classification and object detection to construct semantic representations of indoor environments by fusing the data of a camera and a laser. In order to improve the accuracy of semantic mapping, the temporal-spatial correlation of semantics is leveraged to realize data association of semantic maps. Also, the proposed semantic mapping system is scalable and portable, which can be applied to different indoor scenarios. The proposed system was evaluated with collected datasets captured in indoor environments. Extensive experimental results indicate that the proposed semantic mapping system exhibits great performance in the robustness and accuracy of semantic mapping.

Deep Learning-Based Object Detection, Localisation and Tracking for Smart Wheelchair Healthcare Mobility

This paper deals with the development of an Advanced Driver Assistance System (ADAS) for a smart electric wheelchair in order to improve the autonomy of disabled people. Our use case, built from a formal clinical study, is based on the detection, depth estimation, localization and tracking of objects in wheelchair's indoor environment, namely: door and door handles. The aim of this work is to provide a perception layer to the wheelchair, enabling this way the detection of these keypoints in its immediate surrounding, and constructing of a short lifespan semantic map. Firstly, we present an adaptation of the YOLOv3 object detection algorithm to our use case. Then, we present our depth estimation approach using an Intel RealSense camera. Finally, as a third and last step of our approach, we present our 3D object tracking approach based on the SORT algorithm. In order to validate all the developments, we have carried out different experiments in a controlled indoor environment. Detection, distance estimation and object tracking are experimented using our own dataset, which includes doors and door handles.