Outdoor Map Construction Based on Aerial Photography and Electrical Map Using Multi-Plane Laser Range Scan Data

This research is on personal mobility that estimates its self position on a sensor data map created from sensor data, acquired from laser range scan sensors and/or other sensors, and annotates various multiple items of information on a digital map. This paper describes a method of creating an edge-based grid map from both aerial photography and an electricalmap for this purpose and a way and its realization to estimate position and to construct outdoor maps from multi-plane laser range scan data on the grid map. Since threedimensional scanning is rather difficult and the scan rate is low, we used two-dimensional scanning that enables movement without slowing it down by scanning multiple horizontal and/or slanted planes. Experimental results show that the system is able to ensure the accuracy of accumulated error within 2 m by integrating aerial photography and electrical maps plus multiplane scanning.

Mobile Robot Localization Using Fuzzy Segments

This paper presents the development of a framework based on fuzzy logic for multi-sensor fusion and localization in indoor environments. Such a framework makes use of fuzzy segments to represent uncertain location information from different sources of information. Fuzzy reasoning, based on similarity interpretation from fuzzy logic, is then used to fuse the sensory information represented as fuzzy segments. This approach makes it possible to fuse vague and imprecise information from different sensors at the feature level instead of fusing raw data directly from different sources of information. The resulting fuzzy segments are used to maintain a coherent representation of the environment around the robot. Such an uncertain representation is finally used to estimate the robot position. The proposed multi-sensor fusion localization approach has been validated with a mobile platform using different range sensors.

Simultaneous localization and mapping with multimodal probability distributions

Simultaneous Localization and Mapping (SLAM) has focused on noisy but unique data associations resulting in linear Gaussian uncertainty models. However, a unique decision is often not possible using only local information, giving rise to ambiguities that have to be resolved globally during optimization. To solve this problem, the pose graph data structure is extended here by multimodal constraints modeled by mixtures of Gaussians (MoG). Furthermore, optimization methods for this novel formulation are introduced, namely (a) robust iteratively reweighted least squares, and (b) Prefilter Stochastic Gradient Descent (SGD) where a preprocessing step determines globally consistent modes before applying SGD. In addition, a variant of the Prefilter method (b) is introduced in form of (c) Prefilter Levenberg–Marquardt. The methods are compared with traditional state-of-the-art optimization methods including (d) Stochastic Gradient Descent and (e) Levenberg–Marquardt as well as (f) Particle filter SLAM and with (g) an optimal exhaustive algorithm. Experiments show that ambiguities significantly impact state-of-the-art methods, and that the novel Prefilter methods (b) and (c) perform best. This is further substantiated with experiments using real-world data. To this end, a method to generate MoG constraints from a plane-based registration algorithm is introduced and used for 3D SLAM under ambiguities.

Information-theoretic compression of pose graphs for laser-based SLAM

In graph-based simultaneous localization and mapping (SLAM), the pose graph grows over time as the robot gathers information about the environment. An ever growing pose graph, however, prevents long-term mapping with mobile robots. In this paper, we address the problem of efficient information-theoretic compression of pose graphs. Our approach estimates the mutual information between the laser measurements and the map to discard the measurements that are expected to provide only a small amount of information. Our method subsequently marginalizes out the nodes from the pose graph that correspond to the discarded laser measurements. To maintain a sparse pose graph that allows for efficient map optimization, our approach applies an approximate marginalization technique that is based on Chow–Liu trees. Our contributions allow the robot to effectively restrict the size of the pose graph. Alternatively, the robot is able to maintain a pose graph that does not grow unless the robot explores previously unobserved parts of the environment. Real-world experiments demonstrate that our approach to pose graph compression is well suited for long-term mobile robot mapping.

Scanning 3D full human bodies using Kinects

Depth camera such as Microsoft Kinect, is much cheaper than conventional 3D scanning devices, and thus it can be acquired for everyday users easily. However, the depth data captured by Kinect over a certain distance is of extreme low quality. In this paper, we present a novel scanning system for capturing 3D full human body models by using multiple Kinects. To avoid the interference phenomena, we use two Kinects to capture the upper part and lower part of a human body respectively without overlapping region. A third Kinect is used to capture the middle part of the human body from the opposite direction. We propose a practical approach for registering the various body parts of different views under non-rigid deformation. First, a rough mesh template is constructed and used to deform successive frames pairwisely. Second, global alignment is performed to distribute errors in the deformation space, which can solve the loop closure problem efficiently. Misalignment caused by complex occlusion can also be handled reasonably by our global alignment algorithm. The experimental results have shown the efficiency and applicability of our system. Our system obtains impressive results in a few minutes with low price devices, thus is practically useful for generating personalized avatars for everyday users. Our system has been used for 3D human animation and virtual try on, and can further facilitate a range of home–oriented virtual reality (VR) applications.

Multi-Focal Vision and Gaze Control Improve Navigation Performance

Multi-focal vision systems comprise cameras with various fields of view and measurement accuracies. This article presents a multi-focal approach to localization and mapping of mobile robots with active vision. An implementation of the novel concept is done considering a humanoid robot navigation scenario where the robot is visually guided through a structured environment with several landmarks. Various embodiments of multi-focal vision systems are investigated and the impact on navigation performance is evaluated in comparison to a conventional mono-focal stereo set-up. The comparative studies clearly show the benefits of multi-focal vision for mobile robot navigation: flexibility to assign the different available sensors optimally in each situation, enhancement of the visible field, higher localization accuracy, and, thus, better task performance, i.e. path following behavior of the mobile robot. It is shown that multi-focal vision may strongly improve navigation performance.

RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments

RGB-D cameras (such as the Microsoft Kinect) are novel sensing systems that capture RGB images along with per-pixel depth information. In this paper we investigate how such cameras can be used for building dense 3D maps of indoor environments. Such maps have applications in robot navigation, manipulation, semantic mapping, and telepresence. We present RGB-D Mapping, a full 3D mapping system that utilizes a novel joint optimization algorithm combining visual features and shape-based alignment. Visual and depth information are also combined for view-based loop-closure detection, followed by pose optimization to achieve globally consistent maps. We evaluate RGB-D Mapping on two large indoor environments, and show that it effectively combines the visual and shape information available from RGB-D cameras.

iSAM2: Incremental smoothing and mapping using the Bayes tree

We present a novel data structure, the Bayes tree, that provides an algorithmic foundation enabling a better understanding of existing graphical model inference algorithms and their connection to sparse matrix factorization methods. Similar to a clique tree, a Bayes tree encodes a factored probability density, but unlike the clique tree it is directed and maps more naturally to the square root information matrix of the simultaneous localization and mapping (SLAM) problem. In this paper, we highlight three insights provided by our new data structure. First, the Bayes tree provides a better understanding of the matrix factorization in terms of probability densities. Second, we show how the fairly abstract updates to a matrix factorization translate to a simple editing of the Bayes tree and its conditional densities. Third, we apply the Bayes tree to obtain a completely novel algorithm for sparse nonlinear incremental optimization, named iSAM2, which achieves improvements in efficiency through incremental variable re-ordering and fluid relinearization, eliminating the need for periodic batch steps. We analyze various properties of iSAM2 in detail, and show on a range of real and simulated datasets that our algorithm compares favorably with other recent mapping algorithms in both quality and efficiency.

Neural network-based multiple robot simultaneous localization and mapping

In this paper, a decentralized platform for simultaneous localization and mapping (SLAM) with multiple robots is developed. Each robot performs single robot view-based SLAM using an extended Kalman filter to fuse data from two encoders and a laser ranger. To extend this approach to multiple robot SLAM, a novel occupancy grid map fusion algorithm is proposed. Map fusion is achieved through a multistep process that includes image preprocessing, map learning (clustering) using neural networks, relative orientation extraction using norm histogram cross correlation and a Radon transform, relative translation extraction using matching norm vectors, and then verification of the results. The proposed map learning method is a process based on the self-organizing map. In the learning phase, the obstacles of the map are learned by clustering the occupied cells of the map into clusters. The learning is an unsupervised process which can be done on the fly without any need to have output training patterns. The clusters represent the spatial form of the map and make further analyses of the map easier and faster. Also, clusters can be interpreted as features extracted from the occupancy grid map so the map fusion problem becomes a task of matching features. Results of the experiments from tests performed on a real environment with multiple robots prove the effectiveness of the proposed solution.

Online probabilistic topological mapping

We present a novel algorithm for topological mapping, which is the problem of finding the graph structure of an environment from a sequence of measurements. Our algorithm, called Online Probabilistic Topological Mapping (OPTM), systematically addresses the problem by constructing the posterior on the space of all possible topologies given measurements. With each successive measurement, the posterior is updated incrementally using a Rao—Blackwellized particle filter. We present efficient sampling mechanisms using data-driven proposals and prior distributions on topologies that further enable OPTM’s operation in an online manner. OPTM can incorporate various sensors seamlessly, as is demonstrated by our use of appearance, laser, and odometry measurements. OPTM is the first topological mapping algorithm that is theoretically accurate, systematic, sensor independent, and online, and thus advances the state of the art significantly. We evaluate the algorithm on a robot in diverse environments.

View-based Maps

Robotic systems that can create and use visual maps in real-time have obvious advantages in many applications, from automatic driving to mobile manipulation in the home. In this paper we describe a mapping system based on retaining stereo views of the environment that are collected as the robot moves. Connections among the views are formed by consistent geometric matching of their features. Out-of-sequence matching is the key problem: how to find connections from the current view to other corresponding views in the map. Our approach uses a vocabulary tree to propose candidate views, and a strong geometric filter to eliminate false positives: essentially, the robot continually re-recognizes where it is. We present experiments showing the utility of the approach on video data, including incremental map building in large indoor and outdoor environments, map building without localization, and re-localization when lost.

Measurement selection in untracked freehand 3D ultrasound

In freehand 3D ultrasound, out-of-plane transducer motion can be estimated via speckle decorrelation instead of using a position tracking device. This approach was recently adapted to arbitrary media by predicting elevational decorrelation curves from local image statistics. However, such adaptive models tend to yield biased measurements in the presence of spatially persistent structures. To account for such failures, this paper introduces a new iterative algorithm for probabilistic fusion and selection of correlation measurements. In experiments with imagery of animal tissue, the approach yields significant accuracy improvements over alternatives which do not apply principled measurement selection.

Persistent Navigation and Mapping using a Biologically Inspired SLAM System

The challenge of persistent navigation and mapping is to develop an autonomous robot system that can simultaneously localize, map and navigate over the lifetime of the robot with little or no human intervention. Most solutions to the simultaneous localization and mapping (SLAM) problem aim to produce highly accurate maps of areas that are assumed to be static. In contrast, solutions for persistent navigation and mapping must produce reliable goal-directed navigation outcomes in an environment that is assumed to be in constant flux. We investigate the persistent navigation and mapping problem in the context of an autonomous robot that performs mock deliveries in a working office environment over a two-week period. The solution was based on the biologically inspired visual SLAM system, RatSLAM. RatSLAM performed SLAM continuously while interacting with global and local navigation systems, and a task selection module that selected between exploration, delivery, and recharging modes. The robot performed 1,143 delivery tasks to 11 different locations with only one delivery failure (from which it recovered), traveled a total distance of more than 40 km over 37 hours of active operation, and recharged autonomously a total of 23 times.

Navigating, Recognizing and Describing Urban Spaces With Vision and Lasers

In this paper we describe a body of work aimed at extending the reach of mobile navigation and mapping. We describe how running topological and metric mapping and pose estimation processes concurrently, using vision and laser ranging, has produced a full six-degree-of-freedom outdoor navigation system. It is capable of producing intricate three-dimensional maps over many kilometers and in real time. We consider issues concerning the intrinsic quality of the built maps and describe our progress towards adding semantic labels to maps via scene de-construction and labeling. We show how our choices of representation, inference methods and use of both topological and metric techniques naturally allow us to fuse maps built from multiple sessions with no need for manual frame alignment or data association.

Distributed Localization of Modular Robot Ensembles

Internal localization, the problem of estimating relative pose for each module of a modular robot, is a prerequisite for many shape control, locomotion, and actuation algorithms. In this paper, we propose a robust hierarchical approach that uses normalized cut to identify dense sub-regions with small mutual localization error, then progressively merges those sub-regions to localize the entire ensemble. Our method works well in both two and three dimensions, and requires neither exact measurements nor rigid inter-module connectors. Most of the computations in our method can be distributed effectively. The result is a robust algorithm that scales to large ensembles. We evaluate our algorithm in two- and three-dimensional simulations of scenarios with up to 10,000 modules.

Factoring the Mapping Problem: Mobile Robot Map-building in the Hybrid Spatial Semantic Hierarchy

We propose a factored approach to mobile robot map-building that handles qualitatively different types of uncertainty by combining the strengths of topological and metrical approaches. Our framework is based on a computational model of the human cognitive map; thus it allows robust navigation and communication within several different spatial ontologies. This paper focuses exclusively on the issue of map-building using the framework.

Our approach factors the mapping problem into natural sub-goals: building a metrical representation for local small-scale spaces; finding a topological map that represents the qualitative structure of large-scale space; and (when necessary) constructing a metrical representation for large-scale space using the skeleton provided by the topological map. We describe how to abstract a symbolic description of the robot’s immediate surround from local metrical models, how to combine these local symbolic models in order to build global symbolic models, and how to create a globally consistent metrical map from a topological skeleton by connecting local frames of reference.

Experimental Analysis of Sample-Based Maps for Long-Term SLAM

This paper presents a system for long-term SLAM (simultaneous localization and mapping) by mobile service robots and its experimental evaluation in a real dynamic environment. To deal with the stability-plasticity dilemma (the trade-off between adaptation to new patterns and preservation of old patterns), the environment is represented by multiple timescales simultaneously (five in our experiments). A sample-based representation is proposed, where older memories fade at different rates depending on the timescale and robust statistics are used to interpret the samples. The dynamics of this representation are analyzed in a five-week experiment, measuring the relative influence of short- and long-term memories over time and further demonstrating the robustness of the approach.