Offline and Real-Time Implementation of a Terrain Classification Pipeline for Pushrim-Activated Power-Assisted Wheelchairs

Pushrim-activated power-assisted wheelchairs (PAPAWs) are assistive technologies that provide propulsion assist to wheelchair users and enable access to various indoor and outdoor terrains. Therefore, it is beneficial to use PAPAW controllers that adapt to different terrain conditions. To achieve this objective, terrain classification techniques can be used as an integral part of the control architecture. Previously, the feasibility of using learning-based terrain classification models was investigated for offline applications. In this paper, we examine the effects of three model parameters (i.e., feature characteristics, terrain types, and the length of data segments) on offline and real-time classification accuracy. Our findings revealed that Random Forest classifiers are computationally efficient and can be used effectively for real-time terrain classification. These classifiers have the highest performance accuracy when used with a combination of time- and frequency-domain features. Additionally, we found that increasing the number of data points used for terrain estimation improves the prediction accuracy. Finally, our results revealed that classification accuracy can be improved by considering terrains with similar characteristics under one umbrella group. These findings can contribute to the development of real-time adaptive controllers that enhance PAPAW usability on different terrains.

Learning-Based Analysis of a New Wearable 3D Force System Data to Classify the Underlying Surface of a Walking Robot

Biped humanoid robots that operate in real-world environments need to be able to physically recognize different floors to best adapt their gait. In this work, we describe the preparation of a dataset of contact forces obtained with eight force tactile sensors for determining the underlying surface of a walking robot. The data is acquired for four floors with different coefficient of friction, and different robot gaits and speeds. To classify the different floors, the data is used as input for two common computational intelligence techniques (CITs): Artificial neural network (ANN) and extreme learning machine (ELM). After optimizing the parameters for both CITs, a good mapping between inputs and targets is achieved with classification accuracies of about 99%.

Deep Green Diagnostics: Urban Green Space Analysis Using Deep Learning and Drone Images

Nowadays, more than half of the world’s population lives in urban areas, and this number continues increasing. Consequently, there are more and more scientific publications that analyze health problems of people associated with living in these highly urbanized locations. In particular, some of the recent work has focused on relating people’s health to the quality and quantity of urban green areas. In this context, and considering the huge amount of land area in large cities that must be supervised, our work seeks to develop a deep learning-based solution capable of determining the level of health of the land and to assess whether it is contaminated. The main purpose is to provide health institutions with software capable of creating updated maps that indicate where these phenomena are presented, as this information could be very useful to guide public health goals in large cities. Our software is released as open source code, and the data used for the experiments presented in this paper are also freely available.

What Lies Beneath One’s Feet? Terrain Classification Using Inertial Data of Human Walk

The objective of this study was to investigate if the inertial data collected from normal human walk can be used to reveal the underlying terrain types. For this purpose, we recorded the gait patterns of normal human walk on six different terrain types with variation in hardness and friction using body mounted inertial sensors. We collected accelerations and angular velocities of 40 healthy subjects with two smartphones embedded inertial measurement units (MPU-6500) attached at two different body locations (chest and lower back). The recorded data were segmented with stride based segmentation approach and 194 tempo-spectral features were computed for each stride. We trained two machine learning classifiers, namely random forest and support vector machine, and cross validated the results with 10-fold cross-validation strategy. The classification tasks were performed on indoor–outdoor terrains, hard–soft terrains, and a combination of binary, ternary, quaternary, quinary and senary terrains. From the experimental results, the classification accuracies of 97% and 92% were achieved for indoor–outdoor and hard–soft terrains, respectively. The classification results for binary, ternary, quaternary, quinary and senary class classification were 96%, 94%, 92%, 90%, and 89%, respectively. These results demonstrate that the stride data collected with the low-level signals of a single IMU can be used to train classifiers and predict terrain types with high accuracy. Moreover, the problem at hand can be solved invariant of sensor type and sensor location.

Deep Multi-Layer Perception Based Terrain Classification for Planetary Exploration Rovers

Accurate classification and identification of the detected terrain is the basis for the long-distance patrol mission of the planetary rover. But terrain measurement based on vision and radar is subject to conditions such as light changes and dust storms. In this paper, under the premise of not increasing the sensor load of the existing rover, a terrain classification and recognition method based on vibration is proposed. Firstly, the time-frequency domain transformation of vibration information is realized by fast Fourier transform (FFT), and the characteristic representation of vibration information is given. Secondly, a deep neural network based on multi-layer perception is designed to realize classification of different terrains. Finally, combined with the Jackal unmanned vehicle platform, the XQ unmanned vehicle platform, and the vibration sensor, the terrain classification comparison test based on five different terrains was completed. The results show that the proposed algorithm has higher classification accuracy, and different platforms and running speeds have certain influence on the terrain classification at the same time, which provides support for subsequent practical applications.

Terrain Awareness Using a Tracked Skid-Steering Vehicle With Passive Independent Suspensions

This paper presents a novel approach for terrain characterization based on a tracked skid-steer vehicle with a passive independent suspensions system. A set of physics-based parameters is used to characterize the terrain properties: drive motor electrical currents, the equivalent track, the power spectral density for the vertical accelerations and motor currents. Based on this feature set, the system predicts the type of terrain that the robot traverses. A wide set of experimental results acquired on various surfaces are provided to verify the study in the field, proving its effectiveness for application in autonomous robots.

SURF-BRISK–Based Image Infilling Method for Terrain Classification of a Legged Robot

In this study, we propose adaptive locomotion for an autonomous multilegged walking robot, an image infilling method for terrain classification based on a combination of speeded up robust features, and binary robust invariant scalable keypoints (SURF-BRISK). The terrain classifier is based on the bag-of-words (BoW) model and SURF-BRISK, both of which are fast and accurate. The image infilling method is used for identifying terrain with obstacles and mixed terrain; their features are magnified to help with recognition of different complex terrains. Local image infilling is used to improve low accuracy caused by obstacles and super-pixel image infilling is employed for mixed terrain. A series of experiments including classification of terrain with obstacles and mixed terrain were conducted and the obtained results show that the proposed method can accurately identify all terrain types and achieve adaptive locomotion.

Deep spatiotemporal models for robust proprioceptive terrain classification

Terrain classification is a critical component of any autonomous mobile robot system operating in unknown real-world environments. Over the years, several proprioceptive terrain classification techniques have been introduced to increase robustness or act as a fallback for traditional vision based approaches. However, they lack widespread adaptation due to various factors that include inadequate accuracy, robustness and slow run-times. In this paper, we use vehicle-terrain interaction sounds as a proprioceptive modality and propose a deep long-short term memory based recurrent model that captures both the spatial and temporal dynamics of such a problem, thereby overcoming these past limitations. Our model consists of a new convolution neural network architecture that learns deep spatial features, complemented with long-short term memory units that learn complex temporal dynamics. Experiments on two extensive datasets collected with different microphones on various indoor and outdoor terrains demonstrate state-of-the-art performance compared to existing techniques. We additionally evaluate the performance in adverse acoustic conditions with high-ambient noise and propose a noise-aware training scheme that enables learning of more generalizable models that are essential for robust real-world deployments.

Geometrical analysis of profile of certain heavy terrain sections exerting dynamic loads on the chassis components of off-road vehicles

The paper presents the results of measurements concerned with the changes of profile height observed on selected heavy terrain road sections. Based on the results of direct measurements, primary indicators and amplitude frequency characteristics have been determined. The characteristics of these roads profiles have been presented as a function of path frequency, and also as a function of the vehicle wheels’ excitation frequency observed while driving at a certain, constant speed. The results presented serve as a source of information on specific heavy terrain road sections exerting dynamic loads on the vehicle’s chassis components while driving in off-road conditions.

Dynamic modeling and parameter estimation for traction, rolling, and lateral wheel forces to enhance mobile robot trajectory tracking

Studying wheel and ground interaction during motion has the potential to increase the performance of localization, navigation, and trajectory tracking control of a mobile robot. In this paper, a differential mobile robot is modeled in a way that (traction, rolling, and lateral) wheel forces are included in the overall system dynamics. Lateral wheel forces are included in the mathematical model together with traction and rolling forces. A least square parameter estimation process is proposed to estimate the parameters of the wheel forces. In order to implement the proposed methodologies, an experimental setup is used. The setup contains a differentially driven mobile robot, a specially constructed test surface, and a camera system attached at the top of surface for obtaining ground truth. Models having one or more wheel forces are simulated to find the most realistic model. Simulation results are verified by experiments.

Non-parametric Learning to Aid Path Planning over Slopes

In this paper we address the problem of closing the loop from perception to action selection for unmanned ground vehicles, with a focus on navigating slopes. A new non-parametric learning technique is presented to generate a mobility representation where the maximum feasible speed is used as a criterion to classify the world. The inputs to the algorithm are terrain gradients derived from an elevation map and past observations of wheel slip. It is argued that such a representation can aid in path planning with improved selection of vehicle heading and velocity in off-road slopes. In addition, an information theoretic test is proposed to validate a chosen proprioceptive representation (such as slip) for mobility map generation. Results of mobility map generation and its benefits to path planning are shown.

A Comparison of RBF Neural Network Training Algorithms for Inertial Sensor Based Terrain Classification

This paper introduces a comparison of training algorithms of radial basis function (RBF) neural networks for classification purposes. RBF networks provide effective solutions in many science and engineering fields. They are especially popular in the pattern classification and signal processing areas. Several algorithms have been proposed for training RBF networks. The Artificial Bee Colony (ABC) algorithm is a new, very simple and robust population based optimization algorithm that is inspired by the intelligent behavior of honey bee swarms. The training performance of the ABC algorithm is compared with the Genetic algorithm, Kalman filtering algorithm and gradient descent algorithm. In the experiments, not only well known classification problems from the UCI repository such as the Iris, Wine and Glass datasets have been used, but also an experimental setup is designed and inertial sensor based terrain classification for autonomous ground vehicles was also achieved. Experimental results show that the use of the ABC algorithm results in better learning than those of others.

Robot Navigation in Multi-terrain Outdoor Environments

This paper presents a methodology for motion planning in outdoor environments that takes into account specific characteristics of the terrain. Instead of decomposing the robot configuration space into “free” and “occupied”, we consider the existence of several regions with different navigation costs. In this paper, costs are determined experimentally by navigating the robot through the regions and measuring the influence of the terrain on its motion. We measure the robot's vertical acceleration, which reflects the terrain roughness. The paper presents a hybrid (discrete—continuous) approach to guide and control the robot. After decomposing the map into triangular cells, a path planning algorithm is used to determine a discrete sequence of cells that minimizes the navigation cost. Robot control is accomplished by a fully continuous vector field that drives the robot through the sequence of triangular cells. This vector field allows smooth robot trajectories from any position inside the sequence to the goal, even for a small number of large cells. Moreover, the vector field is terrain dependent in the sense it changes the robot velocity according to the characteristics of the terrain. Experimental results with a differential driven, all-terrain mobile robot illustrate the proposed approach.

A study of visual and tactile terrain classification and classifier fusion for planetary exploration rovers

Knowledge of the physical properties of terrain surrounding a planetary exploration rover can be used to allow a rover system to fully exploit its mobility capabilities. Terrain classification methods provide semantic descriptions of the physical nature of a given terrain region. These descriptions can be associated with nominal numerical physical parameters, and/or nominal traversability estimates, to improve mobility prediction accuracy. Here we study the performance of multisensor classification methods in the context of Mars surface exploration. The performance of two classification algorithms for color, texture, and range features are presented based on maximum likelihood estimation and support vector machines. In addition, a classification method based on vibration features derived from rover wheel–terrain interaction is briefly described. Two techniques for merging the results of these “low-level” classifiers are presented that rely on Bayesian fusion and meta-classifier fusion. The performance of these algorithms is studied using images from NASA's Mars Exploration Rover mission and through experiments on a four-wheeled test-bed rover operating in Mars-analog terrain. Also a novel approach to terrain sensing based on fused tactile and visual features is presented. It is shown that accurate terrain classification can be achieved via classifier fusion from visual and tactile features.