Vision-Based SLAM System for Unmanned Aerial Vehicles

Assessment of Accuracy in Unmanned Aerial Vehicle (UAV) Pose Estimation with the REAL-Time Kinematic (RTK) Method on the Example of DJI Matrice 300 RTK

The growing possibilities offered by unmanned aerial vehicles (UAV) in many areas of life, in particular in automatic data acquisition, spur the search for new methods to improve the accuracy and effectiveness of the acquired information. This study was undertaken on the assumption that modern navigation receivers equipped with real-time kinematic positioning software and integrated with UAVs can considerably improve the accuracy of photogrammetric measurements. The research hypothesis was verified during field measurements with the use of a popular Enterprise series drone. The problems associated with accurate UAV pose estimation were identified. The main aim of the study was to perform a qualitative assessment of the pose estimation accuracy of a UAV equipped with a GNSS RTK receiver. A test procedure comprising three field experiments was designed to achieve the above research goal: an analysis of the stability of absolute pose estimation when the UAV is hovering over a point, and analyses of UAV pose estimation during flight along a predefined trajectory and during continuous flight without waypoints. The tests were conducted in a designated research area. The results were verified based on direct tachometric measurements. The qualitative assessment was performed with the use of statistical methods. The study demonstrated that in a state of apparent stability, horizontal deviations of around 0.02 m occurred at low altitudes and increased with a rise in altitude. Mission type significantly influences pose estimation accuracy over waypoints. The results were used to verify the accuracy of the UAV's pose estimation and to identify factors that affect the pose estimation accuracy of an UAV equipped with a GNSS RTK receiver. The present findings provide valuable input for developing a new method to improve the accuracy of measurements performed with the use of UAVs.

RTOB SLAM: Real-Time Onboard Laser-Based Localization and Mapping

RTOB-SLAM is a new low-computation framework for real-time onboard simultaneous localization and mapping (SLAM) and obstacle avoidance for autonomous vehicles. A low-resolution 2D laser scanner is used and a small form-factor computer perform all computations onboard. The SLAM process is based on laser scan matching with the iterative closest point technique to estimate the vehicle’s current position by aligning the new scan with the map. This paper describes a new method which uses only a small subsample of the global map for scan matching, which improves the performance and allows for a map to adapt to a dynamic environment by partly forgetting the past. A detailed comparison between this method and current state-of-the-art SLAM frameworks is given, together with a methodology to choose the parameters of the RTOB-SLAM. The RTOB-SLAM has been implemented in ROS and perform well in various simulations and real experiments.

A Comprehensive Review of Unmanned Aerial Vehicle Attacks and Neutralization Techniques

Unmanned Aerial Vehicles (UAV) have revolutionized the aircraft industry in this decade. UAVs are now capable of carrying out remote sensing, remote monitoring, courier delivery, and a lot more. A lot of research is happening on making UAVs more robust using energy harvesting techniques to have a better battery lifetime, network performance and to secure against attackers. UAV networks are many times used for unmanned missions. There have been many attacks on civilian, military, and industrial targets that were carried out using remotely controlled or automated UAVs. This continued misuse has led to research in preventing unauthorized UAVs from causing damage to life and property. In this paper, we present a literature review of UAVs, UAV attacks, and their prevention using anti-UAV techniques. We first discuss the different types of UAVs, the regulatory laws for UAV activities, their use cases, recreational, and military UAV incidents. After understanding their operation, various techniques for monitoring and preventing UAV attacks are described along with case studies.

Automated Geolocation in Urban Environments Using a Simple Camera-Equipped Unmanned Aerial Vehicle: A Rapid Mapping Surveying Alternative?

GNSS positioning accuracy can be degraded in areas where the surrounding object geometry and morphology interacts with the GNSS signals. Specifically, urban environments pose challenges to precise GNSS positioning because of signal interference or interruptions. Also, non-GNSS surveying methods, including total stations and laser scanners, involve time consuming practices in the field and costly equipment. The present study proposes the use of an Unmanned Aerial Vehicle (UAV) for autonomous rapid mapping that resolves the problem of localization for the drone itself by acquiring location information of characteristic points on the ground in a local coordinate system using simultaneous localization and mapping (SLAM) and vision algorithms. A common UAV equipped with a camera and at least a single known point, are enough to produce a local map of the scene and to estimate the relative coordinates of pre-defined ground points along with an additional arbitrary point cloud. The resulting point cloud is readily measurable for extracting and interpreting geometric information from the area of interest. Under two novel optimization procedures performing line and plane alignment of the UAV-camera-measured point geometries, a set of experiments determines that the localization of a visual point in distances reaching 15 m from the origin, delivered a level of accuracy under 50 cm. Thus, a simple UAV with an optical sensor and a visual marker, prove quite promising and cost-effective for rapid mapping and point localization in an unknown environment.

Robust Vision-Based Control of a Rotorcraft UAV for Uncooperative Target Tracking

This paper investigates the problem of using an unmanned aerial vehicle (UAV) to track and hover above an uncooperative target, such as an unvisited area or an object that is newly discovered. A vision-based strategy integrating the metrology and the control is employed to achieve target tracking and hovering observation. First, by introducing a virtual camera frame, the reprojected image features can change independently of the rotational motion of the vehicle. The image centroid and an optimal observation area on the virtual image plane are exploited to regulate the relative horizontal and vertical distance. Then, the optic flow and gyro measurements are utilized to estimate the relative UAV-to-target velocity. Further, a gain-switching proportional-derivative (PD) control scheme is proposed to compensate for the external interference and model uncertainties. The closed-loop system is proven to be exponentially stable, based on the Lyapunov method. Finally, simulation results are presented to demonstrate the effectiveness of the proposed vision-based strategy in both hovering and tracking scenarios.

Modification of MTEA-Based Temperature Drift Error Compensation Model for MEMS-Gyros

The conventional temperature drift error (TDE) compensation model cannot decouple temperature dependence of Si-based materials because temperature correlated quantities (TCQ) have not been obtained comprehensively, and Micro-Electro-Mechanical System gyros’ (MEMS-gyros’) environmental adaptability is reduced in diverse, complicated conditions. The study presents modification of TDE compensation model of MEMS-gyros based on microstructure thermal effect analysis (MTEA). First, Si-based materials’ temperature dependence was studied in microstructure with thermal expansion effect and TCQ that determines the structural deformation were extracted to modify the conventional model, including temperature variation and its square. Second, a precise TDE test method was formed by analyzing heat conduction process between MEMS-gyros and thermal chamber, and temperature experiments were designed and conducted. Third, the modified model’s parameters were identified based on radical basis function artificial neural network (RBF ANN) and its performance was evaluated. Last, the conventional and modified models were compared in performance. The experimental results show MEMS-gyros’ bias stability was up to 10% of the conventional model, the temperature dependence of Si-based materials was decoupled better by the modified one and the environmental adaptability of MEMS-gyros was improved to expand their application in diverse complicated conditions.

A Novel Underwater Simultaneous Localization and Mapping Online Algorithm Based on Neural Network

The navigation and localization of autonomous underwater vehicles (AUVs) in seawater are of the utmost importance for scientific research, petroleum engineering, search and rescue, and military missions concerning the special environment of seawater. However, there is still no general method for AUVs navigation and localization, especially in the featureless seabed. The reported approaches to solving AUVs navigation and localization problems employ an expensive inertial navigation system (INS), with cumulative errors and dead reckoning, and a high-cost long baseline (LBL) in a featureless subsea. In this study, a simultaneous localization and mapping (AMB-SLAM) online algorithm, based on acoustic and magnetic beacons, was proposed. The AMB-SLAM online algorithm is based on multiple randomly distributed beacons of low-frequency magnetic fields and a single fixed acoustic beacon for location and mapping. The experimental results show that the performance of the AMB-SLAM online algorithm has a high robustness. The proposed approach (the AMB-SLAM online algorithm) provides a low-complexity, low-cost, and high-precision online solution to the AUVs navigation and localization problem in featureless seawater environments. The AMB-SLAM online solution could enable AUVs to autonomously explore or autonomously intervene in featureless seawater environments, which would enable AUVs to accomplish fully autonomous survey missions.

Testing Procedure of Unmanned Aerial Vehicles (UAVs) Trajectory in Automatic Missions

This paper describes an experimental test campaign while using an Unmanned Aerial Vehicle (UAV) and measuring the obtained UAV positions during different flight tasks and in different operative conditions. A new test procedure has been presented and tested for different devices in various weather conditions. This paper describes and analyses the measurements of the flight trajectory of the UAV that was performed with the use of a robotic total station (RTS), as compared to the design data and the data recorded in the internal memory of the UAV. Five different test tasks have been conducted. The obtained results have allowed for the assessment of the correctness of task performance as compared to the design and to determine the flying accuracy of the entire UAV set. The proposed set of tasks can be successfully utilised to control the correctness of operation of various types of UAVs and it may be implemented as a universal test to verify the algorithms optimising take-offs and landings, test flights of the objects, as well as flight planning in various terrain and weather conditions, which will increase the safety of the flights while using UAVs.

Computer Vision in Autonomous Unmanned Aerial Vehicles—A Systematic Mapping Study

Personal assistant robots provide novel technological solutions in order to monitor people’s activities, helping them in their daily lives. In this sense, unmanned aerial vehicles (UAVs) can also bring forward a present and future model of assistant robots. To develop aerial assistants, it is necessary to address the issue of autonomous navigation based on visual cues. Indeed, navigating autonomously is still a challenge in which computer vision technologies tend to play an outstanding role. Thus, the design of vision systems and algorithms for autonomous UAV navigation and flight control has become a prominent research field in the last few years. In this paper, a systematic mapping study is carried out in order to obtain a general view of this subject. The study provides an extensive analysis of papers that address computer vision as regards the following autonomous UAV vision-based tasks: (1) navigation, (2) control, (3) tracking or guidance, and (4) sense-and-avoid. The works considered in the mapping study—a total of 144 papers from an initial set of 2081—have been classified under the four categories above. Moreover, type of UAV, features of the vision systems employed and validation procedures are also analyzed. The results obtained make it possible to draw conclusions about the research focuses, which UAV platforms are mostly used in each category, which vision systems are most frequently employed, and which types of tests are usually performed to validate the proposed solutions. The results of this systematic mapping study demonstrate the scientific community’s growing interest in the development of vision-based solutions for autonomous UAVs. Moreover, they will make it possible to study the feasibility and characteristics of future UAVs taking the role of personal assistants.

Cooperative Optimization of UAVs Formation Visual Tracking

The use of unmanned vehicles to perform tiring, hazardous, repetitive tasks, is becoming a reality out of the academy laboratories, getting more and more interest for several application fields from the industrial, to the civil, to the military contexts. In particular, these technologies appear quite promising when they employ several low-cost resource-constrained vehicles leveraging their coordination to perform complex tasks with efficiency, flexibility, and adaptation that are superior to those of a single agent (even if more instrumented). In this work, we study one of said applications, namely the visual tracking of an evader (target) by means of a fleet of autonomous aerial vehicles, with the specific aim of focusing on the target so as to perform an accurate position estimation while concurrently allowing a wide coverage over the monitored area so as to limit the probability of losing the target itself. These clearly conflicting objectives call for an optimization approach that is here developed: by considering both aforementioned aspects and the cooperative capabilities of the fleet, the designed algorithm allows controling in real time the single fields of view so as to counteract evasion maneuvers and maximize an overall performance index. The proposed strategy is discussed and finally assessed through the realistic Gazebo-ROS simulation framework.

MallARD: An Autonomous Aquatic Surface Vehicle for Inspection and Monitoring of Wet Nuclear Storage Facilities

Inspection and monitoring of wet nuclear storage facilities such as spent fuel pools or wet silos is performed for a variety of reasons, including nuclear security and characterisation of storage facilities prior to decommissioning. Until now such tasks have been performed by personnel or, if the risk to health is too high, avoided. Tasks are often repetitive, time-consuming and potentially dangerous, making them suited to being performed by an autonomous robot. Previous autonomous surface vehicles (ASVs) have been designed for operation in natural outdoor environments and lack the localisation and tracking accuracy necessary for operation in a wet nuclear storage facility. In this paper the environmental and operational conditions are analysed, applicable localisation technologies selected and a unique aquatic autonomous surface vehicle (ASV) is designed and constructed. The ASV developed is holonomic, uses a LiDAR for localisation and features a robust trajectory tracking controller. In a series of experiments the mean error between the present ASV’s planned path and the actual path is approximately 1 cm, which is two orders of magnitude lower than previous ASVs. As well as lab testing, the ASV has been used in two deployments, one of which was in an active spent fuel pool.

Aircraft Pose Estimation Based on Geometry Structure Features and Line Correspondences

A robust and accurate aircraft pose estimation method is proposed in this paper. The aircraft pose reflects the flight status of the aircraft and accurate pose measurement is of great importance in many aerospace applications. This work aims to establish a universal framework to estimate the aircraft pose based on generic geometry structure features. In our method, line features are extracted to describe the structure of an aircraft in single images and the generic geometry features are exploited to form line groups for aircraft structure recognition. Parallel line clustering is utilized to detect the fuselage reference line and bilateral symmetry property of aircraft provides an important constraint for the extraction of wing edge lines under weak perspective projection. After identifying the main structure of the aircraft, a planes intersection method is used to obtain the 3D pose parameters based on the established line correspondences. Our proposed method can increase the measuring range of binocular vision sensors and has the advantage of not relying on 3D models, cooperative marks or other feature datasets. Experimental results show that our method can obtain reliable and accurate pose information of different types of aircraft.

Relative Altitude Estimation Using Omnidirectional Imaging and Holistic Descriptors

Currently, many tasks can be carried out using mobile robots. These robots must be able to estimate their position in the environment to plan their actions correctly. Omnidirectional vision sensors constitute a robust choice to solve this problem, since they provide the robot with complete information from the environment where it moves. The use of global appearance or holistic methods along with omnidirectional images constitutes a robust approach to estimate the robot position when its movement is restricted to the ground plane. However, in some applications, the robot changes its altitude with respect to this plane, and this altitude must be estimated. This work focuses on this problem. A method based on the use of holistic descriptors is proposed to estimate the relative altitude of the robot when it moves upwards or downwards. This descriptor is constructed from the Radon transform of omnidirectional images captured by a catadioptric vision system. To estimate the altitude, the descriptor of the image captured from the current position is compared with the descriptor of the reference image, previously built. The framework is based on the use of phase correlation to calculate relative orientation and a method based on the compression-expansion of the columns of the holistic descriptor to estimate relative height. Only an omnidirectional vision sensor and image processing techniques are used to solve these problems. This approach has been tested using different sets of images captured both indoors and outdoors under realistic working conditions. The experimental results prove the validity of the method even in the presence of noise or occlusions.

Visual-Based SLAM Configurations for Cooperative Multi-UAV Systems with a Lead Agent: An Observability-Based Approach

In this work, the problem of the cooperative visual-based SLAM for the class of multi-UA systems that integrates a lead agent has been addressed. In these kinds of systems, a team of aerial robots flying in formation must follow a dynamic lead agent, which can be another aerial robot, vehicle or even a human. A fundamental problem that must be addressed for these kinds of systems has to do with the estimation of the states of the aerial robots as well as the state of the lead agent. In this work, the use of a cooperative visual-based SLAM approach is studied in order to solve the above problem. In this case, three different system configurations are proposed and investigated by means of an intensive nonlinear observability analysis. In addition, a high-level control scheme is proposed that allows to control the formation of the UAVs with respect to the lead agent. In this work, several theoretical results are obtained, together with an extensive set of computer simulations which are presented in order to numerically validate the proposal and to show that it can perform well under different circumstances (e.g., GPS-challenging environments). That is, the proposed method is able to operate robustly under many conditions providing a good position estimation of the aerial vehicles and the lead agent as well.

Visual Information Fusion through Bayesian Inference for Adaptive Probability-Oriented Feature Matching

This work presents a visual information fusion approach for robust probability-oriented feature matching. It is sustained by omnidirectional imaging, and it is tested in a visual localization framework, in mobile robotics. General visual localization methods have been extensively studied and optimized in terms of performance. However, one of the main threats that jeopardizes the final estimation is the presence of outliers. In this paper, we present several contributions to deal with that issue. First, 3D information data, associated with SURF (Speeded-Up Robust Feature) points detected on the images, is inferred under the Bayesian framework established by Gaussian processes (GPs). Such information represents a probability distribution for the feature points’ existence, which is successively fused and updated throughout the robot’s poses. Secondly, this distribution can be properly sampled and projected onto the next 2D image frame in t+1, by means of a filter-motion prediction. This strategy permits obtaining relevant areas in the image reference system, from which probable matches could be detected, in terms of the accumulated probability of feature existence. This approach entails an adaptive probability-oriented matching search, which accounts for significant areas of the image, but it also considers unseen parts of the scene, thanks to an internal modulation of the probability distribution domain, computed in terms of the current uncertainty of the system. The main outcomes confirm a robust feature matching, which permits producing consistent localization estimates, aided by the odometer’s prior to estimate the scale factor. Publicly available datasets have been used to validate the design and operation of the approach. Moreover, the proposal has been compared, firstly with a standard feature matching and secondly with a localization method, based on an inverse depth parametrization. The results confirm the validity of the approach in terms of feature matching, localization accuracy, and time consumption.

Application of Crack Identification Techniques for an Aging Concrete Bridge Inspection Using an Unmanned Aerial Vehicle

Bridge inspection using unmanned aerial vehicles (UAV) with high performance vision sensors has received considerable attention due to its safety and reliability. As bridges become obsolete, the number of bridges that need to be inspected increases, and they require much maintenance cost. Therefore, a bridge inspection method based on UAV with vision sensors is proposed as one of the promising strategies to maintain bridges. In this paper, a crack identification method by using a commercial UAV with a high resolution vision sensor is investigated in an aging concrete bridge. First, a point cloud-based background model is generated in the preliminary flight. Then, cracks on the structural surface are detected with the deep learning algorithm, and their thickness and length are calculated. In the deep learning method, region with convolutional neural networks (R-CNN)-based transfer learning is applied. As a result, a new network for the 384 collected crack images of 256 × 256 pixel resolution is generated from the pre-trained network. A field test is conducted to verify the proposed approach, and the experimental results proved that the UAV-based bridge inspection is effective at identifying and quantifying the cracks on the structures.

A Visual-Aided Inertial Navigation and Mapping System

State estimation is a fundamental necessity for any application involving autonomous robots. This paper describes a visual-aided inertial navigation and mapping system for application to autonomous robots. The system, which relies on Kalman filtering, is designed to fuse the measurements obtained from a monocular camera, an inertial measurement unit (IMU) and a position sensor (GPS). The estimated state consists of the full state of the vehicle: the position, orientation, their first derivatives and the parameter errors of the inertial sensors (i.e., the bias of gyroscopes and accelerometers). The system also provides the spatial locations of the visual features observed by the camera. The proposed scheme was designed by considering the limited resources commonly available in small mobile robots, while it is intended to be applied to cluttered environments in order to perform fully vision-based navigation in periods where the position sensor is not available. Moreover, the estimated map of visual features would be suitable for multiple tasks: i) terrain analysis; ii) three-dimensional (3D) scene reconstruction; iii) localization, detection or perception of obstacles and generating trajectories to navigate around these obstacles; and iv) autonomous exploration. In this work, simulations and experiments with real data are presented in order to validate and demonstrate the performance of the proposal.