Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems

Dynamics modeling and nonlinear attitude controller design for a rocket-type unmanned aerial vehicle

This paper presents an altitude and attitude control system for a newly designed rocket-type unmanned aerial vehicle (UAV) propelled by a gimbal-based coaxial rotor system (GCRS) enabling thrust vector control (TVC). The GCRS is the only means of actuation available to control the UAV's orientation, and the flight dynamics identify the primary control difficulty as the highly nonlinear and tightly coupled control distribution problem. To address this, the study presents detailed derivations of attitude flight dynamics and a control strategy to track the desired attitude trajectory. First, a Proportional-Integral-Derivative (PID) control algorithm is developed based on the formulation of linear matrix inequality (LMI) to ensure robust stability and performance. Second, an optimization algorithm using the Levenberg-Marquardt (LM) method is introduced to solve the nonlinear inverse mapping problem between the control law and the actual actuator outputs, addressing the nonlinear coupled control input distribution problem of the GCRS. In summary, the main contribution is the proposal of a new TVC UAV system based on GCRS. The PID control algorithm and LM algorithm were designed to solve the distribution problem of the actuation model and confirm altitude and attitude tracking missions. Finally, to validate the flight properties of the rocket-type UAV and the performance of the proposed control algorithm, several numerical simulations were conducted. The results indicate that the tightly coupled control input nonlinear inverse problem was successfully solved, and the proposed control algorithm achieved effective attitude stabilization even in the presence of disturbances.

Control Schemes for Quadrotor UAV: Taxonomy and Survey

Quadrotor Unmanned Aerial Vehicle (UAV) is an unstable system, so it needs to be controlled efficiently and intelligently. Moreover, due to its non-linear, coupled, and under-actuated nature, the quadrotor has become an important research platform to study and validate various control theories. Different control approaches have been used to control the quadrotor UAV. In this context, a comprehensive study of different control schemes is presented in this research. First, an overview of the working and different applications of quadrotor UAVs is presented. Second, a mathematical model of the quadrotor is discussed. Later, the experimental results of various existing control techniques are discussed and compared. The various control schemes discussed and described for quadrotors are; Proportional Integral and Derivative (PID), Linear Quadratic Regulator (LQR), H-infinity ( H ∞ ), Sliding Mode Control (SMC), Feedback Linearization (FBL), Model Predictive Control (MPC), Fuzzy Logic Control (FLC), Artificial Neural Network (ANN), Iterative Learning Control (ILC), Reinforcement Learning Control (RLC), Brain Emotional Learning Control (BELC), Memory Based Control (MBC), Nested Saturation Control (NSC), and Hybrid Controllers (HC). Comparison is done among all the control techniques and it is concluded that the hybrid control method gives improved results. This survey presents a broad overview of the state-of-the-art in UAV design, control, and implementation for real-life applications.

Safety Flight Control Design of a Quadrotor UAV With Capability Analysis

This article considers the safety control problem of a quadrotor unmanned aerial vehicle (UAV) subject to actuator faults and external disturbances, based on the quantization of system capability and safety margin. First, a trajectory function is constructed online with backpropagation of system dynamics. Therefore, a degraded trajectory is gracefully regenerated, via the tradeoff between the remaining system capability and the expected derivatives (velocity, jerk, and snap) of the trajectory. Second, a control-oriented model is established into a form of strict feedback, integrating actuator malfunctions and disturbances. Therefore, a retrofit dynamic surface control (DSC) scheme based on the control-oriented model is developed to improve the tracking performance. When comparing to the existing control methods, the compensation ability is analyzed to determine whether the faults and disturbances can be handled or not. Finally, simulation and experimental studies are conducted to highlight the efficiency of the proposed safety control scheme.

Perception, Path Planning, and Flight Control for a Drone-Enabled Autonomous Pollination System

The decline of natural pollinators necessitates the development of novel pollination technologies. In this work, we propose a drone-enabled autonomous pollination system (APS) that consists of five primary modules: environment sensing, flower perception, path planning, flight control, and pollination mechanisms. These modules are highly dependent upon each other, with each module relying on inputs from the other modules. In this paper, we focus on approaches to the flower perception, path planning, and flight control modules. First, we briefly introduce a flower perception method from our previous work to create a map of flower locations. With a map of flowers, APS path planning is defined as a variant of the Travelling Salesman Problem (TSP). Two path planning approaches are compared based on mixed-integer programming (MIP) and genetic algorithms (GA), respectively. The GA approach is chosen as the superior approach due to the vast computational savings with negligible loss of optimality. To accurately follow the generated path for pollination, we develop a convex optimization approach to the quadrotor flight control problem (QFCP). This approach solves two convex problems. The first problem is a convexified three degree-of-freedom QFCP. The solution to this problem is used as an initial guess to the second convex problem, which is a linearized six degree-of-freedom QFCP. It is found that changing the objective of the second convex problem to minimize the deviation from the initial guess provides improved physical feasibility and solutions similar to a general-purpose optimizer. The path planning and flight control approaches are then tested within a model predictive control (MPC) framework where significant computational savings and embedded adjustments to uncertainty are observed. Coupling the two modules together provides a simple demonstration of how the entire APS will operate in practice.

Robust Adaptive Control for a Small Unmanned Helicopter Using Reinforcement Learning

This article presents a novel adaptive controller for a small-size unmanned helicopter using the reinforcement learning (RL) control methodology. The helicopter is subject to system uncertainties and unknown external disturbances. The dynamic unmodeling uncertainties of the system are estimated online by the actor network, and the tracking performance function is optimized via the critic network. The estimation error of the actor-critic network and the external unknown disturbances are compensated via the nonlinear robust component based on the sliding mode control method. The stability of the closed-loop system and the asymptotic convergence of the attitude tracking error are proved via the Lyapunov-based stability analysis. Finally, real-time experiments are performed on a helicopter control testbed. The experimental results show that the proposed controller achieves good control performance.

Recent Advances in Tracking Devices for Biomedical Ultrasound Imaging Applications

With the rapid advancement of tracking technologies, the applications of tracking systems in ultrasound imaging have expanded across a wide range of fields. In this review article, we discuss the basic tracking principles, system components, performance analyses, as well as the main sources of error for popular tracking technologies that are utilized in ultrasound imaging. In light of the growing demand for object tracking, this article explores both the potential and challenges associated with different tracking technologies applied to various ultrasound imaging applications, including freehand 3D ultrasound imaging, ultrasound image fusion, ultrasound-guided intervention and treatment. Recent development in tracking technology has led to increased accuracy and intuitiveness of ultrasound imaging and navigation with less reliance on operator skills, thereby benefiting the medical diagnosis and treatment. Although commercially available tracking systems are capable of achieving sub-millimeter resolution for positional tracking and sub-degree resolution for orientational tracking, such systems are subject to a number of disadvantages, including high costs and time-consuming calibration procedures. While some emerging tracking technologies are still in the research stage, their potentials have been demonstrated in terms of the compactness, light weight, and easy integration with existing standard or portable ultrasound machines.

Modified A-Star (A*) Approach to Plan the Motion of a Quadrotor UAV in Three-Dimensional Obstacle-Cluttered Environment

Motion-planning algorithms play a vital role in attaining various levels of autonomy for any ground or flying agent. Three-dimensional (3D) motion-planning is interesting, but rather complex, especially for flying agents such as autonomous unmanned aerial vehicles (UAVs), due to the increased dimensionality of space and consideration of dynamical constraints for a feasible trajectory. Usually, in 3D path search problems, it is hard to avoid extra expanded nodes due to increased dimensionality with various available search options. Therefore, this paper discusses and implements a modified heuristic-based A* formalism that uses a truncation mechanism in order to eradicate the mentioned problem. In this formalism, the complete motion planning is divided into shortest path search problem and smooth trajectory generation. The shortest path search problem is subdivided into an initial naïve guess of the path and the truncation of the extra nodes. To find a naïve shortest path, a conventional two-dimensional (2D) A* algorithm is augmented for 3D space with six-sided search. This approach led to the inclusion of extra expanded nodes and, therefore, it is not the shortest one. Hence, a truncation algorithm is developed to further process this path in order to truncate the extra expanded nodes and then to shorten the path length. This new approach significantly reduces the path length and renders only those nodes that are obstacle-free. The latter is ensured using a collision detection algorithm during the truncation process. Subsequently, the nodes of this shortened path are used to generate a dynamically feasible and optimal trajectory for the quadrotor UAV. Optimal trajectory generation requires that the plant dynamics must be differentially flat. Therefore, the corresponding proof is presented to ensure generation of the optimal trajectory. This optimal trajectory minimizes the control effort and ensures longer endurance. Moreover, quadrotor model and controllers are derived as preliminaries, which are subsequently used to track the desired trajectory generated by the trajectory planner. Finally, results of numerical simulations which ultimately validate the theoretical developments are presented.

Design and Experimental Comparison of PID, LQR and MPC Stabilizing Controllers for Parrot Mambo Mini-Drone

Parrot Mambo mini-drone is a readily available commercial quadrotor platform to understand and analyze the behavior of a quadrotor both in indoor and outdoor applications. This study evaluates the performance of three alternative controllers on a Parrot Mambo mini-drone in an interior environment, including Proportional–Integral–Derivative (PID), Linear Quadratic Regulator (LQR), and Model Predictive Control (MPC). To investigate the controllers’ performance, initially, the MATLAB®/Simulink™ environment was considered as the simulation platform. The successful simulation results finally led to the implementation of the controllers in real-time in the Parrot Mambo mini-drone. Here, MPC surpasses PID and LQR in ensuring the system’s stability and robustness in simulation and real-time experiment results. Thus, this work makes a contribution by introducing the impact of MPC on this quadrotor platform, such as system stability and robustness, and showing its efficacy over PID and LQR. All three controllers demonstrate similar tracking performance in simulations and experiments. In steady state, the maximal pitch deviation for the PID controller is 0.075 rad, for the LQR, it is 0.025 rad, and for the MPC, it is 0.04 rad. The maximum pitch deviation for the PID-based controller is 0.3 rad after the take-off impulse, 0.06 rad for the LQR, and 0.17 rad for the MPC.

A Vision-Based Motion Control Framework for Water Quality Monitoring Using an Unmanned Aerial Vehicle

In this paper, we present a vision-aided motion planning and control framework for the efficient monitoring and surveillance of water surfaces using an Unmanned Aerial Vehicle (UAV). The ultimate goal of the proposed strategy is to equip the UAV with the necessary autonomy and decision-making capabilities to support First Responders during emergency water contamination incidents. Toward this direction, we propose an end-to-end solution, based on which the First Responder indicates visiting and landing waypoints, while the envisioned strategy is responsible for the safe and autonomous navigation of the UAV, the refinement of the way-point locations that maximize the visible water surface area from the onboard camera, as well as the on-site refinement of the appropriate landing region in harsh environments. More specifically, we develop an efficient waypoint-tracking motion-planning scheme with guaranteed collision avoidance, a local autonomous exploration algorithm for refining the way-point location with respect to the areas visible to the drone’s camera, water, a vision-based algorithm for the on-site area selection for feasible landing and finally, a model predictive motion controller for the landing procedure. The efficacy of the proposed framework is demonstrated via a set of simulated and experimental scenarios using an octorotor UAV.

Research on solving heading attitude of airdrop cargo platform based on line features

The present study envisages the development of an improved line features method to accurately estimate the attitude of the airdrop cargo platform during airdrop landing. Therefore, this article uses the geometric characteristics of the line features to improve the traditional line features extraction and removes the locally dense line features in the image, which greatly reduces the number of line features in the image. Then, the improved random sample consensus is used to remove the mismatching of line features, which improves the real-time performance of the algorithm and the accuracy of the attitude angle, and makes up for the problem of difficult extraction of point features or low matching accuracy in the airdrop environment. Finally, a constraint equation is established for the line features that are successfully matched, and using homography to obtain attitude of the airdrop cargo platform. This article also meets the requirements of accurate calculation attitude of airdrop cargo platform. The experiment shows the significance and feasibility of the airdrop cargo platform heading and attitude calculation technology based on the line feature, and it has a good application prospect.

Vision-Based 2D Navigation of Unmanned Aerial Vehicles in Riverine Environments with Imitation Learning

There have been many researchers studying how to enable unmanned aerial vehicles (UAVs) to navigate in complex and natural environments autonomously. In this paper, we develop an imitation learning framework and use it to train navigation policies for the UAV flying inside complex and GPS-denied riverine environments. The UAV relies on a forward-pointing camera to perform reactive maneuvers and navigate itself in 2D space by adapting the heading. We compare the performance of a linear regression-based controller, an end-to-end neural network controller and a variational autoencoder (VAE)-based controller trained with data aggregation method in the simulation environments. The results show that the VAE-based controller outperforms the other two controllers in both training and testing environments and is able to navigate the UAV with a longer traveling distance and a lower intervention rate from the pilots.

Sliding Mode Fault Tolerant Control for a Quadrotor with Varying Load and Actuator Fault

In this paper, an adaptive sliding mode fault-tolerant control scheme based on prescribed performance control and neural networks is developed for an Unmanned Aerial Vehicle (UAV) quadrotor carrying a load to deal with actuator faults. First, a nonsingular fast terminal sliding mode (NFTSM) control strategy is presented. In virtue of the proposed strategy, fast convergence and high robustness can be guaranteed without stimulating chattering. Secondly, to obtain correct fault magnitudes and compensate the failures actively, a radial basis function neural network-based fault estimation scheme is proposed. By combining the proposed fault estimation strategy and the NFTSM controller, an active fault-tolerant control algorithm is established. Then, the uncertainties caused by load variation are explicitly considered and compensated by the presented adaptive laws. Moreover, by synthesizing the proposed sliding mode control and prescribed performance control (PPC), an output error transformation is defined to deal with state constraints and provide better tracking performance. From the Lyapunov stability analysis, the overall system is proven to be uniformly asymptotically stable. Finally, numerical simulation based on a quadrotor helicopter is carried out to validate the effectiveness and superiority of the proposed algorithm.

Vision-Based Target Detection and Tracking for a Miniature Pan-Tilt Inertially Stabilized Platform

This paper presents a novel visual servoing sheme for a miniature pan-tilt intertially stabilized platform (ISP). A fully customized ISP can be mounted on a miniature quadcopter to achieve stationary or moving target detection and tracking. The airborne pan-tilt ISP can effectively isolate a disturbing rotational motion of the carrier, ensuring the stabilization of the optical axis of the camera in order to obtain a clear video image. Meanwhile, the ISP guarantees that the target is always on the optical axis of the camera, so as to achieve the target detection and tracking. The vision-based tracking control design adopts a cascaded control structure based on the mathematical model, which can accurately reflect the dynamic characteristics of the ISP. The inner loop of the proposed controller employs a proportional lag compensator to improve the stability of the optical axis, and the outer loop adopts the feedback linearization-based sliding mode control method to achieve the target tracking. Numerical simulations and laboratory experiments demonstrate that the proposed controller can achieve satisfactory tracking performance.

A seasonally invariant deep transform for visual terrain-relative navigation

Visual terrain-relative navigation (VTRN) is a localization method based on registering a source image taken from a robotic vehicle against a georeferenced target image. With high-resolution imagery databases of Earth and other planets now available, VTRN offers accurate, drift-free navigation for air and space robots even in the absence of external positioning signals. Despite its potential for high accuracy, however, VTRN remains extremely fragile to common and predictable seasonal effects, such as lighting, vegetation changes, and snow cover. Engineered registration algorithms are mature and have provable geometric advantages but cannot accommodate the content changes caused by seasonal effects and have poor matching skill. Approaches based on deep learning can accommodate image content changes but produce opaque position estimates that either lack an interpretable uncertainty or require tedious human annotation. In this work, we address these issues with targeted use of deep learning within an image transform architecture, which converts seasonal imagery to a stable, invariant domain that can be used by conventional algorithms without modification. Our transform preserves the geometric structure and uncertainty estimates of legacy approaches and demonstrates superior performance under extreme seasonal changes while also being easy to train and highly generalizable. We show that classical registration methods perform exceptionally well for robotic visual navigation when stabilized with the proposed architecture and are able to consistently anticipate reliable imagery. Gross mismatches were nearly eliminated in challenging and realistic visual navigation tasks that also included topographic and perspective effects.

A Review on Comparative Remarks, Performance Evaluation and Improvement Strategies of Quadrotor Controllers

The quadrotor is an ideal platform for testing control strategies because of its non-linearity and under-actuated configuration, allowing researchers to evaluate and verify control strategies. Several control strategies are used, including Proportional-Integral-Derivative (PID), Linear Quadratic Regulator (LQR), Backstepping, Feedback Linearization Control (FLC), Sliding Mode Control (SMC), and Model Predictive Control (MPC), Neural Network, H-infinity, Fuzzy Logic, and Adaptive Control. However, due to several drawbacks, such as high computation, a large amount of training data, approximation error, and the existence of uncertainty, the commercialization of those control technologies in various industrial applications is currently limited. This paper conducts a thorough analysis of the current literature on the effects of multiple controllers on quadrotors, focusing on two separate approaches: (i) controller hybridization and (ii) controller development. Besides, the limitations of the previous works are discussed, challenges and opportunities to work in this field are assessed, and potential research directions are suggested.

Trajectory robust control of autonomous quadcopters based on model decoupling and disturbance estimation

In this article, a systematic procedure is given for determining a robust motion control law for autonomous quadcopters, starting from an input–output linearizable model. In particular, the suggested technique can be considered as a robust feedback linearization (FL), where the nonlinear state-feedback terms, which contain the aerodynamic forces and moments and other unknown disturbances, are estimated online by means of extended state observers. Therefore, the control system is made robust against unmodelled dynamics and endogenous as well as exogenous disturbances. The desired closed-loop dynamics is obtained by means of pole assignment. To have a feasible control action, that is, the forces produced by the motors belong to an admissible set of forces, suitable reference signals are generated by means of differentiators supplied by the desired trajectory. The proposed control algorithm is tested by means of simulation experiments on a Raspberry-PI board by means of the hardware-in-the-loop method, showing the effectiveness of the proposed approach. Moreover, it is compared with the standard FL control method, where the above nonlinear terms are computed using nominal parameters and the aerodynamical disturbances are neglected. The comparison shows that the control algorithm based on the online estimation of the above nonlinear state-feedback terms gives better static and dynamic behaviour over the standard FL control method.

Contact transition control with acceleration feedback enhancement for a quadrotor

A stable transition from free-motion to contact phase is extremely significant for a UAV interacting with its objects and environment. Oscillations may occur at the contact point if the force controller of the UAV fails to accommodate to the environment parameters, which are usually unknown to the UAV. In this paper, the acceleration feedback control is proposed to enhance the robustness of a classic force controller. This intends to achieve a smooth and stable contact transition of the UAV while interacting with different stiffness environments with non-zero approaching velocity. Extensive flight tests are conducted on a quadrotor UAV to verify the performance of the proposed algorithm with respect to different-stiffness environments. The experimental results are also compared with those without the acceleration feedback enhancement, to demonstrate the improvement of the proposed scheme.

Data Driven Model-Free Adaptive Control Method for Quadrotor Formation Trajectory Tracking Based on RISE and ISMC Algorithm

In order to solve the problems of complex dynamic modeling and parameters identification of quadrotor formation cooperative trajectory tracking control, this paper proposes a data-driven model-free adaptive control method for quadrotor formation based on robust integral of the signum of the error (RISE) and improved sliding mode control (ISMC). The leader-follower strategy is adopted, and the leader realizes trajectory tracking control. A novel asymptotic tracking data-driven controller of quadrotor is used to control the system using the RISE method. It is divided into two parts: The inner loop is for attitude control and the outer loop for position control. Both use the RISE method in the loop to eliminate interference and this method only uses the input and output data of the unmanned aerial vehicle(UAV) system and does not rely on any dynamics and kinematics model of the UAV. The followers realize formation cooperative control, introducing adaptive update law and saturation function to improve sliding mode control (SMC), and it eliminates the general SMC algorithm controller design dependence on the mathematical model of the UAV and has the chattering problem. Then, the stability of the system is proved by the Lyapunov method, and the effectiveness of the algorithm and the feasibility of the scheme are verified by numerical simulation. The experimental results show that the designed data-driven model-free adaptive control method for the quadrotor formation is effective and can effectively realize the coordinated formation trajectory tracking control of the quadrotor. At the same time, the design of the controller does not depend on the UAV kinematics and dynamics model, and it has high control accuracy, stability, and robustness.

Embedded Computation Architectures for Autonomy in Unmanned Aircraft Systems (UAS)

This paper addresses the challenge of embedded computing resources required by future autonomous Unmanned Aircraft Systems (UAS). Based on an analysis of the required onboard functions that will lead to higher levels of autonomy, we look at most common UAS tasks to first propose a classification of UAS tasks considering categories such as flight, navigation, safety, mission and executing entities such as human, offline machine, embedded system. We then analyse how a given combination of tasks can lead to higher levels of autonomy by defining an autonomy level. We link UAS applications, the tasks required by those applications, the autonomy level and the implications on computing resources to achieve that autonomy level. We provide insights on how to define a given autonomy level for a given application based on a number of tasks. Our study relies on the state-of-the-art hardware and software implementations of the most common tasks currently used by UAS, also expected tasks according to the nature of their future missions. We conclude that current computing architectures are unlikely to meet the autonomy requirements of future UAS. Our proposed approach is based on dynamically reconfigurable hardware that offers benefits in computational performance and energy usage. We believe that UAS designers must now consider the embedded system as a masterpiece of the system.

Improving Motion Safety and Efficiency of Intelligent Autonomous Swarm of Drones

Interest is growing in the use of autonomous swarms of drones in various mission-physical applications such as surveillance, intelligent monitoring, and rescue operations. Swarm systems should fulfill safety and efficiency constraints in order to guarantee dependable operations. To maximize motion safety, we should design the swarm system in such a way that drones do not collide with each other and/or other objects in the operating environment. On other hand, to ensure that the drones have sufficient resources to complete the required task reliably, we should also achieve efficiency while implementing the mission, by minimizing the travelling distance of the drones. In this paper, we propose a novel integrated approach that maximizes motion safety and efficiency while planning and controlling the operation of the swarm of drones. To achieve this goal, we propose a novel parallel evolutionary-based swarm mission planning algorithm. The evolutionary computing allows us to plan and optimize the routes of the drones at the run-time to maximize safety while minimizing travelling distance as the efficiency objective. In order to fulfill the defined constraints efficiently, our solution promotes a holistic approach that considers the whole design process from the definition of formal requirements through the software development. The results of benchmarking demonstrate that our approach improves the route efficiency by up to 10% route efficiency without any crashes in controlling swarms compared to state-of-the-art solutions.

An Embedded Platform for Positioning and Obstacle Detection for Small Unmanned Aerial Vehicles

Unmanned Aerial Vehicles (UAV) with on-board augmentation systems (UAS, Unmanned Aircraft System) have penetrated into civil and general-purpose applications, due to advances in battery technology, control components, avionics and rapidly falling prices. This paper describes the conceptual design and the validation campaigns performed for an embedded precision Positioning, field mapping, Obstacle Detection and Avoiding (PODA) platform, which uses commercial-off-the-shelf sensors, i.e., a 10-Degrees-of-Freedom Inertial Measurement Unit (10-DoF IMU) and a Light Detection and Ranging (LiDAR), managed by an Arduino Mega 2560 microcontroller with Wi-Fi capabilities. The PODA system, designed and tested for a commercial small quadcopter (Parrot Drones SAS Ar.Drone 2.0, Paris, France), estimates position, attitude and distance of the rotorcraft from an obstacle or a landing area, sending data to a PC-based ground station. The main design issues are presented, such as the necessary corrections of the IMU data (i.e., biases and measurement noise), and Kalman filtering techniques for attitude estimation, data fusion and position estimation from accelerometer data. The real-time multiple-sensor optimal state estimation algorithm, developed for the PODA platform and implemented on the Arduino, has been tested in typical aerospace application scenarios, such as General Visual Inspection (GVI), automatic landing and obstacle detection. Experimental results and simulations of various missions show the effectiveness of the approach.

Survey of 8 UAV Set-Covering Algorithms for Terrain Photogrammetry

Remote sensing with unmanned aerial vehicles (UAVs) facilitates photogrammetry for environmental and infrastructural monitoring. Models are created with less computational cost by reducing the number of photos required. Optimal camera locations for reducing the number of photos needed for structure-from-motion (SfM) are determined through eight mathematical set-covering algorithms as constrained by solve time. The algorithms examined are: traditional greedy, reverse greedy, carousel greedy (CG), linear programming, particle swarm optimization, simulated annealing, genetic, and ant colony optimization. Coverage and solve time are investigated for these algorithms. CG is the best method for choosing optimal camera locations as it balances number of photos required and time required to calculate camera positions as shown through an analysis similar to a Pareto Front. CG obtains a statistically significant 3.2 fewer cameras per modeled area than base greedy algorithm while requiring just one additional order of magnitude of solve time. For comparison, linear programming is capable of fewer cameras than base greedy but takes at least three orders of magnitude longer to solve. A grid independence study serves as a sensitivity analysis of the CG algorithms α (iteration number) and β (percentage to be recalculated) parameters that adjust traditional greedy heuristics, and a case study at the Rock Canyon collection dike in Provo, UT, USA, compares the results of all eight algorithms and the uniqueness (in terms of percentage comparisons based on location/angle metadata and qualitative visual comparison) of each selected set. Though this specific study uses SfM, the principles could apply to other instruments such as multi-spectral cameras or aerial LiDAR.

A Nonlinear Double Model for Multisensor-Integrated Navigation Using the Federated EKF Algorithm for Small UAVs

Aimed at improving upon the disadvantages of the single centralized Kalman filter for integrated navigation, including its fragile robustness and low solution accuracy, a nonlinear double model based on the improved decentralized federated extended Kalman filter (EKF) for integrated navigation is proposed. The multisensor error model is established and simplified in this paper according to the near-ground short distance navigation applications of small unmanned aerial vehicles (UAVs). In order to overcome the centralized Kalman filter that is used in the linear Gaussian system, the improved federated EKF is designed for multisensor-integrated navigation. Subsequently, because of the navigation requirements of UAVs, especially for the attitude solution accuracy, this paper presents a nonlinear double model that consists of the nonlinear attitude heading reference system (AHRS) model and nonlinear strapdown inertial navigation system (SINS)/GPS-integrated navigation model. Moreover, the common state parameters of the nonlinear double model are optimized by the federated filter to obtain a better attitude. The proposed algorithm is compared with multisensor complementary filtering (MSCF) and multisensor EKF (MSEKF) using collected flight sensors data. The simulation and experimental tests demonstrate that the proposed algorithm has a good robustness and state estimation solution accuracy.

Real-Time Onboard 3D State Estimation of an Unmanned Aerial Vehicle in Multi-Environments Using Multi-Sensor Data Fusion

The question of how to estimate the state of an unmanned aerial vehicle (UAV) in real time in multi-environments remains a challenge. Although the global navigation satellite system (GNSS) has been widely applied, drones cannot perform position estimation when a GNSS signal is not available or the GNSS is disturbed. In this paper, the problem of state estimation in multi-environments is solved by employing an Extended Kalman Filter (EKF) algorithm to fuse the data from multiple heterogeneous sensors (MHS), including an inertial measurement unit (IMU), a magnetometer, a barometer, a GNSS receiver, an optical flow sensor (OFS), Light Detection and Ranging (LiDAR), and an RGB-D camera. Finally, the robustness and effectiveness of the multi-sensor data fusion system based on the EKF algorithm are verified by field flights in unstructured, indoor, outdoor, and indoor and outdoor transition scenarios.

Magnetometer Calibration for Small Unmanned Aerial Vehicles Using Cooperative Flight Data

This paper presents a new method to improve the accuracy in the heading angle estimate provided by low-cost magnetometers on board of small Unmanned Aerial Vehicles (UAVs). This task can be achieved by estimating the systematic error produced by the magnetic fields generated by onboard electric equipment. To this aim, calibration data must be collected in flight when, for instance, the level of thrust provided by the electric engines (and, consequently, the associated magnetic disturbance) is the same as the one occurring during nominal flight operations. The UAV whose magnetometers need to be calibrated (chief) must be able to detect and track a cooperative vehicle (deputy) using a visual camera, while flying under nominal GNSS coverage to enable relative positioning. The magnetic biases’ determination problem can be formulated as a system of non-linear equations by exploiting the acquired visual and GNSS data. The calibration can be carried out either off-line, using the data collected in flight (as done in this paper), or directly on board, i.e., in real time. Clearly, in the latter case, the two UAVs should rely on a communication link to exchange navigation data. Performance assessment is carried out by conducting multiple experimental flight tests.

Robust State/Output-Feedback Control of Coaxial-Rotor MAVs Based on Adaptive NN Approach

The coaxial-rotor micro-aerial vehicles (CRMAVs) have been proven to be a powerful tool in forming small and agile manned-unmanned hybrid applications. However, the operation of them is usually subject to unpredictable time-varying aerodynamic disturbances and model uncertainties. In this paper, an adaptive robust controller based on a neural network (NN) approach is proposed to reject such perturbations and track both the desired position and orientation trajectories. A complete dynamic model of a CRMAV is first constructed. When all system states are assumed to be available, an NN-based state-feedback controller is proposed through feedback linearization and Lyapunov analysis. Furthermore, to overcome the practical challenge that certain states are not measurable, a high-gain observer is introduced to estimate the unavailable states, and then, an output-feedback controller is developed. Rigorous theoretical analysis verifies the stability of the entire closed-loop system. In addition, extensive simulation studies are conducted to validate the feasibility of the proposed scheme.

UAV Flight and Landing Guidance System for Emergency Situations †

Unmanned aerial vehicles (UAVs) with high mobility can perform various roles such as delivering goods, collecting information, recording videos and more. However, there are many elements in the city that disturb the flight of the UAVs, such as various obstacles and urban canyons which can cause a multi-path effect of GPS signals, which degrades the accuracy of GPS-based localization. In order to empower the safety of the UAVs flying in urban areas, UAVs should be guided to a safe area even in a GPS-denied or network-disconnected environment. Also, UAVs must be able to avoid obstacles while landing in an urban area. For this purpose, we present the UAV detour system for operating UAV in an urban area. The UAV detour system includes a highly reliable laser guidance system to guide the UAVs to a point where they can land, and optical flow magnitude map to avoid obstacles for a safe landing.

Aerial Grasping with a Lightweight Manipulator Based on Multi-Objective Optimization and Visual Compensation

Autonomous grasping with an aerial manipulator in the applications of aerial transportation and manipulation is still a challenging problem because of the complex kinematics/dynamics and motion constraints of the coupled rotors-manipulator system. The paper develops a novel aerial manipulation system with a lightweight manipulator, an X8 coaxial octocopter and onboard visual tracking system. To implement autonomous grasping control, we develop a novel and efficient approach that includes trajectory planning, visual trajectory tracking and kinematic compensation. Trajectory planning for aerial grasping control is formulated as a multi-objective optimization problem, while motion constraints and collision avoidance are considered in the optimization. A genetic method is applied to obtain the optimal solution. A kinematic compensation-based visual trajectory tracking is introduced to address the coupled affection between the manipulator and octocopter, with the advantage of discarding the complex dynamic parameter calibration. Finally, several experiments are performed to verify the effectiveness of the proposed approach.

A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance

Advances in Unmanned Aerial Vehicles (UAVs), also known as drones, offer unprecedented opportunities to boost a wide array of large-scale Internet of Things (IoT) applications. Nevertheless, UAV platforms still face important limitations mainly related to autonomy and weight that impact their remote sensing capabilities when capturing and processing the data required for developing autonomous and robust real-time obstacle detection and avoidance systems. In this regard, Deep Learning (DL) techniques have arisen as a promising alternative for improving real-time obstacle detection and collision avoidance for highly autonomous UAVs. This article reviews the most recent developments on DL Unmanned Aerial Systems (UASs) and provides a detailed explanation on the main DL techniques. Moreover, the latest DL-UAV communication architectures are studied and their most common hardware is analyzed. Furthermore, this article enumerates the most relevant open challenges for current DL-UAV solutions, thus allowing future researchers to define a roadmap for devising the new generation affordable autonomous DL-UAV IoT solutions.

Disturbance suppression for quadrotor UAV using sliding-mode-observer-based equivalent-input-disturbance approach

This paper presents a new control scheme for quadrotor unmanned aerial vehicle attitude control and disturbance suppression. A quadrotor dynamic model is divided into two subsystems: fully-actuated and under-actuated. While the PID method is used to control the fully-actuated subsystem, a sliding-mode-observer-based equivalent-input-disturbance approach is used to control the under-actuated subsystem. The system design is simple, and it is globally uniformly ultimately bounded. Simulations and comparisons demonstrate the effectiveness of the method.

Attitude estimation for collision recovery of a quadcopter unmanned aerial vehicle

An extensive evaluation of attitude estimation algorithms in simulation and experiments is performed to determine their suitability for a collision recovery pipeline of a quadcopter unmanned aerial vehicle. A multiplicative extended Kalman filter (MEKF), unscented Kalman filter (UKF), complementary filter, [Formula: see text] filter, and novel adaptive varieties of the selected filters are compared. The experimental quadcopter uses a PixHawk flight controller, and the algorithms are implemented using data from only the PixHawk inertial measurement unit (IMU). Performance of the aforementioned filters is first evaluated in a simulation environment using modified sensor models to capture the effects of collision on inertial measurements. Simulation results help define the efficacy and use cases of the conventional and novel algorithms in a quadcopter collision scenario. An analogous evaluation is then conducted by post-processing logged sensor data from collision flight tests, to gain new insights into algorithms’ performance in the transition from simulated to real data. The post-processing evaluation compares each algorithm’s attitude estimate, including the stock attitude estimator of the PixHawk controller, to data collected by an offboard infrared motion capture system. Based on this evaluation, two promising algorithms, the MEKF and an adaptive [Formula: see text] filter, are selected for implementation on the physical quadcopter in the control loop of the collision recovery pipeline. Experimental results show an improvement in the metric used to evaluate experimental performance, the time taken to recover from the collision, when compared with the stock attitude estimator on the PixHawk (PX4) software.

A Neural Network Based Landing Method for an Unmanned Aerial Vehicle with Soft Landing Gears

This paper presents the design, implementation, and testing of a soft landing gear together with a neural network-based control method for replicating avian landing behavior on non-flat surfaces. With full consideration of unmanned aerial vehicles and landing gear requirements, a quadrotor helicopter, comprised of one flying unit and one landing assistance unit, is employed. Considering the touchdown speed and posture, a novel design of a soft mechanism for non-flat surfaces is proposed, in order to absorb the remaining landing impact. The framework of the control strategy is designed based on a derived dynamic model. A neural network-based backstepping controller is applied to achieve the desired trajectory. The simulation and outdoor testing results attest to the effectiveness and reliability of the proposed control method.

Hybrid finite-time trajectory tracking control of a quadrotor

In this paper, accurate trajectory tracking control problem of a quadrotor with unknown dynamics and disturbances is addressed by devising a hybrid finite-time control (HFTC) approach. An adaptive integral sliding mode (AISM) control law is proposed for altitude subsystem of the quadrotor, whereby underactuated characteristics can decoupled. Backstepping technique is further deployed to control the horizontal position subsystem. To exactly attenuate external disturbances, a finite-time disturbance observer (FDO) combining with nonsingular terminal sliding mode (NTSM) control strategy is constructed for attitude subsystem, and thereby achieve finite-time stability. Using the compounded control scheme, trajectory tracking errors can be stabilized rapidly. Simulation results and comprehensive comparisons show that the proposed HFTC scheme has remarkably superior performance.

Bio-inspired Autonomous Visual Vertical and Horizontal Control of a Quadrotor Unmanned Aerial Vehicle

Near-ground manoeuvres, such as landing, are key elements in unmanned aerial vehicle navigation. Traditionally, these manoeuvres have been done using external reference frames to measure or estimate the velocity and the height of the vehicle. Complex near-ground manoeuvres are performed by flying animals with ease. These animals perform these complex manoeuvres by exclusively using the information from their vision and vestibular system. In this paper, we use the Tau theory, a visual strategy that, is believed, is used by many animals to approach objects, as a solution for relative ground distance control for unmanned vehicles. In this paper, it is shown how this approach can be used to perform near-ground manoeuvres in a vertical and horizontal manner on a moving target without the knowledge of height and velocity of either the vehicle or the target. The proposed system is tested with simulations. Here, it is shown that, using the proposed methods, the vehicle is able to perform landing on a moving target, and also they enable the user to choose the dynamic characteristics of the approach.

Optimal trajectory generation for time-to-contact based aerial robotic perching

Many biological organisms (e.g. insects, birds, and mammals) rely on the perception of an informational variable called time-to-contact (TTC) to control their motion for various tasks such as avoiding obstacles, landing, or interception. TTC, defined as the required time to contact an object if the current velocity is maintained, has been recently leveraged for robot motion control in various tasks. However, most existing robotic applications of TTC simply control the TTC to be constant or constantly decreasing, without fully exploring the applicability for TTC. In this paper, we propose two-stage TTC based strategies and apply them to aerial robotic perching. With the proposed strategies, we can generate reference trajectories for TTC to realize the non-zero contact velocity required by perching, which is impossible for constant or constantly decreasing TTC strategy, but of critical importance for robust perching performance. We conduct simulations to verify the superiority of the proposed strategies in terms of shorter time for perching and satisfying more constraints. Further, with properly designed controllers, we perform experiments on a palm-size quadcopter to track the planned reference trajectories and realize aerial robotic perching. The research presented in this paper can be readily applied to the control of flying robot for perching with visual feedback, and can inspire more alternative forms of TTC based planning and control for robotic applications.

High-Order Sliding Mode-Based Fixed-Time Active Disturbance Rejection Control for Quadrotor Attitude System

This article presents a fixed-time active disturbance rejection control approach for the attitude control problem of quadrotor unmanned aerial vehicle in the presence of dynamic wind, mass eccentricity and an actuator fault. The control scheme applies the feedback linearization technique and enhances the performance of the traditional active disturbance rejection control (ADRC) based on the fixed-time high-order sliding mode method. A switching-type uniformly convergent differentiator is used to improve the extended state observer for estimating and attenuating the lumped disturbance more accurately. A multivariable high-order sliding mode feedback law is derived to achieve fixed time convergence. The timely convergence of the designed extended state observer and the feedback law is proved theoretically. Mathematical simulations with detailed actuator models and real time experiments are performed to demonstrate the robustness and practicability of the proposed control scheme.

Model-free control of a quadrotor using adaptive proportional derivative-sliding mode control and robust integral of the signum of the error

In this article, a robust model-free trajectory tracking control strategy is developed for a quadrotor in the presence of external disturbances. The proposed strategy has an outer-inner-loop control structure. The outer loop controls the position with adaptive proportional derivative-sliding mode control and generates the desired attitude angles for the inner loop corresponding to the given position, velocity, and heading references, while the robust integral of the signum of the error method is applied to the inner loop to guarantee fast convergence of attitude angles. Asymptotic tracking of the three-dimensional trajectories is proven by the Lyapunov stability theory. The effectiveness of the proposed controller is demonstrated with the simulation results by comparing with other model-free quadrotor trajectory tracking controllers.

Development of Model Predictive Controller for a Tail-Sitter VTOL UAV in Hover Flight

This paper presents a model predictive controller (MPC) for position control of a vertical take-off and landing (VTOL) tail-sitter unmanned aerial vehicle (UAV) in hover flight. A ‘cross’ configuration quad-rotor tail-sitter UAV is designed with the capabilities for both hover and high efficiency level flight. The six-degree-of-freedom (DOF) nonlinear dynamic model of the UAV is built based on aerodynamic data obtained from wind tunnel experiments. The model predictive position controller is then developed with the augmented linearized state-space model. Measured and unmeasured disturbance model are introduced into the modeling and optimization process to improve disturbance rejection ability. The MPC controller is first verified and tuned in the hardware-in-loop (HIL) simulation environment and then implemented in an on-board flight computer for real-time indoor experiments. The simulation and experimental results show that the proposed MPC position controller has good trajectory tracking performance and robust position holding capability under the conditions of prevailing and gusty winds.

Vision-Based Autonomous Landing of a Quadrotor on the Perturbed Deck of an Unmanned Surface Vehicle

Autonomous landing on the deck of an unmanned surface vehicle (USV) is still a major challenge for unmanned aerial vehicles (UAVs). In this paper, a fiducial marker is located on the platform so as to facilitate the task since it is possible to retrieve its six-degrees of freedom relative-pose in an easy way. To compensate interruption in the marker’s observations, an extended Kalman filter (EKF) estimates the current USV’s position with reference to the last known position. Validation experiments have been performed in a simulated environment under various marine conditions. The results confirmed that the EKF provides estimates accurate enough to direct the UAV in proximity of the autonomous vessel such that the marker becomes visible again. Using only the odometry and the inertial measurements for the estimation, this method is found to be applicable even under adverse weather conditions in the absence of the global positioning system.