Drone Mission Definition and Implementation for Automated Infrastructure Inspection Using Airborne Sensors

DCP-SLAM: Distributed Collaborative Partial Swarm SLAM for Efficient Navigation of Autonomous Robots

Collaborative robots represent an evolution in the field of swarm robotics that is pervasive in modern industrial undertakings from manufacturing to exploration. Though there has been much work on path planning for autonomous robots employing floor plans, energy-efficient navigation of autonomous robots in unknown environments is gaining traction. This work presents a novel methodology of low-overhead collaborative sensing, run-time mapping and localization, and navigation for robot swarms. The aim is to optimize energy consumption for the swarm as a whole rather than individual robots. An energy- and information-aware management algorithm is proposed to optimize the time and energy required for a swarm of autonomous robots to move from a launch area to the predefined destination. This is achieved by modifying the classical Partial Swarm SLAM technique, whereby sections of objects discovered by different members of the swarm are stitched together and broadcast to members of the swarm. Thus, a follower can find the shortest path to the destination while avoiding even far away obstacles in an efficient manner. The proposed algorithm reduces the energy consumption of the swarm as a whole due to the fact that the leading robots sense and discover respective optimal paths and share their discoveries with the followers. The simulation results show that the robots effectively re-optimized the previous solution while sharing necessary information within the swarm. Furthermore, the efficiency of the proposed scheme is shown via comparative results, i.e., reducing traveling distance by 13% for individual robots and up to 11% for the swarm as a whole in the performed experiments.

Cortical theta-gamma coupling tracks the mental workload as an indicator of mental schema development during simulated quadrotor UAV operation

Objective. In the emerging field of neuroergonomics, mental workload assessment is one of the most important problems. Previous studies have made some progress on the relationship between task difficulties and mental workload, but how the mental schema, a reflection of the understanding and mastery degree of a task, affects mental workload has not been clearly discussed.Approach. There is emerging appreciation for the role of theta-gamma coupling (TGC) in high-level cognitive functions. Here, we attempt to further our understanding of how mental schema development and task difficulty had an impact on mental workload from the perspective of TGC. Specifically, the variation of TGC coupling strength and coupling pattern was estimated with different test orders and task difficulties performed by 51 students in a ten-day simulated quadrotor unmanned aerial vehicle flight training and test tasks.Main results. During the training, TGC increased with mental schema development. For the test tasks, TGC did not change with increasing task difficulty before the operator formed a mental schema but decreased with the increasing mental workload after the formation of the mental schema.Significance. Our results suggest that TGC was a robust indicator of mental schema development and could be biased by task difficulty. In conclusion, TGC can be a promising measure of mental workload, but only for experienced operators.

Unmanned Aerial Vehicle Surveying and Mapping Trajectory Scheduling and Autonomous Control for Landslide Monitoring

Real-time and efficient monitoring of geological disasters has received extensive attention in the application of UAV surveying and mapping control technology. The application of traditional landslide monitoring methods lacks the accuracy of control algorithms, which has become a hot issue currently facing. Based on the landslide surface subsidence monitoring method, this article designs the UAV trajectory scheduling subsidence monitoring software, which can monitor the UAV’s flight status and navigation information, and draw the flight trajectory in real time. At the same time, the model solves the problem of storage and management of landslide inspection results by the landslide inspection management system, and realizes the functions of entering and querying landslide information, viewing inspection results, landslide safety judgment, generating reports, and autonomous control. The simulation results show that the global accuracy reaches 0.975, and the algorithm recognition degree reaches 99.8%, which promotes the reliability of the landslide monitoring data for the identification of the surveying and mapping trajectory, and provides a decision-making basis for landslide treatment.

Wireless Local Area Network Technologies as Communication Solutions for Unmanned Surface Vehicles

As the number of research activities and practical deployments of unmanned vehicles has shown a rapid growth, topics related to their communication with operator and external infrastructure became of high importance. As a result a trend of employing IP communication for this purpose is emerging and can be expected to bring significant advantages. However, its employment can be expected to be most effective using broadband communication technologies such as Wireless Local Area Networks (WLANs). To verify the effectiveness of such an approach in a specific case of surface unmanned vehicles, the paper includes an overview of IP-based MAVLink communication advantages and requirements, followed by a laboratory and field-experiment study of selected WLAN technologies, compared to popular narrowband communication solutions. The conclusions confirm the general applicability of IP/WLAN communication for surface unmanned vehicles, providing an overview of their advantages and pointing out deployment requirements.

Scheduling PID Attitude and Position Control Frequencies for Time-Optimal Quadrotor Waypoint Tracking under Unknown External Disturbances

Recently, the use of quadrotors has increased in numerous applications, such as agriculture, rescue, transportation, inspection, and localization. Time-optimal quadrotor waypoint tracking is defined as controlling quadrotors to follow the given waypoints as quickly as possible. Although PID control is widely used for quadrotor control, it is not adaptable to environmental changes, such as various trajectories and dynamic external disturbances. In this work, we discover that adjusting PID control frequencies is necessary for adapting to environmental changes by showing that the optimal control frequencies can be different for different environments. Therefore, we suggest a method to schedule the PID position and attitude control frequencies for time-optimal quadrotor waypoint tracking. The method includes (1) a Control Frequency Agent (CFA) that finds the best control frequencies in various environments, (2) a Quadrotor Future Predictor (QFP) that predicts the next state of a quadrotor, and (3) combining the CFA and QFP for time-optimal quadrotor waypoint tracking under unknown external disturbances. The experimental results prove the effectiveness of the proposed method by showing that it reduces the travel time of a quadrotor for waypoint tracking.

FlockAI: A Testing Suite for ML-Driven Drone Applications

Machine Learning (ML) is now becoming a key driver empowering the next generation of drone technology and extending its reach to applications never envisioned before. Examples include precision agriculture, crowd detection, and even aerial supply transportation. Testing drone projects before actual deployment is usually performed via robotic simulators. However, extending testing to include the assessment of on-board ML algorithms is a daunting task. ML practitioners are now required to dedicate vast amounts of time for the development and configuration of the benchmarking infrastructure through a mixture of use-cases coded over the simulator to evaluate various key performance indicators. These indicators extend well beyond the accuracy of the ML algorithm and must capture drone-relevant data including flight performance, resource utilization, communication overhead and energy consumption. As most ML practitioners are not accustomed with all these demanding requirements, the evaluation of ML-driven drone applications can lead to sub-optimal, costly, and error-prone deployments. In this article we introduce FlockAI, an open and modular by design framework supporting ML practitioners with the rapid deployment and repeatable testing of ML-driven drone applications over the Webots simulator. To show the wide applicability of rapid testing with FlockAI, we introduce a proof-of-concept use-case encompassing different scenarios, ML algorithms and KPIs for pinpointing crowded areas in an urban environment.

Power Line Charging Mechanism for Drones

The use of multirotor drones has increased dramatically in the last decade. These days, quadcopters and Vertical Takeoff and Landing (VTOL) drones can be found in many applications such as search and rescue, inspection, commercial photography, intelligence, sports, and recreation. One of the major drawbacks of electric multirotor drones is their limited flight time. Commercial drones commonly have about 20–40 min of flight time. The short flight time limits the overall usability of drones in homeland security applications where long-duration performance is required. In this paper, we present a new concept of a “power-line-charging drone”, the idea being to equip existing drones with a robotic mechanism and an onboard charger in order to allow them to land safely on power lines and then charge from the existing 100–250 V AC (50–60 Hz). This research presents several possible conceptual models for power line charging. All suggested solutions were constructed and submitted to a field experiment. Finally, the paper focuses on the optimal solution and presents the performance and possible future development of such power-line-charging drones.

UAS-Based Plant Phenotyping for Research and Breeding Applications

Unmanned aircraft system (UAS) is a particularly powerful tool for plant phenotyping, due to reasonable cost of procurement and deployment, ease and flexibility for control and operation, ability to reconfigure sensor payloads to diversify sensing, and the ability to seamlessly fit into a larger connected phenotyping network. These advantages have expanded the use of UAS-based plant phenotyping approach in research and breeding applications. This paper reviews the state of the art in the deployment, collection, curation, storage, and analysis of data from UAS-based phenotyping platforms. We discuss pressing technical challenges, identify future trends in UAS-based phenotyping that the plant research community should be aware of, and pinpoint key plant science and agronomic questions that can be resolved with the next generation of UAS-based imaging modalities and associated data analysis pipelines. This review provides a broad account of the state of the art in UAS-based phenotyping to reduce the barrier to entry to plant science practitioners interested in deploying this imaging modality for phenotyping in plant breeding and research areas.

Energy-Efficient Navigation of an Autonomous Swarm with Adaptive Consciousness

The focus of this work is to analyze the behavior of an autonomous swarm, in which only the leader or a dedicated set of agents can take intelligent decisions with other agents just reacting to the information that is received by those dedicated agents, when the swarm comes across stationary or dynamic obstacles. An energy-aware information management algorithm is proposed to avoid over-sensation in order to optimize the sensing energy based on the amount of information obtained from the environment. The information that is needed from each agent is determined by the swarm’s self-awareness in the space domain, i.e., its self-localization characteristics. A swarm of drones as a multi-agent system is considered to be a distributed wireless sensor network that is able to share information inside the swarm and make decisions accordingly. The proposed algorithm reduces the power that is consumed by individual agents due to the use of ranging sensors for observing the environment for safe navigation. This is because only the leader or a dedicated set of agents will turn on their sensors and observe the environment, whereas other agents in the swarm will only be listening to their leader’s translated coordinates and the whereabouts of any detected obstacles w.r.t. the leader. Instead of systematically turning on the sensors to avoid potential collisions with moving obstacles, the follower agents themselves decide on when to turn on their sensors, resulting in further reduction of overall power consumption of the whole swarm. The simulation results show that the swarm maintains the desired formation and efficiently avoids collisions with encountered obstacles, based on the cross-referencing feedback between the swarm agents.

RGDiNet: Efficient Onboard Object Detection with Faster R-CNN for Air-to-Ground Surveillance

An essential component for the autonomous flight or air-to-ground surveillance of a UAV is an object detection device. It must possess a high detection accuracy and requires real-time data processing to be employed for various tasks such as search and rescue, object tracking and disaster analysis. With the recent advancements in multimodal data-based object detection architectures, autonomous driving technology has significantly improved, and the latest algorithm has achieved an average precision of up to 96%. However, these remarkable advances may be unsuitable for the image processing of UAV aerial data directly onboard for object detection because of the following major problems: (1) Objects in aerial views generally have a smaller size than in an image and they are uneven and sparsely distributed throughout an image; (2) Objects are exposed to various environmental changes, such as occlusion and background interference; and (3) The payload weight of a UAV is limited. Thus, we propose employing a new real-time onboard object detection architecture, an RGB aerial image and a point cloud data (PCD) depth map image network (RGDiNet). A faster region-based convolutional neural network was used as the baseline detection network and an RGD, an integration of the RGB aerial image and the depth map reconstructed by the light detection and ranging PCD, was utilized as an input for computational efficiency. Performance tests and evaluation of the proposed RGDiNet were conducted under various operating conditions using hand-labeled aerial datasets. Consequently, it was shown that the proposed method has a superior performance for the detection of vehicles and pedestrians than conventional vision-based methods.

Sensors and Communication Simulation for Unmanned Traffic Management

Unmanned traffic management (UTM) systems will become a key enabler to the future drone market ecosystem, enabling the safe concurrent operation of both manned and unmanned aircrafts. Currently, these systems are usually tested by performing real scenarios that are costly, limited, hardly scalable, and poorly repeatable. As a solution, in this paper we propose an agent-based simulation platform, implemented through a micro service architecture, which may simulate UTM information sources, such as flight plans, telemetry messages, or tracks from a surveillance network. The final objective of this simulator is to use these information streams to perform a system-level evaluation of UTM systems both in the pre-flight and in-flight stages. The proposed platform, with a focus on simulation of communications and sensors, allows to model UTM actors' behaviors and their interactions. In addition, it also considers the manual definition of events to simulate unexpected behaviors/events (contingencies), such as communications failures or pilots' actions. In order to validate our architecture, we implemented a simulator that considers the following actors: drones, pilots, ground control stations, surveillance networks, and communications networks. This platform enables the simulation of the drone trajectory and control, the C2 (command and control) link, drone detection by surveillance sensors, and the communication of all agents by means of a mobile communications network. Our results show that it is possible to truthfully recreate complex scenarios using this simulator, mitigating the disadvantages of real testbeds.

A Framework for Coverage Path Planning Optimization Based on Point Cloud for Structural Inspection

Different practical applications have emerged in the last few years, requiring periodic and detailed inspections to verify possible structural changes. Inspections using Unmanned Aerial Vehicles (UAVs) should minimize flight time due to battery time restrictions and identify the terrain’s topographic features. In this sense, Coverage Path Planning (CPP) aims at finding the best path to coverage of a determined area respecting the operation’s restrictions. Photometric information from the terrain is used to create routes or even refine paths already created. Therefore, this research’s main contribution is developing a methodology that uses a metaheuristic algorithm based on point cloud data to inspect slope and dams structures. The technique was applied in a simulated and real scenario to verify its effectiveness. The results showed an increasing 3D reconstructions’ quality observing optimizing photometric and mission time criteria.

Fisheye-Based Smart Control System for Autonomous UAV Operation

Recently, as UAVs (unmanned aerial vehicles) have become smaller and higher-performance, they play a very important role in the Internet of Things (IoT). Especially, UAVs are currently used not only in military fields but also in various private sectors such as IT, agriculture, logistics, construction, etc. The range is further expected to increase. Drone-related techniques need to evolve along with this change. In particular, there is a need for the development of an autonomous system in which a drone can determine and accomplish its mission even in the absence of remote control from a GCS (Ground Control Station). Responding to such requirements, there have been various studies and algorithms developed for autonomous flight systems. Especially, many ML-based (Machine-Learning-based) methods have been proposed for autonomous path finding. Unlike other studies, the proposed mechanism could enable autonomous drone path finding over a large target area without size limitations, one of the challenges of ML-based autonomous flight or driving in the real world. Specifically, we devised Multi-Layer HVIN (Hierarchical VIN) methods that increase the area applicable to autonomous flight by overlaying multiple layers. To further improve this, we developed Fisheye HVIN, which applied an adaptive map compression ratio according to the drone’s location. We also built an autonomous flight training and verification platform. Through the proposed simulation platform, it is possible to train ML-based path planning algorithms in a realistic environment that takes into account the physical characteristics of UAV movements.

Impact of UAV Surveying Parameters on Mixed Urban Landuse Surface Modelling

Unmanned Aerial Vehicles (UAVs) as a surveying tool are mainly characterized by a large amount of data and high computational cost. This research investigates the use of a small amount of data with less computational cost for more accurate three-dimensional (3D) photogrammetric products by manipulating UAV surveying parameters such as flight lines pattern and image overlap percentages. Sixteen photogrammetric projects with perpendicular flight plans and a variation of 55% to 85% side and forward overlap were processed in Pix4DMapper. For UAV data georeferencing and accuracy assessment, 10 Ground Control Points (GCPs) and 18 Check Points (CPs) were used. Comparative analysis was done by incorporating the median of tie points, the number of 3D point cloud, horizontal/vertical Root Mean Square Error (RMSE), and large-scale topographic variations. The results show that an increased forward overlap also increases the median of the tie points, and an increase in both side and forward overlap results in the increased number of point clouds. The horizontal accuracy of 16 projects varies from ±0.13m to ±0.17m whereas the vertical accuracy varies from ± 0.09 m to ± 0.32 m. However, the lowest vertical RMSE value was not for highest overlap percentage. The tradeoff among UAV surveying parameters can result in high accuracy products with less computational cost.

Genetic Algorithm-Based Tuning of Backstepping Controller for a Quadrotor-Type Unmanned Aerial Vehicle

Backstepping is a control technique based on Lyapunov’s theory that has been successfully implemented in the control of motors and robots by several nonlinear methods. However, there are no standardized methods for tuning control gains (unlike the PIDs). This paper shows the tuning gains of the backstepping controller, using Genetic Algorithms (GA), for an Unmanned Aerial Vehicle (UAV), quadrotor type, designed for autonomous trajectory tracking. First, a dynamic model of the vehicle is obtained through the Newton‒Euler methodology. Then, the control law is obtained, and self-tuning is performed, through which we can obtain suitable values of the gains in order to achieve the design requirements. In this work, the establishment time and maximum impulse are considered as such. The tuning and simulations of the system response were performed using the MATLAB-Simulink environment, obtaining as a result the compliance of the design parameters and the correct tracking of different trajectories. The results show that self-tuning by means of genetic algorithms satisfactorily adjusts for the gains of a backstepping controller applied to a quadrotor and allows for the implementation of a control system that responds appropriately to errors of different magnitude.

Fundamental Elements of an Urban UTM

Urban airspace environments present exciting new opportunities for delivering drone services to an increasingly large global market, including: information gathering; package delivery; air-taxi services. A key challenge is how to model airspace environments over densely populated urban spaces, coupled with the design and development of scalable traffic management systems that may need to handle potentially hundreds to thousands of drone movements per hour. This paper explores the background to Urban unmanned traffic management (UTM), examining high-level initiatives, such as the USA’s Unmanned Air Traffic (UTM) systems and Europe’s U-Space services, as well as a number of contemporary research activities in this area. The main body of the paper describes the initial research outputs of the U-Flyte R&D group, based at Maynooth University in Ireland, who have focused on developing an integrated approach to airspace modelling and traffic management platforms for operating large drone fleets over urban environments. This work proposes pragmatic and innovative approaches to expedite the roll-out of these much-needed urban UTM solutions. These approaches include the certification of drones for urban operation, the adoption of a collaborative and democratic approach to designing urban airspace, the development of a scalable traffic management and the replacement of direct human involvement in operating drones and coordinating drone traffic with machines. The key fundamental elements of airspace architecture and traffic management for busy drone operations in urban environments are described together with initial UTM performance results from simulation studies.

3D Trajectory Planning Method for UAVs Swarm in Building Emergencies

The development in Multi-Robot Systems (MRS) has become one of the most exploited fields of research in robotics in recent years. This is due to the robustness and versatility they present to effectively undertake a set of tasks autonomously. One of the essential elements for several vehicles, in this case, Unmanned Aerial Vehicles (UAVs), to perform tasks autonomously and cooperatively is trajectory planning, which is necessary to guarantee the safe and collision-free movement of the different vehicles. This document includes the planning of multiple trajectories for a swarm of UAVs based on 3D Probabilistic Roadmaps (PRM). This swarm is capable of reaching different locations of interest in different cases (labeled and unlabeled), supporting of an Emergency Response Team (ERT) in emergencies in urban environments. In addition, an architecture based on Robot Operating System (ROS) is presented to allow the simulation and integration of the methods developed in a UAV swarm. This architecture allows the communications with the MavLink protocol and control via the Pixhawk autopilot, for a quick and easy implementation in real UAVs. The proposed method was validated by experiments simulating building emergences. Finally, the obtained results show that methods based on probability roadmaps create effective solutions in terms of calculation time in the case of scalable systems in different situations along with their integration into a versatile framework such as ROS.

Internet of Unmanned Aerial Vehicles-A Multilayer Low-Altitude Airspace Model for Distributed UAV Traffic Management

The rapid adoption of Internet of Things (IoT) has encouraged the integration of new connected devices such as Unmanned Aerial Vehicles (UAVs) to the ubiquitous network. UAVs promise a pragmatic solution to the limitations of existing terrestrial IoT infrastructure as well as bring new means of delivering IoT services through a wide range of applications. Owning to their potential, UAVs are expected to soon dominate the low-altitude airspace over populated cities. This introduces new research challenges such as the safe management of UAVs operation under high traffic demands. This paper proposes a novel way of structuring the uncontrolled, low-altitude airspace, with the aim of addressing the complex problem of UAV traffic management at an abstract level. The work, hence, introduces a model of the airspace as a weighted multilayer network of nodes and airways and presents a set of experimental simulation results using three UAV traffic management heuristics.

A Framework for Automated Acquisition and Processing of As-Built Data with Autonomous Unmanned Aerial Vehicles

Planning and scheduling in construction heavily depend on current information about the state of construction processes. However, the acquisition process for visual data requires human personnel to take photographs of construction objects. We propose using unmanned aerial vehicle (UAVs) for automated creation of images and point cloud data of particular construction objects. The method extracts locations of objects that require inspection from Four Dimensional Building Information Modelling (4D-BIM). With this information at hand viable flight missions around the known structures of the construction site are computed. During flight, the UAV uses stereo cameras to detect and avoid any obstacles that are not known to the model, for example moving humans or machinery. The combination of pre-computed waypoint missions and reactive avoidance ensures deterministic routing from takeoff to landing and operational safety for humans and machines. During flight, an additional software component compares the captured point cloud data with the model data, enabling automatic per-object completion checking or reconstruction. The prototype is developed in the Robot Operating System (ROS) and evaluated in Software-In-The-Loop (SITL) simulations for the sake of being executable on real UAVs.

Two-Layer Routing for High-Voltage Powerline Inspection by Cooperated Ground Vehicle and Drone

A novel high-voltage powerline inspection system was investigated, which consists of the cooperated ground vehicle and drone. The ground vehicle acts as a mobile platform that can launch and recycle the drone, while the drone can fly over the powerline for inspection within limited endurance. This inspection system enables the drone to inspect powerline networks in a very large area. Both vehicle’ route in the road network and drone’s routes along the powerline network have to be optimized for improving the inspection efficiency, which generates a new Two-Layer Point-Arc Routing Problem (2L-PA-RP). Two constructive heuristics were designed based on “Cluster First, Route Second” and “Route First, Split Second”. Then, local search strategies were developed to further improve the quality of the solution. To test the performance of the proposed algorithms, different-scale practical cases were designed based on the road network and powerline network of Ji’an, China. Sensitivity analysis on the parameters related to the drone’s inspection speed and battery capacity was conducted. Computational results indicate that technical improvement on the inspection sensor is more important for the cooperated ground vehicle and drone system.