Three-dimensional mapping of pipelines using laser ranging and a gyroscope

Conventional underground gas pipeline equipment can only map straight pipelines at close distances. As such, a dual-robot mapping technique is proposed in this study, based on laser ranging and a gyroscope, for application to straight, bent, or sloped underground pipelines. This process is facilitated by two robots (a detecting vehicle and a mapping vehicle) alternately moving from the entrance to the exit of a pipe system. The two vehicles acquire data using individual distance sensors and a gyroscope, producing 3D maps of each section. Mapping results can then be produced for the entire pipeline by splicing images from multiple sections. Simulated validation experiments were conducted using horizontal straight, bent horizontal fixed turning radius, sloped uphill or downhill, and composite sections. Results showed this approach could effectively map different types of pipelines with inner diameters of 300-500 mm. Distance errors over 2000 m of travel were within 1 m, while the angular error was within 1.5°, demonstrating our approach to be highly accurate for complex pipeline system mapping.

Architecture for a Mobile Robotic Camera Positioning System for Photogrammetric Data Acquisition in Hydroelectric Tunnels

The structural condition of hydroelectric tunnels is important to the overall performance, safety, and longevity of generating stations. Significant effort is required to inspect, monitor, and maintain these tunnels. Photogrammetry is an effective method of collecting highly accurate visual and spatial data. However, it also presents the complex challenge of positioning a camera at thousands of difficult-to-reach locations throughout the large and varying-diameter tunnels. A semi-automated robotic camera positioning system was developed to enhance the collection of images within hydroelectric tunnels for photogrammetric inspections. A continuous spiral image network was developed to optimize the collection speed within the bounds of photography and capture-in-motion constraints. The positioning system and image network optimization reduce the time and effort required while providing the ability to adapt to different and varying tunnel diameters. To demonstrate, over 28,000 images were captured at a ground sampling distance of 0.4 mm in the 822 m long concrete-lined section of the Grand Falls Generating Station intake tunnel.

Accurate integrated position measurement system for mobile applications in GPS-denied coal mine

The automatic positioning of underground mobile applications plays a crucial role in enabling intelligent coal mining. However, due to the diverse kinematics and dynamics of these applications, various positioning methods have been proposed to match different targets. Nonetheless, the accuracy and applicability of these methods still fall short of meeting the requirements for field applications. Based on the vibration characteristics of underground mobile devices, a multi-sensor fusion positioning system is developed to enhance the accuracy of positioning in long and narrow global positioning system denied (GPS-denied) underground coal mine roadways. The system combines inertial navigation (INS), odometer, and ultra wide band (UWB) technologies through extended Kalman filter (EKF) and unscented Kalman filter (UKF). This approach enables accurate positioning by recognizing target carrier vibrations and facilitating fast conversion between multi-sensor fusion modes. The proposed system is tested on both a small unmanned mine vehicle (UMV) and a large roadheader, demonstrating that UKF enhances stability for roadheaders with strong nonlinear vibrations while EKF is more suitable for flexible UMVs. Detailed results confirm that the proposed system achieves an accuracy level of 0.15 m, meeting most coal mine application requirements.

Versatile articulated aerial robot DRAGON: Aerial manipulation and grasping by vectorable thrust control

Various state-of-the-art works have achieved aerial manipulation and grasping by attaching additional manipulator to aerial robots. However, such a coupled platform has limitations with respect to the interaction force and mobility. In this paper, we present the successful implementation of aerial manipulation and grasping by a novel articulated aerial robot called DRAGON, in which a vectorable rotor unit is embedded in each link. The key to performing stable manipulation and grasping in the air is the usage of rotor vectoring apparatus having two degrees-of-freedom. First, a comprehensive flight control methodology for aerial transformation using the vectorable thrust force is developed with the consideration of the dynamics of vectoring actuators. This proposed control method can suppress the oscillation due to the dynamics of vectoring actuators and also allow the integration with external and internal wrenches for object manipulation and grasping. Second, an online thrust-level planning method for bimanual object grasping using the two ends of this articulated model is presented. The proposed grasping style is unique in that the vectorable thrust force is used as the internal wrench instead of the joint torque. Finally, we show the experimental results of evaluation on the proposed control and planning methods for object manipulation and grasping.

COMPRA: A COMPact Reactive Autonomy Framework for Subterranean MAV Based Search-And-Rescue Operations

This work establishes COMPRA, a compact and reactive autonomy framework for fast deployment of Micro Aerial Vehicles (MAVs) in subterranean Search-and- Rescue (SAR) missions. A COMPRA-enabled MAV is able to autonomously explore previously unknown areas while specific mission criteria are considered e.g. an object of interest is identified and localized, the remaining useful battery life, the overall desired exploration mission duration. The proposed architecture follows a low-complexity algorithmic design to facilitate fully on-board computations, including nonlinear control, state-estimation, navigation, exploration behavior and object localization capabilities. The framework is mainly structured around a reactive local avoidance planner, based on enhanced Potential Field concepts and using instantaneous 3D pointclouds, as well as a computationally efficient heading regulation technique, based on depth images from an instantaneous camera stream. Those techniques decouple the collision-free path generation from the dependency of a global map and are capable of handling imprecise localization occasions. Field experimental verification of the overall architecture is performed in relevant unknown Global Positioning System (GPS)-denied environments.

An Effective Algorithm for Finding Shortest Paths in Tubular Spaces

We propose a novel algorithm to determine the Euclidean shortest path (ESP) from a given point (source) to another point (destination) inside a tubular space. The method is based on the observation data of a virtual particle (VP) assumed to move along this path. In the first step, the geometric properties of the shortest path inside the considered space are presented and proven. Utilizing these properties, the desired ESP can be segmented into three partitions depending on the visibility of the VP. Our algorithm will check which partition the VP belongs to and calculate the correct direction of its movement, and thus the shortest path will be traced. The proposed method is then compared to Dijkstra’s algorithm, considering different types of tubular spaces. In all cases, the solution provided by the proposed algorithm is smoother, shorter, and has a higher accuracy with a faster calculation speed than that obtained by Dijkstra’s method.

Drone Control in AR: An Intuitive System for Single-Handed Gesture Control, Drone Tracking, and Contextualized Camera Feed Visualization in Augmented Reality

Traditional drone handheld remote controllers, although well-established and widely used, are not a particularly intuitive control method. At the same time, drone pilots normally watch the drone video feed on a smartphone or another small screen attached to the remote. This forces them to constantly shift their visual focus from the drone to the screen and vice-versa. This can be an eye-and-mind-tiring and stressful experience, as the eyes constantly change focus and the mind struggles to merge two different points of view. This paper presents a solution based on Microsoft’s HoloLens 2 headset that leverages augmented reality and gesture recognition to make drone piloting easier, more comfortable, and more intuitive. It describes a system for single-handed gesture control that can achieve all maneuvers possible with a traditional remote, including complex motions; a method for tracking a real drone in AR to improve flying beyond line of sight or at distances where the physical drone is hard to see; and the option to display the drone’s live video feed in AR, either in first-person-view mode or in context with the environment.

Design and Experimental Evaluation of an Aerial Solution for Visual Inspection of Tunnel-like Infrastructures

Current railway tunnel inspections rely on expert operators performing a visual examination of the entire infrastructure and manually annotating encountered defects. Automatizing the inspection and maintenance task of such critical and aging infrastructures has the potential to decrease the associated costs and risks. Contributing to this aim, the present work describes an aerial robotic solution designed to perform autonomous inspections of tunnel-like infrastructures. The proposed robotic system is equipped with visual and thermal sensors and uses an inspection-driven path planning algorithm to generate a path that maximizes the quality of the gathered data in terms of photogrammetry goals while optimizing the surface coverage and the total trajectory length. The performance of the planning algorithm is demonstrated in simulation against state-of-the-art methods and a wall-following inspection trajectory. Results of a real inspection test conducted in a railway tunnel are also presented, validating the whole system operation.

Learning high-speed flight in the wild

Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines.

MUSSOL: A Micro-Uas to Survey Ship Cargo hOLds

Because of their high maneuverability and fast deployment times, aerial robots have recently gained popularity for automating inspection tasks. In this paper, we address the visual inspection of vessel cargo holds, aiming at safer, cost-efficient and more intensive visual inspections of ships by means of a multirotor-type platform. To this end, the vehicle is equipped with a sensor suite able to supply the surveyor with imagery from relevant areas, while the control software is supporting the operator during flight with enhanced functionalities and reliable autonomy. All this has been accomplished in the context of the supervised autonomy (SA) paradigm, by means of extensive use of behaviour-based high-level control (including obstacle detection and collision prevention), all specifically devised for visual inspection. The full system has been evaluated both in laboratory and in real environments, on-board two different vessels. Results show the vehicle effective for the referred application, in particular due to the inspection-oriented capabilities it has been fitted with.

A method for autonomous collision-free navigation of a quadrotor UAV in unknown tunnel-like environments

Unmanned aerial vehicles (UAVs) have become essential tools for exploring, mapping and inspection of unknown three-dimensional (3D) tunnel-like environments which is a very challenging problem. A computationally light navigation algorithm is developed in this paper for quadrotor UAVs to autonomously guide the vehicle through such environments. It uses sensors observations to safely guide the UAV along the tunnel axis while avoiding collisions with its walls. The approach is evaluated using several computer simulations with realistic sensing models and practical implementation with a quadrotor UAV. The proposed method is also applicable to other UAV types and autonomous underwater vehicles.

Path Planner for Autonomous Exploration of Underground Mines by Aerial Vehicles

This paper presents a path planner solution that makes it possible to autonomously explore underground mines with aerial robots (typically multicopters). In these environments the operations may be limited by many factors like the lack of external navigation signals, the narrow passages and the absence of radio communications. The designed path planner is defined as a simple and highly computationally efficient algorithm that, only relying on a laser imaging detection and ranging (LIDAR) sensor with Simultaneous localization and mapping (SLAM) capability, permits the exploration of a set of single-level mining tunnels. It performs dynamic planning based on exploration vectors, a novel variant of the open sector method with reinforced filtering. The algorithm incorporates global awareness and obstacle avoidance modules. The first one prevents the possibility of getting trapped in a loop, whereas the second one facilitates the navigation along narrow tunnels. The performance of the proposed solution has been tested in different study cases with a Hardware-in-the-loop (HIL) simulator developed for this purpose. In all situations the path planner logic performed as expected and the used routing was optimal. Furthermore, the path efficiency, measured in terms of traveled distance and used time, was high when compared with an ideal reference case. The result is a very fast, real-time, and static memory capable algorithm, which implemented on the proposed architecture presents a feasible solution for the autonomous exploration of underground mines.

A Drag Model-LIDAR-IMU Fault-Tolerance Fusion Method for Quadrotors

In this paper, a drag model-aided fault-tolerant state estimation method is presented for quadrotors. Firstly, the drag model accuracy was improved by modeling an angular rate related item and an angular acceleration related item, which are related with flight maneuver. Then the drag model, light detection and ranging (LIDAR), and inertial measurement unit (IMU) were fused based on the Federal Kalman filter frame. In the filter, the LIDAR estimation fault was detected and isolated, and the disturbance to the drag model was estimated and compensated. Some experiments were carried out, showing that the velocity and position estimation were improved compared with the traditional LIDAR/IMU fusion scheme.

A Comparison of Adaptive View Techniques for Exploratory 3D Drone Teleoperation

Drone navigation in complex environments poses many problems to teleoperators. Especially in three dimensional (3D) structures such as buildings or tunnels, viewpoints are often limited to the drone’s current camera view, nearby objects can be collision hazards, and frequent occlusion can hinder accurate manipulation.

To address these issues, we have developed a novel interface for teleoperation that provides a user with environment-adaptive viewpoints that are automatically configured to improve safety and provide smooth operation. This real-time adaptive viewpoint system takes robot position, orientation, and 3D point-cloud information into account to modify the user’s viewpoint to maximize visibility. Our prototype uses simultaneous localization and mapping (SLAM) based reconstruction with an omnidirectional camera, and we use the resulting models as well as simulations in a series of preliminary experiments testing navigation of various structures. Results suggest that automatic viewpoint generation can outperform first- and third-person view interfaces for virtual teleoperators in terms of ease of control and accuracy of robot operation.