Reducing the complexity of visual navigation: Optical track controller for long‐range unmanned aerial vehicles

Sparse Optical Flow Implementation Using a Neural Network for Low-Resolution Thermal Aerial Imaging

This study is inspired by the widely used algorithm for real-time optical flow, the sparse Lucas-Kanade, by applying a feature extractor to decrease the computational requirement of optical flow based neural networks from real-world thermal aerial imagery. Although deep-learning-based algorithms have achieved state-of-the-art accuracy and have outperformed most traditional techniques, most of them cannot be implemented on a small multi-rotor UAV due to size and weight constraints on the platform. This challenge comes from the high computational cost of these techniques, with implementations requiring an integrated graphics processing unit with a powerful on-board computer to run in real time, resulting in a larger payload and consequently shorter flight time. For navigation applications that only require a 2D optical flow vector, a dense flow field computed from a deep learning neural network contains redundant information. A feature extractor based on the Shi-Tomasi technique was used to extract only appropriate features from thermal images to compute optical flow. The state-of-the-art RAFT-s model was trained with a full image and with our proposed alternative input, showing a substantial increase in speed while maintain its accuracy in the presence of high thermal contrast where features could be detected.

A ROS Multi-Tier UAV Localization Module Based on GNSS, Inertial and Visual-Depth Data

Uncrewed aerial vehicles (UAVs) are continuously gaining popularity in a wide spectrum of applications, while their positioning and navigation most often relies on Global Navigation Satellite Systems (GNSS). However, numerous conditions and practices require UAV operation in GNSS-denied environments, including confined spaces, urban canyons, vegetated areas and indoor places. For the purposes of this study, an integrated UAV navigation system was designed and implemented which utilizes GNSS, visual, depth and inertial data to provide real-time localization. The implementation is built as a package for the Robotic Operation System (ROS) environment to allow ease of integration in various systems. The system can be autonomously adjusted to the flight environment, providing spatial awareness to the aircraft. This system expands the functionality of UAVs, as it enables navigation even in GNSS-denied environments. This integrated positional system provides the means to support fully autonomous navigation under mixed environments, or malfunctioning conditions. Experiments show the capability of the system to provide adequate results in open, confined and mixed spaces.

A Comparison of Dense and Sparse Optical Flow Techniques for Low-Resolution Aerial Thermal Imagery

It is necessary to establish the relative performance of established optical flow approaches in airborne scenarios with thermal cameras. This study investigated the performance of a dense optical flow algorithm on 14 bit radiometric images of the ground. While sparse techniques that rely on feature matching techniques perform very well with airborne thermal data in high-contrast thermal conditions, these techniques suffer in low-contrast scenes, where there are fewer detectable and distinct features in the image. On the other hand, some dense optical flow algorithms are highly amenable to parallel processing approaches compared to those that rely on tracking and feature detection. A Long-Wave Infrared (LWIR) micro-sensor and a PX4Flow optical sensor were mounted looking downwards on a drone. We compared the optical flow signals of a representative dense optical flow technique, the Image Interpolation Algorithm (I2A), to the Lucas-Kanade (LK) algorithm in OpenCV and the visible light optical flow results from the PX4Flow in both X and Y displacements. The I2A to LK was found to be generally comparable in performance and better in cold-soaked environments while suffering from the aperture problem in some scenes.

A Review of Modern Thermal Imaging Sensor Technology and Applications for Autonomous Aerial Navigation

Limited navigation capabilities of many current robots and UAVs restricts their applications in GPS denied areas. Large aircraft with complex navigation systems rely on a variety of sensors including radio frequency aids and high performance inertial systems rendering them somewhat resistant to GPS denial. The rapid development of computer vision has seen cameras incorporated into small drones. Vision-based systems, consisting of one or more cameras, could arguably satisfy both size and weight constraints faced by UAVs. A new generation of thermal sensors is available that are lighter, smaller and widely available. Thermal sensors are a solution to enable navigation in difficult environments, including in low-light, dust or smoke. The purpose of this paper is to present a comprehensive literature review of thermal sensors integrated into navigation systems. Furthermore, the physics and characteristics of thermal sensors will also be presented to provide insight into challenges when integrating thermal sensors in place of conventional visual spectrum sensors.