Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas

YOLOv8-MPEB small target detection algorithm based on UAV images

Target detection in Unmanned Aerial Vehicle (UAV) aerial images has gained significance within UAV application scenarios. However, UAV aerial images present challenges, including large-scale changes, small target sizes, complex scenes, and variable external factors, resulting in missed or false detections. This study proposes an algorithm for small target detection in UAV images based on an enhanced YOLOv8 model termed YOLOv8-MPEB. Firstly, the Cross Stage Partial Darknet53 (CSPDarknet53) backbone network is substituted with the lightweight MobileNetV3 backbone network, consequently reducing model parameters and computational complexity, while also enhancing inference speed. Secondly, a dedicated small target detection layer is intricately designed to optimize feature extraction for multi-scale targets. Thirdly, the integration of the Efficient Multi-Scale Attention (EMA) mechanism within the Convolution to Feature (C2f) module aims to enhance the extraction of vital features and suppress superfluous ones. Lastly, the utilization of a bidirectional feature pyramid network (BiFPN) in the Neck segment serves to ameliorate detection errors stemming from scale variations and complex scenes, thereby augmenting model generalization. The study provides a thorough examination by conducting ablation experiments and comparing the results with alternative algorithms to substantiate the enhanced effectiveness of the proposed algorithm, with a particular focus on detection performance. The experimental outcomes illustrate that with a parameter count of 7.39 M and a model size of 14.5 MB, the algorithm attains a mean Average Precision (mAP) of 91.9 % on the custom-made helmet and reflective clothing dataset. In comparison to standard YOLOv8 models, this algorithm elevates average accuracy by 2.2 percentage points, reduces model parameters by 34 %, and diminishes model size by 32 %. It outperforms other prevalent detection algorithms in terms of accuracy and speed.

Risks of Drone Use in Light of Literature Studies

This article aims to present the results of a bibliometric analysis of relevant literature and discuss the main research streams related to the topic of risks in drone applications. The methodology of the conducted research consisted of five procedural steps, including the planning of the research, conducting a systematic review of the literature, proposing a classification framework corresponding to contemporary research trends related to the risk of drone applications, and compiling the characteristics of the publications assigned to each of the highlighted thematic groups. This systematic literature review used the PRISMA method. A total of 257 documents comprising articles and conference proceedings were analysed. On this basis, eight thematic categories related to the use of drones and the risks associated with their operation were distinguished. Due to the high content within two of these categories, a further division into subcategories was proposed to illustrate the research topics better. The conducted investigation made it possible to identify the current research trends related to the risk of drone use and pointed out the existing research gaps, both in the area of risk assessment methodology and in its application areas. The results obtained from the analysis can provide interesting material for both industry and academia.

Real-Time Person Detection in Wooded Areas Using Thermal Images from an Aerial Perspective

Detecting people in images and videos captured from an aerial platform in wooded areas for search and rescue operations is a current problem. Detection is difficult due to the relatively small dimensions of the person captured by the sensor in relation to the environment. The environment can generate occlusion, complicating the timely detection of people. There are currently numerous RGB image datasets available that are used for person detection tasks in urban and wooded areas and consider the general characteristics of a person, like size, shape, and height, without considering the occlusion of the object of interest. The present research work focuses on developing a thermal image dataset, which considers the occlusion situation to develop CNN convolutional deep learning models to perform detection tasks in real-time from an aerial perspective using altitude control in a quadcopter prototype. Extended models are proposed considering the occlusion of the person, in conjunction with a thermal sensor, which allows for highlighting the desired characteristics of the occluded person.

For Heart Rate Assessments from Drone Footage in Disaster Scenarios

The ability to use drones to obtain important vital signs could be very valuable for emergency personnel during mass-casualty incidents. The rapid and robust remote assessment of heart rates could serve as a life-saving decision aid for first-responders. With the flight sensor data of a specialized drone, a pipeline was developed to achieve a robust, non-contact assessment of heart rates through remote photoplethysmography (rPPG). This robust assessment was achieved through adaptive face-aware exposure and comprehensive de-noising of a large number of predicted noise sources. In addition, we performed a proof-of-concept study that involved 18 stationary subjects with clean skin and 36 recordings of their vital signs, using the developed pipeline in outdoor conditions. In this study, we could achieve a single-value heart-rate assessment with an overall root-mean-squared error of 14.3 beats-per-minute, demonstrating the basic feasibility of our approach. However, further research is needed to verify the applicability of our approach in actual disaster situations, where remote photoplethysmography readings could be impacted by other factors, such as blood, dirt, and body positioning.

Small-scale location identification in natural environments with deep learning based on biomimetic sonar echoes

Many bat species navigate in complex, heavily vegetated habitats. To achieve this, the animal relies on a sensory basis that is very different from what is typically done in engineered systems that are designed for outdoor navigation. Whereas the engineered systems rely on data-heavy senses such as lidar, bats make do with echoes triggered by short, ultrasonic pulses. Prior work has shown that 'clutter echoes' originating from vegetation can convey information on the environment they were recorded in-despite their unpredictable nature. The current work has investigated the spatial granularity that these clutter echoes can convey by applying deep-learning location identification to an echo data set that resulted from the dense spatial sampling of a forest environment. The Global Positioning System (GPS) location corresponding to the echo collection events was clustered to break the survey area into the number of spatial patches ranging from two to 100. A convolutional neural network (Resnet 152) was used to identify the patch associated with echo sets ranging from one to ten echoes. The results demonstrate a spatial resolution that is comparable to the accuracy of recreation-grade GPS operating under foliage cover. This demonstrates that fine-grained location identification can be accomplished at very low data rates.

Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones

In this paper, we aim to open up new perspectives in the field of autonomous aerial surveillance and target tracking systems, by exploring an alternative that, surprisingly, and to the best of the authors’ knowledge, has not been addressed in that context by the research community thus far. It can be summarized by the following two questions. Under the scope of such applications, what are the implications and possibilities offered by mounting several steerable cameras onboard of each aerial agent? Second, how can optimization algorithms benefit from this new framework, in their attempt to provide more efficient and cost-effective solutions on these areas? The paper presents the idea as an additional degree of freedom to be exploited, which can enable more efficient alternatives in the deployment of such applications. As an initial approach, the problem of the optimal positioning with respect to a set of targets of one single agent, equipped with several onboard tracking cameras with different or variable focal lengths, is addressed. As a consequence of this allowed heterogeneity in focal lengths, the notion of distance needs to be adapted into a notion of optical range, as the agent can trade longer Euclidean distances for correspondingly longer focal lengths. Moreover, the proposed optimization indices try to balance, in an optimal way, the verticality of the viewpoints along with the optical range to the targets. Under these premises, several positioning strategies are proposed and comparatively evaluated.

Drone Control in AR: An Intuitive System for Single-Handed Gesture Control, Drone Tracking, and Contextualized Camera Feed Visualization in Augmented Reality

Traditional drone handheld remote controllers, although well-established and widely used, are not a particularly intuitive control method. At the same time, drone pilots normally watch the drone video feed on a smartphone or another small screen attached to the remote. This forces them to constantly shift their visual focus from the drone to the screen and vice-versa. This can be an eye-and-mind-tiring and stressful experience, as the eyes constantly change focus and the mind struggles to merge two different points of view. This paper presents a solution based on Microsoft’s HoloLens 2 headset that leverages augmented reality and gesture recognition to make drone piloting easier, more comfortable, and more intuitive. It describes a system for single-handed gesture control that can achieve all maneuvers possible with a traditional remote, including complex motions; a method for tracking a real drone in AR to improve flying beyond line of sight or at distances where the physical drone is hard to see; and the option to display the drone’s live video feed in AR, either in first-person-view mode or in context with the environment.

Improving the Model for Person Detection in Aerial Image Sequences Using the Displacement Vector: A Search and Rescue Scenario

Recent results in person detection using deep learning methods applied to aerial images gathered by Unmanned Aerial Vehicles (UAVs) have demonstrated the applicability of this approach in scenarios such as Search and Rescue (SAR) operations. In this paper, the continuation of our previous research is presented. The main goal is to further improve detection results, especially in terms of reducing the number of false positive detections and consequently increasing the precision value. We present a new approach that, as input to the multimodel neural network architecture, uses sequences of consecutive images instead of only one static image. Since successive images overlap, the same object of interest needs to be detected in more than one image. The correlation between successive images was calculated, and detected regions in one image were translated to other images based on the displacement vector. The assumption is that an object detected in more than one image has a higher probability of being a true positive detection because it is unlikely that the detection model will find the same false positive detections in multiple images. Based on this information, three different algorithms for rejecting detections and adding detections from one image to other images in the sequence are proposed. All of them achieved precision value about 80% which is increased by almost 20% compared to the current state-of-the-art methods.

Applications of Uncrewed Aerial Vehicles (UAV) Technology to Support Integrated Coastal Zone Management and the UN Sustainable Development Goals at the Coast

Data and information obtained from low-cost uncrewed aerial vehicles (UAVs), commonly referred to as 'drones', can be used to support integrated coastal zone management (ICZM) and sustainable development at the coast. Several recent studies in various disciplines, including ecology, engineering, and several branches of physical and human geography, describe the applications of UAV technology with practical coastal management potential, yet the extent to which such data can contribute to these activities remains underexplored. The main objective of this paper is to collate this knowledge to highlight the areas in which UAV technology can contribute to ICZM and can influence the achievement of the UN Sustainable Development Goals (SDGs) at the coast. We focus on applications with practical potential for coastal management activities and assess their accessibility in terms of cost, ease of use, and maturity. We identified ten (out of the 17) SDGs to which UAVs can contribute data and information. Examples of applications include surveillance of illegal fishing and aquaculture activities, seaweed resource assessments, cost-estimation of post-storm damages, and documentation of natural and cultural heritage sites under threat from, for example, erosion and sea-level rise. An awareness of how UAVs can contribute to ICZM, as well as the limitations of the technology, can help coastal practitioners to evaluate their options for future management activities.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s12237-021-01001-5.

Using an unmanned aerial system to monitor and assess irrigation water channels susceptible to sediment deposition

The irrigation channel of the Qishan River is among the most crucial agricultural water resource facilities in Qishan District, Kaohsiung City, Taiwan. The channel was blocked by debris due to flood events caused by Typhoon Morakot in 2009. This study analyzed images captured by an unmanned aerial system to identify channel areas susceptible to sediment deposition and propose measures for reducing the effects of natural hazards on irrigation water resources. The analysis results revealed that the channel was located downstream of the Qishan River; however, debris flows, riverbank landslides, and natural dam breaches deposited sediment in the downstream section, preventing the flow of water. Furthermore, the sediment and driftwood blocked the channel. The channel was also blocked due to a hyperconcentrated flow. Sediment deposition areas and volumes were estimated. On the basis of these results, we suggest that the damaged riverbed groundsills and river tributary banks be restored to inhibit erosion. In addition, subsurface water collection and transfer structures should be constructed to maintain the flow of water during the dry season. The study findings are expected to increase the efficiency of agricultural irrigation water management and prevent natural hazards from affecting water resources.

Moving People Tracking and False Track Removing with Infrared Thermal Imaging by a Multirotor

Infrared (IR) thermal imaging can detect the warm temperature of the human body regardless of the light conditions, thus small drones equipped with the IR thermal camera can be utilized to recognize human activity for smart surveillance, road safety, and search and rescue missions. However, the unpredictable motion of the drone poses more challenges than a fixed camera. This paper addresses the detection and tracking of people through IR thermal video captured by a multirotor. For object detection, each frame is first registered with a reference frame to compensate for its coordinates. Then, the objects in each frame are segmented through k-means clustering and morphological operations. Falsely detected objects are removed considering the actual size and the shape of the object. The centroid of the segmented area is considered the measured position for target tracking. The track is initialized with two-point differencing initialization, and the target states are continuously estimated by the interacting multiple model (IMM) filter. The nearest neighbor association rule assigns the measurement to the track. Tracks that move slower than the minimum speed are terminated at the proposed criteria. In the experiments, three videos were captured with a long-wave IR band thermal imaging camera mounted on a multirotor. In the first and second videos, eight pedestrians on a pavement and three hikers on a mountain on winter nights were captured, respectively. In the third video, two walking people with complex backgrounds were captured on a windy summer day. The image characteristics vary between videos depending on the climate and surrounding objects, but the proposed scheme shows the robust performance in all cases; the average root mean squared errors in position and velocity are obtained as 0.08 m and 0.53 m/s, respectively for the first video, 0.06 m and 0.58 m/s, respectively for the second video, and 0.18 m and 1.84 m/s, respectively for the third video. The proposed method reduces false tracks from 10 to 1 in the third video.

Evaluation and Selection of Video Stabilization Techniques for UAV-Based Active Infrared Thermography Application

Unmanned Aerial Vehicles (UAVs) that can fly around an aircraft carrying several sensors, e.g., thermal and optical cameras, to inspect the parts of interest without removing them can have significant impact in reducing inspection time and cost. One of the main challenges in the UAV based active InfraRed Thermography (IRT) inspection is the UAV’s unexpected motions. Since active thermography is mainly concerned with the analysis of thermal sequences, unexpected motions can disturb the thermal profiling and cause data misinterpretation especially for providing an automated process pipeline of such inspections. Additionally, in the scenarios where post-analysis is intended to be applied by an inspector, the UAV’s unexpected motions can increase the risk of human error, data misinterpretation, and incorrect characterization of possible defects. Therefore, post-processing is required to minimize/eliminate such undesired motions using digital video stabilization techniques. There are number of video stabilization algorithms that are readily available; however, selecting the best suited one is also challenging. Therefore, this paper evaluates video stabilization algorithms to minimize/mitigate undesired UAV motion and proposes a simple method to find the best suited stabilization algorithm as a fundamental first step towards a fully operational UAV-IRT inspection system.

Multimodel Deep Learning for Person Detection in Aerial Images

In this paper, we propose a novel method for person detection in aerial images of nonurban terrain gathered by an Unmanned Aerial Vehicle (UAV), which plays an important role in Search And Rescue (SAR) missions. The UAV in SAR operations contributes significantly due to the ability to survey a larger geographical area from an aerial viewpoint. Because of the high altitude of recording, the object of interest (person) covers a small part of an image (around 0.1%), which makes this task quite challenging. To address this problem, a multimodel deep learning approach is proposed. The solution consists of two different convolutional neural networks in region proposal, as well as in the classification stage. Additionally, contextual information is used in the classification stage in order to improve the detection results. Experimental results tested on the HERIDAL dataset achieved precision of 68.89% and a recall of 94.65%, which is better than current state-of-the-art methods used for person detection in similar scenarios. Consequently, it may be concluded that this approach is suitable for usage as an auxiliary method in real SAR operations.