Applications, databases and open computer vision research from drone videos and images: a survey

A Review on the State of the Art in Copter Drones and Flight Control Systems

This paper presents an overview on the state of the art in copter drones and their components. It starts by providing an introduction to unmanned aerial vehicles in general, describing their main types, and then shifts its focus mostly to multirotor drones as the most attractive for individual and research use. This paper analyzes various multirotor drone types, their construction, typical areas of implementation, and technology used underneath their construction. Finally, it looks at current challenges and future directions in drone system development, emerging technologies, and future research topics in the area. This paper concludes by highlighting some key challenges that need to be addressed before widespread adoption of drone technologies in everyday life can occur. By summarizing an up-to-date survey on the state of the art in copter drone technology, this paper will provide valuable insights into where this field is heading in terms of progress and innovation.

Vision-Based Flying Obstacle Detection for Avoiding Midair Collisions: A Systematic Review

This paper presents a systematic review of articles on computer-vision-based flying obstacle detection with a focus on midair collision avoidance. Publications from the beginning until 2022 were searched in Scopus, IEEE, ACM, MDPI, and Web of Science databases. From the initial 647 publications obtained, 85 were finally selected and examined. The results show an increasing interest in this topic, especially in relation to object detection and tracking. Our study hypothesizes that the widespread access to commercial drones, the improvements in single-board computers, and their compatibility with computer vision libraries have contributed to the increase in the number of publications. The review also shows that the proposed algorithms are mainly tested using simulation software and flight simulators, and only 26 papers report testing with physical flying vehicles. This systematic review highlights other gaps to be addressed in future work. Several identified challenges are related to increasing the success rate of threat detection and testing solutions in complex scenarios.

Vehicle Instance Segmentation Polygonal Dataset for a Private Surveillance System

Vehicle identification and re-identification is an essential tool for traffic surveillance. However, with cameras at every corner of the street, there is a requirement for private surveillance. Automated surveillance can be achieved through computer vision tasks such as segmentation of the vehicle, classification of the make and model of the vehicle and license plate detection. To achieve a unique representation of every vehicle on the road with just the region of interest extracted, instance segmentation is applied. With the frontal part of the vehicle segmented for privacy, the vehicle make is identified along with the license plate. To achieve this, a dataset is annotated with a polygonal bounding box of its frontal region and license plate localization. State-of-the-art methods, maskRCNN, is utilized to identify the best performing model. Further, data augmentation using multiple techniques is evaluated for better generalization of the dataset. The results showed improved classification as well as a high mAP for the dataset when compared to previous approaches on the same dataset. A classification accuracy of 99.2% was obtained and segmentation was achieved with a high mAP of 99.67%. Data augmentation approaches were employed to balance and generalize the dataset of which the mosaic-tiled approach produced higher accuracy.

Feature fusion based on joint sparse representations and wavelets for multiview classification

Feature-level-based fusion has attracted much interest. Generally, a dataset can be created in different views, features, or modalities. To improve the classification rate, local information is shared among different views by various fusion methods. However, almost all the methods use the views without considering their common aspects. In this paper, wavelet transform is considered to extract high and low frequencies of the views as common aspects to improve the classification rate. The fusion method for the decomposed parts is based on joint sparse representation in which a number of scenarios can be considered. The presented approach is tested on three datasets. The results obtained by this method prove competitive performance in terms of the datasets compared to the state-of-the-art results.

Building a UAV Based System to Acquire High Spatial Resolution Thermal Imagery for Energy Balance Modelling

High spatial resolution and geolocation accuracy canopy evapotranspiration (ET) maps are well suited tools for evaluation of small plot field trials. While creating such a map by use of an energy balance model is routinely performed, the acquisition of the necessary imagery at a suitable quality is still challenging. An UAV based thermal/RGB integrated imaging system was built using the RaspberryPi (RPi) microcomputer as a central unit. The imagery served as input to the two-source energy balance model pyTSEB to derive the ET map. The setup's flexibility and modularity are based on the multiple interfaces provided by the RPi and the software development kit (SDK) provided for the thermal camera. The SDK was installed on the RPi and used to trigger cameras, retrieve and store images and geolocation information from an onboard GNSS rover for PPK processing. The system allows acquisition of 8 cm spatial resolution thermal imagery from a 60 m height of flight and less than 7 cm geolocation accuracy of the mosaicked RGB imagery. Modelled latent heat flux data have been validated against latent heat fluxes measured by eddy covariance stations at two locations with RMSE of 75 W/m² over a two-year study period.

Simultaneous Localization and Mapping (SLAM) and Data Fusion in Unmanned Aerial Vehicles: Recent Advances and Challenges

This article presents a survey of simultaneous localization and mapping (SLAM) and data fusion techniques for object detection and environmental scene perception in unmanned aerial vehicles (UAVs). We critically evaluate some current SLAM implementations in robotics and autonomous vehicles and their applicability and scalability to UAVs. SLAM is envisioned as a potential technique for object detection and scene perception to enable UAV navigation through continuous state estimation. In this article, we bridge the gap between SLAM and data fusion in UAVs while also comprehensively surveying related object detection techniques such as visual odometry and aerial photogrammetry. We begin with an introduction to applications where UAV localization is necessary, followed by an analysis of multimodal sensor data fusion to fuse the information gathered from different sensors mounted on UAVs. We then discuss SLAM techniques such as Kalman filters and extended Kalman filters to address scene perception, mapping, and localization in UAVs. The findings are summarized to correlate prevalent and futuristic SLAM and data fusion for UAV navigation, and some avenues for further research are discussed.