UAVs and Machine Learning Revolutionising Invasive Grass and Vegetation Surveys in Remote Arid Lands

Preliminary Classification of Selected Farmland Habitats in Ireland Using Deep Neural Networks

Ireland has a wide variety of farmlands that includes arable fields, grassland, hedgerows, streams, lakes, rivers, and native woodlands. Traditional methods of habitat identification rely on field surveys, which are resource intensive, therefore there is a strong need for digital methods to improve the speed and efficiency of identification and differentiation of farmland habitats. This is challenging because of the large number of subcategories having nearly indistinguishable features within the habitat classes. Heterogeneity among sites within the same habitat class is another problem. Therefore, this research work presents a preliminary technique for accurate farmland classification using stacked ensemble deep convolutional neural networks (DNNs). The proposed approach has been validated on a high-resolution dataset collected using drones. The image samples were manually labelled by the experts in the area before providing them to the DNNs for training purposes. Three pre-trained DNNs customized using the transfer learning approach are used as the base learners. The predicted features derived from the base learners were then used to train a DNN based meta-learner to achieve high classification rates. We analyse the obtained results in terms of convergence rate, confusion matrices, and ROC curves. This is a preliminary work and further research is needed to establish a standard technique.

Very High-Resolution Imagery and Machine Learning for Detailed Mapping of Riparian Vegetation and Substrate Types

Riparian zones fulfill diverse ecological and economic functions. Sustainable management requires detailed spatial information about vegetation and hydromorphological properties. In this study, we propose a machine learning classification workflow to map classes of the thematic levels Basic surface types (BA), Vegetation units (VE), Dominant stands (DO) and Substrate types (SU) based on multispectral imagery from an unmanned aerial system (UAS). A case study was carried out in Emmericher Ward on the river Rhine, Germany. The results showed that: (I) In terms of overall accuracy, classification results decreased with increasing detail of classes from BA (88.9%) and VE (88.4%) to DO (74.8%) or SU (62%), respectively. (II) The use of Support Vector Machines and Extreme Gradient Boost algorithms did not increase classification performance in comparison to Random Forest. (III) Based on probability maps, classification performance was lower in areas of shaded vegetation and in the transition zones. (IV) In order to cover larger areas, a gyrocopter can be used applying the same workflow and achieving comparable results as by UAS for thematic levels BA, VE and homogeneous classes covering larger areas. The generated classification maps are a valuable tool for ecologically integrated water management.

Estimating Fuel Moisture in Grasslands Using UAV-Mounted Infrared and Visible Light Sensors

Predicting wildfire behavior is a complex task that has historically relied on empirical models. Physics-based fire models could improve predictions and have broad applicability, but these models require more detailed inputs, including spatially explicit estimates of fuel characteristics. One of the most critical of these characteristics is fuel moisture. Obtaining moisture measurements with traditional destructive sampling techniques can be prohibitively time-consuming and extremely limited in spatial resolution. This study seeks to assess how effectively moisture in grasses can be estimated using reflectance in six wavelengths in the visible and infrared ranges. One hundred twenty 1 m-square field samples were collected in a western Washington grassland as well as overhead imagery in six wavelengths for the same area. Predictive models of vegetation moisture using existing vegetation indices and components from principal component analysis of the wavelengths were generated and compared. The best model, a linear model based on principal components and biomass, showed modest predictive power (r² = 0.45). This model performed better for the plots with both dominant grass species pooled than it did for each species individually. The presence of this correlation, especially given the limited moisture range of this study, suggests that further research using samples across the entire fire season could potentially produce effective models for estimating moisture in this type of ecosystem using unmanned aerial vehicles, even when more than one major species of grass is present. This approach would be a fast and flexible approach compared to traditional moisture measurements.

The Rapid Rise of Next-Generation Natural History

Many ecologists have lamented the demise of natural history and have attributed this decline to a misguided view that natural history is outdated and unscientific. Although there is a perception that the focus in ecology and conservation have shifted away from descriptive natural history research and training toward hypothetico-deductive research, we argue that natural history has entered a new phase that we call “next-generation natural history.” This renaissance of natural history is characterized by technological and statistical advances that aid in collecting detailed observations systematically over broad spatial and temporal extents. The technological advances that have increased exponentially in the last decade include electronic sensors such as camera-traps and acoustic recorders, aircraft- and satellite-based remote sensing, animal-borne biologgers, genetics and genomics methods, and community science programs. Advances in statistics and computation have aided in analyzing a growing quantity of observations to reveal patterns in nature. These robust next-generation natural history datasets have transformed the anecdotal perception of natural history observations into systematically collected observations that collectively constitute the foundation for hypothetico-deductive research and can be leveraged and applied to conservation and management. These advances are encouraging scientists to conduct and embrace detailed descriptions of nature that remain a critically important component of the scientific endeavor. Finally, these next-generation natural history observations are engaging scientists and non-scientists alike with new documentations of the wonders of nature. Thus, we celebrate next-generation natural history for encouraging people to experience nature directly.

Deep Convolutional Neural Network for Large-Scale Date Palm Tree Mapping from UAV-Based Images

Large-scale mapping of date palm trees is vital for their consistent monitoring and sustainable management, considering their substantial commercial, environmental, and cultural value. This study presents an automatic approach for the large-scale mapping of date palm trees from very-high-spatial-resolution (VHSR) unmanned aerial vehicle (UAV) datasets, based on a deep learning approach. A U-Shape convolutional neural network (U-Net), based on a deep residual learning framework, was developed for the semantic segmentation of date palm trees. A comprehensive set of labeled data was established to enable the training and evaluation of the proposed segmentation model and increase its generalization capability. The performance of the proposed approach was compared with those of various state-of-the-art fully convolutional networks (FCNs) with different encoder architectures, including U-Net (based on VGG-16 backbone), pyramid scene parsing network, and two variants of DeepLab V3+. Experimental results showed that the proposed model outperformed other FCNs in the validation and testing datasets. The generalizability evaluation of the proposed approach on a comprehensive and complex testing dataset exhibited higher classification accuracy and showed that date palm trees could be automatically mapped from VHSR UAV images with an F-score, mean intersection over union, precision, and recall of 91%, 85%, 0.91, and 0.92, respectively. The proposed approach provides an efficient deep learning architecture for the automatic mapping of date palm trees from VHSR UAV-based images.

Application of Drone and WorldView-4 Satellite Data in Mapping and Monitoring Grazing Land Cover and Pasture Quality: Pre- and Post-Flooding

Frequent flooding worldwide, especially in grazing environments, requires mapping and monitoring grazing land cover and pasture quality to support land management. Although drones, satellite, and machine learning technologies can be used to map land cover and pasture quality, there have been limited applications in grazing land environments, especially monitoring land cover change and pasture quality pre- and post-flood events. The use of high spatial resolution drone and satellite data such as WorldView-4 can provide effective mapping and monitoring in grazing land environments. The aim of this study was to utilize high spatial resolution drone and WorldView-4 satellite data to map and monitor grazing land cover change and pasture quality pre-and post-flooding. The grazing land cover was mapped pre-flooding using WorldView-4 satellite data and post-flooding using real-time drone data. The machine learning Random Forest classification algorithm was used to delineate land cover types and the normalized difference vegetation index (NDVI) was used to monitor pasture quality. This study found a seven percent (7%) increase in pasture cover and a one hundred percent (100%) increase in pasture quality post-flooding. The drone and WorldView-4 satellite data were useful to detect grazing land cover change at a finer scale.

Classification of the Complex Agricultural Planting Structure with a Semi-Supervised Extreme Learning Machine Framework

Many approaches have been developed to analyze remote sensing images. However, for the classification of large-scale problems, most algorithms showed low computational efficiency and low accuracy. In this paper, the newly developed semi-supervised extreme learning machine (SS-ELM) framework with k-means clustering algorithm for image segmentation and co-training algorithm to enlarge the sample sets was used to classify the agricultural planting structure at large-scale areas. Data sets collected from a small-scale area within the Hetao Irrigation District (HID) at the upper reaches of the Yellow River basin were used to evaluate the SS-ELM framework. The results of the SS-ELM algorithm were compared with those of the random forest (RF), ELM, support vector machine (SVM) and semi-supervised support vector machine (S-SVM) algorithms. Then the SS-ELM algorithm was applied to analyze the complex planting structure of HID in 1986–2010 by comparing the remote sensing estimated results with the statistical data. In the small-scale case, the SS-ELM algorithm performed better than the RF, ELM, SVM, and S-SVM algorithms. For the SS-ELM algorithm, the average overall accuracy (OA) was in a range of 83.00–92.17%. On the contrary, for the other four algorithms, their average OA values ranged from 56.97% to 92.84%. Whereas, in the classification of planting structure in HID, the SS-ELM algorithm had an excellent performance in classification accuracy and computational efficiency for three major planting crops including maize, wheat, and sunflowers. The estimated areas by using the SS-ELM algorithm based on the remote sensing images were consistent with the statistical data, and their difference was within a range of 3–25%. This implied that the SS-ELM framework could be served as an effective method for the classification of complex planting structures with relatively fast training, good generalization, universal approximation capability, and reasonable learning accuracy.

A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV

Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based methods are often greatly influenced by the environment. As a result, a single type of feature derived from aerial monitoring videos is often insufficient to characterize variations among different abnormal crowd behaviors. To address this, we propose combining two types of features to better represent behavior, namely, multitask cascading CNN (MC-CNN) and multiscale infrared optical flow (MIR-OF), capturing both crowd density and average speed and the appearances of the crowd behaviors, respectively. First, an infrared (IR) camera and Nvidia Jetson TX1 were chosen as an infrared vision system. Since there are no published infrared-based aerial abnormal-behavior datasets, we provide a new infrared aerial dataset named the IR-flying dataset, which includes sample pictures and videos in different scenes of public areas. Second, MC-CNN was used to estimate the crowd density. Third, MIR-OF was designed to characterize the average speed of crowd. Finally, considering two typical abnormal crowd behaviors of crowd aggregating and crowd escaping, the experimental results show that the monitoring UAV system can detect abnormal crowd behaviors in public areas effectively.

Supporting Urban Weed Biosecurity Programs with Remote Sensing

Weeds can impact many ecosystems, including natural, urban and agricultural environments. This paper discusses core weed biosecurity program concepts and considerations for urban and peri-urban areas from a remote sensing perspective and reviews the contribution of remote sensing to weed detection and management in these environments. Urban and peri-urban landscapes are typically heterogenous ecosystems with a variety of vectors for invasive weed species introduction and dispersal. This diversity requires agile systems to support landscape-scale detection and monitoring, while accommodating more site-specific management and eradication goals. The integration of remote sensing technologies within biosecurity programs presents an opportunity to improve weed detection rates, the timeliness of surveillance, distribution and monitoring data availability, and the cost-effectiveness of surveillance and eradication efforts. A framework (the Weed Aerial Surveillance Program) is presented to support a structured approach to integrating multiple remote sensing technologies into urban and peri-urban weed biosecurity and invasive species management efforts. It is designed to support the translation of remote sensing science into operational management outcomes and promote more effective use of remote sensing technologies within biosecurity programs.

Technology innovation: advancing capacities for the early detection of and rapid response to invasive species

The 2016–2018National Invasive Species Council (NISC) Management Plan and Executive Order 13751 call for US federal agencies to foster technology development and application to address invasive species and their impacts. This paper complements and draws on an Innovation Summit, review of advanced biotechnologies applicable to invasive species management, and a survey of federal agencies that respond to these high-level directives. We provide an assessment of federal government capacities for the early detection of and rapid response to invasive species (EDRR) through advances in technology application; examples of emerging technologies for the detection, identification, reporting, and response to invasive species; and guidance for fostering further advancements in applicable technologies. Throughout the paper, we provide examples of how federal agencies are applying technologies to improve programmatic effectiveness and cost-efficiencies. We also highlight the outstanding technology-related needs identified by federal agencies to overcome barriers to enacting EDRR. Examples include improvements in research facility infrastructure, data mobilization across a wide range of invasive species parameters (from genetic to landscape scales), promotion of and support for filling key gaps in technological capacity (e.g., portable, field-ready devices with automated capacities), and greater investments in technology prizes and challenge competitions.

Bayesian inverse methods for spatiotemporal characterization of gastric electrical activity from cutaneous multi-electrode recordings

Gastrointestinal (GI) problems give rise to 10 percent of initial patient visits to their physician. Although blockages and infections are easy to diagnose, more than half of GI disorders involve abnormal functioning of the GI tract, where diagnosis entails subjective symptom-based questionnaires or objective but invasive, intermittent procedures in specialized centers. Although common procedures capture motor aspects of gastric function, which do not correlate with symptoms or treatment response, recent findings with invasive electrical recordings show that spatiotemporal patterns of the gastric slow wave are associated with diagnosis, symptoms, and treatment response. We here consider developing non-invasive approaches to extract this information. Using CT scans from human subjects, we simulate normative and disordered gastric surface electrical activity along with associated abdominal activity. We employ Bayesian inference to solve the ill-posed inverse problem of estimating gastric surface activity from cutaneous recordings. We utilize a prior distribution on the spatiotemporal activity pertaining to sparsity in the number of wavefronts on the stomach surface, and smooth evolution of these wavefronts across time. We implement an efficient procedure to construct the Bayes optimal estimate and demonstrate its superiority compared to other commonly used inverse methods, for both normal and disordered gastric activity. Region-specific wave direction information is calculated and consistent with the simulated normative and disordered cases. We apply these methods to cutaneous multi-electrode recordings of two human subjects with the same clinical description of motor function, but different diagnosis of underlying cause. Our method finds statistically significant wave propagation in all stomach regions for both subjects, anterograde activity throughout for the subject with diabetic gastroparesis, and retrograde activity in some regions for the subject with idiopathic gastroparesis. These findings provide a further step towards towards non-invasive phenotyping of gastric function and indicate the long-term potential for enabling population health opportunities with objective GI assessment.

Prediction Model with High-Performance Constitutive Androstane Receptor (CAR) Using DeepSnap-Deep Learning Approach from the Tox21 10K Compound Library

The constitutive androstane receptor (CAR) plays pivotal roles in drug-induced liver injury through the transcriptional regulation of drug-metabolizing enzymes and transporters. Thus, identifying regulatory factors for CAR activation is important for understanding its mechanisms. Numerous studies conducted previously on CAR activation and its toxicity focused on in vivo or in vitro analyses, which are expensive, time consuming, and require many animals. We developed a computational model that predicts agonists for the CAR using the Toxicology in the 21st Century 10k library. Additionally, we evaluate the prediction performance of novel deep learning (DL)-based quantitative structure-activity relationship analysis called the DeepSnap-DL approach, which is a procedure of generating an omnidirectional snapshot portraying three-dimensional (3D) structures of chemical compounds. The CAR prediction model, which applies a 3D structure generator tool, called CORINA-generated and -optimized chemical structures, in the DeepSnap-DL demonstrated better performance than the existing methods using molecular descriptors. These results indicate that high performance in the prediction model using the DeepSnap-DL approach may be important to prepare suitable 3D chemical structures as input data and to enable the identification of modulators of the CAR.

High-Resolution Vegetation Mapping Using eXtreme Gradient Boosting Based on Extensive Features

Accurate mapping of vegetation is a premise for conserving, managing, and sustainably using vegetation resources, especially in conditions of intensive human activities and accelerating global changes. However, it is still challenging to produce high-resolution multiclass vegetation map in high accuracy, due to the incapacity of traditional mapping techniques in distinguishing mosaic vegetation classes with subtle differences and the paucity of fieldwork data. This study created a workflow by adopting a promising classifier, extreme gradient boosting (XGBoost), to produce accurate vegetation maps of two strikingly different cases (the Dzungarian Basin in China and New Zealand) based on extensive features and abundant vegetation data. For the Dzungarian Basin, a vegetation map with seven vegetation types, 17 subtypes, and 43 associations was produced with an overall accuracy of 0.907, 0.801, and 0.748, respectively. For New Zealand, a map of 10 habitats and a map of 41 vegetation classes were produced with 0.946, and 0.703 overall accuracy, respectively. The workflow incorporating simplified field survey procedures outperformed conventional field survey and remote sensing based methods in terms of accuracy and efficiency. In addition, it opens a possibility of building large-scale, high-resolution, and timely vegetation monitoring platforms for most terrestrial ecosystems worldwide with the aid of Google Earth Engine and citizen science programs.

Iterative Learning Method for In-Flight Auto-Tuning of UAV Controllers Based on Basic Sensory Information

With an increasing number of multirotor unmanned aerial vehicles (UAVs), solutions supporting the improvement in their precision of operation and safety of autonomous flights are gaining importance. They are particularly crucial in transportation tasks, where control systems are required to provide a stable and controllable flight in various environmental conditions, especially after changing the total mass of the UAV (by adding extra load). In the paper, the problem of using only available basic sensory information for fast, locally best, iterative real-time auto-tuning of parameters of fixed-gain altitude controllers is considered. The machine learning method proposed for this purpose is based on a modified zero-order optimization algorithm (golden-search algorithm) and bootstrapping technique. It has been validated in numerous simulations and real-world experiments in terms of its effectiveness in such aspects as: the impact of environmental disturbances (wind gusts); flight with change in mass; and change of sensory information sources in the auto-tuning procedure. The main advantage of the proposed method is that for the trajectory primitives repeatedly followed by an UAV (for programmed controller gains), the method effectively minimizes the selected performance index (cost function). Such a performance index might, e.g., express indirect requirements about tracking quality and energy expenditure. In the paper, a comprehensive description of the method, as well as a wide discussion of the results obtained from experiments conducted in the AeroLab for a low-cost UAV (Bebop 2), are included. The results have confirmed high efficiency of the method at the expected, low computational complexity.

WeedMap: A Large-Scale Semantic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Network for Precision Farming

The ability to automatically monitor agricultural fields is an important capability in precision farming, enabling steps towards more sustainable agriculture. Precise, high-resolution monitoring is a key prerequisite for targeted intervention and the selective application of agro-chemicals. The main goal of this paper is developing a novel crop/weed segmentation and mapping framework that processes multispectral images obtained from an unmanned aerial vehicle (UAV) using a deep neural network (DNN). Most studies on crop/weed semantic segmentation only consider single images for processing and classification. Images taken by UAVs often cover only a few hundred square meters with either color only or color and near-infrared (NIR) channels. Although a map can be generated by processing single segmented images incrementally, this requires additional complex information fusion techniques which struggle to handle high fidelity maps due to their computational costs and problems in ensuring global consistency. Moreover, computing a single large and accurate vegetation map (e.g., crop/weed) using a DNN is non-trivial due to difficulties arising from: (1) limited ground sample distances (GSDs) in high-altitude datasets, (2) sacrificed resolution resulting from downsampling high-fidelity images, and (3) multispectral image alignment. To address these issues, we adopt a stand sliding window approach that operates on only small portions of multispectral orthomosaic maps (tiles), which are channel-wise aligned and calibrated radiometrically across the entire map. We define the tile size to be the same as that of the DNN input to avoid resolution loss. Compared to our baseline model (i.e., SegNet with 3 channel RGB (red, green, and blue) inputs) yielding an area under the curve (AUC) of [background=0.607, crop=0.681, weed=0.576], our proposed model with 9 input channels achieves [0.839, 0.863, 0.782]. Additionally, we provide an extensive analysis of 20 trained models, both qualitatively and quantitatively, in order to evaluate the effects of varying input channels and tunable network hyperparameters. Furthermore, we release a large sugar beet/weed aerial dataset with expertly guided annotations for further research in the fields of remote sensing, precision agriculture, and agricultural robotics.