Uncovering Ecological Patterns with Convolutional Neural Networks

Assessment of American Bullfrog (Lithobates catesbeianus) spreading in the Republic of Korea using rule learning of elementary cellular automata

The spread of American Bullfrog has a significant impact on the surrounding ecosystem. It is important to study the mechanisms of their spreading so that proper mitigation can be applied when needed. This study analyzes data from national surveys on bullfrog distribution. We divided the data into 25 regional clusters. To assess the spread within each cluster, we constructed temporal sequences of spatial distribution using the agglomerative clustering method. We employed Elementary Cellular Automata (ECA) to identify rules governing the changes in spatial patterns. Each cell in the ECA grid represents either the presence or absence of bullfrogs based on observations. For each cluster, we counted the number of presence location in the sequence to quantify spreading intensity. We used a Convolutional Neural Network (CNN) to learn the ECA rules and predict future spreading intensity by estimating the expected number of presence locations over 400 simulated generations. We incorporated environmental factors by obtaining habitat suitability maps using Maxent. We multiplied spreading intensity by habitat suitability to create an overall assessment of bullfrog invasion risk. We estimated the relative spreading assessment and classified it into four categories: rapidly spreading, slowly spreading, stable populations, and declining populations.

Evaluate effect of 126 pre-processing methods on various artificial intelligence models accuracy versus normal mode to predict groundwater level (case study: Hamedan-Bahar Plain, Iran)

The estimation of groundwater levels is crucial and an important step in ensuring sustainable management of water resources. In this paper, selected piezometers of the Hamedan-Bahar plain located in west of Iran. The main objective of this study is to compare effect of various pre-processing methods on input data for different artificial intelligence (AI) models to predict groundwater levels (GWLs). The observed GWL, evaporation, precipitation, and temperature were used as input variables in the AI algorithms. Firstly, 126 method of data pre-processing was done by python programming which are classified into three classes: 1- statistical methods, 2- wavelet transform methods and 3- decomposition methods; later, various pre-processed data used by four types of widely used AI models with different kernels, which includes: Support Vector Machine (SVR), Artificial Neural Network (ANN), Long-Short Term memory (LSTM), and Pelican Optimization Algorithm (POA) - Artificial Neural Network (POA-ANN) are classified into three classes: 1- machine learning (SVR and ANN), 2- deep learning (LSTM) and 3- hybrid-ML (POA-ANN) models, to predict groundwater levels (GWLs). Akaike Information Criterion (AIC) were used to evaluate and validate the predictive accuracy of algorithms. According to the results, based on summation (train and test phases) of AIC value of 1778 models, average of AIC values for ML, DL, hybrid-ML classes, was decreased to -25.3%, -29.6% and -57.8%, respectively. Therefore, the results showed that all data pre-processing methods do not lead to improvement of prediction accuracy, and they should be selected very carefully by trial and error. In conclusion, wavelet-ANN model with daubechies 13 and 25 neurons (db13_ANN_25) is the best model to predict GWL that has -204.9 value for AIC which has grown by 5.23% (-194.7) compared to the state without any pre-processing method (ANN_Relu_25).

Ecological inferences about marine mammals from passive acoustic data

Monitoring on the basis of sound recordings, or passive acoustic monitoring, can complement or serve as an alternative to real-time visual or aural monitoring of marine mammals and other animals by human observers. Passive acoustic data can support the estimation of common, individual-level ecological metrics, such as presence, detection-weighted occupancy, abundance and density, population viability and structure, and behaviour. Passive acoustic data also can support estimation of some community-level metrics, such as species richness and composition. The feasibility of estimation and certainty of estimates is highly context dependent, and understanding the factors that affect the reliability of measurements is useful for those considering whether to use passive acoustic data. Here, we review basic concepts and methods of passive acoustic sampling in marine systems that often are applicable to marine mammal research and conservation. Our ultimate aim is to facilitate collaboration among ecologists, bioacousticians, and data analysts. Ecological applications of passive acoustics require one to make decisions about sampling design, which in turn requires consideration of sound propagation, sampling of signals, and data storage. One also must make decisions about signal detection and classification and evaluation of the performance of algorithms for these tasks. Investment in the research and development of systems that automate detection and classification, including machine learning, are increasing. Passive acoustic monitoring is more reliable for detection of species presence than for estimation of other species-level metrics. Use of passive acoustic monitoring to distinguish among individual animals remains difficult. However, information about detection probability, vocalisation or cue rate, and relations between vocalisations and the number and behaviour of animals increases the feasibility of estimating abundance or density. Most sensor deployments are fixed in space or are sporadic, making temporal turnover in species composition more tractable to estimate than spatial turnover. Collaborations between acousticians and ecologists are most likely to be successful and rewarding when all partners critically examine and share a fundamental understanding of the target variables, sampling process, and analytical methods.

Towards the fully automated monitoring of ecological communities

High-resolution monitoring is fundamental to understand ecosystems dynamics in an era of global change and biodiversity declines. While real-time and automated monitoring of abiotic components has been possible for some time, monitoring biotic components-for example, individual behaviours and traits, and species abundance and distribution-is far more challenging. Recent technological advancements offer potential solutions to achieve this through: (i) increasingly affordable high-throughput recording hardware, which can collect rich multidimensional data, and (ii) increasingly accessible artificial intelligence approaches, which can extract ecological knowledge from large datasets. However, automating the monitoring of facets of ecological communities via such technologies has primarily been achieved at low spatiotemporal resolutions within limited steps of the monitoring workflow. Here, we review existing technologies for data recording and processing that enable automated monitoring of ecological communities. We then present novel frameworks that combine such technologies, forming fully automated pipelines to detect, track, classify and count multiple species, and record behavioural and morphological traits, at resolutions which have previously been impossible to achieve. Based on these rapidly developing technologies, we illustrate a solution to one of the greatest challenges in ecology: the ability to rapidly generate high-resolution, multidimensional and standardised data across complex ecologies.

An Outlook for Deep Learning in Ecosystem Science

Rapid advances in hardware and software, accompanied by public- and private-sector investment, have led to a new generation of data-driven computational tools. Recently, there has been a particular focus on deep learning—a class of machine learning algorithms that uses deep neural networks to identify patterns in large and heterogeneous datasets. These developments have been accompanied by both hype and scepticism by ecologists and others. This review describes the context in which deep learning methods have emerged, the deep learning methods most relevant to ecosystem ecologists, and some of the problem domains they have been applied to. Deep learning methods have high predictive performance in a range of ecological contexts, leveraging the large data resources now available. Furthermore, deep learning tools offer ecosystem ecologists new ways to learn about ecosystem dynamics. In particular, recent advances in interpretable machine learning and in developing hybrid approaches combining deep learning and mechanistic models provide a bridge between pure prediction and causal explanation. We conclude by looking at the opportunities that deep learning tools offer ecosystem ecologists and assess the challenges in interpretability that deep learning applications pose.

Automated mapping of Portulacaria afra canopies for restoration monitoring with convolutional neural networks and heterogeneous unmanned aerial vehicle imagery

Ecosystem restoration and reforestation often operate at large scales, whereas monitoring practices are usually limited to spatially restricted field measurements that are (i) time- and labour-intensive, and (ii) unable to accurately quantify restoration success over hundreds to thousands of hectares. Recent advances in remote sensing technologies paired with deep learning algorithms provide an unprecedented opportunity for monitoring changes in vegetation cover at spatial and temporal scales. Such data can feed directly into adaptive management practices and provide insights into restoration and regeneration dynamics. Here, we demonstrate that convolutional neural network (CNN) segmentation algorithms can accurately classify the canopy cover of Portulacaria afra Jacq. in imagery acquired using different models of unoccupied aerial vehicles (UAVs) and under variable light intensities. Portulacaria afra is the target species for the restoration of Albany Subtropical Thicket vegetation, endemic to South Africa, where canopy cover is challenging to measure due to the dense, tangled structure of this vegetation. The automated classification strategy presented here is widely transferable to restoration monitoring as its application does not require any knowledge of the CNN model or specialist training, and can be applied to imagery generated by a range of UAV models. This will reduce the sampling effort required to track restoration trajectories in space and time, contributing to more effective management of restoration sites, and promoting collaboration between scientists, practitioners and landowners.

Seeing the System from Above: The Use and Potential of Remote Sensing for Studying Ecosystem Dynamics

Remote sensing techniques are increasingly used for studying ecosystem dynamics, delivering spatially explicit information on the properties of Earth over large spatial and multi-decadal temporal extents. Yet, there is still a gap between the more technology-driven development of novel remote sensing techniques and their applications for studying ecosystem dynamics. Here, I review the existing literature to explore how addressing these gaps might enable recent methods to overcome longstanding challenges in ecological research. First, I trace the emergence of remote sensing as a major tool for understanding ecosystem dynamics. Second, I examine recent developments in the field of remote sensing that are of particular importance for studying ecosystem dynamics. Third, I consider opportunities and challenges for emerging open data and software policies and suggest that remote sensing is at its most powerful when it is theoretically motivated and rigorously ground-truthed. I close with an outlook on four exciting new research frontiers that will define remote sensing ecology in the upcoming decade.

Spatial point patterns generation on remote sensing data using convolutional neural networks with further statistical analysis

Continuous technological growth and the corresponding environmental implications are triggering the enhancement of advanced environmental monitoring solutions, such as remote sensing. In this paper, we propose a new method for the spatial point patterns generation by classifying remote sensing images using convolutional neural network. To increase the accuracy, the training samples are extended by the suggested data augmentation scheme based on the similarities of images within the same part of the landscape for a limited observation time. The image patches are classified in accordance with the labels of previously classified images of the manually prepared training and test samples. This approach has improved the accuracy of image classification by 7% compared to current best practices of data augmentation. A set of image patch centers of a particular class is considered as a random point configuration, while the class labels are used as marks for every point. A marked point pattern is regarded as a combination of several subpoint patterns with the same qualitative marks. We analyze the bivariate point pattern to identify the relationships between points of different types using the features of a marked random point pattern.

Machine-Learning for Mapping and Monitoring Shallow Coral Reef Habitats

Mapping and monitoring coral reef benthic composition using remotely sensed imagery provides a large-scale inference of spatial and temporal dynamics. These maps have become essential components in marine science and management, with their utility being dependent upon accuracy, scale, and repeatability. One of the primary factors that affects the utility of a coral reef benthic composition map is the choice of the machine-learning algorithm used to classify the coral reef benthic classes. Current machine-learning algorithms used to map coral reef benthic composition and detect changes over time achieve moderate to high overall accuracies yet have not demonstrated spatio-temporal generalisation. The inability to generalise limits their scalability to only those reefs where in situ reference data samples are present. This limitation is becoming more pronounced given the rapid increase in the availability of high temporal (daily) and high spatial resolution (<5 m) multispectral satellite imagery. Therefore, there is presently a need to identify algorithms capable of spatio-temporal generalisation in order to increase the scalability of coral reef benthic composition mapping and change detection. This review focuses on the most commonly used machine-learning algorithms applied to map coral reef benthic composition and detect benthic changes over time using multispectral satellite imagery. The review then introduces convolutional neural networks that have recently demonstrated an ability to spatially and temporally generalise in relation to coral reef benthic mapping; and recurrent neural networks that have demonstrated spatio-temporal generalisation in the field of land cover change detection. A clear conclusion of this review is that existing convolutional neural network and recurrent neural network frameworks hold the most potential in relation to increasing the spatio-temporal scalability of coral reef benthic composition mapping and change detection due to their ability to spatially and temporally generalise.

Deep Species Distribution Modeling From Sentinel-2 Image Time-Series: A Global Scale Analysis on the Orchid Family

Species distribution models (SDMs) are widely used numerical tools that rely on correlations between geolocated presences (and possibly absences) and environmental predictors to model the ecological preferences of species. Recently, SDMs exploiting deep learning and remote sensing images have emerged and have demonstrated high predictive performance. In particular, it has been shown that one of the key advantages of these models (called deep-SDMs) is their ability to capture the spatial structure of the landscape, unlike prior models. In this paper, we examine whether the temporal dimension of remote sensing images can also be exploited by deep-SDMs. Indeed, satellites such as Sentinel-2 are now providing data with a high temporal revisit, and it is likely that the resulting time-series of images contain relevant information about the seasonal variations of the environment and vegetation. To confirm this hypothesis, we built a substantial and original dataset (called DeepOrchidSeries) aimed at modeling the distribution of orchids on a global scale based on Sentinel-2 image time series. It includes around 1 million occurrences of orchids worldwide, each being paired with a 12-month-long time series of high-resolution images (640 x 640 m RGB+IR patches centered on the geolocated observations). This ambitious dataset enabled us to train several deep-SDMs based on convolutional neural networks (CNNs) whose input was extended to include the temporal dimension. To quantify the contribution of the temporal dimension, we designed a novel interpretability methodology based on temporal permutation tests, temporal sampling, and temporal averaging. We show that the predictive performance of the model is greatly increased by the seasonality information contained in the temporal series. In particular, occurrence-poor species and diversity-rich regions are the ones that benefit the most from this improvement, revealing the importance of habitat's temporal dynamics to characterize species distribution.

Deep Learning in Plant Phenological Research: A Systematic Literature Review

Climate change represents one of the most critical threats to biodiversity with far-reaching consequences for species interactions, the functioning of ecosystems, or the assembly of biotic communities. Plant phenology research has gained increasing attention as the timing of periodic events in plants is strongly affected by seasonal and interannual climate variation. Recent technological development allowed us to gather invaluable data at a variety of spatial and ecological scales. The feasibility of phenological monitoring today and in the future depends heavily on developing tools capable of efficiently analyzing these enormous amounts of data. Deep Neural Networks learn representations from data with impressive accuracy and lead to significant breakthroughs in, e.g., image processing. This article is the first systematic literature review aiming to thoroughly analyze all primary studies on deep learning approaches in plant phenology research. In a multi-stage process, we selected 24 peer-reviewed studies published in the last five years (2016-2021). After carefully analyzing these studies, we describe the applied methods categorized according to the studied phenological stages, vegetation type, spatial scale, data acquisition- and deep learning methods. Furthermore, we identify and discuss research trends and highlight promising future directions. We present a systematic overview of previously applied methods on different tasks that can guide this emerging complex research field.

Airborne HySpex Hyperspectral Versus Multitemporal Sentinel-2 Images for Mountain Plant Communities Mapping

Climate change and anthropopression significantly impact plant communities by leading to the spread of expansive and alien invasive plants, thus reducing their biodiversity. Due to significant elevation gradients, high-mountain plant communities in a small area allow for the monitoring of the most important environmental changes. Additionally, being a tourist attraction, they are exposed to direct human influence (e.g., trampling). Airborne hyperspectral remote sensing is one of the best data sources for vegetation mapping, but flight campaign costs limit the repeatability of surveys. A possible alternative approach is to use satellite data from the Copernicus Earth observation program. In our study, we compared multitemporal Sentinel-2 data with HySpex airborne hyperspectral images to map the plant communities on Tatra Mountains based on open-source R programing implementation of Random Forest and Support Vector Machine classifiers. As high-mountain ecosystems are adapted to topographic conditions, the input of Digital Elevation Model (DEM) derivatives on the classification accuracy was analyzed and the effect of the number of training pixels was tested to procure practical information for field campaign planning. For 13 classes (from rock scree communities and alpine grasslands to montane conifer and deciduous forests), we achieved results in the range of 76–90% F1-score depending on the data set. Topographic features: digital terrain model (DTM), normalized digital surface model (nDSM), and aspect and slope maps improved the accuracy of HySpex spectral images, transforming their minimum noise fraction (MNF) bands and Sentinel-2 data sets by 5–15% of the F1-score. Maps obtained on the basis of HySpex imagery (2 m; 430 bands) had a high similarity to maps obtained on the basis of multitemporal Sentinel-2 data (10 m; 132 bands; 11 acquisition dates), which was less than one percentage point for classifications based on 500–1000 pixels; for sets consisting of 50–100 pixels, Random Forest (RF) offered better accuracy.

Mapping Key Indicators of Forest Restoration in the Amazon Using a Low-Cost Drone and Artificial Intelligence

Monitoring the vegetation structure and species composition of forest restoration (FR) in the Brazilian Amazon is critical to ensuring its long-term benefits. Since remotely piloted aircrafts (RPAs) associated with deep learning (DL) are becoming powerful tools for vegetation monitoring, this study aims to use DL to automatically map individual crowns of Vismia (low resilience recovery indicator), Cecropia (fast recovery indicator), and trees in general (this study refers to individual crowns of all trees regardless of species as All Trees). Since All Trees can be accurately mapped, this study also aims to propose a tree crown heterogeneity index (TCHI), which estimates species diversity based on: the heterogeneity attributes/parameters of the RPA image inside the All Trees results; and the Shannon index measured by traditional fieldwork. Regarding the DL methods, this work evaluated the accuracy of the detection of individual objects, the quality of the delineation outlines and the area distribution. Except for Vismia delineation (IoU = 0.2), DL results presented accurate values in general, as F1 and IoU were always greater than 0.7 and 0.55, respectively, while Cecropia presented the most accurate results: F1 = 0.85 and IoU = 0.77. Since All Trees results were accurate, the TCHI was obtained through regression analysis between the canopy height model (CHM) heterogeneity attributes and the field plot data. Although TCHI presented robust parameters, such as p-value < 0.05, its results are considered preliminary because more data are needed to include different FR situations. Thus, the results of this work show that low-cost RPA has great potential for monitoring FR quality in the Amazon, because Vismia, Cecropia, and All Trees can be automatically mapped. Moreover, the TCHI preliminary results showed high potential in estimating species diversity. Future studies must assess domain adaptation methods for the DL results and different FR situations to improve the TCHI range of action.

Perspectives in machine learning for wildlife conservation

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into relevant information. We argue that animal ecologists can capitalize on large datasets generated by modern sensors by combining machine learning approaches with domain knowledge. Incorporating machine learning into ecological workflows could improve inputs for ecological models and lead to integrated hybrid modeling tools. This approach will require close interdisciplinary collaboration to ensure the quality of novel approaches and train a new generation of data scientists in ecology and conservation.

The flowering of Atlantic Forest Pleroma trees

Mapping the spatial distribution of a plant is a current challenge in ecology. Here, a convolutional neural network (CNN) and 33,798 Sentinel-2 satellite images were used to detect and map forest stands dominated by trees of the genus Pleroma by their magenta-to-deep-purple blossoms in the entire Brazilian Atlantic Forest domain, from June 2016 to July 2020. The Pleroma genus, known for its pioneer behaviour, was detected in an area representing 10.8% of the Atlantic Forest, associated negatively with temperature and positively with elevation, slope, tree cover and precipitation. The detection of another genus by the model, 18% of all the detections contained only pink blooming Handroanthus trees, highlighted that botanical identification from space must be taken with caution, particularly outside the known distribution range of the species. The Pleroma blossom seasonality occurred over a period of ~5–6 months centered on the March equinox and populations with distinct blossom timings were found. Our results indicate that in the Atlantic Forest, the remaining natural forest is less diverse than expected but is at least recovering from degradation. Our study suggests a method to produce ecological-domain scale maps of tree genera and species based on their blossoms that could be used for tree studies and biodiversity assessments.

Understanding and Modeling Forest Disturbance Interactions at the Landscape Level

Disturbances, both natural and anthropogenic, affect the configuration, composition, and function of forested ecosystems. Complex system behaviors emerge from the interactions between disturbance regimes, the vegetation response to those disturbances, and their interplay with multiple drivers (climate, topography, land use, etc.) across spatial and temporal scales. Here, we summarize conceptual advances and empirical approaches to disturbance interaction investigation, and used those insights to evaluate and categorize 146 landscape modeling studies emerging from a systematic review of the literature published since 2010. Recent conceptual advances include formal disaggregation of disturbances into their constituent components, embedding disturbance processes into system dynamics, and clarifying terminology for interaction factors, types, and ecosystem responses. Empirical studies investigating disturbance interactions now span a wide range of approaches, including (most recently) advanced statistical methods applied to an expanding set of spatial and temporal datasets. Concurrent development in spatially-explicit landscape models, informed by these empirical insights, integrate the interactions among natural and anthropogenic disturbances by coupling these processes to account for disturbance stochasticity, disturbance within and across scales, and non-linear landscape responses to climate change. Still, trade-offs between model elegance and complexity remain. We developed an index for the degree of process integration (i.e., balance of static vs. dynamic components) within a given disturbance agent and applied it to the studies from our systematic review. Contemporary model applications in this line of research have applied a wide range process integration, depending on the specific question, but also limited in part by data and knowledge. Non-linear “threshold” behavior and cross-scaled interactions remain a frontier in temperate, boreal, and alpine regions of North America and Europe, while even simplistic studies are lacking from other regions of the globe (e.g., subtropical and tropical biomes). Understanding and planning for uncertainty in system behavior—including disturbance interactions—is paramount at a time of accelerated anthropogenic change. While progress in landscape modeling studies in this area is evident, work remains to increase model transparency and confidence, especially for understudied regions and processes. Moving forward, a multi-dimensional approach is recommended to address the uncertainties of complex human-ecological dynamics.

Temporally Generalizable Land Cover Classification: A Recurrent Convolutional Neural Network Unveils Major Coastal Change through Time

The ability to accurately classify land cover in periods before appropriate training and validation data exist is a critical step towards understanding subtle long-term impacts of climate change. These trends cannot be properly understood and distinguished from individual disturbance events or decadal cycles using only a decade or less of data. Understanding these long-term changes in low lying coastal areas, home to a huge proportion of the global population, is of particular importance. Relatively simple deep learning models that extract representative spatiotemporal patterns can lead to major improvements in temporal generalizability. To provide insight into major changes in low lying coastal areas, our study (1) developed a recurrent convolutional neural network that incorporates spectral, spatial, and temporal contexts for predicting land cover class, (2) evaluated this model across time and space and compared this model to conventional Random Forest and Support Vector Machine methods as well as other deep learning approaches, and (3) applied this model to classify land cover across 20 years of Landsat 5 data in the low-lying coastal plain of North Carolina, USA. We observed striking changes related to sea level rise that support evidence on a smaller scale of agricultural land and forests transitioning into wetlands and “ghost forests”. This work demonstrates that recurrent convolutional neural networks should be considered when a model is needed that can generalize across time and that they can help uncover important trends necessary for understanding and responding to climate change in vulnerable coastal regions.

Deep learning and citizen science enable automated plant trait predictions from photographs

Plant functional traits ('traits') are essential for assessing biodiversity and ecosystem processes, but cumbersome to measure. To facilitate trait measurements, we test if traits can be predicted through visible morphological features by coupling heterogeneous photographs from citizen science (iNaturalist) with trait observations (TRY database) through Convolutional Neural Networks (CNN). Our results show that image features suffice to predict several traits representing the main axes of plant functioning. The accuracy is enhanced when using CNN ensembles and incorporating prior knowledge on trait plasticity and climate. Our results suggest that these models generalise across growth forms, taxa and biomes around the globe. We highlight the applicability of this approach by producing global trait maps that reflect known macroecological patterns. These findings demonstrate the potential of Big Data derived from professional and citizen science in concert with CNN as powerful tools for an efficient and automated assessment of Earth's plant functional diversity.

Fusion neural networks for plant classification: learning to combine RGB, hyperspectral, and lidar data

Airborne remote sensing offers unprecedented opportunities to efficiently monitor vegetation, but methods to delineate and classify individual plant species using the collected data are still actively being developed and improved. The Integrating Data science with Trees and Remote Sensing (IDTReeS) plant identification competition openly invited scientists to create and compare individual tree mapping methods. Participants were tasked with training taxon identification algorithms based on two sites, to then transfer their methods to a third unseen site, using field-based plant observations in combination with airborne remote sensing image data products from the National Ecological Observatory Network (NEON). These data were captured by a high resolution digital camera sensitive to red, green, blue (RGB) light, hyperspectral imaging spectrometer spanning the visible to shortwave infrared wavelengths, and lidar systems to capture the spectral and structural properties of vegetation. As participants in the IDTReeS competition, we developed a two-stage deep learning approach to integrate NEON remote sensing data from all three sensors and classify individual plant species and genera. The first stage was a convolutional neural network that generates taxon probabilities from RGB images, and the second stage was a fusion neural network that “learns” how to combine these probabilities with hyperspectral and lidar data. Our two-stage approach leverages the ability of neural networks to flexibly and automatically extract descriptive features from complex image data with high dimensionality. Our method achieved an overall classification accuracy of 0.51 based on the training set, and 0.32 based on the test set which contained data from an unseen site with unknown taxa classes. Although transferability of classification algorithms to unseen sites with unknown species and genus classes proved to be a challenging task, developing methods with openly available NEON data that will be collected in a standardized format for 30 years allows for continual improvements and major gains for members of the computational ecology community. We outline promising directions related to data preparation and processing techniques for further investigation, and provide our code to contribute to open reproducible science efforts.

The Rapid Rise of Next-Generation Natural History

Many ecologists have lamented the demise of natural history and have attributed this decline to a misguided view that natural history is outdated and unscientific. Although there is a perception that the focus in ecology and conservation have shifted away from descriptive natural history research and training toward hypothetico-deductive research, we argue that natural history has entered a new phase that we call “next-generation natural history.” This renaissance of natural history is characterized by technological and statistical advances that aid in collecting detailed observations systematically over broad spatial and temporal extents. The technological advances that have increased exponentially in the last decade include electronic sensors such as camera-traps and acoustic recorders, aircraft- and satellite-based remote sensing, animal-borne biologgers, genetics and genomics methods, and community science programs. Advances in statistics and computation have aided in analyzing a growing quantity of observations to reveal patterns in nature. These robust next-generation natural history datasets have transformed the anecdotal perception of natural history observations into systematically collected observations that collectively constitute the foundation for hypothetico-deductive research and can be leveraged and applied to conservation and management. These advances are encouraging scientists to conduct and embrace detailed descriptions of nature that remain a critically important component of the scientific endeavor. Finally, these next-generation natural history observations are engaging scientists and non-scientists alike with new documentations of the wonders of nature. Thus, we celebrate next-generation natural history for encouraging people to experience nature directly.

Detect, Consolidate, Delineate: Scalable Mapping of Field Boundaries Using Satellite Images

Digital agriculture services can greatly assist growers to monitor their fields and optimize their use throughout the growing season. Thus, knowing the exact location of fields and their boundaries is a prerequisite. Unlike property boundaries, which are recorded in local council or title records, field boundaries are not historically recorded. As a result, digital services currently ask their users to manually draw their field, which is time-consuming and creates disincentives. Here, we present a generalized method, hereafter referred to as DECODE (DEtect, COnsolidate, and DElinetate), that automatically extracts accurate field boundary data from satellite imagery using deep learning based on spatial, spectral, and temporal cues. We introduce a new convolutional neural network (FracTAL ResUNet) as well as two uncertainty metrics to characterize the confidence of the field detection and field delineation processes. We finally propose a new methodology to compare and summarize field-based accuracy metrics. To demonstrate the performance and scalability of our method, we extracted fields across the Australian grains zone with a pixel-based accuracy of 0.87 and a field-based accuracy of up to 0.88 depending on the metric. We also trained a model on data from South Africa instead of Australia and found it transferred well to unseen Australian landscapes. We conclude that the accuracy, scalability and transferability of DECODE shows that large-scale field boundary extraction based on deep learning has reached operational maturity. This opens the door to new agricultural services that provide routine, near-real time field-based analytics.

Repeatable Semantic Reef-Mapping through Photogrammetry and Label-Augmentation

In an endeavor to study natural systems at multiple spatial and taxonomic resolutions, there is an urgent need for automated, high-throughput frameworks that can handle plethora of information. The coalescence of remote-sensing, computer-vision, and deep-learning elicits a new era in ecological research. However, in complex systems, such as marine-benthic habitats, key ecological processes still remain enigmatic due to the lack of cross-scale automated approaches (mms to kms) for community structure analysis. We address this gap by working towards scalable and comprehensive photogrammetric surveys, tackling the profound challenges of full semantic segmentation and 3D grid definition. Full semantic segmentation (where every pixel is classified) is extremely labour-intensive and difficult to achieve using manual labeling. We propose using label-augmentation, i.e., propagation of sparse manual labels, to accelerate the task of full segmentation of photomosaics. Photomosaics are synthetic images generated from a projected point-of-view of a 3D model. In the lack of navigation sensors (e.g., a diver-held camera), it is difficult to repeatably determine the slope-angle of a 3D map. We show this is especially important in complex topographical settings, prevalent in coral-reefs. Specifically, we evaluate our approach on benthic habitats, in three different environments in the challenging underwater domain. Our approach for label-augmentation shows human-level accuracy in full segmentation of photomosaics using labeling as sparse as 0.1%, evaluated on several ecological measures. Moreover, we found that grid definition using a leveler improves the consistency in community-metrics obtained due to occlusions and topology (angle and distance between objects), and that we were able to standardise the 3D transformation with two percent error in size measurements. By significantly easing the annotation process for full segmentation and standardizing the 3D grid definition we present a semantic mapping methodology enabling change-detection, which is practical, swift, and cost-effective. Our workflow enables repeatable surveys without permanent markers and specialized mapping gear, useful for research and monitoring, and our code is available online. Additionally, we release the Benthos data-set, fully manually labeled photomosaics from three oceanic environments with over 4500 segmented objects useful for research in computer-vision and marine ecology.

Using U-Net-Like Deep Convolutional Neural Networks for Precise Tree Recognition in Very High Resolution RGB (Red, Green, Blue) Satellite Images

Very high resolution satellite imageries provide an excellent foundation for precise mapping of plant communities and even single plants. We aim to perform individual tree recognition on the basis of very high resolution RGB (red, green, blue) satellite images using deep learning approaches for northern temperate mixed forests in the Primorsky Region of the Russian Far East. We used a pansharpened satellite RGB image by GeoEye-1 with a spatial resolution of 0.46 m/pixel, obtained in late April 2019. We parametrized the standard U-Net convolutional neural network (CNN) and trained it in manually delineated satellite images to solve the satellite image segmentation problem. For comparison purposes, we also applied standard pixel-based classification algorithms, such as random forest, k-nearest neighbor classifier, naive Bayes classifier, and quadratic discrimination. Pattern-specific features based on grey level co-occurrence matrices (GLCM) were computed to improve the recognition ability of standard machine learning methods. The U-Net-like CNN allowed us to obtain precise recognition of Mongolian poplar (Populus suaveolens Fisch. ex Loudon s.l.) and evergreen coniferous trees (Abies holophylla Maxim., Pinus koraiensis Siebold & Zucc.). We were able to distinguish species belonging to either poplar or coniferous groups but were unable to separate species within the same group (i.e. A. holophylla and P. koraiensis were not distinguishable). The accuracy of recognition was estimated by several metrics and exceeded values obtained for standard machine learning approaches. In contrast to pixel-based recognition algorithms, the U-Net-like CNN does not lead to an increase in false-positive decisions when facing green-colored objects that are similar to trees. By means of U-Net-like CNN, we obtained a mean accuracy score of up to 0.96 in our computational experiments. The U-Net-like CNN recognizes tree crowns not as a set of pixels with known RGB intensities but as spatial objects with a specific geometry and pattern. This CNN’s specific feature excludes misclassifications related to objects of similar colors as objects of interest. We highlight that utilization of satellite images obtained within the suitable phenological season is of high importance for successful tree recognition. The suitability of the phenological season is conceptualized as a group of conditions providing highlighting objects of interest over other components of vegetation cover. In our case, the use of satellite images captured in mid-spring allowed us to recognize evergreen fir and pine trees as the first class of objects (“conifers”) and poplars as the second class, which were in a leafless state among other deciduous tree species.

Near-real time aboveground carbon emissions in Peru

Monitoring aboveground carbon stocks and fluxes from tropical deforestation and forest degradation is important for mitigating climate change and improving forest management. However, high temporal and spatial resolution analyses are rare. This study presents the most detailed tracking of aboveground carbon over time, with yearly, quarterly and monthly estimations of emissions using the stock-difference approach and masked by the forest loss layer of Global Forest Watch. We generated high spatial resolution (1-ha) monitoring of aboveground carbon density (ACD) and emissions (ACE) in Peru by incorporating hundreds of thousands of Planet Dove satellite images, Sentinel-1 radar, topography and airborne LiDAR, embedded into a deep learning regression workflow using high-performance computing. Consistent ACD results were obtained for all quarters and months analyzed, with R² values of 0.75–0.78, and root mean square errors (RMSE) between 20.6 and 22.0 Mg C ha^-1. A total of 7.138 Pg C was estimated for Peru with annual ACE of 20.08 Tg C between the third quarters of 2017 and 2018, respectively, or 23.4% higher than estimates from the FAO Global Forest Resources Assessment. Analyzed quarterly, the spatial evolution of ACE revealed 11.5 Tg C, 6.6 Tg C, 8.6 Tg C, and 10.1 Tg C lost between the third quarters of 2017 and 2018. Moreover, our monthly analysis for the dry season reveals the evolution of ACE at unprecedented temporal detail. We discuss environmental controls over ACE and provide a spatially explicit tool for enhanced forest carbon management at scale.

Quantifying Tropical Plant Diversity Requires an Integrated Technological Approach

Tropical biomes are the most diverse plant communities on Earth, and quantifying this diversity at large spatial scales is vital for many purposes. As macroecological approaches proliferate, the taxonomic uncertainties in species occurrence data are easily neglected and can lead to spurious findings in downstream analyses. Here, we argue that technological approaches offer potential solutions, but there is no single silver bullet to resolve uncertainty in plant biodiversity quantification. Instead, we propose the use of artificial intelligence (AI) approaches to build a data-driven framework that integrates several data sources - including spectroscopy, DNA sequences, image recognition, and morphological data. Such a framework would provide a foundation for improving species identification in macroecological analyses while simultaneously improving the taxonomic process of species delimitation.

A New Method for Counting Reproductive Structures in Digitized Herbarium Specimens Using Mask R-CNN

Phenology-the timing of life-history events-is a key trait for understanding responses of organisms to climate. The digitization and online mobilization of herbarium specimens is rapidly advancing our understanding of plant phenological response to climate and climatic change. The current practice of manually harvesting data from individual specimens, however, greatly restricts our ability to scale-up data collection. Recent investigations have demonstrated that machine-learning approaches can facilitate this effort. However, present attempts have focused largely on simplistic binary coding of reproductive phenology (e.g., presence/absence of flowers). Here, we use crowd-sourced phenological data of buds, flowers, and fruits from >3,000 specimens of six common wildflower species of the eastern United States (Anemone canadensis L., A. hepatica L., A. quinquefolia L., Trillium erectum L., T. grandiflorum (Michx.) Salisb., and T. undulatum Wild.) to train models using Mask R-CNN to segment and count phenological features. A single global model was able to automate the binary coding of each of the three reproductive stages with >87% accuracy. We also successfully estimated the relative abundance of each reproductive structure on a specimen with ≥90% accuracy. Precise counting of features was also successful, but accuracy varied with phenological stage and taxon. Specifically, counting flowers was significantly less accurate than buds or fruits likely due to their morphological variability on pressed specimens. Moreover, our Mask R-CNN model provided more reliable data than non-expert crowd-sourcers but not botanical experts, highlighting the importance of high-quality human training data. Finally, we also demonstrated the transferability of our model to automated phenophase detection and counting of the three Trillium species, which have large and conspicuously-shaped reproductive organs. These results highlight the promise of our two-phase crowd-sourcing and machine-learning pipeline to segment and count reproductive features of herbarium specimens, thus providing high-quality data with which to investigate plant responses to ongoing climatic change.

Regional Mapping and Spatial Distribution Analysis of Canopy Palms in an Amazon Forest Using Deep Learning and VHR Images

Mapping plant species at the regional scale to provide information for ecologists and forest managers is a challenge for the remote sensing community. Here, we use a deep learning algorithm called U-net and very high-resolution multispectral images (0.5 m) from GeoEye satellite to identify, segment and map canopy palms over ∼3000 km 2 of Amazonian forest. The map was used to analyse the spatial distribution of canopy palm trees and its relation to human disturbance and edaphic conditions. The overall accuracy of the map was 95.5% and the F1-score was 0.7. Canopy palm trees covered 6.4% of the forest canopy and were distributed in more than two million patches that can represent one or more individuals. The density of canopy palms is affected by human disturbance. The post-disturbance density in secondary forests seems to be related to the type of disturbance, being higher in abandoned pasture areas and lower in forests that have been cut once and abandoned. Additionally, analysis of palm trees’ distribution shows that their abundance is controlled naturally by local soil water content, avoiding both flooded and waterlogged areas near rivers and dry areas on the top of the hills. They show two preferential habitats, in the low elevation above the large rivers, and in the slope directly below the hill tops. Overall, their distribution over the region indicates a relatively pristine landscape, albeit within a forest that is critically endangered because of its location between two deforestation fronts and because of illegal cutting. New tree species distribution data, such as the map of all adult canopy palms produced in this work, are urgently needed to support Amazon species inventory and to understand their distribution and diversity.

U-Net-Id, an Instance Segmentation Model for Building Extraction from Satellite Images—Case Study in the Joanópolis City, Brazil

Currently, there exists a growing demand for individual building mapping in regions of rapid urban growth in less-developed countries. Most existing methods can segment buildings but cannot discriminate adjacent buildings. Here, we present a new convolutional neural network architecture (CNN) called U-net-id that performs building instance segmentation. The proposed network is trained with WorldView-3 satellite RGB images (0.3 m) and three different labeled masks. The first is the building mask; the second is the border mask, which is the border of the building segment with 4 pixels added outside and 3 pixels inside; and the third is the inner segment mask, which is the segment of the building diminished by 2 pixels. The architecture consists of three parallel paths, one for each mask, all starting with a U-net model. To accurately capture the overlap between the masks, all activation layers of the U-nets are copied and concatenated on each path and sent to two additional convolutional layers before the output activation layers. The method was tested with a dataset of 7563 manually delineated individual buildings of the city of Joanópolis-SP, Brazil. On this dataset, the semantic segmentation showed an overall accuracy of 97.67% and an F1-Score of 0.937 and the building individual instance segmentation showed good performance with a mean intersection over union (IoU) of 0.582 (median IoU = 0.694).

Challenges in Estimating Tropical Forest Canopy Height from Planet Dove Imagery

Monitoring tropical forests using spaceborne and airborne remote sensing capabilities is important for informing environmental policies and conservation actions. Developing large-scale machine learning estimation models of forest structure is instrumental in bridging the gap between retrospective analysis and near-real-time monitoring. However, most approaches use moderate spatial resolution satellite data with limited capabilities of frequent updating. Here, we take advantage of the high spatial and temporal resolutions of Planet Dove images and aim to automatically estimate top-of-canopy height (TCH) for the biologically diverse country of Peru from satellite imagery at 1 ha spatial resolution by building a model that associates Planet Dove textural features with airborne light detection and ranging (LiDAR) measurements of TCH. We use and modify features derived from Fourier textural ordination (FOTO) of Planet Dove images using spectral projection and train a gradient boosted regression for TCH estimation. We discuss the technical and scientific challenges involved in the generation of reliable mechanisms for estimating TCH from Planet Dove satellite image spectral and textural features. Our developed software toolchain is a robust and generalizable regression model that provides a root mean square error (RMSE) of 4.36 m for Peru. This represents a helpful advancement towards better monitoring of tropical forests and improves efforts in reducing emissions from deforestation and forest degradation (REDD+), an important climate change mitigation approach.

Automatic Windthrow Detection Using Very-High-Resolution Satellite Imagery and Deep Learning

Wind disturbances are significant phenomena in forest spatial structure and succession dynamics. They cause changes in biodiversity, impact on forest ecosystems at different spatial scales, and have a strong influence on economics and human beings. The reliable recognition and mapping of windthrow areas are of high importance from the perspective of forest management and nature conservation. Recent research in artificial intelligence and computer vision has demonstrated the incredible potential of neural networks in addressing image classification problems. The most efficient algorithms are based on artificial neural networks of nested and complex architecture (e.g., convolutional neural networks (CNNs)), which are usually referred to by a common term—deep learning. Deep learning provides powerful algorithms for the precise segmentation of remote sensing data. We developed an algorithm based on a U-Net-like CNN, which was trained to recognize windthrow areas in Kunashir Island, Russia. We used satellite imagery of very-high spatial resolution (0.5 m/pixel) as source data. We performed a grid search among 216 parameter combinations defining different U-Net-like architectures. The best parameter combination allowed us to achieve an overall accuracy for recognition of windthrow sites of up to 94% for forested landscapes by coniferous and mixed coniferous forests. We found that the false-positive decisions of our algorithm correspond to either seashore logs, which may look similar to fallen tree trunks, or leafless forest stands. While the former can be rectified by applying a forest mask, the latter requires the usage of additional information, which is not always provided by satellite imagery.

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

Treating Cancer as an Invasive Species

To cure a patient's cancer is to eradicate invasive cells from the ecosystem of the body. However, the ecologic complexity of this challenge is not well understood. Here we show how results from eradications of invasive mammalian species from islands-one of the few contexts in which invasive species have been regularly cleared-inform new research directions for treating cancer. We first summarize the epidemiologic characteristics of island invader eradications and cancer treatments by analyzing recent datasets from the Database of Invasive Island Species Eradications and The Cancer Genome Atlas, detailing the superior successes of island eradication projects. Next, we compare how genetic and environmental factors impact success in each system. These comparisons illuminate a number of promising cancer research and treatment directions, such as heterogeneity engineering as motivated by gene drives and adaptive therapy; multiscale analyses of how population heterogeneity potentiates treatment resistance; and application of ecological data mining techniques to high-throughput cancer data. We anticipate that interdisciplinary comparisons between tumor progression and invasive species would inspire development of novel paradigms to cure cancer.

Convolutional Neural Networks enable efficient, accurate and fine-grained segmentation of plant species and communities from high-resolution UAV imagery

Recent technological advances in remote sensing sensors and platforms, such as high-resolution satellite imagers or unmanned aerial vehicles (UAV), facilitate the availability of fine-grained earth observation data. Such data reveal vegetation canopies in high spatial detail. Efficient methods are needed to fully harness this unpreceded source of information for vegetation mapping. Deep learning algorithms such as Convolutional Neural Networks (CNN) are currently paving new avenues in the field of image analysis and computer vision. Using multiple datasets, we test a CNN-based segmentation approach (U-net) in combination with training data directly derived from visual interpretation of UAV-based high-resolution RGB imagery for fine-grained mapping of vegetation species and communities. We demonstrate that this approach indeed accurately segments and maps vegetation species and communities (at least 84% accuracy). The fact that we only used RGB imagery suggests that plant identification at very high spatial resolutions is facilitated through spatial patterns rather than spectral information. Accordingly, the presented approach is compatible with low-cost UAV systems that are easy to operate and thus applicable to a wide range of users.