Evaluating the predictive performance of presence-absence models: Why can the same model appear excellent or poor?

When comparing multiple models of species distribution, models yielding higher predictive performance are clearly to be favored. A more difficult question is how to decide whether even the best model is "good enough". Here, we clarify key choices and metrics related to evaluating the predictive performance of presence-absence models. We use a hierarchical case study to evaluate how four metrics of predictive performance (AUC, Tjur's R 2, max-Kappa, and max-TSS) relate to each other, the random and fixed effects parts of the model, the spatial scale at which predictive performance is measured, and the cross-validation strategy chosen. We demonstrate that the very same metric can achieve different values for the very same model, even when similar cross-validation strategies are followed, depending on the spatial scale at which predictive performance is measured. Among metrics, Tjur's R 2 and max-Kappa generally increase with species' prevalence, whereas AUC and max-TSS are largely independent of prevalence. Thus, Tjur's R 2 and max-Kappa often reach lower values when measured at the smallest scales considered in the study, while AUC and max-TSS reaching similar values across the different spatial levels included in the study. However, they provide complementary insights on predictive performance. The very same model may appear excellent or poor not only due to the applied metric, but also how predictive performance is exactly calculated, calling for great caution on the interpretation of predictive performance. The most comprehensive evaluation of predictive performance can be obtained by evaluating predictive performance through the combination of measures providing complementary insights. Instead of following simple rules of thumb or focusing on absolute values, we recommend comparing the achieved predictive performance to the researcher's own a priori expectations on how easy it is to make predictions related to the same question that the model is used for.

The joint effects of forest habitat area and fragmentation on dung beetles

Habitat loss and habitat fragmentation usually occur together, at the same time and place. However, while there is a consensus that habitat loss is the preeminent threat to biodiversity, the effects of fragmentation are contentious. Some argue that habitat fragmentation is not bad for biodiversity, and even that it is good. Generally, the studies that find no harm or positive outcomes of fragmentation invariably assume that it is independent of habitat loss. However, dissociating the effects of habitat fragmentation from habitat loss is questionable because the two are essentially coupled. Accordingly, we evaluated how forest area and fragmentation (via edge effects) influenced dung beetles per se, and through their effects on the abundance of mammals, using structural equation modeling (SEM). Dung beetles are very sensitive to forest habitat loss and fragmentation and to changes in the abundance of mammals on which they depend for dung. Our study area was in the Tana River, Kenya, where forest fragments are depauperated of mammals except for two endemic species of monkeys. We mapped 12 forests, counted the resident monkeys, and sampled 113,955 beetles from 288 plots. Most of the 87 species of beetles found were small tunnellers. After implementing a fully latent Structural Regression SEM, the optimal model explained a significant 26% of the variance in abundance, and 89% of diversity. The main drivers of beetle abundance were positive, direct, effects of forest area and number of monkeys, and negative edge effects. The main drivers of diversity were the direct effects of the beetle abundance, indirect effects of forest area and abundance of mammals, and indirect negative edge effects. Thus, forest area, fragmentation (via edge effects), and the number of monkeys jointly influenced the abundance and diversity of the beetles directly and indirectly.

Wood construction more strongly shapes deadwood microbial communities than spatial location over 5 years of decay

Diverse communities of fungi and bacteria in deadwood mediate wood decay. While rates of decomposition vary greatly among woody species and spatially distinct habitats, the relative importance of these factors in structuring microbial communities and whether these shift over time remains largely unknown. We characterized fungal and bacterial diversity within pieces of deadwood that experienced 6.3-98.8% mass loss while decaying in common garden 'rotplots' in a temperate oak-hickory forest in the Ozark Highlands, MO, USA. Communities were isolated from 21 woody species that had been decomposing for 1-5 years in spatially distinct habitats at the landscape scale (top and bottom of watersheds) and within stems (top and bottom of stems). Microbial community structure varied more strongly with wood traits than with spatial locations, mirroring the relative role of these factors on decay rates on the same pieces of wood even after 5 years. Co-occurring fungal and bacterial communities persistently influenced one another independently from their shared environmental conditions. However, the relative influence of wood construction versus spatial locations differed between fungi and bacteria, suggesting that life history characteristics of these clades structure diversity differently across space and time in decomposing wood.

Responses of generalist and specialist species to fragmented landscapes

Empirical studies have shown that, unlike species with specialized resource requirements, generalist species may benefit from habitat destruction. We use a family of models to probe the causes of the contrasting responses of these two types of species to habitat destruction. Our approach allows a number of mechanisms to be switched on and off, thereby making it possible to study their marginal and joint effects. Unlike many previous models, we do not assume any intrinsic competitive asymmetry between the species, and we assume pre-emptive rather than displacement competition. Under these assumptions, in the mean-field model the prevalences of all species decrease monotonically with decreasing habitat availability, independently of the degree of specialization. However, in the stochastic and spatial individual-based simulations of the same model, the specialists dominate in landscapes of high quality, whereas generalists thrive in landscapes of intermediate quality; no species persist in very poor landscapes. The same pattern also occurs in a non-spatial stochastic model but not in a deterministic spatial model, showing that demographic stochasticity plays a key role in shaping the outcome of competitive interactions.

How to make more out of community data? A conceptual framework and its implementation as models and software

Community ecology aims to understand what factors determine the assembly and dynamics of species assemblages at different spatiotemporal scales. To facilitate the integration between conceptual and statistical approaches in community ecology, we propose Hierarchical Modelling of Species Communities (HMSC) as a general, flexible framework for modern analysis of community data. While non-manipulative data allow for only correlative and not causal inference, this framework facilitates the formulation of data-driven hypotheses regarding the processes that structure communities. We model environmental filtering by variation and covariation in the responses of individual species to the characteristics of their environment, with potential contingencies on species traits and phylogenetic relationships. We capture biotic assembly rules by species-to-species association matrices, which may be estimated at multiple spatial or temporal scales. We operationalise the HMSC framework as a hierarchical Bayesian joint species distribution model, and implement it as R- and Matlab-packages which enable computationally efficient analyses of large data sets. Armed with this tool, community ecologists can make sense of many types of data, including spatially explicit data and time-series data. We illustrate the use of this framework through a series of diverse ecological examples.