Monte Carlo assessments of goodness-of-fit for ecological simulation models

Conceptual and methodological advances in habitat-selection modeling: guidelines for ecology and evolution

Habitat selection is a fundamental animal behavior that shapes a wide range of ecological processes, including animal movement, nutrient transfer, trophic dynamics and population distribution. Although habitat selection has been a focus of ecological studies for decades, technological, conceptual and methodological advances over the last 20 yr have led to a surge in studies addressing this process. Despite the substantial literature focused on quantifying the habitat-selection patterns of animals, there is a marked lack of guidance on best analytical practices. The conceptual foundations of the most commonly applied modeling frameworks can be confusing even to those well versed in their application. Furthermore, there has yet to be a synthesis of the advances made over the last 20 yr. Therefore, there is a need for both synthesis of the current state of knowledge on habitat selection, and guidance for those seeking to study this process. Here, we provide an approachable overview and synthesis of the literature on habitat-selection analyses (HSAs) conducted using selection functions, which are by far the most applied modeling framework for understanding the habitat-selection process. This review is purposefully non-technical and focused on understanding without heavy mathematical and statistical notation, which can confuse many practitioners. We offer an overview and history of HSAs, describing the tortuous conceptual path to our current understanding. Through this overview, we also aim to address the areas of greatest confusion in the literature. We synthesize the literature outlining the most exciting conceptual advances in the field of habitat-selection modeling, discussing the substantial ecological and evolutionary inference that can be made using contemporary techniques. We aim for this paper to provide clarity for those navigating the complex literature on HSAs while acting as a reference and best practices guide for practitioners.

From theory to practice in pattern-oriented modelling: identifying and using empirical patterns in predictive models

To robustly predict the effects of disturbance and ecosystem changes on species, it is necessary to produce structurally realistic models with high predictive power and flexibility. To ensure that these models reflect the natural conditions necessary for reliable prediction, models must be informed and tested using relevant empirical observations. Pattern-oriented modelling (POM) offers a systematic framework for employing empirical patterns throughout the modelling process and has been coupled with complex systems modelling, such as in agent-based models (ABMs). However, while the production of ABMs has been rising rapidly, the explicit use of POM has not increased. Challenges with identifying patterns and an absence of specific guidelines on how to implement empirical observations may limit the accessibility of POM and lead to the production of models which lack a systematic consideration of reality. This review serves to provide guidance on how to identify and apply patterns following a POM approach in ABMs (POM-ABMs), specifically addressing: where in the ecological hierarchy can we find patterns; what kinds of patterns are useful; how should simulations and observations be compared; and when in the modelling cycle are patterns used? The guidance and examples provided herein are intended to encourage the application of POM and inspire efficient identification and implementation of patterns for both new and experienced modellers alike. Additionally, by generalising patterns found especially useful for POM-ABM development, these guidelines provide practical help for the identification of data gaps and guide the collection of observations useful for the development and verification of predictive models. Improving the accessibility and explicitness of POM could facilitate the production of robust and structurally realistic models in the ecological community, contributing to the advancement of predictive ecology at large.

Fatty acids as chemotaxonomic and ecophysiological traits in green microalgae (desmids, Zygnematophyceae, Streptophyta): A discriminant analysis approach

Desmids (Zygnematophyceae) are a group of poorly studied green microalgae. The aim of the present study was to identify fatty acids (FAs) that could be used as biomarkers in desmids in general, and to determine FAs as traits within different ecophysiological desmid groups. FA profiles of 29 desmid strains were determined and analysed with respect to their geographic origin, trophic preference and age of cultivation. It appeared that merely FAs present in relatively large proportions such as palmitic, linoleic, α-linolenic and hexadecatrienoic acids could be used as biomarkers for reliable categorization of this microalgal group. Linear discriminant analysis applied to three a priori defined groups of desmids, revealed clear strain-specific characteristics regarding FA distribution, influenced by climate and trophic conditions at the source sites as well as by the age of culture and growth phase. Accordingly, when considering FAs for the determination of lower taxonomic ranks we recommend using the term "trait" instead of "biomarker", as the latter designates unchangeable "fingerprint" of a specific taxon. Furthermore, despite that desmids were regarded as microalgae having stable genomes, long-term cultivation appeared to cause modifications in FA metabolic pathways, evident as a larger proportion of stearidonic acid in desmid strains cultivated over extensive time periods (>35 years).

Pattern separation of spiketrains in hippocampal neurons

Pattern separation is a process that minimizes overlap between patterns of neuronal activity representing similar experiences. Theoretical work suggests that the dentate gyrus (DG) performs this role for memory processing but a direct demonstration is lacking. One limitation is the difficulty to measure DG inputs and outputs simultaneously. To rigorously assess pattern separation by DG circuitry, we used mouse brain slices to stimulate DG afferents and simultaneously record DG granule cells (GCs) and interneurons. Output spiketrains of GCs are more dissimilar than their input spiketrains, demonstrating for the first time temporal pattern separation at the level of single neurons in the DG. Pattern separation is larger in GCs than in fast-spiking interneurons and hilar mossy cells, and is amplified in CA3 pyramidal cells. Analysis of the neural noise and computational modelling suggest that this form of pattern separation is not explained by simple randomness and arises from specific presynaptic dynamics. Overall, by reframing the concept of pattern separation in dynamic terms and by connecting it to the physiology of different types of neurons, our study offers a new window of understanding in how hippocampal networks might support episodic memory.

Is the diversification of Mediterranean Basin plant lineages coupled to karyotypic changes?

The Mediterranean Basin region, home to 25,000 plant species, is included in the worldwide list of hotspots of biodiversity. Despite the indisputably important role of chromosome transitions in plant evolution and diversification, no reference study to date has dealt with the possible relationship between chromosome evolution and lineage diversification in the Mediterranean Basin. Here we study patterns of diversification, patterns of chromosome number transition (either polyploidy or dysploidy) and the relationship between the two for 14 Mediterranean Basin angiosperm lineages using previously published phylogenies. We found a mixed pattern, with half of the lineages displaying a change in chromosome transition rates after the onset of the Mediterranean climate (six increases, one decrease) and the other half (six) experiencing constant rates of chromosome transitions through time. We have also found a heterogeneous pattern regarding diversification rates, with lineages exhibiting moderate (five phylogenies) or low (six) initial diversification rates that either increased (six) or declined (five) through time. Our results reveal no clear link between diversification rates and chromosome number transition rates. By promoting the formation of new habitats and driving the extinction of many species, the Mediterranean onset and the posterior Quaternary climatic oscillations could have been key for the establishment of new chromosomal variants in some plant phylogenies but not in others. While the biodiversity of the Mediterranean Basin may be partly influenced by the chromosomal diversity of its lineages, this study concludes that lineage diversification in the region is largely decoupled from karyotypic evolution.

Rarefaction and extrapolation of species richness using an area-based Fisher's logseries

Fisher's logseries is widely used to characterize species abundance pattern, and some previous studies used it to predict species richness. However, this model, derived from the negative binomial model, degenerates at the zero‐abundance point (i.e., its probability mass fully concentrates at zero abundance, leading to an odd situation that no species can occur in the studied sample). Moreover, it is not directly related to the sampling area size. In this sense, the original Fisher's alpha (correspondingly, species richness) is incomparable among ecological communities with varying area sizes. To overcome these limitations, we developed a novel area‐based logseries model that can account for the compounding effect of the sampling area. The new model can be used to conduct area‐based rarefaction and extrapolation of species richness, with the advantage of accurately predicting species richness in a large region that has an area size being hundreds or thousands of times larger than that of a locally observed sample, provided that data follow the proposed model. The power of our proposed model has been validated by extensive numerical simulations and empirically tested through tree species richness extrapolation and interpolation in Brazilian Atlantic forests. Our parametric model is data parsimonious as it is still applicable when only the information on species number, community size, or the numbers of singleton and doubleton species in the local sample is available. Notably, in comparison with the original Fisher's method, our area‐based model can provide asymptotically unbiased variance estimation (therefore correct 95% confidence interval) for species richness. In conclusion, the proposed area‐based Fisher's logseries model can be of broad applications with clear and proper statistical background. Particularly, it is very suitable for being applied to hyperdiverse ecological assemblages in which nonparametric richness estimators were found to greatly underestimate species richness.

Bayesian Methods for Calibrating Health Policy Models: A Tutorial

Mathematical simulation models are commonly used to inform health policy decisions. These health policy models represent the social and biological mechanisms that determine health and economic outcomes, combine multiple sources of evidence about how policy alternatives will impact those outcomes, and synthesize outcomes into summary measures salient for the policy decision. Calibrating these health policy models to fit empirical data can provide face validity and improve the quality of model predictions. Bayesian methods provide powerful tools for model calibration. These methods summarize information relevant to a particular policy decision into (1) prior distributions for model parameters, (2) structural assumptions of the model, and (3) a likelihood function created from the calibration data, combining these different sources of evidence via Bayes' theorem. This article provides a tutorial on Bayesian approaches for model calibration, describing the theoretical basis for Bayesian calibration approaches as well as pragmatic considerations that arise in the tasks of creating calibration targets, estimating the posterior distribution, and obtaining results to inform the policy decision. These considerations, as well as the specific steps for implementing the calibration, are described in the context of an extended worked example about the policy choice to provide (or not provide) treatment for a hypothetical infectious disease. Given the many simplifications and subjective decisions required to create prior distributions, model structure, and likelihood, calibration should be considered an exercise in creating a reasonable model that produces valid evidence for policy, rather than as a technique for identifying a unique theoretically optimal summary of the evidence.

Coupled Heterogeneities and Their Impact on Parasite Transmission and Control

Most host-parasite systems exhibit remarkable heterogeneity in the contribution to transmission of certain individuals, locations, host infectious states, or parasite strains. While significant advancements have been made in the understanding of the impact of transmission heterogeneity in epidemic dynamics and parasite persistence and evolution, the knowledge base of the factors contributing to transmission heterogeneity is limited. We argue that research efforts should move beyond considering the impact of single sources of heterogeneity and account for complex couplings between conditions with potential synergistic impacts on parasite transmission. Using theoretical approaches and empirical evidence from various host-parasite systems, we investigate the ecological and epidemiological significance of couplings between heterogeneities and discuss their potential role in transmission dynamics and the impact of control.

Calibrating Parameters for Microsimulation Disease Models: A Review and Comparison of Different Goodness-of-Fit Criteria

BACKGROUND

Calibration (estimation of model parameters) compares model outcomes with observed outcomes and explores possible model parameter values to identify the set of values that provides the best fit to the data. The goodness-of-fit (GOF) criterion quantifies the difference between model and observed outcomes. There is no consensus on the most appropriate GOF criterion, because a direct performance comparison of GOF criteria in model calibration is lacking.

METHODS

We systematically compared the performance of commonly used GOF criteria (sum of squared errors [SSE], Pearson chi-square, and a likelihood-based approach [Poisson and/or binomial deviance functions]) in the calibration of selected parameters of the MISCAN-Colon microsimulation model for colorectal cancer. The performance of each GOF criterion was assessed by comparing the 1) root mean squared prediction error (RMSPE) of the selected parameters, 2) computation time of the calibration procedure of various calibration scenarios, and 3) impact on estimated cost-effectiveness ratios.

RESULTS

The likelihood-based deviance resulted in the lowest RMSPE in 4 of 6 calibration scenarios and was close to best in the other 2. The SSE had a 25 times higher RMSPE in a scenario with considerable differences in the values of observed outcomes, whereas the Pearson chi-square had a 60 times higher RMSPE in a scenario with multiple studies measuring the same outcome. In all scenarios, the SSE required the most computation time. The likelihood-based approach estimated the cost-effectiveness ratio most accurately (up to -0.15% relative difference versus 0.44% [SSE] and 13% [Pearson chi-square]).

CONCLUSIONS

The likelihood-based deviance criteria lead to accurate estimation of parameters under various circumstances. These criteria are recommended for calibration in microsimulation disease models in contrast with other commonly used criteria.

Sequential modelling of the effects of mass drug treatments on anopheline-mediated lymphatic filariasis infection in Papua New Guinea

BACKGROUND

Lymphatic filariasis (LF) has been targeted by the WHO for global eradication leading to the implementation of large scale intervention programs based on annual mass drug administrations (MDA) worldwide. Recent work has indicated that locality-specific bio-ecological complexities affecting parasite transmission may complicate the prediction of LF extinction endpoints, casting uncertainty on the achievement of this initiative. One source of difficulty is the limited quantity and quality of data used to parameterize models of parasite transmission, implying the important need to update initially-derived parameter values. Sequential analysis of longitudinal data following annual MDAs will also be important to gaining new understanding of the persistence dynamics of LF. Here, we apply a Bayesian statistical-dynamical modelling framework that enables assimilation of information in human infection data recorded from communities in Papua New Guinea that underwent annual MDAs, into our previously developed model of parasite transmission, in order to examine these questions in LF ecology and control.

RESULTS

Biological parameters underlying transmission obtained by fitting the model to longitudinal data remained stable throughout the study period. This enabled us to reliably reconstruct the observed baseline data in each community. Endpoint estimates also showed little variation. However, the updating procedure showed a shift towards higher and less variable values for worm kill but not for any other drug-related parameters. An intriguing finding is that the stability in key biological parameters could be disrupted by a significant reduction in the vector biting rate prevailing in a locality.

CONCLUSIONS

Temporal invariance of biological parameters in the face of intervention perturbations indicates a robust adaptation of LF transmission to local ecological conditions. The results imply that understanding the mechanisms that underlie locally adapted transmission dynamics will be integral to identifying points of system fragility, and thus countermeasures to reliably facilitate LF extinction both locally and globally.

Uncertainties in a Baltic sea food-web model reveal challenges for future projections

Models that can project ecosystem dynamics under changing environmental conditions are in high demand. The application of such models, however, requires model validation together with analyses of model uncertainties, which are both often overlooked. We carried out a simplified model uncertainty and sensitivity analysis on an Ecopath with Ecosim food-web model of the Baltic Proper (BaltProWeb) and found the model sensitive to both variations in the input data of pre-identified key groups and environmental forcing. Model uncertainties grew particularly high in future climate change scenarios. For example, cod fishery recommendations that resulted in viable stocks in the original model failed after data uncertainties were introduced. In addition, addressing the trophic control dynamics produced by the food-web model proved as a useful tool for both model validation, and for studying the food-web function. These results indicate that presenting model uncertainties is necessary to alleviate ecological surprises in marine ecosystem management.

Foraging sequence, energy intake and torpor: an individual-based field study of energy balancing in desert golden spiny mice

We studied the relationship between sequence of foraging, energy acquired and use of torpor as an energy-balancing strategy in diurnally active desert golden spiny mice. We hypothesised that individuals that arrive earlier to forage will get higher returns and consequently spend less time torpid. If that is the case, then early foragers can be viewed as more successful; if the same individuals arrive repeatedly early, they are likely to have higher fitness under conditions of resource limitation. For the first time, we show a relationship between foraging sequence and amount of resources removed, with individuals that arrive later to a foraging patch tending to receive lower energetic returns and to spend more time torpid. Torpor bears not only benefits but also significant costs, so these individuals pay a price both in lower energy intake and in extended periods of torpor, in what may well be a positive feedback loop.

Goodness-of-fit measures for individual-level models of infectious disease in a Bayesian framework

In simple models there are a variety of tried and tested ways to assess goodness-of-fit. However, in complex non-linear models, such as spatio-temporal individual-level models, less research has been done on how best to ascertain goodness-of-fit. Often such models are fitted within a Bayesian statistical framework, since such a framework is ideally placed to account for the many areas of data uncertainty. Within a Bayesian context, a major tool for assessing goodness-of-fit is the posterior predictive distribution. That is, a distribution for a test statistic is found through simulation from the posterior distribution and then compared with the observed test statistic for the data. Here, we examine different test statistics and ascertain how well they can detect model misspecification via a simulation study.

Epidemiological prediction of the distribution of insects of medical significance: comparative distributions of fleas and sucking lice on the rat host Rattus norvegicus in Yunnan Province, China

Determining the distribution patterns of ectoparasites is important for predicting the spread of vector-borne diseases. A simple epidemiological model was used to compare the distributions of two different taxa of ectoparasitic insects, sucking lice (Insecta: Siphonaptera) and fleas (Insecta: Anoplura), on the same rodent host, Rattus norvegicus Berkenhout (Rodentia: Muridae), in Yunnan Province, China. Correlations between mean abundance and prevalence were determined. Both fleas and sucking lice were aggregated on their hosts, and sucking lice showed a higher degree of aggregation than fleas. The prevalence of both fleas and sucking lice increased with log-transformed mean abundance and a highly linear correlation and modelling efficiency of predicted prevalence against observed prevalence were obtained. The results demonstrate that prevalence can be explained simply by mean abundance.

Population and life-history consequences of within-cohort individual variation

The consequences of within-cohort (i.e., among-individual) variation for population dynamics are poorly understood, in particular for the case where life history is density dependent. We develop a physiologically structured population model that incorporates individual variation among and within cohorts and allows us to explore the intertwined relationship between individual life history and population dynamics. Our model is parameterized for the lizard Zootoca vivipara and reproduces well the species' dynamics and life history. We explore two common mechanisms that generate within-cohort variation: variability in food intake and variability in birth date. Predicted population dynamics are inherently very stable and do not qualitatively change when either of these sources of individual variation is introduced. However, increased within-cohort variation in food intake leads to changes in morphology, with longer but skinnier individuals, even though mean food intake does not change. Morphological changes result from a seemingly universal nonlinear relationship between growth and resource availability but may become apparent only in environments with strongly fluctuating resources. Overall, our results highlight the importance of using a mechanistic framework to gain insights into how different sources of intraspecific variability translate into life-history and population-dynamic changes.

[Spatial cluster detection without point source specification: the use of five methods and comparison of their results]

BACKGROUND

Various statistical methods have been developed to describe spatial heterogeneity, in terms of high risk zones. If no source can be determined, this heterogeneity can be globally or locally described. Global methods test a statistic estimated over the whole studied geographical area, whereas local methods estimate a statistic on each spatial unit (or regrouping unit). This paper aimed to present, and to compare results of an epidemiological application, of five methods of spatial cluster detection.

METHODS

The two global detection methods were: 1) Moran's coefficient, a classically used autocorrelation coefficient; 2) Tango's statistic, a spatial generalization of the Chi(2) statistic. The three local methods were: 1) the local application of Moran's coefficient, proposed by Anselin; 2) the scan statistic, which searches for grouping of spatial units; 3) the oblique regression tree, which splits the studied zone into sub-zones of different risks. These five methods were applied to the description of the spatial heterogeneity of the malaria risk over a hyperendemic village, in Mali.

RESULTS

All the methods highlighted a significant spatial heterogeneity. Both global methods (Moran's coefficient and Tango's statistic) showed weak spatial correlations. Local Moran's coefficient (with Bonferronis' adjustment) highlighted five spatial units. The scan statistic identified a single high risk cluster. The regression oblique tree split the study area into six sub-zones; the sub-zone with the higher risk was consistent with the cluster identified by the scan statistic.

CONCLUSION

These presented methods do not require any previous knowledge of a source. They allow evaluating spatial risk heterogeneity over the entire geographical area under study. It is noteworthy that shape, size, and spatial heterogeneity characteristics (either global or local) of the study area, as well as the definition of the proximity, significantly influence the spatial risk analysis' outcome. Although their results should be cautiously interpreted, these methods are useful for preliminary field studies or epidemiological surveys.

A priori prediction of disease invasion dynamics in a novel environment

Directly transmitted infectious diseases spread through wildlife populations as travelling waves away from the sites of original introduction. These waves often become distorted through their interaction with environmental and population heterogeneities and by long-distance translocation of infected individuals. Accurate a priori predictions of travelling waves of infection depend upon understanding and quantifying these distorting factors. We assess the effects of anisotropies arising from the orientation of rivers in relation to the direction of disease-front propagation and the damming effect of mountains on disease movement in natural populations. The model successfully predicts the local and large-scale prevaccination spread of raccoon rabies through New York State, based on a previous spatially heterogeneous model of raccoon-rabies invasion across the state of Connecticut. Use of this model provides a rare example of a priori prediction of an epidemic invasion over a naturally heterogeneous landscape. Model predictions matched to data can also be used to evaluate the most likely points of disease introduction. These results have general implications for predicting future pathogen invasions and evaluating potential containment strategies.