How to critically read ecological meta-analyses

ROBITT: A tool for assessing the risk-of-bias in studies of temporal trends in ecology

Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this can undermine statistical inference. In other disciplines, it is common for researchers to complete 'risk-of-bias' assessments to expose and document the potential for biases to undermine conclusions. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar assessments are urgently needed in ecology.We introduce ROBITT, a structured tool for assessing the 'Risk-Of-Bias In studies of Temporal Trends in ecology'. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define study inferential goal(s) and relevant statistical target populations. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate biases, then users must clearly describe them and/or explain what mitigating action will be taken.Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis.We propose that researchers should be strongly encouraged to include a ROBITT assessment when publishing studies of biodiversity trends, especially when using aggregated data. This will help researchers to structure their thinking, clearly acknowledge potential sampling issues, highlight where expert consultation is required and provide an opportunity to describe data checks that might go unreported. ROBITT will also enable reviewers, editors and readers to establish how well research conclusions are supported given a dataset combined with some analytical approach. In turn, it should strengthen evidence-based policy and practice, reduce differing interpretations of data and provide a clearer picture of the uncertainties associated with our understanding of reality.

Limitations on Development of Polychlorinated Biphenyl Tissue Concentration Thresholds for Survival, Growth, and Reproduction in Fish

Polychlorinated biphenyl exposure-response relationships for ecologically relevant endpoints in fish vary greatly whether based on lowest-effect thresholds (Berninger and Tillitt 2019) or all-response data (sensitivity analyses), which precludes use of a single fitted model per endpoint to predict risk or injury to mixed fish populations. PCB = polychlorinated biphenyl.

A meta-analysis of responses in floral traits and flower-visitor interactions to water deficit

Alterations in water availability and drought events as predicted by climate change scenarios will increasingly impact natural communities with effects already emerging at present. Water deficit leads to increasing physiological stress in plants, likely affecting floral development and causing changes in floral morphology, nectar and pollen production or scent. Understanding how these floral traits are altered by water deficit is necessary to predict changes in plant-pollinator interactions and how communities are impacted in the future. Here we employ a meta-analysis approach to synthesize the current evidence of experimental water deficit on floral traits and plant-pollinator interactions. Furthermore, we explore experimental factors potentially increasing heterogeneity between studies and provide ideas how to enhance comparability between studies. In the end, we highlight future directions and knowledge gaps for floral traits and plant-pollinator interactions under water deficit. Our analysis showed consistent decreases in floral size, number of flowers and nectar volume to reduced water availability. Other floral traits such as the start of flowering or herkogamy showed no consistent pattern. This indicates that effects of reduced water availability differ between specific traits that are potentially involved in different functions such as pollinator attraction or efficiency. We found no general decreasing visitation rates with water deficit for flower-visitor interactions. Furthermore, the comparison of available studies suggests that increased reporting of plant stress severity and including more hydraulic and physiological measurements will improve the comparability across experiments and aid a more mechanistic understanding of plant-pollinator interactions under altered environmental conditions. Overall, our results show that water deficit has the potential to strongly affect plant-pollinator interactions via changes in specific floral traits. Linking these changes to pollination services and pollinator performance is one crucial step for understanding how changing water availability and drought events under climate change will alter plant and pollinator communities.

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Neuromuscular-blocking agents for tracheal intubation in pediatric patients (0-12 years): A systematic review and meta-analysis

BACKGROUND

The benefit of using neuromuscular-blocking agents to facilitate tracheal intubation in pediatric patients remains unclear due to variations in design, treatments, and results among trials. By combining the available evidence, we aimed to establish whether scientific findings are consistent and can be generalized across various populations, settings, and treatments.

METHODS

A systematic search for randomized controlled trials, related to the use of neuromuscular-blocking agents for tracheal intubation in American Society of Anesthesiologists class I-II participants (0-12 years), was performed. We considered all randomized controlled trials that studied whether intubation conditions and hemodynamics obtained by using neuromuscular-blocking agents were equivalent to those that were achieved without neuromuscular-blocking agents. We combined the outcomes in Review Manager 5.3 (RevMan, The Cochrane Collaboration) by pairwise random-effects meta-analysis using a risk ratio (RR) for intubation conditions and mean difference for hemodynamic values (mean [95% Confidence Intervals]). Heterogeneity among trials was explored using sensitivity analyses.

RESULTS

We identified 22 eligible randomized controlled trials with 1651 participants. Overall, the use of a neuromuscular-blocking agent was associated with a clinically important increase in the likelihood of both excellent (RR = 1.41 [1.19-1.68], I²  = 76%) and acceptable (RR = 1.13 [1.07-1.19], I²  = 68%) intubating conditions. There is strong evidence that both unacceptable intubation conditions (RR = 0.35 [0.22-0.46], I²  = 23%) and failed first intubation attempts (RR = 0.25 [0.14-0.42], I²  = 0%) were less likely to occur when a neuromuscular-blocking agent was used compared with when it was not. Higher systolic or mean arterial pressures (mean difference = 13.3 [9.1-17.5] mm Hg, I²  = 69%) and heart rates (mean difference = 15.9 [11.0-20.8] beats/min, I²  = 75%) as well as a lower incidence of arrhythmias were observed when tracheal intubation was facilitated by neuromuscular-blocking agents.

CONCLUSION

The use of a neuromuscular-blocking agent during light-to-moderate depth of anesthesia can improve the quality as well as the success rate of tracheal intubation and is associated with better hemodynamic stability during induction of anesthesia.

Global sensitivity analysis of biological multi-scale models

Mathematical models of biological systems need to both reflect and manage the inherent complexities of biological phenomena. Through their versatility and ability to capture behavior at multiple scales, multi-scale models offer a valuable approach. Due to the typically nonlinear and stochastic nature of multi-scale models as well as unknown parameter values, various types of uncertainty are present; thus, effective assessment and quantification of such uncertainty through sensitivity analysis is important. In this review, we discuss global sensitivity analysis in the context of multi-scale and multi-compartment models and highlight its value in model development and analysis. We present an overview of sensitivity analysis methods, approaches for extending such methods to a multi-scale setting, and examples of how sensitivity analysis can inform model reduction. Through schematics and references to past work, we aim to emphasize the advantages and usefulness of such techniques.

Main factors affecting lacertid lizard thermal ecology

The thermal ecology of ectotherms has been studied for almost 2 centuries, but additional attention is currently being paid to it, to understand how organisms deal with the environment in a climate change context. A consensus is still far away due to the large number of factors involved and their complex interactions. In this context, 3 analyses in lacertid lizards were carried out: (i) a meta-analysis, to test for differences between body and air temperatures from 71 populations; (ii) a meta-analysis concerning correlations between body and air temperatures from 60 populations; and (iii) a multimodel inference of thermoregulation effectiveness indices from 45 populations. The importance of different factors, including body size, habitat, insularity, altitude, climate and season, was evaluated in all analyses to model the response variables. A strong seasonality effect was observed, with a consistent pattern of less effective thermoregulation in summer compared to other seasons. Altitude was the second most important factor, with a consistent higher thermoregulation effort in populations occurring at high elevations (>1000 m above sea level). Other factors, such as insularity or body size, can also be important, but did not exhibit a clear pattern. Finally, thermoregulation was less affected by climate and habitat type.

Dissecting the null model for biological invasions: A meta-analysis of the propagule pressure effect

A consistent determinant of the establishment success of alien species appears to be the number of individuals that are introduced to found a population (propagule pressure), yet variation in the form of this relationship has been largely unexplored. Here, we present the first quantitative systematic review of this form, using Bayesian meta-analytical methods. The relationship between propagule pressure and establishment success has been evaluated for a broad range of taxa and life histories, including invertebrates, herbaceous plants and long-lived trees, and terrestrial and aquatic vertebrates. We found a positive mean effect of propagule pressure on establishment success to be a feature of every hypothesis we tested. However, establishment success most critically depended on propagule pressures in the range of 10–100 individuals. Heterogeneity in effect size was associated primarily with different analytical approaches, with some evidence of larger effect sizes in animal rather than plant introductions. Conversely, no variation was accounted for in any analysis by the scale of study (field to global) or methodology (observational, experimental, or proxy) used. Our analyses reveal remarkable consistency in the form of the relationship between propagule pressure and alien population establishment success.

Alien species are a major contributor to human-induced global environmental change. The probability of whether or not an alien species will successfully establish in a novel environment is often related to the number of times a species is introduced and the number of individuals that are introduced each time, collectively termed ‘propagule pressure’. Despite this evidence, we don’t yet know whether this is a universal characteristic of species invasions, and the role of propagule pressure continues to be questioned. Here, we present a quantitative meta-analysis of the relationship between propagule pressure and establishment success across a broad range of species and geographies. We found that propagule pressure was consistently and positively associated with the establishment success of alien species. We conclude that propagule pressure is indeed the most consistent and strongest determinant of alien species establishment. No other factors suggested to explain establishment success can claim such universal support. Our results underpin a clear policy and management target for slowing invasion rates by reducing propagule pressure—ideally to single figures or zero—regardless of any other feature of the invasion.

Meta-evaluation of meta-analysis: ten appraisal questions for biologists

Meta-analysis is a statistical procedure for analyzing the combined data from different studies, and can be a major source of concise up-to-date information. The overall conclusions of a meta-analysis, however, depend heavily on the quality of the meta-analytic process, and an appropriate evaluation of the quality of meta-analysis (meta-evaluation) can be challenging. We outline ten questions biologists can ask to critically appraise a meta-analysis. These questions could also act as simple and accessible guidelines for the authors of meta-analyses. We focus on meta-analyses using non-human species, which we term 'biological' meta-analysis. Our ten questions are aimed at enabling a biologist to evaluate whether a biological meta-analysis embodies 'mega-enlightenment', a 'mega-mistake', or something in between.

Introducing Meta-Partition, a Useful Methodology to Explore Factors That Influence Ecological Effect Sizes

The study of the heterogeneity of effect sizes is a key aspect of ecological meta-analyses. Here we propose a meta-analytic methodology to study the influence of moderators in effect sizes by splitting heterogeneity: meta-partition. To introduce this methodology, we performed a meta-partition of published data about the traits that influence species sensitivity to habitat loss, that have been previously analyzed through meta-regression. Thus, here we aim to introduce meta-partition and to make an initial comparison with meta-regression. Meta-partition algorithm consists of three steps. Step 1 is to study the heterogeneity of effect sizes under the assumption of fixed effect model. If heterogeneity is found, we perform step 2, that is, to partition the heterogeneity by the moderator that minimizes heterogeneity within a subset while maximizing heterogeneity between subsets. Then, if effect sizes of the subset are still heterogeneous, we repeat step 1 and 2 until we reach final subsets. Finally, step 3 is to integrate effect sizes of final subsets, with fixed effect model if there is homogeneity, and with random effects model if there is heterogeneity. Results show that meta-partition is valuable to assess the importance of moderators in explaining heterogeneity of effect sizes, as well as to assess the directions of these relations and to detect possible interactions between moderators. With meta-partition we have been able to evaluate the importance of moderators in a more objective way than with meta-regression, and to visualize the complex relations that may exist between them. As ecological issues are often influenced by several factors interacting in complex ways, ranking the importance of possible moderators and detecting possible interactions would make meta-partition a useful exploration tool for ecological meta-analyses.

Community-level impacts of white-tailed deer on understorey plants in North American forests: a meta-analysis

The impacts of introduced or overabundant large herbivores are a concern for the conservation of forest plant communities and the sustainability of ecosystem function. White-tailed deer (Odocoileus virginianus) are considered ecologically overabundant in much of North America. Previous work suggests that impacts of deer overabundance are broadly negative and are consequently degrading forests at multiple ecological and taxonomic levels. However, no quantitative synthesis currently exists to verify the generality or magnitude of these impacts. Here, we report the results of a meta-analysis quantifying the effects of deer exclusion on the diversity, cover and abundance of woody, herbaceous and whole community components of forest understories in North America. In addition, we explore the relationships of environmental and experimental factors on the direction and magnitude of plant community outcomes using meta-regression. Using 119 calculated effect sizes sourced from 25 peer-reviewed articles, we constructed 10 community-specific data sets and found strongly positive diversity, cover and abundance responses of the woody community to deer exclusion, but no significant effects for the herbaceous or whole community components of forest understories. Local deer density and time since exclusion were significant moderators of both whole community and woody community richness. Local deer density also moderated the effects of deer exclusion on whole community cover. Plot area, in contrast, showed no relationship to any of the community response outcomes. We suggest that the use of inadequate diversity indices, non-native species replacement or legacy effects of chronic deer overabundance might explain why the herbaceous and whole community components of forest understories showed no diversity or cover responses to deer exclusion. We also suggest some strategies to increase opportunities for future quantitative syntheses of deer impacts on forests, including providing better access to existing and future data. Ultimately, we show that white-tailed deer have strongly negative impacts on forest understorey plant communities in North America, but these impacts are not ubiquitous for all components of the plant community.