Use of Systematic Review and Meta-Analysis in Environmental Health Epidemiology: a Systematic Review and Comparison with Guidelines

Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children's health: a methodological survey

BACKGROUND

Translating findings from systematic reviews assessing associations between environmental exposures and reproductive and children's health into policy recommendations requires valid and transparent evidence grading.

METHODS

We aimed to evaluate systems for grading bodies of evidence used in systematic reviews of environmental exposures and reproductive/ children's health outcomes, by conducting a methodological survey of air pollution research, comprising a comprehensive search for and assessment of all relevant systematic reviews. To evaluate the frameworks used for rating the internal validity of primary studies and for grading bodies of evidence (multiple studies), we considered whether and how specific criteria or domains were operationalized to address reproductive/children's environmental health, e.g., whether the timing of exposure assessment was evaluated with regard to vulnerable developmental stages.

RESULTS

Eighteen out of 177 (9.8%) systematic reviews used formal systems for rating the body of evidence; 15 distinct internal validity assessment tools for primary studies, and nine different grading systems for bodies of evidence were used, with multiple modifications applied to the cited approaches. The Newcastle Ottawa Scale (NOS) and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework, neither developed specifically for this field, were the most commonly used approaches for rating individual studies and bodies of evidence, respectively. Overall, the identified approaches were highly heterogeneous in both their comprehensiveness and their applicability to reproductive/children's environmental health research.

CONCLUSION

Establishing the wider use of more appropriate evidence grading methods is instrumental both for strengthening systematic review methodologies, and for the effective development and implementation of environmental public health policies, particularly for protecting pregnant persons and children.

Methodological and reporting quality of systematic reviews on health effects of air pollutants were higher than extreme temperatures: a comparative study

BACKGROUND

An increasing number of systematic reviews (SRs) in the environmental field have been published in recent years as a result of the global concern about the health impacts of air pollution and temperature. However, no study has assessed and compared the methodological and reporting quality of SRs on the health effects of air pollutants and extreme temperatures. This study aims to assess and compare the methodological and reporting quality of SRs on the health effects of ambient air pollutants and extreme temperatures.

METHODS

PubMed, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Library, Web of Science, and Epistemonikos databases were searched. Two researchers screened the literature and extracted information independently. The methodological quality of the SRs was assessed through A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2). The reporting quality was assessed through Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA).

RESULTS

We identified 405 SRs (286 for air pollution, 108 for temperature, and 11 for the synergistic effects). The methodological and reporting quality of the included SRs were suboptimal, with major deficiencies in protocol registration. The methodological quality of SRs of air pollutants was better than that of temperature, especially in terms of satisfactory explanations for any heterogeneity (69.6% v. 45.4%). The reporting quality of SRs of air pollution was better than temperature, however, adherence to the reporting of the assessment results of risk of bias in all SRs (53.5% v. 34.3%) was inadequate.

CONCLUSIONS

Methodological and reporting quality of SRs on the health effect of air pollutants were higher than those of temperatures. However, deficiencies in protocol registration and the assessment of risk of bias remain an issue for both pollutants and temperatures. In addition, developing a risk-of-bias assessment tool applicable to the temperature field may improve the quality of SRs.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Intrauterine exposure to perfluorinated compounds and overweight at age 4: A case-control study

Aims

The aims were to investigate the association between maternal serum levels of perfluorooctane sulfonate (PFOS), perfluorooctanoic acid (PFOA), perfluorohexane sulfonate (PFHxS) and perfluorononanoic acid (PFNA) in early pregnancy and overweight in the child at 4 years and to assess potential heterogeneity in exposure effect between strata with different levels of other risk factors for overweight.

Methods

We used a case-control design and included 354 cases (ISO-BMI ≥ 18 kg/m²) and 2 controls per case (ISO-BMI ≤17 kg/m²) from child health care centers in Malmö, Sweden. Controls were selected stratified on risk scores for overweight in a propensity score framework. Maternal serum levels were analyzed in biobanked samples collected by routine around gestational week 14. Logistic regression was used to estimate odds ratios between quartiles of maternal serum levels and child overweight at age 4.

Results

There were no consistent monotonic exposure-response relationships. We found some significant odds ratios in specific quartiles but these were regarded as spurious findings. The absence of an effect was consistent over risk strata.

Conclusions

We did not find evidence of an association between maternal serum levels of PFOS, PFOA, PFHxS and PFNA in early pregnancy and child overweight at age 4. The level of other risk factors for overweight did not affect children’s susceptibility to prenatal PFAS exposure.

Practical recommendations to conduct a neuroimaging meta-analysis for neuropsychiatric disorders

Over the past decades, neuroimaging has become widely used to investigate structural and functional brain abnormality in neuropsychiatric disorders. The results of individual neuroimaging studies, however, are frequently inconsistent due to small and heterogeneous samples, analytical flexibility, and publication bias toward positive findings. To consolidate the emergent findings toward clinically useful insight, meta-analyses have been developed to integrate the results of studies and identify areas that are consistently involved in pathophysiology of particular neuropsychiatric disorders. However, it should be considered that the results of meta-analyses could also be divergent due to heterogeneity in search strategy, selection criteria, imaging modalities, behavioral tasks, number of experiments, data organization methods, and statistical analysis with different multiple comparison thresholds. Following an introduction to the problem and the concepts of quantitative summaries of neuroimaging findings, we propose practical recommendations for clinicians and researchers for conducting transparent and methodologically sound neuroimaging meta-analyses. This should help to consolidate the search for convergent regional brain abnormality in neuropsychiatric disorders.

Blue-collar work and women's health: A systematic review of the evidence from 1990 to 2015

Despite the implications of gender and sex differences for health risks associated with blue-collar work, adverse health outcomes among blue-collar workers has been most frequently studied among men. The present study provides a "state-of-the-field" systematic review of the empiric evidence published on blue-collar women's health. We systematically reviewed literature related to the health of blue-collar women published between January 1, 1990 and December 31, 2015. We limited our review to peer-reviewed studies published in the English language on the health or health behaviors of women who were presently working or had previously worked in a blue-collar job. Studies were eligible for inclusion regardless of the number, age, or geographic region of blue-collar women in the study sample. We retained 177 studies that considered a wide range of health outcomes in study populations from 40 different countries. Overall, these studies suggested inferior health among female blue-collar workers as compared with either blue-collar males or other women. However, we noted several methodological limitations in addition to heterogeneity in study context and design, which inhibited comparison of results across publications. Methodological limitations of the extant literature, alongside the rapidly changing nature of women in the workplace, motivate further study on the health of blue-collar women. Efforts to identify specific mechanisms by which blue-collar work predisposes women to adverse health may be particularly valuable in informing future workplace-based and policy-level interventions.

Inadequate diversity of information resources searched in US-affiliated systematic reviews and meta-analyses: 2005-2016

OBJECTIVE

Systematic reviews and meta-analyses (SRMAs) rely upon comprehensive searches into diverse resources that catalog primary studies. However, since what constitutes a comprehensive search is unclear, we examined trends in databases searched from 2005-2016, surrounding the publication of search guidelines in 2013, and associations between resources searched and evidence of publication bias in SRMAs involving human subjects.

STUDY DESIGN

To ensure comparability of included SRMAs over the 12 years in the face of a near 100-fold increase of international SRMAs (mainly genetic studies from China) during this period, we focused on USA-affiliated SRMAs, manually reviewing 100 randomly selected SRMAs from those published in each year. After excluding articles (mainly for inadequate detail or out-of-scope methods), we identified factors associated with the databases searched, used network analysis to see which resources were simultaneously searched, and used logistic regression to link information sources searched with a lower chance of finding publication bias.

RESULTS

Among 817 SRMA articles studied, the common resources used were Medline (95%), EMBASE (44%), and Cochrane (41%). Methods journal SRMAs were most likely to use registries and grey literature resources. We found substantial co-searching of resources with only published materials, and not complemented by searches of registries and the grey literature. The 2013 guideline did not substantially increase searching of registries and grey literature resources to retrieve primary studies for the SRMAs. When used to augment Medline, Scopus (in all SRMAs) and ClinicalTrials.gov (in SRMAs with safety outcomes) were negatively associated with publication bias.

CONCLUSIONS

Even SRMAs that search multiple sources tend to search similar resources. Our study supports searching Scopus and CTG in addition to Medline to reduce the chance of publication bias.

Indoor air quality of environments used for physical exercise and sports practice: Systematic review

BACKGROUND

Systematic reviews have the potential to contribute substantially to environmental health and risk assessment.

OBJECTIVE

This study aimed to investigate indoor air quality of environments used for physical exercise and sports practice through a systematic review.

METHODS

The systematic review followed the PRISMA guidelines and was recorded in the PROSPERO registry (CRD42016036057). The search was performed using the SciELO, Science Direct, Scopus, LILACS, MEDLINE via PubMed, and SPORTDiscus databases, from their inception through April 2017. The search terms used in the databases were {air pollution" OR "air pollutants" OR "air quality"} AND {"physical exercise" OR "physical activity" OR "sport"}. The results of selected studies were divided into 5 categories for analysis: monitoring of air quality in the environment according to international guidelines, indoor-to-outdoor ratio (I/O), air quality during physical exercise, impact of air quality on health, and interventions to improve indoor air quality.

RESULTS

Among 1281 studies screened, 34 satisfied the inclusion criteria. The monitoring of pollutants was conducted in 20 studies. CO and NO₂ were the most investigated pollutants, and guidelines were discussed in most studies. The I/O ratio was investigated in 12 studies, of which 9 showed a higher concentration of some pollutants in indoor rather than outdoor environments. Among the 34 studies selected, only 7 investigated the impact of indoor air pollution on human health. The population in most of these studies consisted of hockey players.

CONCLUSION

Most studies conducted monitoring of pollutants in indoor environments used for physical exercise and sports practice. The earliest studies were conducted in ice skating rinks and the most recent evaluated gymnasiums, fitness centers, and sports centers. The CO, particulate matter, and NO₂ concentrations were the most investigated and have the longest history of investigation. These pollutants were within the limits established by guidelines in most studies. Studies that examined the association between air quality documented the adverse effects of pollution. There is a need for more studies focused on the relationship between pollution and health.

Association between Exposure to p,p'-DDT and Its Metabolite p,p'-DDE with Obesity: Integrated Systematic Review and Meta-Analysis

BACKGROUND

The prevalence of obesity is increasing in all countries, becoming a substantial public health concern worldwide. Increasing evidence has associated obesity with persistent pollutants such as the pesticide DDT and its metabolite p,p'-DDE.

OBJECTIVES

Our objective was to systematically review the literature on the association between exposure to the pesticide DDT and its metabolites and obesity to develop hazard identification conclusions.

METHODS

We applied a systematic review-based strategy to identify and integrate evidence from epidemiological, in vivo, and in vitro studies. The evidence from prospective epidemiological studies was quantitatively synthesized by meta-analysis. We rated the body of evidence and integrated the streams of evidence to systematically develop hazard identification conclusions.

RESULTS

We identified seven epidemiological studies reporting prospective associations between exposure to p,p'-DDE and adiposity assessed by body mass index (BMI) z-score. The results from the meta-analysis revealed positive associations between exposure to p,p'-DDE and BMI z-score (β=0.13 BMI z-score (95% CI: 0.01, 0.25) per log increase of p,p'-DDE). Two studies constituted the primary in vivo evidence. Both studies reported positive associations between exposure to p,p'-DDT and increased adiposity in rodents. We identified 19 in vivo studies and 7 in vitro studies that supported the biological plausibility of the obesogenic effects of p,p'-DDT and p,p'-DDE.

CONCLUSIONS

We classified p,p'-DDT and p,p'-DDE as "presumed" to be obesogenic for humans, based on a moderate level of primary human evidence, a moderate level of primary in vivo evidence, and a moderate level of supporting evidence from in vivo and in vitro studies. https://doi.org/10.1289/EHP527.

Short-term effects of fine particulate matter pollution on daily health events in Latin America: a systematic review and meta-analysis

OBJECTIVES

Ambient air pollution is among the leading risks for health worldwide and by 2050 will largely overcome deaths due to unsafe sanitation and malaria, but local evidence from Latin America (LA) is scarce. We aimed to summarize the effect of short-term exposure to fine particulate air pollution (PM_2.5) on morbidity and mortality in Latin America and evaluate evidence coverage and quality, using systematic review and meta-analysis.

METHODS

The comprehensive search (six online databases and hand-searching) identified studies investigating the short-term associations between PM_2.5 and daily health events in LA. Two reviewers independently accessed the internal validity of the studies and used random-effect models in the meta-analysis.

RESULTS

We retrieved 1628 studies. Nine were elected for the qualitative analysis and seven for the quantitative analyses. Each 10 µg/m³ increments in daily PM_2.5 concentrations was significantly associated with increased risk for respiratory and cardiovascular mortality in all-ages (polled RR = 1.02, 95% CI, 1.02-1.02 and RR = 1.01, 95% CI , 1.01-1.02, respectively).

CONCLUSIONS

Short-term exposure to PM_2.5 in LA is significantly associated with increased risk for respiratory and cardiovascular mortality. Evidence is concentrated in few cities and some presented high risk of bias.

Ambient air pollution epidemiology systematic review and meta-analysis: A review of reporting and methods practice

BACKGROUND

Systematic review and meta-analysis (SRMA) are increasingly employed in environmental health (EH) epidemiology and, provided methods and reporting are sound, contribute to translating science evidence to policy. Ambient air pollution (AAP) is both among the leading environmental causes of mortality and morbidity worldwide, and of growing policy relevance due to health co-benefits associated with greenhouse gas emissions reductions.

OBJECTIVES

We reviewed the published AAP SRMA literature (2009 to mid-2015), and evaluated the consistency of methods, reporting and evidence evaluation using a 22-point questionnaire developed from available best-practice consensus guidelines and emerging recommendations for EH. Our goal was to contribute to enhancing the utility of AAP SRMAs to EH policy.

RESULTS AND DISCUSSION

We identified 43 studies that used both SR and MA techniques to examine associations between the AAPs PM2.5, PM10, NO2, SO2, CO and O3, and various health outcomes. On average AAP SRMAs partially or thoroughly addressed 16 of 22 questions (range 10-21), and thoroughly addressed 13 of 22 (range 5-19). We found evidence of an improving trend over the period. However, we observed some weaknesses, particularly infrequent formal reviews of underlying study quality and risk-of-bias that correlated with lower frequency of thorough evaluation for key study quality parameters. Several other areas for enhanced reporting are highlighted.

CONCLUSIONS

The AAP SRMA literature, in particular more recent studies, indicate broad concordance with current and emerging best practice guidance. Development of an EH-specific SRMA consensus statement including a risk-of-bias evaluation tool, would be a contribution to enhanced reliability and robustness as well as policy utility.

Environmental Chemical Assessment in Clinical Practice: Unveiling the Elephant in the Room

A growing body of evidence suggests chemicals present in air, water, soil, food, building materials and household products are toxicants that contribute to the many chronic diseases typically seen in routine medical practice. Yet, despite calls from numerous organisations to provide clinicians with more training and awareness in environmental health, there are multiple barriers to the clinical assessment of toxic environmental exposures. Recent developments in the fields of systems biology, innovative breakthroughs in biomedical research encompassing the "-omics" fields, and advances in mobile sensing, peer-to-peer networks and big data, provide tools that future clinicians can use to assess environmental chemical exposures in their patients. There is also a need for concerted action at all levels, including actions by individual patients, clinicians, medical educators, regulators, government and non-government organisations, corporations and the wider civil society, to understand the "exposome" and minimise the extent of toxic exposures on current and future generations. Clinical environmental chemical risk assessment may provide a bridge between multiple disciplines that uses new technologies to herald in a new era in personalised medicine that unites clinicians, patients and civil society in the quest to understand and master the links between the environment and human health.