Adjustment of meta-analyses on the basis of quality scores should be abandoned

Quality of reporting inflammatory bowel disease randomised controlled trials: a systematic review

OBJECTIVE

Our objective was to perform a systemic evaluation of the risk of bias in randomised controlled trial (RCT) reports published on inflammatory bowel disease (IBD).

DESIGN

We assessed the risk of bias using the Cochrane tool, as indicators of poor methodology or subsequently poor reporting. We systematically selected, with dual independent judgements, all studies published on IBD with no time limits and assessed the methodological quality of included studies again using independent dual ratings.

RESULTS

563 full texts were included after selection and review. No abstract publications were free of any source of bias. Full-text publications still fared badly, as only 103 full-text papers exhibited a low risk of bias in all reporting domains when excluding blinding. RCTs published in journals with higher impact factor (IF) were associated with an overall reduced rate of being at high risk. However, only 6% of full RCT publications in journals with an IF greater than 10, published in the past 5 years, were free of bias.The trend over time is towards improved reporting in all areas. Trials published by larger author teams, in full-text form and by industry and public sponsorship were positively correlated with a lower risk of bias. Only allocation concealment showed a statistically significant improvement with time (p=0.037).

CONCLUSION

These findings are consistent with those of other specialties in the literature. While this unclear risk of bias may represent poor reporting of methods instead of poor methodological quality, it leaves readers and future secondary researchers with significant questions regarding such key issues.

Interventions to reduce Salmonella and Campylobacter during chilling and post-chilling stages of poultry processing: a systematic review and meta-analysis

Salmonella and Campylobacter are common bacterial hazards causing foodborne illnesses worldwide. A large proportion of Salmonella and Campylobacter illnesses are attributed to contaminated poultry products that are mishandled or under cooked. Processing interventions such as chilling and post-chill dip are critical to reducing microbial contamination of poultry. A comprehensive search of the literature published between 2000 and 2021 was conducted in the databases Web of Science, Academic Search Complete, and Academic OneFile. Studies were included if they were in English and investigated the effects of interventions against Salmonella and/or Campylobacter on whole carcasses and/or parts during the chilling or post-chill stages of poultry processing. Random-effects meta-analyses were performed using the "meta" package in the R programming language. Subgroup analyses were assessed according to outcome measure reported, microorganism tested, processing stage assessed, and chemical treatment used. The results included 41 eligible studies. Eighteen studies reported results of 28 separate interventions against Salmonella and 31 reported results of 50 separate interventions against Campylobacter. No significant difference (P> 0.05) was observed when comparing the combined mean difference of all interventions targeting Salmonella to the combined mean difference of all interventions targeting Campylobacter or when comparing chilling times within each pathogen subgroup. For analyses examining antimicrobial additives, peroxyacetic acid (PAA) had the largest reduction against Salmonella population regardless of chilling time (P< 0.05). PAA also had the largest reduction against Campylobacter population and prevalence during primary chilling (P< 0.01). Air chilling showed a lower reduction for Campylobacter than any immersion chilling intervention (P< 0.05). Chilling time and antimicrobial used during poultry processing had varying effects depending on the pathogen and outcome measure investigated (concentration or prevalence). High heterogeneity and low sample numbers in most analyses suggest that more high-quality research that is well-designed and has transparent reporting of methodology and results is needed to corroborate the results.

Assessing risk of bias in the meta-analysis of round 1 of the Health Care Innovation Awards

BACKGROUND

Systematic reviews of observational studies can be affected by biases that lead to under- or over-estimates of true intervention effects. Several tools have been reported in the literature that attempt to characterize potential bias. Our objective in this study was to determine the extent to which study-specific bias may have influenced intervention impacts on total costs of care (TCOC) in round 1 of the Health Care Innovation Awards.

METHODS

We reviewed 82 statistical evaluations of innovation impacts on Medicare TCOC. We developed five risk-of-bias measures and assessed their influence on TCOC impacts using meta-regression.

RESULTS

The majority of evaluations used propensity score matching to create their comparison groups. One third of the non-randomized interventions were judged to have some risk of biased effects due largely to the way they recruited their treatment groups, and 35% had some degree of covariate imbalance remaining after propensity score adjustments. However, in the multivariable analysis of TCOC effects, none of the bias threats we examined (comparison group construction method, risk of bias, or degree of covariate imbalance) had a major impact on the magnitude of HCIA1 innovation effects. Evaluations using propensity score weighting produced larger but imprecise savings effects compared to propensity score matching.

DISCUSSION

Our results suggest that it is unlikely that HCIA1 TCOC effect sizes were systematically affected by the types of bias we considered. Assessing the risk of bias based on specific study design features is likely to be more useful for identifying problematic characteristics than the subjective quality ratings used by existing risk tools.

A systematic review and meta-analysis of the sources of Campylobacter in poultry production (preharvest) and their relative contributions to the microbial risk of poultry meat

A systematic review and meta-analysis were conducted to idetnify the relative contributions of the sources of Campylobacter in poultry live production to Campylobacter prevalence of broiler meat. The keywords of Campylobacter, prevalence, live production, and broiler were used in Google Scholar to address the research interest. A total of 16,800 citations were identified, and 63 relevant citations were included in the meta-analysis after applying predetermined inclusion and exclusion criteria. A generalized linear mixed model approach combined with logit transformation was used in the current meta-analysis to stabilize the variance. The analysis revealed that Campylobacter is ubiquitous in the poultry house exterior environment including surroundings, wildlife, domestic animals, and farm vehicle, with a predicted prevalence of 14%. The recovery of Campylobacter in the interior environment of the poultry house is far less abundant than in the exterior, with a prevalence of 2%, including litter, water, insects, mice, feed, and air. A lack of evidence was observed for vertical transmission due to the day-old chicks being free of Campylobacter from 4 studies identified. Live birds are the predominant carrier of Campylobacter, with a predicted prevalence of 41%. Transportation equipment used for live haul had an overall prevalence of 39%, with vehicles showing a predicted prevalence of 44% and crates with a predicted prevalence of 22%. The results of this meta-analysis highlight the need to implement effective biosecurity measures to minimize the risk of Campylobacter in poultry meat, as human activity appears to be the primary factor for Campylobacter introduction.

The Methodological Quality Scale (MQS) for intervention programs: validity evidence

Introduction

A wide variety of instruments are used when assessing the methodological quality (MQ) of intervention programs. Nevertheless, studies on their metric quality are often not available. In order to address this shortcoming, the methodological quality scale (MQS) is presented as a simple and useful tool with adequate reliability, validity evidence, and metric properties.

Methods

Two coders independently applied the MQS to a set of primary studies. The number of MQ facets was determined in parallel analyses before performing factor analyses. For each facet of validity obtained, mean and standard deviation are presented jointly with reliability and average discrimination. Additionally, the validity facet scores are interpreted based on Shadish, Cook, and Campbell's validity model.

Results and discussion

An empirical validation of the three facets of the MQ (external, internal, and construct validity) and the interpretation of the scores were obtained based on a theoretical framework. Unlike other existing scales, MQS is easy to apply and presents adequate metric properties. In addition, MQ profiles can be obtained in different areas of intervention using different methodologies and proves useful for both researchers doing meta-analysis and for evaluators and professionals designing a new intervention.

Interventions for ADHD in childhood and adolescence: A systematic umbrella review and meta-meta-analysis

There are several meta-analyses of treatment effects for children and adolescents with attention deficit hyperactivity disorder (ADHD). The conclusions of these meta-analyses vary considerably. Our aim was to synthesize the latest evidence of the effectiveness of psychological, pharmacological treatment options and their combination in a systematic overview and meta-meta-analyses. A systematic literature search until July 2022 to identify meta-analyses investigating effects of treatments for children and adolescents with ADHD and ADHD symptom severity as primary outcome (parent and teacher rated) yielded 16 meta-analyses for quantitative analyses. Meta-meta-analyses of pre-post data showed significant effects for pharmacological treatment options for parent (SMD = 0.67, 95% CI 0.60 to 0.74) and teacher ADHD symptom ratings (SMD = 0.68, 95% CI 0.54 to 0.82) as well as for psychological interventions for parent (SMD = 0.42, 95% CI 0.33 to 0.51) and teacher rated symptoms (SMD = 0.25, 95% CI 0.12 to 0.38). We were unable to calculate effect sizes for combined treatments due to the lack of meta-analyses. Our analyses revealed that there is a lack of research on combined treatments and for therapy options for adolescents. Finally, future research efforts should adhere to scientific standards as this allows comparison of effects across meta-analyses.

A systematic review and meta-analysis of the sources of Salmonella in poultry production (pre-harvest) and their relative contributions to the microbial risk of poultry meat

Salmonella is a major foodborne pathogen associated with poultry and poultry products and a leading cause for human salmonellosis. Salmonella is known to transmit in poultry flocks both vertically and horizontally. However, there is a lack of knowledge on relative contribution of the factors on Salmonella prevalence in poultry live production system including hatchery, feed, water, environment-interior, and -exterior. Therefore, a systematic review and meta-analysis was conducted to quantify the potential sources of Salmonella during preharvest and their relative contributions to the microbial risk of poultry meat. A total of 16,800 studies identified from Google Scholar and 37 relevant studies were included in the meta-analysis for relative contributions to Salmonella positivity on broilers after applying exclusion criteria. A generalized linear mixed model approach combined with logit transformation was used in the current study to stabilize the variance. The analysis revealed that the hatchery is the most significant contributor of Salmonella with a prevalence of 48.5%. Litter, feces, and poultry house internal environment were the other 3 major contributing factors with a prevalence of 25.4, 16.3, and 7.9%, respectively. Moreover, poultry house external environment (4.7%), feed (4.8%), chicks (4.7%), and drinker water also contributed to the Salmonella positivity. Results from this meta-analysis informed the urgent need for controls in live production to further reduce Salmonella in fresh, processed poultry. The control strategies can include eliminating the sources of Salmonella and incorporating interventions in live production to reduce Salmonella concentrations in broilers.

Subjective cognitive complaints and subjective cognition following electroconvulsive therapy for depression: A systematic review and meta-analysis

BACKGROUND

Subjective cognitive complaints represent a leading reason for suboptimal prescription of electroconvulsive therapy, the most acutely effective treatment for depression. However, research findings regarding their presentation are conflicting.

OBJECTIVE

To describe, quantify and explain the variability in subjective cognitive complaints and subjective cognition following electroconvulsive therapy for depression.

METHODS

We searched systematically PubMed, Embase, PsycARTICLES, CINAHL and relevant reviews for research published from 1 January 1972 to 31 August 2020. We included all independent studies of patients whose subjective cognition was assessed following brief-pulse electroconvulsive therapy for depression. Three meta-analyses were conducted to quantify: (1) the prevalence of patients presenting with cognitive complaints following electroconvulsive therapy; (2) the pre-post electroconvulsive therapy change in subjective cognition and (3) the association between change in depressive symptoms and change in cognitive complaints. Effect sizes were calculated using random-effects models. Eight prespecified demographic and clinical moderators of between-study variability were assessed using sub-group analyses and meta-regressions.

RESULTS

Data from 72 studies including 5699 participants were meta-analysed. Weighted mean prevalence of patients with cognitive complaints was 48.1% (95% confidence interval = [42.3%, 53.9%]). The type of subjective measure used significantly impacted the pre-post electroconvulsive therapy change in subjective cognition, showing, respectively, large improvement (g = 0.81; 95% confidence interval = [0.59, 1.03]) after electroconvulsive therapy with the Squire Self-Rating Scale of Memory and small worsening (g = -0.48; 95% confidence interval = [-0.70, -0.26]) with 1-item memory scales. The strongest predictors of improved post-electroconvulsive therapy subjective cognition were increased age and larger post-electroconvulsive therapy decrease in depressive symptoms. The weighted mean correlation between pre-post electroconvulsive therapy decrease in depressive symptoms and pre-post electroconvulsive therapy decrease in cognitive complaints was 0.46 (95% confidence interval = [0.39, 0.53]).

CONCLUSION

Variability in subjective cognition following electroconvulsive therapy is mostly explained by the degree of post-treatment persisting depression and by the subjective assessment used. Scales measuring exclusively subjective worsening present limited clinical utility and instruments allowing the detection of both improvement and worsening in subjective cognition should be preferred.

Discrepancies in Ratings of Behavioral Healthcare Interventions Among Evidence-Based Program Resources Websites

Decision makers in the behavioral health disciplines could benefit from tools to assist them in identifying and implementing evidence-based interventions. One tool is an evidence-based program resources website (EBPR). Prior studies documented that when multiple EBPRs rate an intervention, they may disagree. Prior research concerning the reason for such conflicts is sparse. The present study examines how EBPRs rate interventions and the sources of disagreement between EBPRs when rating the same intervention. This study hypothesizes that EBPRs may disagree about intervention ratings because they either use different rating paradigms or they use different studies as evidence of intervention effectiveness (or both). This study identified 15 EBPRs for inclusion. One author (M.J.L.E.) coded the EBPRs for which "tiers of evidence" each EBPR used to classify behavioral health interventions and which criteria they used when rating interventions. The author then computed one Jaccard index of similarity for the criteria shared between each pair of EBPRs that co-rated interventions, and one for the studies used by EBPR rating pairs when rating the same program. The authors used a combination of chi-square, correlation, and binary logistic regression analyses to analyze the data. There was a statistically significant negative correlation between the number of Cochrane Risk of Bias criteria shared between 2 EBPRs and the likelihood of those 2 EBPRs agreeing on an intervention rating (r = -.12, P ≤ .01). There was no relationship between the number of studies evaluated by 2 EBPRs and the likelihood of those EBPRs agreeing on an intervention rating. The major reason for disagreements between EBPRs when rating the same intervention in this study was due to differences in the rating criteria used by the EBPRs. The studies used by the EBPRs to rate programs does not appear to have an impact.

Living through the psychological consequences of COVID-19 pandemic: a systematic review of effective mitigating interventions

OBJECTIVE

This review assesses interventions and their effectiveness in mitigating psychological consequences from pandemic.

METHOD

Published English literatures were searched from four databases (Medline, PubMed, Embase and PsycINFO) from January 2020 and September 2021. A total of 27 papers with 29 studies (one paper reported three studies) met inclusion criteria. Cochrane risk-of-bias tool is applied to assess the quality of all randomised controlled trials (RCT).

RESULTS

All studies were recently conducted in 2020. Publications were from high-income (13, 44.8%), upper middle-income (12, 41.4%) and lower middle-income countries (3, 10.3%) and global (1, 3.5%). Half of the studies conducted for general population (51.7%). One-third of studies (8, 27.6%) provided interventions to patients with COVID-19 and 20.7% to healthcare workers. Of the 29 studies, 14 (48.3%) were RCT. All RCTs were assessed for risk of biases; five studies (15, 35.7%) had low risk as measured against all six dimensions reflecting high-quality study.Of these 29 studies, 26 diagnostic or screening measures were applied; 8 (30.9%) for anxiety, 7 (26.9%) for depression, 5 (19.2%) for stress, 5 (19.2%) for insomnia and 1 (3.8%) for suicide. Measures used to assess the baseline and outcomes of interventions were standardised and widely applied by other studies with high level of reliability and validity. Of 11 RCT studies, 10 (90.9%) showed that anxiety interventions significantly lowered anxiety in intervention groups. Five of the six RCT studies (83.3%) had significantly reduced the level of depression. Most interventions for anxiety and stress were mindfulness and meditation based.

CONCLUSIONS

Results from RCT studies (11%, 78.6%) were effective in mitigating psychological consequences from COVID-19 pandemic when applied to healthcare workers, patients with COVID-19 and general population. These effective interventions can be applied and scaled up in other country settings through adaptation of modes of delivery suitable to country resources, pandemic and health system context.

Trial-level characteristics associate with treatment effect estimates: a systematic review of meta-epidemiological studies

BACKGROUND

To summarize the up-to-date empirical evidence on trial-level characteristics of randomized controlled trials associated with treatment effect estimates.

METHODS

A systematic review searched three databases up to August 2020. Meta-epidemiological (ME) studies of randomized controlled trials on intervention effect were eligible. We assessed the methodological quality of ME studies using a self-developed criterion. Associations between treatment effect estimates and trial-level characteristics were presented using forest plots.

RESULTS

Eighty ME studies were included, with 25/80 (31%) being published after 2015. Less than one-third ME studies critically appraised the included studies (26/80, 33%), published a protocol (23/80, 29%), and provided a list of excluded studies with justifications (12/80, 15%). Trials with high or unclear (versus low) risk of bias on sequence generation (3/14 for binary outcome and 1/6 for continuous outcome), allocation concealment (11/18 and 1/6), double blinding (5/15 and 2/4) and smaller sample size (4/5 and 2/2) significantly associated with larger treatment effect estimates. Associations between high or unclear risk of bias on allocation concealment (5/6 for binary outcome and 1/3 for continuous outcome), double blinding (4/5 and 1/3) and larger treatment effect estimates were more frequently observed for subjective outcomes. The associations between treatment effect estimates and non-blinding of outcome assessors were removed in trials using multiple observers to reach consensus for both binary and continuous outcomes. Some trial characteristics in the Cochrane risk-of-bias (RoB2) tool have not been covered by the included ME studies, including using validated method for outcome measures and selection of the reported results from multiple outcome measures or multiple analysis based on results (e.g., significance of the results).

CONCLUSIONS

Consistently significant associations between larger treatment effect estimates and high or unclear risk of bias on sequence generation, allocation concealment, double blinding and smaller sample size were found. The impact of allocation concealment and double blinding were more consistent for subjective outcomes. The methodological and reporting quality of included ME studies were dissatisfactory. Future ME studies should follow the corresponding reporting guideline. Specific guidelines for conducting and critically appraising ME studies are needed.

Authors' rebuttal to Integrated Risk Information System (IRIS) response to "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools"

This letter responds to the US Environmental Protection Agency's Integrated Risk Information System (IRIS) program letter by Radke et al. (2021) that was published in response to the application of the IRIS risk of bias tool in our recent study "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools." Their letter stated that we misrepresented the IRIS approach. Here, we respond to their three points raised and how we did not misrepresent their tool and also identified areas for improvement: (1) why it should be expected that different reviewers could reach different conclusions with the IRIS tool, as ratings are subject to reviewer judgment; (2) why our interpretation that "low" or "uninformative" studies could be excluded from a body of evidence was reasonable; and (3) why we believe the use of a rating system that generates an overall rating based on an individual domain or a combination of identified deficiencies essentially acts as a score and assumes that we know empirically how much each risk of bias domain should contribute to the overall rating for that study. We have elaborated on these points in our letter.

Nutritional interventions to ameliorate the effect of endocrine disruptors on human reproductive health: A semi-structured review from FIGO

Background

Endocrine disrupting chemicals have harmful effects on reproductive, perinatal, and obstetric outcomes.

Objective

To analyze the evidence on nutritional interventions to reduce the negative effects of endocrine disruptors on reproductive, perinatal, and obstetric outcomes.

Search strategy

A search of MEDLINE (PubMed), Allied Health Literature (CINAHL), EMBASE, Web of Science, and the Cochrane Database was conducted from inception to May 2021.

Selection criteria

Experimental studies on human populations.

Data collection and analysis

Data were collected from eligible studies. Risk of bias assessment was completed using the Cochrane risk of bias tool and the ROBINS‐I Tool.

Results

Database searches yielded 15 362 articles. Removing 11 181 duplicates, 4181 articles underwent abstract screening, 26 articles were eligible for full manuscript review, and 16 met full inclusion criteria. Several interventions were found to be effective in reducing exposure to endocrine disruptors: avoidance of plastic containers, bottles, and packaging; avoidance of canned food/beverages; consumption of fresh and organic food; avoidance of fast/processed foods; and supplementation with vitamin C, iodine, and folic acid. There were some interventional studies examining therapies to improve clinical outcomes related to endocrine disruptors.

Conclusion

Dietary alterations can reduce exposure to endocrine disruptors, with limited data on interventions to improve endocrine‐disruptor–related clinical outcomes. This review provides useful instruction to women, their families, healthcare providers, and regulatory bodies.

Nutritional interventions shown to reduce exposure to endocrine disruptors include avoidance of canned/processed or plastic‐packaged foods. Consumption of fresh/organic foods and vitamin C, iodine, and folic acid also reduce exposure.

Is meta-analysis of RCTs assessing the efficacy of interventions a reliable source of evidence for therapeutic decisions?

Literature-based meta-analysis is a standard technique applied to pool results of individual studies used in medicine and social sciences. It has been criticized for being too malleable to constrain results, averaging incomparable values, lacking a measure of evidence's strength, and problems with a systematic bias of individual studies. We argue against using literature-based meta-analysis of RCTs to assess treatment efficacy and show that therapeutic decisions based on meta-analytic average are not optimal given the full scope of existing evidence. The argument proceeds with discussing examples and analyzing the properties of some standard meta-analytic techniques. First, we demonstrate that meta-analysis can lead to reporting statistically significant results despite the treatment's limited efficacy. Second, we show that meta-analytic confidence intervals are too narrow compared to the variability of treatment outcomes reported by individual studies. Third, we argue that literature-based meta-analysis is not a reliable measurement instrument. Finally, we show that meta-analysis averages out the differences among studies and leads to a loss of information. Despite these problems, literature-based meta-analysis is useful for the assessment of harms. We support two alternative approaches to evidence amalgamation: meta-analysis of individual patient data (IPD) and qualitative review employing mechanistic evidence.

Preferred reporting items for systematic reviews and meta-analyses in ecology and evolutionary biology: a PRISMA extension

Since the early 1990s, ecologists and evolutionary biologists have aggregated primary research using meta-analytic methods to understand ecological and evolutionary phenomena. Meta-analyses can resolve long-standing disputes, dispel spurious claims, and generate new research questions. At their worst, however, meta-analysis publications are wolves in sheep's clothing: subjective with biased conclusions, hidden under coats of objective authority. Conclusions can be rendered unreliable by inappropriate statistical methods, problems with the methods used to select primary research, or problems within the primary research itself. Because of these risks, meta-analyses are increasingly conducted as part of systematic reviews, which use structured, transparent, and reproducible methods to collate and summarise evidence. For readers to determine whether the conclusions from a systematic review or meta-analysis should be trusted - and to be able to build upon the review - authors need to report what they did, why they did it, and what they found. Complete, transparent, and reproducible reporting is measured by 'reporting quality'. To assess perceptions and standards of reporting quality of systematic reviews and meta-analyses published in ecology and evolutionary biology, we surveyed 208 researchers with relevant experience (as authors, reviewers, or editors), and conducted detailed evaluations of 102 systematic review and meta-analysis papers published between 2010 and 2019. Reporting quality was far below optimal and approximately normally distributed. Measured reporting quality was lower than what the community perceived, particularly for the systematic review methods required to measure trustworthiness. The minority of assessed papers that referenced a guideline (~16%) showed substantially higher reporting quality than average, and surveyed researchers showed interest in using a reporting guideline to improve reporting quality. The leading guideline for improving reporting quality of systematic reviews is the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Here we unveil an extension of PRISMA to serve the meta-analysis community in ecology and evolutionary biology: PRISMA-EcoEvo (version 1.0). PRISMA-EcoEvo is a checklist of 27 main items that, when applicable, should be reported in systematic review and meta-analysis publications summarising primary research in ecology and evolutionary biology. In this explanation and elaboration document, we provide guidance for authors, reviewers, and editors, with explanations for each item on the checklist, including supplementary examples from published papers. Authors can consult this PRISMA-EcoEvo guideline both in the planning and writing stages of a systematic review and meta-analysis, to increase reporting quality of submitted manuscripts. Reviewers and editors can use the checklist to assess reporting quality in the manuscripts they review. Overall, PRISMA-EcoEvo is a resource for the ecology and evolutionary biology community to facilitate transparent and comprehensively reported systematic reviews and meta-analyses.

Camouflaging in autism: A systematic review

Some autistic people employ strategies and behaviours to cope with the everyday social world, thereby 'camouflaging' their autistic differences and difficulties. This review aimed to systematically appraise and synthesise the current evidence base pertaining to autistic camouflaging. Following a systematic search of eight databases, 29 studies quantifying camouflaging in children and adults with autism diagnoses or high levels of autistic traits were reviewed. The multiple methods used to measure camouflaging broadly fell under two different approaches: internal-external discrepancy or self-report. These approaches appear to relate to two distinct but potentially connected elements of camouflaging: observable behavioural presentations and self-perceived camouflaging efforts. While significant variation was noted across individual study findings, much of the existing literature supported three preliminary findings about the nature of autistic camouflaging: (1) adults with more self-reported autistic traits report greater engagement in camouflaging; (2) sex and gender differences exist in camouflaging; and (3) higher self-reported camouflaging is associated with worse mental health outcomes. However, the research base was limited regarding participant characterisation and representativeness, which suggests that conclusions cannot be applied to the autistic community as a whole. We propose priorities for future research in refining the current understanding of camouflaging and improving measurement methods.

Dry Needling for the Treatment of Tension-Type, Cervicogenic, or Migraine Headaches: A Systematic Review and Meta-Analysis

OBJECTIVE

Dry needling is a treatment technique used by clinicians to relieve symptoms in patients with tension-type headache (TTH), cervicogenic headache (CGH), or migraine. This systematic review's main objective was to assess the effectiveness of dry needling on headache pain intensity and related disability in patients with TTH, CGH, or migraine.

METHODS

Medline/PubMed, Scopus, Embase, PEDro, Web of Science, Ovid, Allied and Complementary Medicine Database/EBSCO, Cochrane Central Register of Controlled Clinical Trials, Google Scholar, and trial registries were searched until April 1, 2020, along with reference lists of eligible studies and related reviews. Randomized clinical trials or observational studies that compared the effectiveness of dry needling with any other interventions were eligible for inclusion. Three reviewers independently selected studies, extracted data, and assessed risk of bias. Random-effects meta-analyses were performed to produce pooled-effect estimates (Morris dppc2) and their respective CIs.

RESULTS

Of 2715 identified studies, 11 randomized clinical trials were eligible for qualitative synthesis and 9 for meta-analysis. Only 4 trials were of high quality. Very low-quality evidence suggested that dry needling is not statistically better than other interventions for improving headache pain intensity in the short term in patients with TTH (SMD -1.27, 95% CI = -3.56 to 1.03, n = 230), CGH (SMD -0.41, 95% CI = -4.69 to 3.87, n = 104), or mixed headache (TTH and migraine; SMD 0.03; 95% CI = -0.42 to 0.48, n = 90). Dry needling provided significantly greater improvement in related disability in the short term in patients with TTH (SMD -2.28, 95% CI = -2.66 to -1.91, n = 160) and CGH (SMD -0.72, 95% CI = -1.09 to -0.34, n = 144). The synthesis of results showed that dry needling could significantly improve headache frequency, health-related quality of life, trigger point tenderness, and cervical range of motion in TTH and CGH.

CONCLUSIONS

Dry needling produces similar effects to other interventions for short-term headache pain relief, whereas dry needling seems to be better than other therapies for improvement in related disability in the short term.

IMPACT

Although further high-methodological quality studies are warranted to provide a more robust conclusion, our systematic review suggested that for every 1 or 2 patients with TTH treated by dry needling, 1 patient will likely show decreased headache intensity (number needed to treat [NNT] = 2; large effect) and improved related disability (NNT = 1; very large effect). In CGH, for every 3 or 4 patients treated by dry needling, 1 patient will likely exhibit decreased headache intensity (NNT = 4; small effect) and improved related disability (NNT = 3; medium effect).

Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health

Systematic reviews are powerful tools for drawing causal inference for evidence-based decision-making. Published systematic reviews and meta-analyses of environmental and occupational epidemiology studies have increased dramatically in recent years; however, the quality and utility of published reviews are variable. Most methodologies were adapted from clinical epidemiology and have not been adequately modified to evaluate and integrate evidence from observational epidemiology studies assessing environmental and occupational hazards, especially in evaluating the quality of exposure assessments. Although many reviews conduct a systematic and transparent assessment for the potential for bias, they are often deficient in subsequently integrating across a body of evidence. A cohesive review considers the impact of the direction and magnitude of potential biases on the results, systematically evaluates important scientific issues such as study sensitivity and effect modifiers, identifies how different studies complement each other, and assesses other potential sources of heterogeneity. Given these challenges of conducting informative systematic reviews of observational studies, we provide a series of specific recommendations based on practical examples for cohesive evidence integration to reach an overall conclusion on a body of evidence to better support policy making in public health.

A unified framework for bias assessment in clinical research

Methodological flaws, limitations, and inadequate practices in research are well known and pose threats to the internal validity of any research study. However, there are ways of safeguarding research conduct to reduce the chance of research producing distorted results. Numerous tools now exist to assess the incorporation of such safeguards into primary research studies (also known as quality and/or risk-of-bias assessment). These tools typically include a variety of items that are then checked against those implemented in the study. Despite a lot of research in this area, no comprehensive generic classification of safeguards across study designs exist, although attempts have been made to clarify aspects of this. We review the developments in this area as well as use preliminary data from 100 methodological studies to illustrate our proposed approach. We conclude by proposing a new framework for identifying research studies at risk of being biased and the information in this article will promote a unification of the diverse approaches to facilitating bias assessment in clinical research.

Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools

BACKGROUND

Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program's Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency's (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB.

METHODS

We selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence.

RESULTS

The time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall "low" or "uninformative" using IRIS, due to "deficient" or "critically deficient" ratings in one or two domains. Similarly, all studies were rated "unacceptable" using the TSCA tool because of one "unacceptable" rating in a metric related to statistical power. Approximately half of the studies had "low" or "probably low ROB" ratings across all domains with the OHAT and Navigation Guide tools.

CONCLUSIONS

Tools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide.

SYSTEMATIC REVIEW REGISTRATION

This review has not been registered as it is not a systematic review.

Real-world adherence to, and persistence with, once- and twice-daily oral disease-modifying drugs in patients with multiple sclerosis: a systematic review and meta-analysis

BACKGROUND

Nonadherence to disease-modifying drugs (DMDs) for multiple sclerosis (MS) is associated with poorer clinical outcomes, including higher rates of relapse and disease progression, and higher medical resource use. A systematic review and quantification of adherence and persistence with oral DMDs would help clarify the extent of nonadherence and nonpersistence in patients with MS to help prescribers make informed treatment plans and optimize patient care. The objectives were to: 1) conduct a systematic literature review to assess the availability and variability of oral DMD adherence and/or persistence rates across 'real-world' data sources; and 2) conduct meta-analyses of the rates of adherence and persistence for once- and twice-daily oral DMDs in patients with MS using real-world data.

METHODS

A systematic review of studies published between January 2010 and April 2018 in the PubMed database was performed. Only studies assessing once- and twice-daily oral DMDs were available for inclusion in the analysis. Study quality was evaluated using a modified version of the Newcastle-Ottawa Scale, a tool for assessing quality of observational studies. The random effects model evaluated pooled summary estimates of nonadherence.

RESULTS

From 510 abstracts, 31 studies comprising 16,398 patients with MS treated with daily oral DMDs were included. Overall 1-year mean medication possession ratio (MPR; n = 4 studies) was 83.3% (95% confidence interval [CI] 74.5-92.1%) and proportion of days covered (PDC; n = 4 studies) was 76.5% (95% CI 72.0-81.1%). Pooled 1-year MPR ≥80% adherence (n = 6) was 78.5% (95% CI 63.5-88.5%) and PDC ≥80% (n = 5 studies) was 71.8% (95% CI 59.1-81.9%). Pooled 1-year discontinuation (n = 20) was 25.4% (95% CI 21.6-29.7%).

CONCLUSIONS

Approximately one in five patients with MS do not adhere to, and one in four discontinue, daily oral DMDs before 1 year. Opportunities to improve adherence and ultimately patient outcomes, such as patient education, medication support/reminders, simplified dosing regimens, and reducing administration or monitoring requirements, remain. Implementation of efforts to improve adherence are essential to improving care of patients with MS.

Prevalence of Salmonella and Campylobacter spp. in Alternative and Conventionally Produced Chicken in the United States: A Systematic Review and Meta-Analysis

ABSTRACT

The burden of foodborne illness linked to the consumption of contaminated broiler meat is high in the United States. With the increase in popularity of alternative poultry rearing and production systems, it is important to identify the differences in food safety risks presented by alternative systems compared with conventional methods. Although many studies have been conducted that surveyed foodborne pathogen prevalence along the broiler supply chain, a systematic overview of all of the results is lacking. In the current study, a systematic review and meta-analysis were conducted to quantify the differences in prevalence of Salmonella and Campylobacter spp. in farm environment, rehang, prechill, postchill, and retail samples between conventional and alternative production systems. A systematic search of Web of Science and PubMed databases was conducted to identify eligible studies. Studies were then evaluated by inclusion criteria, and the included studies were qualitatively and quantitatively analyzed. In total, 137 trials from 72 studies were used in the final meta-analysis. Meta-analysis models were individually constructed for subgroups that were determined by sample type, pathogen, and production type. All subgroups possessed high amounts of heterogeneity (I2 > 75%). For environmental sample subgroups, Campylobacter prevalence was estimated to be 15.8 and 52.8% for conventional and alternative samples, respectively. Similar prevalence estimates for both production types were observed for Salmonella environmental samples and all retail samples. For conventional samples, Campylobacter and Salmonella prevalence was highest in prechill samples followed by rehang and postchill samples, respectively. The results herein will be useful in future quantitative microbial risk assessments for characterizing the differences in foodborne illness risks presented by different broiler production systems.

HIGHLIGHTS

The Nordic Nutrition Recommendations 2022 - structure and rationale of qualified systematic reviews

BACKGROUND

Qualified systematic reviews (SRs) will form the main basis for evaluating causal effects of nutrients or food groups on health outcomes in the sixth edition of Nordic Nutrition Recommendations to be published in 2022 (NNR2022).

OBJECTIVE

To describe rationale and structure of SRs used in NNR2022.

DESIGN

The SR methodologies of the previous edition of NNR were used as a starting point. Methodologies of recent SRs commissioned by leading national food and health authorities or international food and health organizations were examined and scrutinized. Methodologies for developing SRs were agreed by the NNR2022 Committee in a consensus-driven process.

RESULTS

Qualified SRs will be developed by a cross-disciplinary group of experts and reported according to the requirements of the EQUATOR network. A number of additional requirements must also be fulfilled, including 1) a clearly stated set of objectives and research questions with pre-defined eligibility criteria for the studies, 2) an explicit, reproducible methodology, 3) a systematic search that attempts to identify all studies that would meet the eligibility criteria, 4) an assessment of the validity of the findings of the included studies through an assessment of 'risk of bias' of the studies, 5) a systematic presentation and synthesis of the characteristics and findings of the included studies, and 6) a grading of the overall evidence. The complete definition and requirements of a qualified SR are described.

DISCUSSION

Most SRs published in scientific journals do not fulfill all criteria of the qualified SRs in the NNR2022 project. This article discusses the structure and rationale for requirements of qualified SRs in NNR2022. National food and health authorities have only recently begun to use qualified SRs as a basis for nutrition recommendations.

CONCLUSION

Qualified SRs will be used to inform dietary reference values (DRVs) and food-based dietary guidelines (FBDGs) in the NNR2022 project.

Alcohol Consumption and Risk of Liver Cirrhosis: A Systematic Review and Meta-Analysis

OBJECTIVES

To systematically summarize the risk relationship between different levels of alcohol consumption and incidence of liver cirrhosis.

METHODS

MEDLINE and Embase were searched up to March 6, 2019, to identify case-control and cohort studies with sex-specific results and more than 2 categories of drinking in relation to the incidence of liver cirrhosis. Study characteristics were extracted and random-effects meta-analyses and meta-regressions were conducted.

RESULTS

A total of 7 cohort studies and 2 case-control studies met the inclusion criteria, providing data from 2,629,272 participants with 5,505 cases of liver cirrhosis. There was no increased risk for occasional drinkers. Consumption of one drink per day in comparison to long-term abstainers showed an increased risk for liver cirrhosis in women, but not in men. The risk for women was consistently higher compared to men. Drinking ≥5 drinks per day was associated with a substantially increased risk in both women (relative risk [RR] = 12.44, 95% confidence interval [CI]: 6.65-23.27 for 5-6 drinks, and RR = 24.58, 95% CI: 14.77-40.90 for ≥7 drinks) and men (RR = 3.80, 95% CI: 0.85-17.02, and RR = 6.93, 95% CI: 1.07-44.99, respectively). Heterogeneity across studies indicated an additional impact of other risk factors.

DISCUSSION

Alcohol is a major risk factor for liver cirrhosis with risk increasing exponentially. Women may be at higher risk compared to men even with little alcohol consumption. More high-quality research is necessary to elucidate the role of other risk factors, such as genetic vulnerability, body weight, metabolic risk factors, and drinking patterns over the life course. High alcohol consumption should be avoided, and people drinking at high levels should receive interventions to reduce their intake.

Comparison of bias adjustment methods in meta-analysis suggests that quality effects modeling may have less limitations than other approaches

BACKGROUND

The quality of primary research is commonly assessed before inclusion in meta-analyses. Findings are discussed in the context of the quality appraisal by categorizing studies according to risk of bias. The impact of appraised risk of bias on study outcomes is typically judged by the reader; however, several methods have been developed to quantify this risk of bias assessment and incorporate it into the pooled results of meta-analysis, a process known as bias adjustment. The advantages, potential limitations, and applicability of these methods are not well defined.

STUDY DESIGN AND SETTING

Comparative evaluation of the applicability of the various methods and their limitations are discussed using two examples from the literature. These methods include weighting, stratification, regression, use of empirically based prior distributions, and elicitation by experts.

RESULTS

Use of the two examples from the literature suggest that all methods provide similar adjustment. Methods differed mainly in applicability and limitations.

CONCLUSION

Bias adjustment is a feasible process in meta-analysis with several strategies currently available. Quality effects modelling was found to be easily implementable with fewer limitations in comparison to other methods.

Phthalate exposure and female reproductive and developmental outcomes: a systematic review of the human epidemiological evidence

OBJECTIVE

We performed a systematic review of the epidemiology literature to identify the female reproductive and developmental effects associated with phthalate exposure.

DATA SOURCES AND STUDY ELIGIBILITY CRITERIA

Six phthalates were included in the review: di(2-ethylhexyl) phthalate (DEHP), diisononyl phthalate (DINP), dibutyl phthalate (DBP), diisobutyl phthalate (DIBP), butyl benzyl phthalate (BBP), and diethyl phthalate (DEP). The initial literature search (of PubMed, Web of Science, and Toxline) included all studies of female reproductive and developmental effects in humans, and outcomes were selected for full systematic review based on data availability.

STUDY EVALUATION AND SYNTHESIS METHODS

For each outcome, studies were evaluated using criteria defined a priori for risk of bias and sensitivity by two reviewers using a domain-based approach. Evidence was synthesized by outcome and phthalate and strength of evidence was summarized using a structured framework.

RESULTS

The primary outcomes reviewed here are (number of included/excluded studies in parentheses): pubertal development (5/13), time to pregnancy (3/4), preterm birth (8/12), and spontaneous abortion (5/0). Among these outcomes, preterm birth had moderate evidence of a positive association with phthalate exposure (specifically DEHP, DBP, and DEP). Exposure levels for BBP, DIBP, and DINP were generally lower than for the phthalates with an observed effect, which may partially explain the difference due to lower sensitivity. Other phthalate/outcome combinations were considered to have slight or indeterminate evidence of an association.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

Overall, these results support that some phthalates may be associated with higher odds of preterm birth in humans, though there is some remaining inconsistency. More evidence is needed on the mechanism and relevant exposure window for this association. The views expressed are those of the authors and do not necessarily represent the views or policies of the U.S. EPA.

The Consensus on Exercise Reporting Template (CERT) in a systematic review of exercise-based rehabilitation effectiveness: completeness of reporting, rater agreement, and utility

BACKGROUND

Rehabilitation interventions are diverse - making decisions about pooling data in meta-analyses challenging. Intervention reporting templates such as the Consensus on Exercise Reporting Template (CERT) may help reviewers document intervention variability.

AIM

To assess inter-rater agreement and utility of CERT used to assess completeness of reporting of one rehabilitation exercise intervention: pelvic floor muscle training (PFMT).

DESIGN

A non-experimental agreement study.

SETTING

Update of the Cochrane systematic review comparing different approaches to PFMT for urinary incontinence in women.

POPULATION

Two PFMT arms from 21 newly identified trials.

METHODS

Five raters independently used CERT to assess sufficiency of reporting of each arm (experimental PFMT and control PFMT) of each trial. One rater, PFMT non-expert, rated all trials. Four raters, all PFMT experts, assessed a mutually exclusive subgroup of the trials. In addition to rating sufficiency - "Yes" compared to No" or "Uncertain" - raters also reported on CERT utility. Expert ratings were used to determine the proportion of CERT items rated as sufficiently reported. Rater agreement was estimated using coefficient kappa and McNemar's test.

RESULTS

The range of CERT items rated as sufficiently reported was 0 to 15 of 19 items, and the mean for both trial arms was 5.5. For agreement, 11 of 19 items had sufficient data to estimate coefficient kappa and only 3 of 11 had a kappa >0.4 (moderate agreement). From the 12 of 19 items for which McNemar's test could be performed, five had evidence that PFMT experts more often rated the reporting as sufficient than the non-expert. Raters reported the CERT template was comprehensive but not complete and needed contextualizing for PFMT.

CONCLUSIONS

Completeness of reporting was poor for this example of a rehabilitation exercise intervention, and equally poor in both trial arms. Inter-rater agreement of completeness of reporting was also poor. Using a data extraction tool with poor rater-agreement may add unnecessary burden in a review. However, using a data extraction tool that enables assessment of intervention homogeneity has benefits in making decisions about which data to pool or not.

CLINICAL REHABILITATION IMPACT

Researchers reporting clinical trials must pay more attention to completeness of rehabilitation exercise reporting.

Effects of Motivational Interviewing on Adherence to Treatment Regimens Among Patients With Type 1 Diabetes: A Systematic Review

BACKGROUND

Motivational interviewing (MI) is a strategy for promoting adherence to treatment regimens among patients with diabetes. However, limited evidence exists regarding its effectiveness in reducing A1C.

OBJECTIVE

To identify and synthesize evidence about the effectiveness of patient, provider, and health system interventions to improve diabetes care among patients with type 1 diabetes.

DESIGN

This was a systematic review of randomized controlled trials.

METHODS

A search was conducted of the scientific databases MEDLINE, Elsevier, CINAHL, Google Scholar, ProQuest, Ovid, and PubMed without imposing any time limit. Only four documents met the inclusion criteria and were included in the final analysis. The methodological quality of these four articles was reviewed by three reviewers using the Jadad Scale. The main intervention and the primary outcome in this study were MI or motivational enhancement therapy and A1C, respectively.

RESULTS

The retrieved studies reported that MI promotes self-monitoring of blood glucose and reduces A1C.

CONCLUSION

MI is effective in enhancing patients' adherence to the treatment regimen and thereby decreasing A1C. Given the fact that the reviewed studies had not eliminated the effects of confounding factors, further studies are needed to assess the pure effects of MI on adherence to treatment regimens and A1C levels.

Comparing Automated Office Blood Pressure Readings With Other Methods of Blood Pressure Measurement for Identifying Patients With Possible Hypertension: A Systematic Review and Meta-analysis

IMPORTANCE

Automated office blood pressure (AOBP) measurement involves recording several blood pressure (BP) readings using a fully automated oscillometric sphygmomanometer with the patient resting alone in a quiet place. Although several studies have shown AOBP measurement to be more accurate than routine office BP measurement and not subject to a "white coat effect," the cumulative evidence has not yet been systematically reviewed.

OBJECTIVE

To perform a systematic review and meta-analysis to examine the association between AOBP and office BP readings measured in routine clinical practice and in research studies, and ambulatory BP recorded during awake hours, as the latter is a standard for predicting future cardiovascular events.

DATA SOURCES

The MEDLINE, Embase, and Cochrane Library were searched from 2003 to April 25, 2018.

STUDY SELECTION

Studies on systolic and diastolic BP measurement by AOBP in comparison with awake ambulatory BP, routine office BP, and research BP measurements were included if they contained 30 patients or more.

DATA EXTRACTION AND SYNTHESIS

Study characteristics were abstracted independently and random effects meta-analyses and meta-regressions were conducted.

MAIN OUTCOMES AND MEASURES

Pooled mean differences (95% CI) of systolic and diastolic BP between types of BP measurement.

RESULTS

Data were compiled from 31 articles comprising 9279 participants (4736 men and 4543 women). In samples with systolic AOBP of 130 mm Hg or more, routine office and research systolic BP readings were substantially higher than AOBP readings, with a pooled mean difference of 14.5 mm Hg (95% CI, 11.8-17.2 mm Hg; n = 9; I2 = 94.3%; P < .001) for routine office systolic BP readings and 7.0 mm Hg (95% CI, 4.9-9.1 mm Hg; n = 9; I2 = 85.7%; P < .001) for research systolic BP readings. Systolic awake ambulatory BP and AOBP readings were similar, with a pooled mean difference of 0.3 mm Hg (95% CI, -1.1 to 1.7 mm Hg; n = 19; I2 = 90%; P < .001).

CONCLUSIONS AND RELEVANCE

Automated office blood pressure readings, only when recorded properly with the patient sitting alone in a quiet place, are more accurate than office BP readings in routine clinical practice and are similar to awake ambulatory BP readings, with mean AOBP being devoid of any white coat effect. There has been some reluctance among physicians to adopt this technique because of uncertainty about its advantages compared with more traditional methods of recording BP during an office visit. Based on the evidence, AOBP should now be the preferred method for recording BP in routine clinical practice.

Primary Study Quality in Psychological Meta-Analyses: An Empirical Assessment of Recent Practice

As meta-analytic research has come to occupy a sizeable contingent of published work in the psychological sciences, clarity in the reporting of such work is crucial to its interpretability and reproducibility. This is especially true regarding the assessment of primary study quality, as notions of study quality can vary across research domains. The present study examines the general state of reporting practices related to primary study quality in a sample of 382 published psychological meta-analyses, as well as the reporting decisions and motivations of the authors that published them. Our findings suggest adherence to reporting standards has remained poor for assessments of primary study quality and that the discipline remains inconsistent in its reporting practices generally. We discuss several potential reasons for the poor adherence to reporting standards in our sample, including whether quality assessments are being conducted in the first place, whether standards are well-known within the discipline, and the potential conflation of assessing primary study quality with other facets of conducting a meta-analysis. The implications of suboptimal reporting practices related to primary study quality are discussed.

The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures

BACKGROUND

Systematic reviews, which assess the risk of bias in included studies, are increasingly used to develop environmental hazard assessments and public health guidelines. These research areas typically rely on evidence from human observational studies of exposures, yet there are currently no universally accepted standards for assessing risk of bias in such studies. The risk of bias in non-randomised studies of exposures (ROBINS-E) tool has been developed by building upon tools for risk of bias assessment of randomised trials, diagnostic test accuracy studies and observational studies of interventions. This paper reports our experience with the application of the ROBINS-E tool.

METHODS

We applied ROBINS-E to 74 exposure studies (60 cohort studies, 14 case-control studies) in 3 areas: environmental risk, dietary exposure and drug harm. All investigators provided written feedback, and we documented verbal discussion of the tool. We inductively and iteratively classified the feedback into 7 themes based on commonalities and differences until all the feedback was accounted for in the themes. We present a description of each theme.

RESULTS

We identified practical concerns with the premise that ROBINS-E is a structured comparison of the observational study being rated to the 'ideal' randomised controlled trial. ROBINS-E assesses 7 domains of bias, but relevant questions related to some critical sources of bias, such as exposure and funding source, are not assessed. ROBINS-E fails to discriminate between studies with a single risk of bias or multiple risks of bias. ROBINS-E is severely limited at determining whether confounders will bias study outcomes. The construct of co-exposures was difficult to distinguish from confounders. Applying ROBINS-E was time-consuming and confusing.

CONCLUSIONS

Our experience suggests that the ROBINS-E tool does not meet the need for an international standard for evaluating human observational studies for questions of harm relevant to public and environmental health. We propose that a simpler tool, based on empirical evidence of bias, would provide accurate measures of risk of bias and is more likely to meet the needs of the environmental and public health community.

Effectiveness of probiotics in the management of inflammatory arthritis: a systematic review protocol

REVIEW QUESTIONS

The questions of this systematic review are: i) What is the effectiveness of probiotics as a therapeutic intervention for individuals living with inflammatory arthritis? ii) What are the reported adverse effects of probiotics when used as a therapeutic intervention for individuals living with inflammatory arthritis?

Transradial Versus Transfemoral Access for Percutaneous Coronary Intervention of Unprotected Left Main Coronary Artery Stenosis: A Systematic Review and Meta-Analysis

INTRODUCTION

PCI of ULMS is frequently performed through TFA because of technical complexity and safety concern. Studies have shown comparable efficacy and safety of TRA versus TFA, however, these studies are few in number. We intended to compare the clinical outcomes between transradial access (TRA) and transfemoral access (TFA) in patients undergoing percutaneous coronary intervention (PCI) for unprotected left main coronary artery stenosis (ULMS) by performing a meta-analysis.

METHOD

A systematic search of database, including, PubMed, Web of Science, Google scholar and Cochrane Database were performed by two independent reviewers. Studies were included comparing "TRA" versus "TFA" in patients undergoing PCI in ULMS. The primary outcome was a procedural success rate. Secondary outcomes were major bleeding, access site complications, in-hospital and long term: major adverse cardiac events (MACE), myocardial infarction (MI) and cardiovascular mortality.

RESULTS

Eight studies were included in the analysis. The procedural success rate was 97.3% and there was no statistically significant difference between TRA and TFA groups (OR, 1.41 [CI 0.64, 3.12], I² = 26%). The rates of access site complications (OR, 0.17 [CI 0.07, 0.41], I² = 16%), major bleeding (OR, 0.39 [CI 0.17, 0.86], I² = 0%) and all-cause mortality (OR, 0.28 [CI 0.12, 0.64], I² = 0%) were lower in the TRA group. There were no significant differences in in-hospital and long term cardiovascular mortality, MI and MACE between the two groups.

CONCLUSION

In contrast to TFA, TRA is associated with reduced bleeding and access site complications, with similar procedural success rate in patients undergoing PCI of ULMS.

A Systematic Review and Meta-Analysis on the Strength and Consistency of the Associations between Dupuytren Disease and Diabetes Mellitus, Liver Disease, and Epilepsy

Supplemental Digital Content is available in the text.

Background:

The role of diabetes mellitus, liver disease, and epilepsy as risk factors for Dupuytren disease remains unclear. In this systematic review and meta-analysis, the strength and consistency of these associations were examined.

Methods:

The MEDLINE, EMBASE, and Web of Science databases were searched for articles reporting an association between Dupuytren disease and diabetes mellitus, liver disease, and epilepsy published before September 26, 2016. The frequencies of Dupuytren disease and diabetes mellitus, liver disease, and epilepsy were extracted, as was information on potential confounders. Generalized linear mixed models were applied to estimate pooled odds ratios, adjusted for confounders. Heterogeneity between studies was quantified using an intraclass correlation coefficient and was accounted for by a random effect for study.

Results:

One thousand two hundred sixty unique studies were identified, of which 32 were used in the meta-analyses. An association between Dupuytren disease and diabetes mellitus was observed (OR, 3.06; 95 percent CI, 2.69 to 3.48, adjusted for age), which was stronger for type 1 diabetes mellitus than for type 2 diabetes mellitus but was not statistically significant (p = 0.24). An association between Dupuytren disease and liver disease was observed (OR, 2.92; 95 percent CI, 2.08 to 4.12, adjusted for sex). Dupuytren disease and epilepsy were associated, yielding an OR of 2.80 (95 percent CI, 2.49 to 3.15). Heterogeneity between studies was moderate to low.

Conclusions:

These findings demonstrate an association between Dupuytren disease and diabetes mellitus, liver disease, and epilepsy. Prospective, longitudinal studies are needed to elucidate the pathways causing these associations.

Sex-Specific Associations Between Alcohol Consumption and Incidence of Hypertension: A Systematic Review and Meta-Analysis of Cohort Studies

BACKGROUND

Although it is well established that heavy alcohol consumption increases the risk of hypertension, the risk associated with low levels of alcohol intake in men and women is unclear.

METHODS AND RESULTS

We searched Medline and Embase for original cohort studies on the association between average alcohol consumption and incidence of hypertension in people without hypertension. Random-effects meta-analyses and metaregressions were conducted. Data from 20 articles with 361 254 participants (125 907 men and 235 347 women) and 90 160 incident cases of hypertension (32 426 men and 57 734 women) were included. In people drinking 1 to 2 drinks/day (12 g of pure ethanol per drink), incidence of hypertension differed between men and women (relative riskwomen vs men=0.79; 95% confidence interval, 0.67-0.93). In men, the risk for hypertension in comparison with abstainers was relative risk=1.19 (1.07-1.31; I2=59%), 1.51 (1.30-1.76), and 1.74 (1.35-2.24) for consumption of 1 to 2, 3 to 4, and 5 or more standard drinks per day, respectively. In women, there was no increased risk for 1 to 2 drinks/day (relative risk=0.94; 0.88-1.01; I2=73%), and an increased risk for consumption beyond this level (relative risk=1.42; 1.22-1.66).

CONCLUSIONS

Any alcohol consumption was associated with an increase in the risk for hypertension in men. In women, there was no risk increase for consumption of 1 to 2 drinks/day and an increased risk for higher consumption levels. We did not find evidence for a protective effect of alcohol consumption in women, contrary to earlier meta-analyses.

Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations

Background

Systematic reviews and meta-analyses of observational studies are frequently performed, but no widely accepted guidance is available at present. We performed a systematic scoping review of published methodological recommendations on how to systematically review and meta-analyse observational studies.

Methods

We searched online databases and websites and contacted experts in the field to locate potentially eligible articles. We included articles that provided any type of recommendation on how to conduct systematic reviews and meta-analyses of observational studies. We extracted and summarised recommendations on pre-defined key items: protocol development, research question, search strategy, study eligibility, data extraction, dealing with different study designs, risk of bias assessment, publication bias, heterogeneity, statistical analysis. We summarised recommendations by key item, identifying areas of agreement and disagreement as well as areas where recommendations were missing or scarce.

Results

The searches identified 2461 articles of which 93 were eligible. Many recommendations for reviews and meta-analyses of observational studies were transferred from guidance developed for reviews and meta-analyses of RCTs. Although there was substantial agreement in some methodological areas there was also considerable disagreement on how evidence synthesis of observational studies should be conducted. Conflicting recommendations were seen on topics such as the inclusion of different study designs in systematic reviews and meta-analyses, the use of quality scales to assess the risk of bias, and the choice of model (e.g. fixed vs. random effects) for meta-analysis.

Conclusion

There is a need for sound methodological guidance on how to conduct systematic reviews and meta-analyses of observational studies, which critically considers areas in which there are conflicting recommendations.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0495-9) contains supplementary material, which is available to authorized users.

Association Between Risk-of-Bias Assessments and Results of Randomized Trials in Cochrane Reviews: The ROBES Meta-Epidemiologic Study

Flaws in the design of randomized trials may bias intervention effect estimates and increase between-trial heterogeneity. Empirical evidence suggests that these problems are greatest for subjectively assessed outcomes. For the Risk of Bias in Evidence Synthesis (ROBES) Study, we extracted risk-of-bias judgements (for sequence generation, allocation concealment, blinding, and incomplete data) from a large collection of meta-analyses published in the Cochrane Library (issue 4; April 2011). We categorized outcome measures as mortality, other objective outcome, or subjective outcome, and we estimated associations of bias judgements with intervention effect estimates using Bayesian hierarchical models. Among 2,443 randomized trials in 228 meta-analyses, intervention effect estimates were, on average, exaggerated in trials with high or unclear (versus low) risk-of-bias judgements for sequence generation (ratio of odds ratios (ROR) = 0.91, 95% credible interval (CrI): 0.86, 0.98), allocation concealment (ROR = 0.92, 95% CrI: 0.86, 0.98), and blinding (ROR = 0.87, 95% CrI: 0.80, 0.93). In contrast to previous work, we did not observe consistently different bias for subjective outcomes compared with mortality. However, we found an increase in between-trial heterogeneity associated with lack of blinding in meta-analyses with subjective outcomes. Inconsistency in criteria for risk-of-bias judgements applied by individual reviewers is a likely limitation of routinely collected bias assessments. Inadequate randomization and lack of blinding may lead to exaggeration of intervention effect estimates in randomized trials.

Patient Characteristics Associated With Quality of Colonoscopy Preparation: A Systematic Review and Meta-analysis

BACKGROUND & AIMS

Some features of patients are associated with inadequate bowel preparation, which reduces the effectiveness of colonoscopy examination. We performed a systematic review and meta-analysis of the association between patients' sociodemographic characteristics, health conditions, and medications with inadequate bowel preparation.

METHODS

We searched the PubMed, Scopus, and Cochrane Review databases for randomized controlled trials cohort (prospective and retrospective), case-control, and cross-sectional studies published through March 2016. We collected information on study design, study population, and bowel preparation. For each factor, we obtained the odds ratio (OR) for inadequate bowel preparation. We conducted the meta-analyses using the random-effects approach and investigated any identified heterogeneity and publication bias via graphical methods, stratification, and meta-regression.

RESULTS

We performed a meta-analysis of 67 studies, comprising 75,818 patients. The estimated pooled OR for inadequate bowel preparation was small for sociodemographic characteristics: 1.14 for age, and 1.23 for male sex (excluding studies in Asia, which had substantial heterogeneity and publication bias), and 1.49 for low education. The effect of high body mass index differed significantly in studies with mostly female patients (OR, 1.05) vs those with mostly male patients (OR, 1.30) (P = .013 for the difference). ORs for constipation and cirrhosis were heterogeneous; adjusted ORs were larger than unadjusted ORs (1.97 vs 1.29 for constipation and 3.41 vs 1.36 for cirrhosis). Diabetes (OR, 1.79), hypertension (OR, 1.25), stroke or dementia (OR, 2.09), and opioid use (OR, 1.70) were associated with inadequate bowel preparation. History of abdominal surgery (OR, 0.99) did not associate with inadequate bowel preparation. Use of tricyclic antidepressants had a larger effect on risk of inadequate bowel preparation in studies of mostly female patients (OR, 2.62) than studies of mostly male patients (OR, 1.42) (P = .085 for the difference).

CONCLUSIONS

In a systematic review and meta-analysis, we found no single patient-related factor to be solely associated with inadequate bowel preparation. Health conditions and use of some medications appear to be stronger predictors than sociodemographic characteristics.

Evidence Synthesis in Harm Assessment of Medicines Using the Example of Rosiglitazone and Myocardial Infarction

The current system of harm assessment of medicines has been criticized for relying on intuitive expert judgment. There is a call for more quantitative approaches and transparency in decision-making. Illustrated with the case of cardiovascular safety concerns for rosiglitazone, we aimed to explore a structured procedure for the collection, quality assessment, and statistical modeling of safety data from observational and randomized studies. We distinguished five stages in the synthesis process. In Stage I, the general research question, population and outcome, and general inclusion and exclusion criteria are defined and a systematic search is performed. Stage II focusses on the identification of sub-questions examined in the included studies and the classification of the studies into the different categories of sub-questions. In Stage III, the quality of the identified studies is assessed. Coding and data extraction are performed in Stage IV. Finally, meta-analyses on the study results per sub-question are performed in Stage V. A Pubmed search identified 30 randomized and 14 observational studies meeting our search criteria. From these studies, we identified 4 higher level sub-questions and 4 lower level sub-questions. We were able to categorize 29 individual treatment comparisons into one or more of the sub-question categories, and selected study duration as an important covariate. We extracted covariate, outcome, and sample size information at the treatment arm level of the studies. We extracted absolute numbers of myocardial infarctions from the randomized study, and adjusted risk estimates with 95% confidence intervals from the observational studies. Overall, few events were observed in the randomized studies that were frequently of relatively short duration. The large observational studies provided more information since these were often of longer duration. A Bayesian random effects meta-analysis on these data showed no significant increase in risk of rosiglitazone for any of the sub-questions. The proposed procedure can be of additional value for drug safety assessment because it provides a stepwise approach that guides the decision-making in increasing process transparency. The procedure allows for the inclusion of results from both randomized an observational studies, which is especially relevant for this type of research.

Recommendations for assessing the risk of bias in systematic reviews of health-care interventions

OBJECTIVES

Risk-of-bias assessment is a central component of systematic reviews, but little conclusive empirical evidence exists on the validity of such assessments. In the context of such uncertainty, we present pragmatic recommendations that promote transparency and reproducibility in processes, address methodological advances in the risk-of-bias assessment, and can be applied consistently across review topics.

STUDY DESIGN AND SETTING

Epidemiological study design principles; available empirical evidence, risk-of-bias tools, and guidance; and workgroup consensus.

RESULTS

We developed recommendations for assessing the risk of bias of studies of health-care interventions specific to framing the focus and scope of risk-of-bias assessment; selecting the risk-of-bias categories; choosing assessment instruments; and conducting, analyzing, and presenting results of risk-of-bias assessments. Key recommendations include transparency and reproducibility of judgments, separating risk of bias from other constructs such as applicability and precision, and evaluating the risk of bias per outcome. We recommend against certain past practices, such as focusing on reporting quality, relying solely on study design or numerical quality scores, and automatically downgrading for industry sponsorship.

CONCLUSION

Risk-of-bias assessment remains a challenging but essential step in systematic reviews. We presented standards to promote transparency of judgments.

Childhood obesity and adult cardiovascular disease risk factors: a systematic review with meta-analysis

BACKGROUND

Overweight and obesity is a major public health concern that includes associations with the development of cardiovascular disease (CVD) risk factors during childhood and adolescence as well as premature mortality in adults. Despite the high prevalence of childhood and adolescent obesity as well as adult CVD, individual studies as well as previous systematic reviews examining the relationship between childhood obesity and adult CVD have yielded conflicting results. The purpose of this study was to use the aggregate data meta-analytic approach to address this gap.

METHODS

Studies were included if they met the following criteria: (1) longitudinal and cohort studies (including case-cohort), (2) childhood exposure and adult outcomes collected on the same individual over time, (3) childhood obesity, as defined by the original study authors, (4) English-language articles, (5) studies published up to June, 2015, (6) one or more of the following CVD risk factors [systolic blood pressure (SBP), diastolic blood pressure (DBP), total cholesterol (TC), high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL), non-high-density lipoprotein cholesterol (non-HDL), and triglycerides (TG)], (7) outcome(s) not self-reported, and (8) exposure measurements (child's adiposity) assessed by health professionals, trained investigators, or self-reported. Studies were retrieved by searching three electronic databases as well as citation tracking. Fisher's r to z score was calculated for each study for each outcome. Pooled effect sizes were calculated using random-effects models while risk of bias was assessed using the STROBE instrument. In order to try and identify sources of heterogeneity, random-effects meta-regression was also performed.

RESULTS

Of the 4840 citations reviewed, a total of 23 studies were included in the systematic review and 21 in the meta-analysis. The findings suggested that childhood obesity is significantly and positively associated with adult SBP (Zr = 0.11; 95% CI: 0.07, 0.14), DBP (Zr = 0.11; 95% CI: 0.07, 0.14), and TG (Zr =0.08; 95% CI: 0.03, 0.13), and significantly and inversely associated with adult HDL (Zr = -0.06; 95% CI: -0.10, -0.02). For those studies that adjusted for adult body mass index (BMI), associations were reversed, suggesting that adult BMI may be a potential mediator. Nine studies had more than 33% of items that placed them at an increased risk for bias.

CONCLUSIONS

The results of this study suggest that childhood obesity may be a risk factor for selected adult CVD risk factors. However, a need exists for additional, higher-quality studies that include, but are not limited to, both unadjusted and adjusted measures such as BMI before any definitive conclusions can be reached.

SYSTEMATIC REVIEW AND META-ANALYSIS

PROSPERO 2015: CRD42015019763 .

Childhood obesity and adult cardiovascular disease risk factors: a systematic review with meta-analysis

BACKGROUND

Overweight and obesity is a major public health concern that includes associations with the development of cardiovascular disease (CVD) risk factors during childhood and adolescence as well as premature mortality in adults. Despite the high prevalence of childhood and adolescent obesity as well as adult CVD, individual studies as well as previous systematic reviews examining the relationship between childhood obesity and adult CVD have yielded conflicting results. The purpose of this study was to use the aggregate data meta-analytic approach to address this gap.

METHODS

Studies were included if they met the following criteria: (1) longitudinal and cohort studies (including case-cohort), (2) childhood exposure and adult outcomes collected on the same individual over time, (3) childhood obesity, as defined by the original study authors, (4) English-language articles, (5) studies published up to June, 2015, (6) one or more of the following CVD risk factors [systolic blood pressure (SBP), diastolic blood pressure (DBP), total cholesterol (TC), high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL), non-high-density lipoprotein cholesterol (non-HDL), and triglycerides (TG)], (7) outcome(s) not self-reported, and (8) exposure measurements (child's adiposity) assessed by health professionals, trained investigators, or self-reported. Studies were retrieved by searching three electronic databases as well as citation tracking. Fisher's r to z score was calculated for each study for each outcome. Pooled effect sizes were calculated using random-effects models while risk of bias was assessed using the STROBE instrument. In order to try and identify sources of heterogeneity, random-effects meta-regression was also performed.

RESULTS

Of the 4840 citations reviewed, a total of 23 studies were included in the systematic review and 21 in the meta-analysis. The findings suggested that childhood obesity is significantly and positively associated with adult SBP (Zr = 0.11; 95% CI: 0.07, 0.14), DBP (Zr = 0.11; 95% CI: 0.07, 0.14), and TG (Zr =0.08; 95% CI: 0.03, 0.13), and significantly and inversely associated with adult HDL (Zr = -0.06; 95% CI: -0.10, -0.02). For those studies that adjusted for adult body mass index (BMI), associations were reversed, suggesting that adult BMI may be a potential mediator. Nine studies had more than 33% of items that placed them at an increased risk for bias.

CONCLUSIONS

The results of this study suggest that childhood obesity may be a risk factor for selected adult CVD risk factors. However, a need exists for additional, higher-quality studies that include, but are not limited to, both unadjusted and adjusted measures such as BMI before any definitive conclusions can be reached.

SYSTEMATIC REVIEW AND META-ANALYSIS

PROSPERO 2015: CRD42015019763 .

The Use of Deceased Controls in Epidemiologic Research: A Systematic Review

Scholarly debate on the use of deceased controls in epidemiologic research continues. This systematic review examined published epidemiologic research using deceased persons as a control group. A systematic search of 5 major biomedical literature databases (MEDLINE, CINAHL, PsycINFO, Scopus, and EMBASE) was conducted, using variations of the search terms "deceased" and "controls" to identify relevant peer-reviewed journal articles. Information was sought on study design, rationale for using deceased controls, application of theoretical principles of control selection, and discussion of the use of deceased controls. The review identified 134 studies using deceased controls published in English between 1978 and 2015. Common health outcomes under investigation included cancer (n = 31; 23.1%), nervous system diseases (n = 26; 19.4%), and injury and other external causes (n = 22; 16.4%). The majority of studies used deceased controls for comparison with deceased cases (n = 95; 70.9%). Investigators rarely presented their rationale for control selection (n = 25/134; 18.7%); however, common reasons included comparability of information on exposures, lack of appropriate controls from other sources, and counteracting bias associated with living controls. Comparable accuracy was the most frequently observed principle of control selection (n = 92; 68.7%). This review highlights the breadth of research using deceased controls and indicates their appropriateness in studies using deceased cases.