Systematic Review: A Method at Risk for Being Corrupted

From agenda to action: JBI Evidence Synthesis and the United Nations Sustainable Development Goals

OBJECTIVE

This pilot study sought to map the alignment of publications in JBI Evidence Synthesis to the United Nations Sustainable Development Goal (SDG) 3 (global health and well-being) and to understand JBI authors' awareness of and engagement with the goals. This will contribute to a larger-scale analysis to assist the JBI Collaboration in engaging with and working toward achieving the SDGs.

INTRODUCTION

In 2015, the United Nations agreed on 17 SDGs as the central framework for sustainable development worldwide. However, in the wake of the global pandemic, the 2030 SDG agenda is in jeopardy due to multiple cascading and intersecting crises. The Global Commission on Evidence has urged the evidence community to consider how it might more meaningfully contribute to the evidence architecture. As we pass the midway point to the SDG 2030 target date, it is timely to reflect on the contribution of those in the evidence synthesis community to achieving these important global objectives.

METHODS

A 2-phase study was conducted utilizing desktop audit methods. SDG 3 targets were mapped across systematic and scoping reviews published in JBI Evidence Synthesis using established key terms, followed by a brief author survey and thematic analysis.

RESULTS

The results of this pilot study indicate that 28.5% of syntheses published in JBI Evidence Synthesis address 11 of the 13 targets of SDG 3. SDGs are not currently a priority consideration for most JBI Evidence Synthesis authors, but there is a desire to learn more and integrate the goals into their prioritization processes.

CONCLUSIONS

While this was only a small pilot study, it is indicative of a need to reset and recommit to mutual global agendas to transform the evidence ecosystem, and to maximize the limited resources available in order to truly have a global impact on health outcomes.

Experiences of family members when a parent is hospitalized for their mental illness: a qualitative systematic review

BACKGROUND

A considerable proportion of people attending mental health services are parents with dependent children. Parental mental illness can be challenging for all family members including the parent's children and partner. The hospitalization of the parent and subsequent separation from dependent children may be a particularly challenging time for all family members. The aim of this paper was to review qualitative studies of family members' experiences when parents, who have dependent children, were hospitalized for their mental illness. The experiences of parents themselves, their children aged 0-18 (including retrospective accounts of adults describing their childhoods), and other family members are included.

METHODS

This systematic review followed Cochrane Collaboration and PRISMA guidelines. A search was performed with keywords relating to parents, mental illness, psychiatric treatment, inpatient units, family members and experiences. Databases included CINAHL Plus, PsycINFO, ProQuest, MEDLINE, PubMed and Scopus. Quality assessment was undertaken using an expanded version of the Critical Appraisal Skills Programme. Thematic synthesis was conducted on the included papers.

RESULTS

Eight papers were identified. The quality assessment was rated as high in some papers, in terms of the clarity of research aims, justification of the methodology employed, recruitment strategy and consideration of ethics. In others, the study design, inclusion criteria and reporting of participant demographics were unclear. Family experiences of pressure and additional responsibilities associated with the parent receiving inpatient treatment were identified along with the family's need for psychoeducational information, and guidance when visiting the parent in hospital. Children expressed various emotions and the need to connect with others. The final theme related to adverse impacts on the parent-child bond when the parent was hospitalized.

CONCLUSION

The limited research in this area indicates that the needs of families are not being met when a parent is hospitalized for their mental illness. There is a considerable need for adequate models of care, family-focused training for staff, and psychoeducational resources for families. Additional research in this area is essential to understand the experiences of different family members during this vulnerable time.

Conducting evaluations of evidence that are transparent, timely and can lead to health-protective actions

BACKGROUND

In February 2021, over one hundred scientists and policy experts participated in a web-based Workshop to discuss the ways that divergent evaluations of evidence and scientific uncertainties are used to delay timely protection of human health and the environment from exposures to hazardous agents. The Workshop arose from a previous workshop organized by the European Environment Agency (EEA) in 2008 and which also drew on case studies from the EEA reports on 'Late Lessons from Early Warnings' (2001, 2013). These reports documented dozens of hazardous agents including many chemicals, for which risk reduction measures were delayed for decades after scientists and others had issued early and later warnings about the harm likely to be caused by those agents.

RESULTS

Workshop participants used recent case studies including Perfluorooctanoic acid (PFOA), Extremely Low Frequency - Electrical Magnetic Fields (ELF-EMF fields), glyphosate, and Bisphenol A (BPA) to explore myriad reasons for divergent outcomes of evaluations, which has led to delayed and inadequate protection of the public's health. Strategies to overcome these barriers must, therefore, at a minimum include approaches that 1) Make better use of existing data and information, 2) Ensure timeliness, 3) Increase transparency, consistency and minimize bias in evidence evaluations, and 4) Minimize the influence of financial conflicts of interest.

CONCLUSION

The recommendations should enhance the production of "actionable evidence," that is, reliable evaluations of the scientific evidence to support timely actions to protect health and environments from exposures to hazardous agents. The recommendations are applicable to policy and regulatory settings at the local, state, federal and international levels.

Efficacy and safety of electronic cigarettes as a smoking cessation intervention: A systematic review and network meta-analysis

INTRODUCTION

This systematic review of randomized controlled trials (RCTs) evaluated the efficacy and safety of electronic cigarettes (e-cigarettes, ENDS) in helping people who smoke to achieve abstinence compared with electronic non-nicotine delivery systems (ENNDS, no nicotine) or any smoking cessation comparator treatment or combination of treatments at 24–26 weeks and at 52 weeks.

METHODS

Systematic review techniques involved searches of three databases in February 2020 with update searches run on 14 May 2021, two-person independent screening, two-person independent assessment of bias, formal extraction of data with verification by a second person, a feasibility assessment to decide if meta-analysis was appropriate, and network meta-analysis (NMA) of data at 24–26 weeks. Data at 52 weeks were narratively summarized.

RESULTS

Ten RCTs met the inclusion criteria, eight for efficacy and ten for safety. Eight of the nine RCTs were assessed as at high risk of bias. The sample sizes of the RCTs were 30–2012. Using nicotine replacement therapy (NRT) as the reference treatment, the incidences of smoking cessation at 24–26 weeks were comparable between ENDS and NRT groups (RR=1.17; 95% CrI: 0.66–1.86). Three sensitivity analyses were carried out indicating the main findings for 24–26 weeks were robust to assumptions. The findings at 52 weeks were inconclusive.

CONCLUSIONS

This systematic review and NMA indicates that there is no clear evidence of a difference in effect between nicotine containing e-cigarettes and NRT on incidences of smoking cessation at 24–26 weeks, and substantial uncertainty remains.

Reviews in environmental health: How systematic are they?

BACKGROUND

Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review.

OBJECTIVES

To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health.

METHODS

We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews.

RESULTS

We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement.

DISCUSSION

Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.

International Society for the Study of Fatty Acids and Lipids 2016 Debate: For Science-Based Dietary Guidelines on Fats, Meta-Analysis and Systematic Reviews Are Decisive

This paper summarizes a debate on whether meta-analyses and systematic reviews are decisive in formulating guidelines for dietary fat. Held during the 12th congress of the International Society for the Study of Fatty Acids and Lipids in Stellenbosch, South Africa, September 7, 2016, the debate was hosted by the International Union of Nutritional Sciences and the International Expert Movement to Improve Dietary Fat Quality (IEM, www.theiem.org). Clemens von Schacky, Ludwig Maximilians-University, Munich, Germany, supported the statement, describing the types of weaknesses in individual studies and clinical trials. With examples of how to overcome such limitations, he concluded that nutritional guidelines on fat need a proper scientific basis in which randomized controlled trials (RCTs) with clinical endpoints and their meta-analyses are essential and decisive. In contention, Ingeborg Brouwer, Vrije Universiteit, Amsterdam, declared that recommendations on dietary fat intake should always be based on the totality of the evidence, including physiologic and biochemical knowledge and associations from observational epidemiology. RCTs and meta-analyses have their shortcomings, but well-conducted systematic reviews and meta-analyses support a transparent process for developing dietary fat guidelines. Participants agreed that evidence-based decision-making for dietary guidance should consider all the best available evidence using a transparent, systematic review.