Implications of a complexity perspective for systematic reviews and guideline development in health decision making

Integrating rights and evidence: a technical advance in abortion guideline development

As part of its core work, the WHO generates, translates and disseminates knowledge, including through guideline development. In recent years, substantial work has been undertaken to revise the Evidence to Decision framework in order to fully integrate inter alia human rights. This paper describes an innovative methodological approach taken by the authors to inform law and policy recommendations for the forthcoming third edition of the Safe Abortion: Technical and Policy Guidance for Health Systems. The methodology described here effectively integrates human rights protection and enjoyment as part of health outcomes and analysis, ensuring that subsequent recommendations are consistent with international human rights standards. This will allow guideline users to make informed decisions on interventions, including legal and policy reform, to fulfil relevant human rights including the right to health.

The case for developing a cohesive systems approach to research across unhealthy commodity industries

OBJECTIVES

Most non-communicable diseases are preventable and largely driven by the consumption of harmful products, such as tobacco, alcohol, gambling and ultra-processed food and drink products, collectively termed unhealthy commodities. This paper explores the links between unhealthy commodity industries (UCIs), analyses the extent of alignment across their corporate political strategies, and proposes a cohesive systems approach to research across UCIs.

METHODS

We held an expert consultation on analysing the involvement of UCIs in public health policy, conducted an analysis of business links across UCIs, and employed taxonomies of corporate political activity to collate, compare and illustrate strategies employed by the alcohol, ultra-processed food and drink products, tobacco and gambling industries.

RESULTS

There are clear commonalities across UCIs' strategies in shaping evidence, employing narratives and framing techniques, constituency building and policy substitution. There is also consistent evidence of business links between UCIs, as well as complex relationships with government agencies, often allowing UCIs to engage in policy-making forums. This knowledge indicates that the role of all UCIs in public health policy would benefit from a common approach to analysis. This enables the development of a theoretical framework for understanding how UCIs influence the policy process. It highlights the need for a deeper and broader understanding of conflicts of interests and how to avoid them; and a broader conception of what constitutes strong evidence generated by a wider range of research types.

CONCLUSION

UCIs employ shared strategies to shape public health policy, protecting business interests, and thereby contributing to the perpetuation of non-communicable diseases. A cohesive systems approach to research across UCIs is required to deepen shared understanding of this complex and interconnected area and also to inform a more effective and coherent response.

Care-experienced cHildren and young people's Interventions to improve Mental health and wEll-being outcomes: Systematic review (CHIMES) protocol

INTRODUCTION

The mental health and well-being of children and young people who have been in care (ie, care-experienced) are a priority. There are a range of interventions aimed at addressing these outcomes, but the international evidence-base remains ambiguous. There is a paucity of methodologically robust systematic reviews of intervention effectiveness, with few considering the contextual conditions under which evaluations were conducted. This is important in understanding the potential transferability of the evidence-base across contexts. The present systematic review will adopt a complex systems perspective to synthesise evidence reporting evaluations of mental health and well-being interventions for care-experienced children and young people. It will address impact, equity, cost-effectiveness, context, implementation and acceptability. Stakeholder consultation will prioritise a programme theory, and associated intervention, that may progress to further development and evaluation in the UK.

METHODS AND ANALYSIS

We will search 16 bibliographic databases from 1990 to June 2020. Supplementary searching will include citation tracking, author recommendation, and identification of evidence clusters relevant to included evaluations. The eligible population is children and young people (aged ≤25 years) with experience of being in care. Outcomes are (1) mental, behavioural or neurodevelopmental disorders; (2) subjective well-being; (3) self-harm; suicidal ideation; suicide. Study quality will be appraised with methodologically appropriate tools. We will construct a taxonomy of programme theories and intervention types. Thematic synthesis will be used for qualitative data reporting context, implementation and acceptability. If appropriate, meta-analysis will be conducted with outcome and economic data. Convergent synthesis will be used to integrate syntheses of qualitative and quantitative data.

ETHICS AND DISSEMINATION

We have a comprehensive strategy for engagement with care-experienced children and young people, carers and social care professionals. Dissemination will include academic and non-academic publications and conference presentations. Ethical approval from Cardiff University's School of Social Sciences REC will be obtained if necessary.

PROSPERO REGISTRATION NUMBER

CRD42020177478.

WICID framework version 1.0: criteria and considerations to guide evidence-informed decision-making on non-pharmacological interventions targeting COVID-19

Introduction

Public health decision-making requires the balancing of numerous, often conflicting factors. However, participatory, evidence-informed decision-making processes to identify and weigh these factors are often not possible- especially, in the context of the SARS-CoV-2 pandemic. While evidence-to-decision frameworks are not able or intended to replace stakeholder participation, they can serve as a tool to approach relevancy and comprehensiveness of the criteria considered.

Objective

To develop a decision-making framework adapted to the challenges of decision-making on non-pharmacological interventions to contain the global SARS-CoV-2 pandemic.

Methods

We employed the ‘best fit’ framework synthesis technique and used the WHO-INTEGRATE framework as a starting point. First, we adapted the framework through brainstorming exercises and application to case studies. Next, we conducted a content analysis of comprehensive strategy documents intended to guide policymakers on the phasing out of applied lockdown measures in Germany. Based on factors and criteria identified in this process, we developed the WICID (WHO-INTEGRATE COVID-19) framework version 1.0.

Results

Twelve comprehensive strategy documents were analysed. The revised framework consists of 11+1 criteria, supported by 48 aspects, and embraces a complex systems perspective. The criteria cover implications for the health of individuals and populations due to and beyond COVID-19, infringement on liberties and fundamental human rights, acceptability and equity considerations, societal, environmental and economic implications, as well as implementation, resource and feasibility considerations.

Discussion

The proposed framework will be expanded through a comprehensive document analysis focusing on key stakeholder groups across the society. The WICID framework can be a tool to support comprehensive evidence-informed decision-making processes.

Methods for the health technology assessment of complex interventions: a protocol for a scoping review

INTRODUCTION

In healthcare policy and economic literature, research on the health technology assessment (HTA) of complex interventions (CIs) is becoming increasingly important. In many developed countries, HTA guides decision-making to help achieve greater value for money when funding health care. However, research has yet to identify the forms of evidence and evaluation criteria that should be used in the HTA of CIs. Previous research has established that the HTA of CIs requires multiple factors to be evaluated but there is no agreement on which factors ought always to be considered. There is equally little agreement on which forms of evidence ought to be collected or synthesised and how. We plan to perform a systematic scoping review in order to identify the range of evaluation criteria and types of evidence currently used in the HTA of CIs.

METHOD AND ANALYSIS

This protocol was developed to guide the methodological framework for the conduct of a scoping review on health technology assessment (HTA) of complex interventions (CIs), using the Joanna Briggs Institute guidelines and the six-stage framework proposed by Arksey and O'Malley, in addition to more recent innovations in scoping review methodology. A grey literature search will supplement the primary searches of seven electronic databases for studies available in English between January 2000 and August 2020. Two reviewers will independently screen all search results for inclusion and data will be extracted using a customised data extraction or charting form. Any dispute will be resolved by consensus or through arbitration by a third author. The mnemonic Population, Concept and Context will be adopted to establish criteria for selecting relevant literature, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses: Extension for Scoping Review will be used for reporting the results. Several explanatory-descriptive methods will be used for analysing the extracted data including frequency and trend analyses as well as reflexive thematic coding and analysis.Mapping evidence on the HTA of CIs will allow us to gain a better understanding of both established and emerging practices, including the information types, requirements, values and parameters that are incorporated in the HTA of CIs. We also expect the findings of the scoping review to help identify research gaps that will guide future studies. As healthcare becomes more complex in its delivery, it is timely to determine how these complex interventions should be assessed so that policy decisions can be made about whether implementation and public funding is warranted.

ETHICS AND DISSEMINATION

This scoping review will involve secondary analysis of already collected data, and thus, does not require ethics approval. The research findings will be submitted to peer-reviewed journals for publication and will also be disseminated at conferences and seminars.

Systems integration to promote the mental health of Aboriginal and Torres Strait Islander children: protocol for a community-driven continuous quality improvement approach

BACKGROUND

Systems integration to promote the mental health of Aboriginal and Torres Strait Islander children works towards developing a spectrum of effective, community-based services and supports. These services and supports are organised into a coordinated network, build meaningful partnerships with families and address their cultural and linguistic needs, to help children to function better at home, in school, in the community, and throughout life. This study is conducted in partnership with primary healthcare (PHC) and other services in three diverse Indigenous Australian communities. It entails conceptualising, co-designing, implementing, and evaluating the effectiveness of systems integration to promote the mental health and wellbeing of Indigenous school-aged children (4-17 years). This paper outlines a protocol for implementing such complex community-driven research.

METHODS/DESIGN

Using continuous quality improvement processes, community co-designed strategies for improved systems integration will be informed by narratives from yarning circles with Indigenous children and service providers, and quantitative data from surveys of service providers and audits of PHC client records and intersectoral systems. Agreed strategies to improve the integration of community-based services and supports will be modelled using microsimulation software, with a preferred model implemented in each community. The evaluation will investigate changes in the: 1) availability of services that are community-driven, youth-informed and culturally competent; 2) extent of collaborative service networks; 3) identification by PHC services of children's social and emotional wellbeing concerns; and 4) ratio of children receiving services to identified need. Costs and benefits of improvements to systems integration will also be calculated.

DISCUSSION

The study will provide evidence-informed, community-driven, and tested models that can be used for implementing systems integration to promote the mental health and wellbeing of Indigenous children. It will identify the situational enablers and barriers that impact systems integration and determine the extent to which systems integration improves service availability, systems and child outcomes. Evidence for the cost effectiveness of systems-level integration will contribute to national mental health policy reform.

Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators

BACKGROUND

Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A "complex systems approach" has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations.

METHODS AND FINDINGS

We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014-September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study.

CONCLUSION

This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.

Bridging the research-practice gap in healthcare: a rapid review of research translation centres in England and Australia

Background

Large-scale partnerships between universities and health services are widely seen as vehicles for bridging the evidence–practice gap and for accelerating the adoption of new evidence in healthcare. Recently, different versions of these partnerships – often called academic health science centres – have been established across the globe. Although they differ in structure and processes, all aim to improve the integration of research and education with health services. Collectively, these entities are often referred to as Research Translation Centres (RTCs) and both England and Australia have developed relatively new and funded examples of these collaborative centres.

Methods

This paper presents findings from a rapid review of RTCs in Australia and England that aimed to identify their structures, leadership, workforce development and strategies for involving communities and service users. The review included published academic and grey literature with a customised search of the Google search engine and RTC websites.

Results

RTCs are complex system-level interventions that will need to disrupt the current paradigms and silos inherent in healthcare, education and research in order to meet their aims. This will require vision, leadership, collaborations and shared learnings, alongside structures, processes and strategies to deliver impact in the face of complexity. The impact of RTCs in overcoming the deeply entrenched silos across organisations, disciplines and sectors needs to be captured at the systems, organisation and individual levels. This includes workforce capacity and public and patient involvement that are vital to understanding the evolution of RTCs. In addition, new models of leadership are needed to support the brokering and mobilisation of knowledge in complex organisations.

Conclusions

The development and funding of RTCs represents one of the most significant shifts in the health research landscape and it is imperative that we continue to explore how we can progress the integration of research and healthcare and ensure research meets stakeholder needs and is translated via the collaborations supported by these organisations. Because RTCs are a recent addition to the healthcare landscape in Australia, it is instructive to review the processes and infrastructure needed to support their implementation and applied health research in England.

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

BACKGROUND

Guidelines on public health and health system interventions often involve considerations beyond effectiveness and safety to account for the impact that these interventions have on the wider systems in which they are implemented. This paper describes how a complexity perspective may be adopted in guideline development to facilitate a more nuanced consideration of a range of factors pertinent to decisions regarding public health and health system interventions. These factors include acceptability and feasibility, and societal, economic, and equity and equality implications of interventions.

MAIN MESSAGE

A 5-step process describes how to incorporate a complexity perspective in guideline development with examples to illustrate each step. The steps include: (i) guideline scoping, (ii) formulating questions, (iii) retrieving and synthesising evidence, (iv) assessing the evidence, and (v) developing recommendations. Guideline scoping using stakeholder consultations, complexity features, evidence mapping, logic modelling, and explicit decision criteria is emphasised as a key step that informs all subsequent steps.

CONCLUSIONS

Through explicit consideration of a range of factors and enhanced understanding of the specific circumstances in which interventions work, a complexity perspective can yield guidelines with better informed recommendations and facilitate local adaptation and implementation. Further work will need to look into the methods of collecting and assessing different types of evidence beyond effectiveness and develop procedural guidance for prioritising across a range of decision criteria.

Impact of population tobacco control interventions on socioeconomic inequalities in smoking: a systematic review and appraisal of future research directions

BACKGROUND

While price increases and targeted cessation support have been found to reduce inequalities in smoking by socioeconomic status (SES), evidence on other measures is mixed. We aimed to update the most recent (2014) previous review by identifying and appraising evidence published since 2013 on the equity impact of population tobacco control measures.

METHODS

Systematic searching of 10 electronic databases and hand-searching of four key journals identified 68 primary research articles published since 2013 that sought to examine the equity impact of population tobacco control measures in high-income countries with a negative socioeconomic gradient in smoking. Reported equity impacts were categorised as positive (greater impact among lower SES), neutral (no difference by SES), negative (greater impact among higher SES) or mixed/unclear.

RESULTS

There was substantial growth in research seeking to evaluate the equity impact of tobacco control interventions, but the majority of new studies showed mixed/unclear results. Findings for price increases and targeted cessation support continue to suggest an equity-positive impact, but limitations in the available evidence make further assessment difficult. Substantial differences in the context, scale and implementation of tobacco control policies make straightforward comparison of findings from the previous 2014 and current reviews problematic.

CONCLUSION

Researchers need to adopt more sophisticated, multidisciplinary approaches in evaluating the equity impact of tobacco control measures-developing robust measures of equity effect and using frameworks that take account of context, existing systems/processes and the likely mechanisms of action. Socioeconomic differences in intervention impact within low-income and middle-income countries require evaluation.

Context and the evidence-based paradigm: The potential for participatory research and systems thinking in oral health

The implementation of research evidence to promote oral health is critical, given the intransigent and emerging challenges for policymakers at a population level. Despite this, little attention has been paid to implementation research within the evidence-based paradigm. This is important as getting research evidence into clinical practice is not a linear path that consists of simple sequential steps. In this article, we argue that we need to consider a broader range of conceptual and methodological approaches to increase the value of information generated. This should be undertaken either in parallel with empirical and experimental designs, or in some cases, instead of. This is important if we are going to understand the complexity and contextual knowledge of the 'system', within which interventions are implemented. Involving key stakeholders alongside empirical and experimental designs is one helpful approach. Examples of these approaches include Patient and Public Involvement and the development of Core Outcome Sets, where the views of those that will be potentially affected by the research, are included. The use of theoretical frameworks and process evaluations alongside trials are also important, if they are fully integrated into the approach taken to address the research question. A more radical approach is using participatory designs and 'systems thinking'. Participatory approaches include subject matter 'experts by experience'. These include patients, their families, carers, healthcare professionals, services managers, policymakers, commissioners and researchers. Participatory approaches raise important questions about who facilitates the process, when it should happen and how the diverse actors become meaningfully engaged so that their involvement is active, democratic and ongoing. We argue that the issues of control, power and language are central to this and represent a paradigmatic shift to conventional approaches. Systems thinking captures the idea that public health problems commonly involve multiple interdependent and interconnected factors, which interact with each other dynamically. This approach challenges the simplicity of the hierarchy of evidence and linear sequential logic, when it does not account for context. In contrast, systems thinking accepts complexity de novo and emphasizes the need to understand the whole system rather than its individual component parts. We conclude with the idea that participatory and systems thinking help to unpack the diverse agents that are often involved in the generation and translation of evidence into clinical dental practice. It moves our conception of research away from a simple exchange between 'knowledge producers' and 'knowledge users' and raises both methodological and epistemological challenges.

Understanding the Problem of Access to Public Health Insurance Schemes among Cross-Border Migrants in Thailand through Systems Thinking

Thailand has become a popular destination for international migrant workers, particularly from Cambodia, Lao PDR, and Myanmar. However, only a fraction of these migrant workers were insured by public health insurance. The objective of this study was to apply systems thinking to explore contextual factors affecting access to public health insurance among cross-border migrants in Thailand. A group model building approach was applied. Participants (n = 20) were encouraged to share ideas about underlying drivers and barriers of migrants' access to health insurance. The causal loop diagram and stock and flow diagram were synthesised to identify the dynamics of access to migrant health insurance. Results showed that nationality verification is an important mechanism to deal with the precarious citizenship status of undocumented migrants. However, some migrants are still left uninsured. The likely explanations are the semi-voluntary nature of the Health Insurance Card Scheme, administrative delay of the enrollment process, and resistance of some employers to hiring migrants. As a result, findings suggest that effective communication is required to raise acceptance towards insurance among migrants and their employers. A participatory public policy process is needed to create a good balance of migrant policies among diverse authorities.

Virtual follow-up and care for patients with cardiac electronic implantable devices: protocol for a systematic review

BACKGROUND

Capacity to deliver outpatient care for patients with cardiac implantable electronic devices (CIEDs) may soon be outweighed by need. This systematic review aims to investigate the comparative effectiveness, safety, and cost for virtual or remote clinic interventions for patients with CIEDs and explores how outcomes may be influenced by patient or system factors in-depth.

METHODS

We will perform a systematic literature search in MEDLINE, Embase, PsycINFO, CINAHL, Proquest Dissertations & Theses, other EBM Reviews, and trial registry databases. Two authors will independently screen titles and abstracts for eligibility. We will include randomized and non-randomized controlled trials, quasi-randomized and experimental studies, cohort, and case-control studies. Study populations of interest are individuals with a CIED (pacemaker, ICD, CRT). Eligibility will be restricted to virtual or remote follow-up or care interventions compared to any other approach. The co-primary outcomes of interest are mortality and patient satisfaction. Secondary outcomes include clinical effectiveness (e.g., ICD shock, time-to-detection of medical event, hospitalizations), safety (e.g., serious or device-related adverse events), device efficacy (e.g., transmissions, malfunctions), costs, workflow (e.g., resources, process outcomes, time-saved), and patient reported (e.g., burden, quality of life). Data will be extracted by one author and checked by a second using a standardized template. We will use published frameworks to capture data relevant to intervention effects that may be influenced by intervention definition or complexity, context and setting, or in socially disadvantaged populations. Detailed descriptive results will be presented for all included studies and outcomes, and where feasible, synthesized using meta-analysis. Risk of bias will be assessed by two review authors independently using Cochrane Risk of Bias tools. Certainty of evidence will be assessed using the GRADE approach.

DISCUSSION

Increases in number of CIEDs implanted, combined with an aging population and finite health resource allocations at the system-level may lead to increased reliance on virtual follow-up or care models in the future. These models must prioritize consistent, equitable, and timely care as a priority. Results from this systematic review will provide important insight into the potential contextual factors which moderate or mediate the effectiveness, safety, and cost of virtual follow-up or care models for patients.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO registration number CRD42020145210.

Patient and healthcare provider reported barriers and enablers to virtual or remote-only follow-up models for cardiovascular implantable electronic devices: protocol for a qualitative framework synthesis

BACKGROUND

Virtual care models are used to follow-up patients with cardiovascular implantable electronic devices (CIED), including pacemakers, implantable cardioverter defibrillators, and cardiac resynchronization therapy. There is increasing interest in the expansion of virtual, or even remote-only, CIED care models to alleviate resource and economic burden to both patients and specialty device clinics and to maintain or improve equity and access to high-quality cardiovascular care. This qualitative framework synthesis aims to identify barriers and enablers to virtual care models from both the perspective of the patient and device clinics. How setting, context, equity factors or other aspects influence these factors, or satisfaction with care, will also be investigated.

METHODS

We will perform a systematic literature search in MEDLINE, Embase, PsycINFO, CINAHL, Proquest Dissertations & Theses, other EBM Reviews, and trial registry databases. Screening will be completed by two independent review authors. Original research articles having a qualitative component (i.e., qualitative, mixed-, or multi-method) are eligible. Study populations of interest are (a) individuals with a CIED or (b) healthcare providers involved in any aspect of virtual or remote follow-up of patients with CIEDs. Eligibility will be restricted to studies published after January 1, 2000 in English or French. Data will be captured using standardized templates based on the domains and constructs of the Theoretical Domains Framework and the Warwick Patient Experiences Framework. The Joanna Briggs Institute Critical Appraisal Checklist for Qualitative Research will be applied to all included studies. The GRADE-CERQual approach will be applied to assess and summarize confidence in key findings. Reporting will follow the enhancing transparency in reporting the synthesis of qualitative research (ENTREQ) statement. Detailed descriptive results will be presented, and summary of qualitative findings tables will be produced.

DISCUSSION

While a number of trials have captured the clinical effectiveness and safety of virtual follow-up for CIEDs, there has been less attention given to factors affecting use and implementation of remote care by patients and healthcare providers or satisfaction with care. Results from this qualitative framework synthesis will provide important lived experience data from both patients and healthcare providers which will be essential to incorporate in clinical guidelines.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42020160533.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies

BACKGROUND

Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

METHODS

A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included.

RESULTS

Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence.

CONCLUSION

This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

Dimensions of the Complexity of Health Interventions: What Are We Talking About? A Review

Many recommendations and innovative approaches are available for the development and evaluation of complex health interventions. We investigated the dimensions of complexity described in health research and how these descriptions may affect the adopted research methodology (e.g., the choice of designs and methods). We used a mixed method approach to review the scientific literature evaluating complex interventions in the health field. Of 438 articles identified, 179 were subjected to descriptive analysis and 48 to content analysis. The three principal dimensions of complexity were: stakeholder characteristics, intervention multimodality and context. Recognition of such dimensions influenced the methodological choices made during evaluation of the interventions with their use of designs and methods, which aimed to address the complexity. We analysed not only how researchers view complexity but also the effects of such views on researcher practices. Our results highlight the need for clarification of what complexity means and to consider complexity when deciding how to evaluate research interventions.

Health Inequality as a Large-Scale Outcome of Complex Social Systems: Lessons for Action on the Sustainable Development Goals

Action on the Sustainable Development Goals (SDGs) needs to become real and impactful, taking a "whole systems" perspective on levers for systems change. This article reviews what we have learned over the past century about the large-scale outcome of health inequality, and what we know about the behaviour of complex social systems. This combined knowledge provides lessons on the nature of inequality and what effective action on our big goals, like the SDGs, might look like. It argues that economic theories and positivist social theories which have dominated the last 150 years have largely excluded the nature of human connections to each other, and the environment. This exclusion of intimacy has legitimatised arguments that only value-free economic processes matter for macro human systems, and only abstract measurement constitutes valuable social science. Theories of complex systems provide an alternative perspective. One where health inequality is viewed as emergent, and causes are systemic and compounding. Action therefore needs to be intensely local, with power relationships key to transformation. This requires conscious and difficult intervention on the intolerable accumulation of resources; improved reciprocity between social groups; and reversal of system flows, which at present ebb away from the local and those already disadvantaged.

Prevention and control of cholera with household and community water, sanitation and hygiene (WASH) interventions: A scoping review of current international guidelines

INTRODUCTION

Cholera remains a frequent cause of outbreaks globally, particularly in areas with inadequate water, sanitation and hygiene (WASH) services. Cholera is spread through faecal-oral routes, and studies demonstrate that ingestion of Vibrio cholerae occurs from consuming contaminated food and water, contact with cholera cases and transmission from contaminated environmental point sources. WASH guidelines recommending interventions for the prevention and control of cholera are numerous and vary considerably in their recommendations. To date, there has been no review of practice guidelines used in cholera prevention and control programmes.

METHODS

We systematically searched international agency websites to identify WASH intervention guidelines used in cholera programmes in endemic and epidemic settings. Recommendations listed in the guidelines were extracted, categorised and analysed. Analysis was based on consistency, concordance and recommendations were classified on the basis of whether the interventions targeted within-household or community-level transmission.

RESULTS

Eight international guidelines were included in this review: three by non-governmental organisations (NGOs), one from a non-profit organisation (NPO), three from multilateral organisations and one from a research institution. There were 95 distinct recommendations identified, and concordance among guidelines was poor to fair. All categories of WASH interventions were featured in the guidelines. The majority of recommendations targeted community-level transmission (45%), 35% targeted within-household transmission and 20% both.

CONCLUSIONS

Recent evidence suggests that interventions for effective cholera control and response to epidemics should focus on case-centred approaches and within-household transmission. Guidelines did consistently propose interventions targeting transmission within households. However, the majority of recommendations listed in guidelines targeted community-level transmission and tended to be more focused on preventing contamination of the environment by cases or recurrent outbreaks, and the level of service required to interrupt community-level transmission was often not specified. The guidelines in current use were varied and interpretation may be difficult when conflicting recommendations are provided. Future editions of guidelines should reflect on the inclusion of evidence-based approaches, cholera transmission models and resource-efficient strategies.

Adapting evidence-informed complex population health interventions for new contexts: a systematic review of guidance

BACKGROUND

Adapting interventions that have worked elsewhere can save resources associated with developing new interventions for each specific context. While a developing body of evidence shows benefits of adapted interventions compared with interventions transported without adaptation, there are also examples of interventions which have been extensively adapted, yet have not worked in the new context. Decisions on when, to what extent, and how to adapt interventions therefore are not straightforward, particularly when conceptualising intervention effects as contingent upon contextual interactions in complex systems. No guidance currently addresses these questions comprehensively. To inform development of an overarching guidance on adaptation of complex population health interventions, this systematic review synthesises the content of the existing guidance papers.

METHODS

We searched for papers published between January 2000 and October 2018 in 7 bibliographic databases. We used citation tracking and contacted authors and experts to locate further papers. We double screened all the identified records. We extracted data into the following categories: descriptive information, key concepts and definitions, rationale for adaptation, aspects of adaptation, process of adaptation, evaluating and reporting adapted interventions. Data extraction was conducted independently by two reviewers, and retrieved data were synthesised thematically within pre-specified and emergent categories.

RESULTS

We retrieved 6694 unique records. Thirty-eight papers were included in the review representing 35 sources of guidance. Most papers were developed in the USA in the context of implementing evidence-informed interventions among different population groups within the country, such as minority populations. We found much agreement on how the papers defined key concepts, aims, and procedures of adaptation, including involvement of key stakeholders, but also identified gaps in scope, conceptualisation, and operationalisation in several categories.

CONCLUSIONS

Our review found limitations that should be addressed in future guidance on adaptation. Specifically, future guidance needs to be reflective of adaptations in the context of transferring interventions across countries, including macro- (e.g. national-) level interventions, better theorise the role of intervention mechanisms and contextual interactions in the replicability of effects and accordingly conceptualise key concepts, such as fidelity to intervention functions, and finally, suggest evidence-informed strategies for adaptation re-evaluation and reporting.

TRIAL REGISTRATION

PROSPERO 2018, CRD42018112714.

Characterising innovations in maternal and newborn health based on a common theory of change: lessons from developing and applying a characterisation framework in Nigeria, Ethiopia and India

Government leadership is key to enhancing maternal and newborn survival. In low/middle-income countries, donor support is extensive and multiple actors add complexity. For policymakers and others interested in harmonising diverse maternal and newborn health efforts, a coherent description of project components and their intended outcomes, based on a common theory of change, can be a valuable tool. We outline an approach to developing such a tool to describe the work and the intended effect of a portfolio of nine large-scale maternal and newborn health projects in north-east Nigeria, Ethiopia and Uttar Pradesh in India. Teams from these projects developed a framework, the 'characterisation framework', based on a common theory of change. They used this framework to describe their innovations and their intended outcomes. Individual project characterisations were then collated in each geography, to identify what innovations were implemented where, when and at what scale, as well as the expected health benefit of the joint efforts of all projects. Our study had some limitations. It would have been enhanced by a more detailed description and analysis of context and, by framing our work in terms of discrete innovations, we may have missed some synergistic aspects of the combination of those innovations. Our approach can be valuable for building a programme according to a commonly agreed theory of change, as well as for researchers examining the effectiveness of the combined work of a range of actors. The exercise enables policymakers and funders, both within and between countries, to enhance coordination of efforts and to inform decision-making about what to fund, when and where.

Clarifying differences between reviews within evidence ecosystems

This paper builds on a 2012 paper by the same authors which argued that the types and brands of systematic review do not sufficiently differentiate between the many dimensions of different review questions and review methods (Gough et al., Syst Rev 1:28, 2012). The current paper extends this argument by considering the dynamic contexts, or 'evidence ecosystems', within which reviews are undertaken; the fact that these ecosystems are constantly changing; and the relevance of this broader context for understanding 'dimensions of difference' in the unfolding development and refinement of review methods.The concept of an evidence ecosystem is used to consider particular issues within the three key dimensions of difference outlined in the 2012 paper of (1) review aims and approach, (2) structure and components of reviews, and (3) breadth, depth, and 'work done' by reviews.

Obstacles and opportunities to using research evidence in local public health decision-making in England

BACKGROUND

Local public health service delivery and policy-setting in England was overhauled in 2013, with local government now responsible for the complex tasks involved in protecting and improving population health and addressing health inequalities. Since 2013, public health funding per person has declined, adding to the challenge of public health decision-making. In a climate of austerity, research evidence could help to guide the more effective use of resources, although there are concerns that the reorganisation of public health decision-making structures has disrupted traditional evidence use patterns. This study aimed to explore local public health evidence use and needs in this new decision-making climate.

METHODS

Semi-structured interviews with Public Health Practitioners across three Local Authorities were conducted, with sites purposefully selected to represent urban, suburban and county Local Authorities, and to reflect a range of public health issues that might be encountered. A topic guide was developed that allowed participants to reflect on their experience and involvement in providing evidence for, or making a decision around, commissioning a public health service. Data were transcribed and template analysis was employed to understand the findings, which involved developing a coding template based on an initial transcript and applying this to subsequent transcripts.

RESULTS

Increased political involvement in local public health decision-making, while welcomed by some participants as a form of democratising public health, has influenced evidence preferences in a number of ways. Political and individual ideologies of locally elected officials meant that certain forms of evidence could be overlooked in favour of evidence that corresponded to decision-makers' preferences. Political involvement at the local level has increased the appetite for local knowledge and evidence. Research evidence needs to demonstrate its local salience if it is to contribute to decision-making alongside competing sources, particularly anecdotal information.

CONCLUSION

To better meet decision-making needs of politicians and practitioners, a shift in the scope of public health evidence is required. At a systematic review level, this could involve moving away from producing evidence that reflects broad global generalisations about narrow and simple questions, and instead towards producing forms of evidence that have local applicability and can support complex policy-focussed decisions.

Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health

Public health interventions and health technologies are commonly described as 'complex', as they involve multiple interacting components and outcomes, and their effects are largely influenced by contextual interactions and system-level processes. Systematic reviewers and guideline developers evaluating the effects of these complex interventions and technologies report difficulties in using existing methods and frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). As part of a special series of papers on implications of complexity in the WHO guideline development, this paper serves as a primer on how to consider sources of complexity when using the GRADE approach to rate certainty of evidence. Relevant sources of complexity in systematic reviews, health technology assessments and guidelines of public health are outlined and mapped onto the reported difficulties in rating the estimates of the effect of these interventions. Recommendations on how to address these difficulties are further outlined, and the need for an integrated use of GRADE from the beginning of the review or guideline development is emphasised. The content of this paper is informed by the existing GRADE guidance, an ongoing research project on considering sources of complexity when applying the GRADE approach to rate certainty of evidence in systematic reviews and the review authors' own experiences with using GRADE.

The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective

INTRODUCTION

Evidence-to-decision (EtD) frameworks intend to ensure that all criteria of relevance to a health decision are systematically considered. This paper, part of a series commissioned by the WHO, reports on the development of an EtD framework that is rooted in WHO norms and values, reflective of the changing global health landscape, and suitable for a range of interventions and complexity features. We also sought to assess the value of this framework to decision-makers at global and national levels, and to facilitate uptake through suggestions on how to prioritise criteria and methods to collect evidence.

METHODS

In an iterative, principles-based approach, we developed the framework structure from WHO norms and values. Preliminary criteria were derived from key documents and supplemented with comprehensive subcriteria obtained through an overview of systematic reviews of criteria employed in health decision-making. We assessed to what extent the framework can accommodate features of complexity, and conducted key informant interviews among WHO guideline developers. Suggestions on methods were drawn from the literature and expert consultation.

RESULTS

The new WHO-INTEGRATE (INTEGRATe Evidence) framework comprises six substantive criteria-balance of health benefits and harms, human rights and sociocultural acceptability, health equity, equality and non-discrimination, societal implications, financial and economic considerations, and feasibility and health system considerations-and the meta-criterion quality of evidence. It is intended to facilitate a structured process of reflection and discussion in a problem-specific and context-specific manner from the start of a guideline development or other health decision-making process. For each criterion, the framework offers a definition, subcriteria and example questions; it also suggests relevant primary research and evidence synthesis methods and approaches to assessing quality of evidence.

CONCLUSION

The framework is deliberately labelled version 1.0. We expect further modifications based on focus group discussions in four countries, example applications and input across concerned disciplines.

Synthesising quantitative evidence in systematic reviews of complex health interventions

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods

This paper is one of a series exploring the implications of complexity for systematic reviews and guideline development, commissioned by the WHO. The paper specifically explores the role of qualitative evidence synthesis. Qualitative evidence synthesis is the broad term for the group of methods used to undertake systematic reviews of qualitative research evidence. As an approach, qualitative evidence synthesis is increasingly recognised as having a key role to play in addressing questions relating to intervention or system complexity, and guideline development processes. This is due to the unique role qualitative research can play in establishing the relative importance of outcomes, the acceptability, fidelity and reach of interventions, their feasibility in different settings and potential consequences on equity across populations. This paper outlines the purpose of qualitative evidence synthesis, provides detail of how qualitative evidence syntheses can help establish understanding and explanation of the complexity that can occur in relation to both interventions and systems, and how qualitative evidence syntheses can contribute to evidence to decision frameworks. It provides guidance for the choice of qualitative evidence synthesis methods in the context of guideline development for complex interventions, giving 'real life' examples of where this has occurred. Information to support decision-making around choice qualitative evidence synthesis methods in the context of guideline development is provided. Approaches for reporting qualitative evidence syntheses are discussed alongside mechanisms for assessing confidence in the findings of a review.

Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods

Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

Formulating questions to explore complex interventions within qualitative evidence synthesis

When making decisions about complex interventions, guideline development groups need to factor in the sociocultural acceptability of an intervention, as well as contextual factors that impact on the feasibility of that intervention. Qualitative evidence synthesis offers one method of exploring these issues. This paper considers the extent to which current methods of question formulation are meeting this challenge. It builds on a rapid review of 38 different frameworks for formulating questions. To be useful, a question framework should recognise context (as setting, environment or context); acknowledge the criticality of different stakeholder perspectives (differentiated from the target population); accommodate elements of time/timing and place; be sensitive to qualitative data (eg, eliciting themes or findings). None of the identified frameworks satisfied all four of these criteria. An innovative question framework, PerSPEcTiF, is proposed and retrospectively applied to a published WHO guideline for a complex intervention. Further testing and evaluation of the PerSPEcTiF framework is required.