Anticipating & assessing adverse consequences of public health interventions - CONSEQUENT framework

Introduction

Despite the best intentions, public health (PH) interventions can have adverse and other unintended consequences (AUCs). AUCs may arise in novel PH interventions, as well as from known and tested PH interventions implemented in a new context. Despite their importance, this topic has been largely overlooked. Therefore, we used a structured value-guided as well as evidence-based approach, to develop a framework to support researchers, practitioners, and policy-makers in anticipating and assessing AUCs of PH interventions.

Methods

We employed the ‘best-fit’ synthesis approach starting with an a priori framework and iteratively revising this based on systematically identified evidence. The a priori framework was derived from both the WHO-INTEGRATE framework and the Behaviour Change Wheel, to root the framework in global health norms and values, established mechanisms of PH interventions, and a complexity perspective. The a priori framework was advanced based on theoretical and conceptual publications and systematic reviews on the topic of AUCs in PH. Thematic analysis was used to revise the framework and identify new themes. To validate the framework, it was coded against four selected systematic reviews of AUCs of PH interventions.

Results

The CONSEQUENT framework includes two components: the first focuses on AUCs and serves to categorise them; the second component highlights the mechanisms through which AUCs may arise. The first component comprises eight domains of consequences - health-related, health system, human and fundamental rights, acceptability- and adherence-related, equality- and equity-related, social and institutional, economic and resource-related, and environmental.

Conclusions

Both over- and underestimation of AUCs of PH intervention poses risks. The CONSEQUENT framework may facilitate classification and conceptualization of AUCs of PH interventions during their development or evaluation to support evidence-informed decision-making.

Key messages

Public Health Guideline to prevent and control SARS-CoV-2 in schools: development and evaluation

Issue/problem

In times of high demand for scientific evidence for decision-making on COVID-19 mitigation measures, guidelines can be useful for translating scientific evidence into policy and practice. While guidelines are widely used in medical decision-making, they are novel to public health in Germany.

Description of the problem

Since December 2020, a guideline group has been working on a living, evidence- and consensus-based public health guideline on preventing and controlling SARS-CoV-2 transmission in schools. The group includes scientists across multiple disciplines as well as a broad range of stakeholders, including from the school family. Key features in the development of recommendations included a Cochrane rapid review and the WHO-INTEGRATE evidence-to-decision framework. The development and usefulness of the guideline for decision-making are being evaluated using a multi-method approach.

Results

The first version of the guideline containing nine recommendations was published in February 2021. The WHO-INTEGRATE framework facilitated the consideration of factors such as health benefits and harms, feasibility, acceptability and financial constraints. Preliminary findings of the evaluation suggest that under time pressure, developing few essential, consensus-based recommendations while assessing their societal implications is warranted. A shared understanding of evidence and of the purpose and limitations of guidelines is critical. To remain relevant, continuous integration of new evidence and updating of the guideline is necessary.

Key messages

Implementation of integrated knowledge translation in NCD research: Examining intervention components

Background

Integrated knowledge translation (IKT) has been a cornerstone of the Collaboration for Evidence-based Healthcare and Public Health in Africa (CEBHA+) with partners in Ethiopia, Germany, Malawi, Rwanda, South Africa, and Uganda. The consortium conducts research on preventing and treating non-communicable diseases as well as road traffic injuries. IKT is understood as the continuous engagement of decision-makers throughout the research process in order to build equitable, mutually beneficial partnerships to conduct policy-relevant research and, ultimately, strengthen evidence-informed decision-making (EIDM). Gradually, a structured “CEBHA+ IKT approach” was developed, including systematic stakeholder mapping and analysis, and the development of local IKT strategies.

Methods

We conducted a mixed-methods process and outcome evaluation of this IKT intervention. This comprised structured interviews, an online survey, and document analyses at two time points, two and four years after IKT initiation.

Results

Preliminary results show that partnerships with decision-makers were successfully established or strengthened. While continuous engagement was implemented, fidelity to formalised IKT strategies was variable. The development, monitoring and updating of the IKT strategies, originally conceptualised as an essential intervention component, has been helpful for some CEBHA+ researchers and may facilitate implementation. However, the vision for decision-maker engagement as well as emphasis on continuous engagement (defined as a deliverable) emerged as more important intervention drivers and may be conceptualised as intervention components.

Conclusions

A strong vision and continuous engagement with decision-makers are critical for strengthening EIDM. Formalised IKT strategies proved to be of moderate importance in current CEBHA+ research activities, but may turn out to be an essential intervention component if implemented from the start of a research project.

Key messages

• Continuous engagement with decision-makers has successfully strengthened or established partnerships between researchers and decision-makers.

• Formal IKT strategies to plan stakeholder engagement were developed but continuous stakeholder engagement and a strong vision proved to be of greater importance.

Lessons learnt: Undertaking rapid reviews on public health and social measures during a global pandemic

Public health and social measures (PHSM) have been central to the COVID‐19 response. Consequently, there has been much pressure on decision‐makers to make evidence‐informed decisions and on researchers to synthesize the evidence regarding these measures. This article describes our experiences, responses and lessons learnt regarding key challenges when planning and conducting rapid reviews of PHSM during the COVID‐19 pandemic. Stakeholder consultations and scoping reviews to obtain an overview of the evidence inform the scope of reviews that are policy‐relevant and feasible. Multiple complementary reviews serve to examine the benefits and harms of PHSM across different populations and contexts. Conceiving reviews of effectiveness as adaptable living reviews helps to respond to evolving evidence needs and an expanding evidence base. An appropriately skilled review team and good planning, coordination and communication ensures smooth and rigorous processes and efficient use of resources. Scientific rigor, the practical implications of PHSM‐related complexity and likely time savings should be carefully weighed in deciding on methodological shortcuts. Making the best possible use of modeling studies represents a particular challenge, and methods should be carefully chosen, piloted and implemented. Our experience raises questions regarding the nature of rapid reviews and regarding how different types of evidence should be considered in making decisions about PHSM during a global pandemic. We highlight the need for readily available protocols for conducting studies on the effectiveness, unintended consequences and implementation of PHSM in a timely manner, as well as the need for rapid review standards tailored to “rapid” versus “emergency” mode reviewing.

Evaluating food-related policies in Germany using the Food Environment Policy Index (Food-EPI)

Background

Germany has seen as rising burden of diet-related adverse health outcomes. Policies for improving food environments and related infrastructure are essential to reduce this burden. A growing awareness of the interlinkages between sustainability and nutrition underlines the need for healthy, sustainable food environments. Therefore, we applied the Food Environment Policy Index (Food-EPI) with an additional domain on sustainability to monitor German food policies and to formulate actions for improvement.

Methods

For the evidence analysis information on food environment and infrastructure was collected from various sources. Results were validated with an expert group consisting of representatives of government agencies, non-governmental organizations and academia. The level of implementation, compared to international best practice examples, was ranked by the expert group through an online survey. A one-day workshop was conducted to identify actions to fill the gaps identified in the online ranking and to prioritize these actions in terms of importance and achievability.

Results

The application of the Food-EPI to German policies and interventions shows a nuanced picture of strengths and weaknesses of the German food environment and infrastructure. Some domains such as monitoring and surveillance are in line with international best practice, however other domains show considerable room for improvement. The recommended actions to improve the shortcomings of the German food environment includes a list with priority actions.

Conclusions

The results from the Food-EPI provides an in-depth analysis of German food environment policies and infrastructure, as well as a list of prioritized actions to improve it.

Key message

Integrating a sustainability domain into the Food-EPI offers opportunities to identify priority actions for the creation of healthy, sustainable food systems.

Collaborative strategies for knowledge translation: the African-German CEBHA+ research network

Background

The Collaboration for Evidence-based Healthcare and Public Health in Africa (CEBHA+) is an NCD research consortium that seeks to engage policy-makers and practitioners throughout the research process in order to build lasting relationships, enhance evidence uptake and build long-term capacity among partner institutions in Ethiopia, Malawi, Rwanda, South Africa and Uganda. This integrated knowledge translation (IKT) approach includes the formal development and implementation of country-specific engagement strategies.

Methods

An early-stage evaluation is taking place in Mid-2020. Online surveys and qualitative interviews with researchers and policy-and-practice partners will inform adaptation of country-specific strategies, advance the initial programme theory and contribute to the science of IKT.

Results

We present three pertinent observations based on the development and implementation of an overarching CEBHA+ IKT approach and five country-specific strategies over the last two years: Despite being informed by an overarching IKT programme theory, the site-specific strategies and resulting partnerships vary markedly, representing the whole continuum of integrated knowledge translation.The diversity of approaches is due to different understandings of IKT, discontinuity of staff, lack of IKT training, and perceptions of usefulness (compared to ongoing research activities) among CEBHA+ researchers.The individual, dynamic and often pre-existing relationships of researchers and partners from policy and practice are central to IKT, but capturing these within the programme theory and monitoring them remains challenging.

Conclusions

These observations are useful to guide further evaluation and cross-country comparison. Close examination of relationships and conceptualisation of IKT as a continuum may provide valuable insights into the circumstances that make IKT efforts worthwhile.

Key messages

Translating evidence into policy and practice is reliant on partnerships between researchers and policy-and-practice partners. These can be formalised but the relationships remain complex and dynamic.

GH:ADT Project: developing a framework of Global Health concepts and a learning objectives catalogue

Background

Global Health (GH) is receiving increasing attention, in Germany, Europe and internationally. Despite an increased demand, teaching on GH remains fragmented and heterogeneous in the university landscape in German-speaking countries. Moreover, there is a lack of common understanding of GH and interpretations range from GH as tropical medicine to GH as everything related to health. This project aims to develop (A) a framework of different GH concepts, as a basis for (B) a set of GH learning objectives for German-speaking countries.

Methods

A review of publications providing GH definitions was carried out. Following a concept analysis approach, we developed a general framework of different and partially overlapping GH understandings. Building on these results we conducted an overview of reviews on learning objectives and curricula for GH to develop a set of learning objectives.

Results

The general framework distinguishes between GH concepts describing GH as (1)a set of global health issues (e.g. COVID-19), (2) a global ideal to strive for, (3 + 4) any activities relating to said global health issues or ideals (e.g. all activities contributing to health equity), and (5) sets of activities meeting various characteristics (e.g. being transdisciplinary and transnational). Within each concept, multiple sets of axes allow a more granular distinction of the concepts (e.g. regarding the relation to public health and tropical medicine). The list of key learning objectives shows considerable overlap with widely accepted public health core competencies. Differences include e.g. a (stronger) emphasis on intercultural and interdisciplinary collaboration, globalisation and transnational determinants of health.

Conclusions

A systematic reflection on commonalities and differences in the understanding of GH is important for a shared understanding. Based on the GH learning objectives we will develop core competencies for GH, including through an online survey of GH experts.

Key messages

The GH:ADT framework of global health concepts supports a systematic reflection on and recognition of the multi-dimensional nature of Global Health.

The learning objectives identified in the GH:ADT project can be a valuable tool for curriculum development.

More harm than good? Building a framework to identify adverse effects of public health interventions

Background

While the importance of adverse events of medical interventions is widely recognized, adverse effects of public health interventions remain a neglected topic. This project aims to develop a framework to guide researchers and decision-makers to systematically reflect on and identify potential adverse effects of public health interventions.

Methods

We conducted a mixed-method systematic review of theoretical and conceptual publications on adverse events of public health interventions to develop a preliminary framework employing best-fit framework synthesis. We used the WHO-INTEGRATE framework as a starting point for the synthesis, a multidimensional evidence-to-decision framework developed for complex interventions in complex systems.

Results

The framework includes two interlinked parts: The first maps domains in which potential adverse events might arise. Drawing on the WHO-INTEGRATE framework, these domains include aspects related to health, but also domains related to societal, economic, and environmental implications. The second part maps general mechanisms through which public health interventions can lead to adverse effects (e.g. reactive behaviour change, increase of labelling and stigmatization, and exposure to environmental risk-factors).

Conclusions

The framework will be advanced in the second phase of the project through empirical studies of harmful effects in public health interventions, which we will identify through an overview of systematic reviews. Adverse effects of public health interventions are currently not sufficiently considered in research and practice. Taking them into account is essential for informed decision-making and establishing appropriate countermeasures. Our framework could be a valuable asset for researchers and policy makers in developing, implementing and evaluating public health interventions.

Key messages

Awareness of the adverse effects of public health interventions is essential for informed decision-making and establishing countermeasures. This framework supports researchers and decision-makers in systematically reflecting on and identifying adverse events when developing, piloting, implementing or evaluating public health interventions.

Development of the WHO-INTEGRATE evidence-to-decision framework: an overview of systematic reviews of decision criteria for health decision-making

Background

Decision-making in public health and health policy is complex and requires careful deliberation of many and sometimes conflicting normative and technical criteria. Several approaches and tools, such as multi-criteria decision analysis, health technology assessments and evidence-to-decision (EtD) frameworks, have been proposed to guide decision-makers in selecting the criteria most relevant and appropriate for a transparent decision-making process. This study forms part of the development of the WHO-INTEGRATE EtD framework, a framework rooted in global health norms and values as reflected in key documents of the World Health Organization and the United Nations system. The objective of this study was to provide a comprehensive overview of criteria used in or proposed for real-world decision-making processes, including guideline development, health technology assessment, resource allocation and others.

Methods

We conducted an overview of systematic reviews through a combination of systematic literature searches and extensive reference searches. Systematic reviews reporting criteria used for real-world health decision-making by governmental or non-governmental organization on a supranational, national, or programme level were included and their quality assessed through a bespoke critical appraisal tool. The criteria reported in the reviews were extracted, de-duplicated and sorted into first-level (i.e. criteria), second-level (i.e. sub-criteria) and third-level (i.e. decision aspects) categories. First-level categories were developed a priori using a normative approach; second- and third-level categories were developed inductively.

Results

We included 36 systematic reviews providing criteria, of which one met all and another eleven met at least five of the items of our critical appraisal tool. The criteria were subsumed into 8 criteria, 45 sub-criteria and 200 decision aspects. The first-level of the category system comprised the following seven substantive criteria: “Health-related balance of benefits and harms”; “Human and individual rights”; “Acceptability considerations”; “Societal considerations”; “Considerations of equity, equality and fairness”; “Cost and financial considerations”; and “Feasibility and health system considerations”. In addition, we identified an eight criterion “Evidence”.

Conclusion

This overview of systematic reviews provides a comprehensive overview of criteria used or suggested for real-world health decision-making. It also discusses key challenges in the selection of the most appropriate criteria and in seeking to implement a fair decision-making process.

Interventions to reduce ambient air pollution and their effects on health: An abridged Cochrane systematic review

BACKGROUND

A broad range of interventions have been implemented to improve ambient air quality, and many of these have been evaluated. Yet to date no systematic review has been conducted to identify and synthesize these studies. In this systematic review, we assess the effectiveness of interventions in reducing ambient particulate matter air pollution and improving adverse health outcomes.

METHODS

We searched a range of electronic databases across multiple disciplines, as well as grey literature databases, trial registries, reference lists of included studies and the contents of relevant journals, through August 2016. Eligible for inclusion were randomized and cluster randomized controlled trials, as well as several non-randomized study designs often used for evaluating air quality interventions. We included studies that evaluated interventions targeting industrial, residential, vehicular and multiple sources, with respect to their effect on mortality, morbidity and the concentrations of particulate matter (PM - including PM₁₀, PM_2.5, coarse particulate matter and combustion-related PM), as well as several criteria pollutants, including ozone, carbon monoxide, nitrogen oxides, nitrogen dioxide, nitric oxide and sulphur dioxide. We did not restrict studies based on the population, setting or comparison. Two authors independently assessed studies for inclusion, extracted data and assessed risk of bias. We assessed risk of bias using the Graphic Appraisal Tool for Epidemiological studies (GATE) for correlation studies, as modified and employed by the UK National Institute for Health and Care Excellence. We synthesized evidence narratively, as well as graphically using harvest plots. We assessed the certainty of evidence using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system.

RESULTS

We included 42 studies assessing 38 unique interventions. These comprised a heterogeneous mix of interventions, including those aiming to address industrial sources (n = 5; e.g. the closure of a factory), residential sources (n = 7; e.g. coal ban), vehicular sources (n = 22; e.g. low emission zones), and multiple sources (n = 4; e.g. tailored measures that target both local traffic and industrial polluters). Evidence for effectiveness was mixed. Most included studies observed either no significant association or an association favoring the intervention, with little evidence that the assessed interventions might be harmful.

CONCLUSIONS

Given the heterogeneity across interventions, outcomes, and methods, it was difficult to derive overall conclusions regarding the effectiveness of interventions in terms of improved air quality or health. Some evidence suggests that interventions are associated with improvements in air quality and human health, with very little evidence suggesting interventions were harmful. The evidence base highlights the challenges related to establishing the effectiveness of specific air pollution interventions on outcomes. It also points to the need for improved study design and analysis methods, as well as more uniform evaluations. The prospective planning of evaluations and an evaluation component built into the design and implementation of interventions may also be particularly beneficial.

'Try to make good hay in the shade - it won't work!' A qualitative interview study on the perspectives of Bavarian farmers regarding primary prevention of skin cancer

BACKGROUND

Extensive exposure to solar ultraviolet radiation (UVR) is the main risk factor for keratinocyte carcinoma (KC), making outdoor workers, including farmers, a high-risk population for KC. The use of sun protection is crucial for KC prevention but is not typically implemented by outdoor workers during their daily tasks.

OBJECTIVES

To explore the attitudes of Bavarian farmers regarding sun-protective measures in their daily work and to understand perceived barriers and unmet needs.

METHODS

Farmers were recruited through the Bavarian Farmers Association in Bavaria, Southern Germany. Qualitative semi structured interviews were conducted with participants between December 2017 and March 2018. Interviews were recorded, transcribed verbatim and analysed using qualitative content analysis.

RESULTS

Twenty farmers (11 women, nine men; nine aged 18-30 years, 11 aged > 60 years) participated. Knowledge and awareness of UVR exposure and KC, perceived individual barriers to implementing sun-protective measures, individual experiences and farm life-specific circumstances emerged as key areas influencing the perspectives of farmers regarding the primary prevention of KC. Female farmers tended to take a more positive stance on sun protection, whereas male farmers showed a lower overall interest.

CONCLUSIONS

Knowledge and awareness of KC and UVR exposure is very limited in Bavarian farmers with serious perceived barriers due to the demands of daily agricultural work. Further qualitative studies are needed to identify intervention options that can increase skin cancer awareness and that can successfully overcome real barriers to implementing sun protection.

Teachers' competence, school policy and social context-HIV prevention needs of primary schools in Kagera, Tanzania

Sub-Saharan Africa carries a high burden of the HIV epidemic, with young adults being particularly affected. Well-designed school-based HIV prevention interventions can contribute to establishing protective behaviour. The aim of this study was to explore the needs of primary school students and teachers in the region of Kagera, Tanzania, regarding an HIV prevention intervention in the primary school setting. The needs assessment was structured according to the PRECEDE component of the PRECEDE-PROCEED model. Qualitative data was collected in six focus group discussions with primary school students and teachers. Key informant interviews with seven experts were conducted. We employed qualitative content analysis to analyse data in MAXQDA. The findings suggest that teachers need to be adequately trained to provide HIV and sex education and to support HIV-positive students. Enabling structural factors, such as an appropriate syllabus, are required. Stigmatization has been reported a major barrier to HIV prevention in schools. Teachers and students identified a more trustful relationship between each other as well as to legal guardians of children as a basis for HIV prevention. These findings will inform the development of a tailored HIV prevention intervention.

Exploring the relationship between sanitation and mental and social well-being: A systematic review and qualitative synthesis

The WHO defines health not as the absence of disease but as a "state of complete physical, mental, and social well-being." To date, public health research on sanitation has focused mainly on the impact of sanitation on infectious diseases and related sequelae, such as diarrhea and malnutrition. This review focuses on the mental and social well-being implications of sanitation. We systematically searched leading databases to identify eligible studies. Qualitative studies were assessed using a 17-point checklist adapted from existing tools, while quantitative studies were assessed using the Liverpool Quality Appraisal Tool. We followed a best-fit framework synthesis approach using six a priori well-being dimensions (privacy, shame, anxiety, fear, assault, and safety), which were examined using line-by-line coding. Two additional dimensions (dignity and embarrassment) inductively emerged during coding for a total of eight well-being outcomes. We then synthesized coded text for each dimension into descriptive themes using thematic analysis. For quantitative studies, we extracted any measures of association between sanitation and well-being. We identified 50 eligible studies covering a variety of populations and sanitation contexts but many studies were conducted in India (N = 14) and many examined the sanitation experience for women and girls (N = 19). Our synthesis results in a preliminary conceptual model in which privacy and safety, including assault, are root well-being dimensions. When people perceive or experience a lack of privacy or safety during open defecation or when using sanitation infrastructure, this can negatively influence their mental and social well-being. We found that perceptions and experiences of privacy and safety are influenced by contextual and individual factors, such as location of sanitation facilities and user's gender identity, respectively. Privacy and safety require thorough examination when developing sanitation interventions and policy to ensure a positive influence on the user's mental and social well-being.

Looking beyond the forest: Using harvest plots, gap analysis, and expert consultations to assess effectiveness, engage stakeholders, and inform policy

We describe a combination of methods for assessing the effectiveness of complex interventions, especially where substantial heterogeneity with regard to the population, intervention, comparison, outcomes, and study design of interest is expected. We applied these methods in a recent systematic review of the effectiveness of reinforced home-based palliative care (rHBPC) interventions, which included home-based care with an additional and explicit component of lay caregiver support. We first summarized the identified evidence, deemed inappropriate for statistical pooling, graphically by creating harvest plots. Although very useful as a tool for summary and presentation of overall effectiveness, such graphical summary approaches may obscure relevant differences between studies. Thus, we then used a gap analysis and conducted expert consultations to look beyond the aggregate level at how the identified evidence of effectiveness may be explained. The goal of these supplemental methods was to step outside of the conventional systematic review and explore this heterogeneity from a broader perspective, based on the experience of palliative care researchers and practitioners. The gap analysis and expert consultations provided valuable input into possible underlying explanations in the evidence, which could be helpful in the further adaptation and testing of existing rHBPC interventions or the development and evaluation of new ones. We feel that such a combination of methods could prove accessible, understandable, and useful in informing decisions and could thus help increase the relevance of systematic reviews to the decision-making process.

Solid fuel use and cooking practices as a major risk factor for ALRI mortality among African children

BACKGROUND

Almost half of global child deaths due to acute lower respiratory infections (ALRIs) occur in sub-Saharan Africa, where three-quarters of the population cook with solid fuels. This study aims to quantify the impact of fuel type and cooking practices on childhood ALRI mortality in Africa, and to explore implications for public health interventions.

METHODS

Early-release World Health Survey data for the year 2003 were pooled for 16 African countries. Among 32,620 children born during the last 10 years, 1455 (4.46%) were reported to have died prior to their fifth birthday. Survival analysis was used to examine the impact of different cooking-related parameters on ALRI mortality, defined as cough accompanied by rapid breathing or chest indrawing based on maternal recall of symptoms prior to death.

RESULTS

Solid fuel use increases the risk of ALRI mortality with an adjusted hazard ratio of 2.35 (95% CI 1.22 to 4.52); this association grows stronger with increasing outcome specificity. Differences between households burning solid fuels on a well-ventilated stove and households relying on cleaner fuels are limited. In contrast, cooking with solid fuels in the absence of a chimney or hood is associated with an adjusted hazard ratio of 2.68 (1.38 to 5.23). Outdoor cooking is less harmful than indoor cooking but, overall, stove ventilation emerges as a more significant determinant of ALRI mortality.

CONCLUSIONS

This study shows substantial differences in ALRI mortality risk among African children in relation to cooking practices, and suggests that stove ventilation may be an important means of reducing indoor air pollution.