Assessing the impact of humanitarian assistance in the health sector

How is the implementation of empirical research results documented in conflict-affected settings? Findings from a scoping review of peer-reviewed literature

Implementation science scholars argue that knowing 'what works' in public health is insufficient to change practices, without understanding 'how', 'where' and 'why' something works. In the peer reviewed literature on conflict-affected settings, challenges to produce research, make decisions informed by evidence, or deliver services are documented, but what about the understanding of 'how', 'where' and 'why' changes occur? We explored these questions through a scoping review of peer-reviewed literature based on core dimensions of the Extended Normalization Process Theory. We selected papers that provided data on how something might work (who is involved and how?), where (in what organizational arrangements or contexts?) and why (what was done?). We searched the Global Health, Medline, Embase databases. We screened 2054 abstracts and 128 full texts. We included 22 papers (of which 15 related to mental health interventions) and analysed them thematically. We had the results revised critically by co-authors experienced in operational research in conflict-affected settings. Using an implementation science lens, we found that: (a) implementing actors are often engaged after research is produced to discuss feasibility; (b) new interventions or delivery modalities need to be flexible; (c) disruptions affect how research findings can lead to sustained practices; (d) strong leadership and stable resources are crucial for frontline actors; (e) creating a safe learning space to discuss challenges is difficult; (f) feasibility in such settings needs to be balanced. Lastly, communities and frontline actors need to be engaged as early as possible in the research process. We used our findings to adapt the Extended Normalization Process Theory for operational research in settings affected by conflicts. Other theories used by researchers to document the implementation processes need to be studied further.

Healthcare governance during humanitarian responses: a survey of current practice among international humanitarian actors

BACKGROUND

Large international humanitarian actors support and directly deliver health services for millions of people in crises annually, and wield considerable power to decide which health services to provide, how and to whom, across a vast spectrum of health areas. Despite decades of reform aiming to improve accountability in the sector, public health practice among humanitarian actors is not heavily scrutinized in either the countries where they are headquartered or those where they provide healthcare. We surveyed current healthcare governance practice among large international humanitarian actors to better understand what organisations are doing to ensure oversight and accountability for health services in humanitarian responses.

METHODS

The term 'healthcare governance' was defined and categorised into seven domains: implementation of health management information systems (HMIS) and use of resulting data; professional development of health sector staff; audits of health service performance; management of clinical incidents; evidence-based practice; pharmaceutical supply; and beneficiary engagement. Senior health professionals at 32 leading international actors providing humanitarian health services were contacted between July and August 2019 to complete a 109-question online survey about their organisation's practice in these domains.

RESULTS

Respondents from 13 organisations completed the questionnaire. Healthcare governance practices were undertaken to varying degrees by all organisations but were often driven by donor requirements and external factors rather than improvement of programme performance. Common strengths were the inclusion of governance in organisational policies, high availability of technical guidelines, and close monitoring of pharmaceutical services. Recurring weaknesses were poor beneficiary engagement, inconsistent use of health information for decision making, unsystematic implementation of healthcare audits, inconsistent management of clinical incidents, and lack of training and professional development opportunities.

CONCLUSIONS

To our knowledge, this is the first study to describe healthcare governance practice among humanitarian actors. Leading international humanitarian healthcare providers are already implementing many healthcare governance activities; however, these are inconsistently applied and generally not reflective of systematic policies or earmarked organisational resources. There is a need for sector-wide consensus on how the humanitarian sector defines healthcare governance, the domains that constitute it, which actors in the humanitarian system are implicated, and how malpractice should be systematically addressed.

Building safety in humanitarian programmes that support post-disaster shelter self-recovery: an evidence review

The humanitarian sector is increasingly aware of the role that good quality evidence plays in the underpinning of effective and accountable practice. This review addresses the need for reliable evidence by evaluating current knowledge about the intersection of two key outcome targets of post-disaster shelter response: supporting shelter self-recovery and building back safer. Evidence about post-disaster shelter programmes that aim to improve hazard resistance while supporting shelter self-recovery has been systematically analysed and evaluated. Technical support, especially training in safer construction techniques, was found to be a central programme feature, but the impact of this and other programme attributes on building safety was largely not ascertainable. Programme reports and studies lack sufficient detail, especially on the hazard resistance of repaired houses. Accounts of shelter programmes need to include more reliable reporting of key activities and assessment of outcomes, in order to contribute to the growing evidence base in this field.

A Bayesian hierarchical model for mortality data from cluster-sampling household surveys in humanitarian crises

Background

The crude death rate (CDR) is one of the defining indicators of humanitarian emergencies. When data from vital registration systems are not available, it is common practice to estimate the CDR from household surveys with cluster-sampling design. However, sample sizes are often too small to compare mortality estimates to emergency thresholds, at least in a frequentist framework. Several authors have proposed Bayesian methods for health surveys in humanitarian crises. Here, we develop an approach specifically for mortality data and cluster-sampling surveys.

Methods

We describe a Bayesian hierarchical Poisson-Gamma mixture model with generic (weakly informative) priors that could be used as default in absence of any specific prior knowledge, and compare Bayesian and frequentist CDR estimates using five different mortality datasets. We provide an interpretation of the Bayesian estimates in the context of an emergency threshold and demonstrate how to interpret parameters at the cluster level and ways in which informative priors can be introduced.

Results

With the same set of weakly informative priors, Bayesian CDR estimates are equivalent to frequentist estimates, for all practical purposes. The probability that the CDR surpasses the emergency threshold can be derived directly from the posterior of the mean of the mixing distribution. All observation in the datasets contribute to the estimation of cluster-level estimates, through the hierarchical structure of the model.

Conclusions

In a context of sparse data, Bayesian mortality assessments have advantages over frequentist ones already when using only weakly informative priors. More informative priors offer a formal and transparent way of combining new data with existing data and expert knowledge and can help to improve decision-making in humanitarian crises by complementing frequentist estimates.

Setting global research priorities for child protection in humanitarian action: Results from an adapted CHNRI exercise

BACKGROUND

Armed conflict, natural disaster, and forced displacement affect millions of children each year. Such humanitarian crises increase the risk of family separation, erode existing support networks, and often result in economic loss, increasing children's vulnerability to violence, exploitation, neglect, and abuse. Research is needed to understand these risks and vulnerabilities and guide donor investment towards the most effective interventions for improving the well-being of children in humanitarian contexts.

METHODS

The Assessment, Measurement & Evidence (AME) Working Group of the Alliance for Child Protection in Humanitarian Action (ACPHA) identified experts to participate in a research priority setting exercise adapted from the Child Health and Nutrition Research Initiative (CHNRI). Experts individually identified key areas for research investment which were subsequently ranked by participants using a Likert scale. Research Priority Scores (RPS) and Average Expert Agreement (AEA) were calculated for each identified research topic, the top fifteen of which are presented within this paper.

RESULTS

Intervention research, which aims to rigorously evaluate the effectiveness of standard child protection activities in humanitarian settings, ranked highly. Child labor was a key area of sector research with two of the top ten priorities examining the practice. Respondents also prioritized research efforts to understand how best to bridge humanitarian and development efforts for child protection as well as identifying most effective way to build the capacity of local systems in order to sustain child protection gains after a crisis.

CONCLUSIONS

Rigorous, scientific research that assesses the scope of child protection risks, examines the effectiveness of interventions to improve child well-being, and translates evidence to practice is critical. Findings from this research priority setting exercise offer guidance for a global research agenda on child protection in humanitarian settings, encouraging cooperation among donors, implementers, and academics to pursue a coordinated approach to evidence generation.

Public health information in crisis-affected populations: a review of methods and their use for advocacy and action

Valid and timely information about various domains of public health underpins the effectiveness of humanitarian public health interventions in crises. However, obstacles including insecurity, insufficient resources and skills for data collection and analysis, and absence of validated methods combine to hamper the quantity and quality of public health information available to humanitarian responders. This paper, the second in a Series of four papers, reviews available methods to collect public health data pertaining to different domains of health and health services in crisis settings, including population size and composition, exposure to armed attacks, sexual and gender-based violence, food security and feeding practices, nutritional status, physical and mental health outcomes, public health service availability, coverage and effectiveness, and mortality. The paper also quantifies the availability of a minimal essential set of information in large armed conflict and natural disaster crises since 2010: we show that information was available and timely only in a small minority of cases. On the basis of this observation, we propose an agenda for methodological research and steps required to improve on the current use of available methods. This proposition includes setting up a dedicated interagency service for public health information and epidemiology in crises.

Evidence on public health interventions in humanitarian crises

Recognition of the need for evidence-based interventions to help to improve the effectiveness and efficiency of humanitarian responses has been increasing. However, little is known about the breadth and quality of evidence on health interventions in humanitarian crises. We describe the findings of a systematic review with the aim of examining the quantity and quality of evidence on public health interventions in humanitarian crises to identify key research gaps. We identified 345 studies published between 1980 and 2014 that met our inclusion criteria. The quantity of evidence varied substantially by health topic, from communicable diseases (n=131), nutrition (n=77), to non-communicable diseases (n=8), and water, sanitation, and hygiene (n=6). We observed common study design and weaknesses in the methods, which substantially reduced the ability to determine causation and attribution of the interventions. Considering the major increase in health-related humanitarian activities in the past three decades and calls for a stronger evidence base, this paper highlights the limited quantity and quality of health intervention research in humanitarian contexts and supports calls to scale up this research.

The Responsibility to Protect (R2P) in Libya and Syria

Purpose

– Against the backdrop of the Responsibility to Protect (R2P) policy – an instrument with which the UN seeks to protect vulnerable civilians from gross violations of human rights – this study examines the application of R2P in the Libyan intervention and the various efforts to replicate similar claim to intervene in Syria. While proposing that the roles of Brazil, Russia, India, China and South Africa (BRICS) is increasingly influential to the success of an intervention, this study asks the question: what are the general conditions for success of R2P application in Libya and Syria during the period 2011-2014?

Design/methodology/approach

– In its examination of the policy and scholarly works that have informed, justified and evaluated the processes and outcomes of the principles of R2P policy, this paper used relevant search terms for conditions for success of humanitarian military intervention (COSI). Specific keywords such as R2P, BRICS and humanitarian intervention are scrutinised for relevance to the research question. Documents that failed to satisfy the criteria of research quality were excluded, whereas the key problems and findings identified in each studied document were tabulated into inclusion and exclusion.

Findings

– Despite the role of BRICS in the Libyan and Syrian interventions, existing literature failed to explicitly make this connection, although much of the literature agreed on a number of general conditions for success. This paper problematise the relationship between success and BRICS role. One of the reasons for this is the emerging nature of the literature that is beginning to appreciate the plausibility that the BRICS influences the success of an intervention.

Originality/value

– This piece synthesises studies that focus on COSI with preference for works that engaged this study’s case countries. Much rich data which even until now are always in need of close examination emerged during data collection, making it useful to craft a third part for BRICS-focused literature that has informed the R2P debate.

Needs, acceptability, and value of humanitarian medical assistance in remote Peruvian Amazon riverine communities

Much debate exists regarding the need, acceptability, and value of humanitarian medical assistance. We conducted a cross-sectional study on 457 children under 5 years from four remote riverine communities in the Peruvian Amazon and collected anthropometric measures, blood samples (1-4 years), and stool samples. Focus groups and key informant interviews assessed perspectives regarding medical aid delivered by foreigners. The prevalence of stunting, anemia, and intestinal parasites was 20%, 37%, and 62%, respectively. Infection with multiple parasites, usually geohelminths, was detected in 41% of children. The prevalence of intestinal parasites both individual and polyparasitism increased with age. Participants from smaller communities less exposed to foreigners expressed lack of trust and fear of them. However, participants from all communities were positive about foreigners visiting to provide health support. Prevalent health needs such as parasitic infections and anemia may be addressed by short-term medical interventions. There is a perceived openness to and acceptability of medical assistance delivered by foreign personnel.

Funding based on needs? A study on the use of needs assessment data by a major humanitarian health assistance donor in its decisions to allocate funds

Background: International humanitarian assistance is essential for disaster-affected populations, particularly in resource scarce settings. To target such assistance, needs assessments are required. According to internationally endorsed principles, donor governments should provide funding for humanitarian assistance based on need. Aim: The aim of this study is to explore a major donor’s use of needs assessment data in decision-making for allocations of funds for health-related humanitarian assistance contributions. Setting: This is a case study of the Swedish International Development Cooperation Agency (Sida), a major and respected international donor of humanitarian assistance. Methods: To explore Sida’s use of needs assessment data in practice for needs-based allocations, we reviewed all decision documents and assessment memoranda for humanitarian assistance contributions for 2012 using content analysis; this was followed by interviews with key personnel at Sida. Results: Our document analysis found that needs assessment data was not systematically included in Sida’s assessment memoranda and decision documents. In the interviews, we observed various descriptions of the concept of needs assessments, the importance of contextual influences as well as previous collaborations with implementing humanitarian assistance organizations. Our findings indicate that policies guiding funding decisions on humanitarian assistance need to be matched with available needs assessment data and that terminologies and concepts have to be clearly defined. Conclusion: Based on the document analysis and the interviews, it is unclear how well Sida used needs assessment data for decisions to allocate funds. However, although our observations show that needs assessments are seldom used in decision making, Sida’s use of needs assessments has improved compared to a previous study. To improve project funds allocations based on needs assessment data, it will be critical to develop distinct frameworks for allocation distributions based on needs assessment and clear definitions, measurements and interpretations of needs. Key words: Needs assessment, humanitarian assistance, disasters, donor decision-making

Implementation of Evidence-based Humanitarian Programs in Military-led Missions: Part II. The Impact Assessment Model

The Department of Defense does not implement health-sector humanitarian assistance impact assessments to complement those of the international humanitarian aid community. This oversight fails to meet the recent Department of Defense Directive 3000.05 mandate calling for the application of measures of effectiveness. The decision by the Department of Defense to incorporate humanitarian assistance in stability operations should be supported by evidence-based impact assessments. This article proposes implementation of an impact assessment model in Department of Defense humanitarian assistance operations. The use of an impact assessment model will refocus previously identified information gaps from traditional military input-output management toward a greater emphasis on outcomes. This will help answer which humanitarian activities are successful, which are not, and why. Over time, the use of an impact assessment model will ensure that the Department of Defense and its operational units are learning as an organization while applying evidence-based lessons learned to future stability operations. Most important, the use of this model will both provide better understanding of the role that the Department of Defense has in humanitarian activities and help interpret and transfer these activities to the host nation and international aid community in a timely and efficient manner. (Disaster Med Public Health Preparedness. 2008;2:237–244)

Advances in Monitoring Have Not Translated into Improvements in Humanitarian Health Services

Background:

This paper is an attempt to review the advances and shortfalls in data collection and use of health data that have occurred during health emergencies in recent decades for the opening session of the Humanitarian and Health Conference at Dartmouth University in September of 2006.

Methods:

Examples of various kinds of successes and failures associated with health data collection are given to highlight advances with an effort to emphasize multi-agency efforts reviewed by outside scholars.

Results:

Health data, particularly surveillance data, have allowed relief workers to set priorities for life-saving humanitarian programs. The main guidelines widely utilized such as those of the US Centers for Disease Control and Prevention, Médecins sans Frontières, and the Sphere Project have considerable similarity due to the consistency of data collected in various crises. Moreover, difficult to see problems and successes have been revealed by coherent surveillance efforts. Yet, these data collection efforts can not show significant improvements in the quality of humanitarian aid in recent years. Moreover, health data often do not appear to have meaningful influence on the prioritizing of relief resources globally or on those political issues that trigger emergencies.

Conclusions:

The field of humanitarian relief is relatively nascent. Methods for documenting basic health measures on the local level have been developed and general health priorities have been documented. Some technical improvements in monitoring still are needed but decision-making is most often limited by the lack of data rather than the problems with data. The ability of health data to influence spending global priorities, legal or political actions undertaken by international organizations, remains very limited.

The livelihoods of Haitian health-care providers after the january 2010 earthquake: a pilot study of the economic and quality-of-life impact of emergency relief

Introduction

An effective international response to a disaster requires cooperation and coordination with the existing infrastructure. In some cases, however, international relief efforts can compete with the local work force and affect the balance of health-care systems already in place. This study seeks to evaluate the impact of the international humanitarian response to the 12 January 2010 earthquake on Haitian health-care providers (HHP).

Methods

Fifty-nine HHPs were surveyed in August of 2010 using a modified World Health Organization Quality of Life-Brief questionnaire (WHOQoL-B) that included questions on respondents' workload before the earthquake, immediately after, and presently. The study population consisted of physicians, nurses, and technicians at public hospitals, non-governmental organization (NGO) clinics, and private offices in Port-au-Prince, Haiti.

Results

Following the earthquake, public hospital and NGO providers reported a significant increase in their workload (15 of 17 and 22 of 26 respondents, respectively). Conversely, 12 of 16 private providers reported a significant decrease in workload (p < 0.0001). Although all groups reported working a similar number of hours prior to the earthquake (average 40 h/week), they reported working significantly different amounts following the earthquake. Public hospital and NGO providers averaged more than 50 h/week, and private providers averaged just over 33 h/week of employment (p < 0.001).

Health-care providers working at public hospitals and NGOs, however, had significantly lower scores on the WHOQoL-B when answering questions about their environment (p < 0.001), and in open-ended responses often commented about the lack of potable water and poor access to toilets. Providers from all groups expressed dissatisfaction with the scope and quality of care provided at public hospitals and NGO clinics, as well as disappointment with the reduction in patient volume at private practices.

Conclusions

The emergency medical response to the January 2010 earthquake in Haiti had the unintended consequence of poorly distributing work among HHPs. To create a robust health-care system in the long term while meeting short-term needs, humanitarian responses should seek to better integrate existing systems and involve local providers in the design and implementation of an emergency program.

Impact of public health emergencies on modern disaster taxonomy, planning, and response

Current disaster taxonomy describes diversity, distinguishing characteristics, and common relations in disaster event classifications. The impact of compromised public health infrastructure and systems on health consequences defines and greatly influences the manner in which disasters are observed, planned for, and managed, especially those that are geographically widespread, population dense, and prolonged. What may first result in direct injuries and death may rapidly change to excess indirect illness and subsequent death as essential public health resources are destroyed, deteriorate, or are systematically denied to vulnerable populations. Public health and public health infrastructure and systems in developed and developing countries must be seen as strategic and security issues that deserve international public health resource monitoring attention from disaster managers, urban planners, the global humanitarian community, World Health Organization authorities, and participating parties to war and conflict. We posit here that disaster frameworks be reformed to emphasize and clarify the relation of public health emergencies and modern disasters.

Fighting anaemia and malnutrition in Hebron (Palestine): impact evaluation of a humanitarian project

Iron supplementation in children is recommended by WHO when prevalence of anaemia is above 40%. In Palestine, due to the emergency situation caused by the outburst of the second intifada, the nutritional situation of children is worsening and iron deficiency anaemia represent one of the most serious problem. An emergency project which included universal iron supplementation (for 12,300 children), free treatment for common diseases and food supplementation to vulnerable families (for 3,275 children found anaemic or/and malnourished) and community health education was implemented in part of Hebron governorate. To evaluate project's impact we performed two random surveys: one before and one after the project. Before the project 30.1% of children (CI 95%=24.5-35.6%) were found anaemic (<11 g/dl), while after the prevalence was 18.8% (CI 95%=14.8-22.8%; decrease of 38%, p=0.001). Levels and reduction in anaemia prevalence were different according to geographical areas: where prevention activities had been implemented in previous years, prevalence of anaemia was 17.1% and remained stable at lower levels (14.7%). In other areas it sharply declined from 47.4 to 21.7% (p<0.0005): the declined was present only for children who received iron supplementation. Comparison of anthropometrics index before versus after the project showed that low "weight for age" declined from 10.9 to 3.8% (p=0.0006), low "weight for height" declined from 6.0 to 1.4% after (p=0.0025). With simple epidemiological tools we could demonstrate and measure the effectiveness of our interventions on the health status of the general population: a 50% reduction of anaemia and a 70% reduction of global acute malnutrition.

Expanding the scope of humanitarian program evaluation

The effectiveness of humanitarian programs normally is evaluated according to a limited number of pre-defined objectives. These objectives typically represent only selected positive expected impacts of program interventions and as such, are inadequate benchmarks for understanding the overall effectiveness of aid. This is because programs also have unexpected impacts (both positive and negative) as well as expected negative impacts and expected positive impacts beyond the program objectives. The authors contend that these other categories of program impacts also should be assessed, and suggest a methodology for doing so that draws on input from the perspectives of beneficiaries. This paper includes examples of the use of this methodology in the field. Finally, the authors suggest future directions for improving this type of expanded assessment and advocate for its widespread use, both within and without the field of disaster response.

Information for disasters, information disasters, and disastrous information

Information is needed to support humanitarian responses in every phase of a disaster. Participants of a multilateral working group convened to examine how best to meet these information needs. Although information systems based on routine reporting of diseases are desirable because they have the potential to identify trends, these systems usually do not deliver on their promise due to inadequate organization and management to support them. To identify organizational and management characteristics likely to be associated with successful information systems in disaster settings, evaluations of the Integrated Disease Surveillance and Response (IDSR) programs in 12 participating countries were reviewed. Characteristics that were mentioned repeatedly in the evaluations as associated with success were grouped into nine categories: (1) human resources management and supervision; (2) political support; (3) strengthened laboratory capacity; (4) communication and feedback (through many mechanisms); (5) infrastructure and resources; (6) system design and capacity; (7) coordination and partnerships with stakeholders; (8) community input; and (9) evaluation. Selected characteristics and issues within each category are discussed. Based on the review of the IDSR evaluations and selected articles in the published literature, recommendations are provided for improving the short- and long-term organization and management of information systems in humanitarian responses associated with disasters. It is suggested that information systems that follow these recommendations are more likely to yield quality information and be sustainable even in disaster settings.

Public health, conflict and human rights: toward a collaborative research agenda

Although epidemiology is increasingly contributing to policy debates on issues of conflict and human rights, its potential is still underutilized. As a result, this article calls for greater collaboration between public health researchers, conflict analysts and human rights monitors, with special emphasis on retrospective, population-based surveys. The article surveys relevant recent public health research, explains why collaboration is useful, and outlines possible future research scenarios, including those pertaining to the indirect and long-term consequences of conflict; human rights and security in conflict prone areas; and the link between human rights, conflict, and International Humanitarian Law.

Epidemiology in conflict - A call to arms

In this first special theme issue, Emerging Themes in Epidemiology publishes a collection of articles on the theme of Epidemiology in conflict. Violent conflict is an issue of great sensitivity within public health, but more structured research and reasoned discussion will allow us to better mitigate the public health impacts of war, and place the public health community in a more informed position in discussions about possible interventions in future conflicts.