Methods for evaluating delivery systems for scaling-up malaria control intervention

Development and Application of the Scale-Up Reflection Guide (SRG)

Scaling up effective interventions in public health is complex and comprehensive, and published accounts of the scale-up process are scarce. Key aspects of the scale-up experience need to be more comprehensively captured. This study describes the development of a guide for reflecting on and documenting the scale-up of public health interventions, to increase the depth of practice-based information of scaling up. Reviews of relevant scale-up frameworks along with expert input informed the development of the guide. We evaluated its acceptability with potential end-users and applied it to two real-world case studies. The Scale-up Reflection Guide (SRG) provides a structure and process for reflecting on and documenting key aspects of the scale-up process of public health interventions. The SRG is comprised of eight sections: context of completion; intervention delivery, history/background; intervention components; costs/funding strategies and partnership arrangements; the scale-up setting and delivery; scale-up process; and evidence of effectiveness and long-term outcomes. Utilization of the SRG may improve the consistency and reporting for the scale-up of public health interventions and facilitate knowledge sharing. The SRG can be used by a variety of stakeholders including researchers, policymakers or practitioners to more comprehensively reflect on and document scale-up experiences and inform future practice.

Coverage of intermittent preventive treatment of malaria in pregnancy in four sub-Saharan countries: findings from household surveys

Background

Intermittent preventive treatment in pregnancy (IPTp) with sulphadoxine-pyrimethamine (SP) is a key malaria prevention strategy in areas with moderate to high transmission. As part of the TIPTOP (Transforming IPT for Optimal Pregnancy) project, baseline information about IPTp coverage was collected in eight districts from four sub-Saharan countries: Democratic Republic of Congo (DRC), Madagascar, Mozambique and Nigeria.

Methods

Cross-sectional household surveys were conducted using a multistage cluster sampling design to estimate the coverage of IPTp and antenatal care attendance. Eligible participants were women of reproductive age who had ended a pregnancy in the 12 months preceding the interview and who had resided in the selected household during at least the past 4 months of pregnancy. Coverage was calculated using percentages and 95% confidence intervals.

Results

A total of 3911 women were interviewed from March to October 2018. Coverage of at least three doses of IPTp (IPTp3+) was 22% and 24% in DRC project districts; 23% and 12% in Madagascar districts; 11% and 16% in Nigeria local government areas; and 63% and 34% in Mozambique districts. In DRC, Madagascar and Nigeria, more than two-thirds of women attending at least four antenatal care visits during pregnancy received less than three doses of IPTp.

Conclusions

The IPTp3+ uptake in the survey districts was far from the universal coverage. However, one of the study districts in Mozambique showed a much higher coverage of IPTp3+ than the other areas, which was also higher than the 2018 average national coverage of 41%. The reasons for the high IPTp3+ coverage in this Mozambican district are unclear and require further study.

Taking account of context in systematic reviews and guidelines considering a complexity perspective

Systematic review teams and guideline development groups face considerable challenges when considering context within the evidence production process. Many complex interventions are context-dependent and are frequently evaluated within considerable contextual variation and change. This paper considers the extent to which current tools used within systematic reviews and guideline development are suitable in meeting these challenges. The paper briefly reviews strengths and weaknesses of existing approaches to specifying context. Illustrative tools are mapped to corresponding stages of the systematic review process. Collectively, systematic review and guideline production reveals a rich diversity of frameworks and tools for handling context. However, current approaches address only specific elements of context, are derived from primary studies which lack information or have not been tested within systematic reviews. A hypothetical example is used to illustrate how context could be integrated throughout the guideline development process. Guideline developers and evidence synthesis organisations should select an appropriate level of contextual detail for their specific guideline that is parsimonious and yet sensitive to health systems contexts and the values, preferences and needs of their target populations.

Contextual factors in maternal and newborn health evaluation: a protocol applied in Nigeria, India and Ethiopia

Background

Understanding the context of a health programme is important in interpreting evaluation findings and in considering the external validity for other settings. Public health researchers can be imprecise and inconsistent in their usage of the word “context” and its application to their work. This paper presents an approach to defining context, to capturing relevant contextual information and to using such information to help interpret findings from the perspective of a research group evaluating the effect of diverse innovations on coverage of evidence-based, life-saving interventions for maternal and newborn health in Ethiopia, Nigeria, and India.

Methods

We define “context” as the background environment or setting of any program, and “contextual factors” as those elements of context that could affect implementation of a programme. Through a structured, consultative process, contextual factors were identified while trying to strike a balance between comprehensiveness and feasibility. Thematic areas included demographics and socio-economics, epidemiological profile, health systems and service uptake, infrastructure, education, environment, politics, policy and governance. We outline an approach for capturing and using contextual factors while maximizing use of existing data. Methods include desk reviews, secondary data extraction and key informant interviews. Outputs include databases of contextual factors and summaries of existing maternal and newborn health policies and their implementation. Use of contextual data will be qualitative in nature and may assist in interpreting findings in both quantitative and qualitative aspects of programme evaluation.

Discussion

Applying this approach was more resource intensive than expected, in part because routinely available information was not consistently available across settings and more primary data collection was required than anticipated. Data was used only minimally, partly due to a lack of evaluation results that needed further explanation, but also because contextual data was not available for the precise units of analysis or time periods of interest. We would advise others to consider integrating contextual factors within other data collection activities, and to conduct regular reviews of maternal and newborn health policies. This approach and the learnings from its application could help inform the development of guidelines for the collection and use of contextual factors in public health evaluation.

Electronic supplementary material

The online version of this article (10.1186/s12982-018-0071-0) contains supplementary material, which is available to authorized users.

Evaluation of the implementation fidelity of the seasonal malaria chemoprevention intervention in Kaya health district, Burkina Faso

BACKGROUND

Burkina Faso implemented the seasonal malaria chemoprevention (SMC) in 2014 in seven pilot health districts, following the new recommendation by the WHO in 2012 for the prevention of the disease in children under five years old, for areas of highly seasonal malaria transmission.The objective of this study was to assess the implementation fidelity of the seasonal malaria chemoprevention strategy in one of the districts, Kaya Health District.

METHODOLOGY

We conducted a case study, with a quantitative and qualitative mixed methods. Data were collected after two campaigns of implementation of the intervention, in 2014 and 2015, through a review of specific documents of SMC intervention, and individual interview with key informants (n = 21) involved at various levels in the implementation of the strategy and a household survey with the parents (n = 284) of eligible children for the SMC strategy in 2015 in the Kaya health district. The analysis framework focused on the fidelity of the intervention's content, its coverage, and its schedule, as well as the potential moderating factors, using the model proposed by Hasson, originally from Carroll.

RESULTS

All components of the intervention were implemented. Villages and sectors were covered at 100%. In terms of intervention doses received, less than one-third of eligible children (32.3%) received the recommended four doses in 2015. Implementation of the strategy faced some difficulties due to insufficient training of community distributors, inadequate supply of inputs and insufficient financial resources for remuneration, advocacy and supervision, but also because of the contextual constraints due to the rainy season. Moreover, an interaction between the different moderating factors, influencing the degree of implementation of the strategy was noted.

CONCLUSION

Taking into account the moderating factors of the implementation is necessary for achieving the highest possible degree of implementation fidelity and then, reach the expected beneficial effects.

Evaluation of a universal long-lasting insecticidal net (LLIN) distribution campaign in Ghana: cost effectiveness of distribution and hang-up activities

BACKGROUND

Between May 2010 and October 2012, approximately 12.5 million long-lasting insecticidal nets (LLINs) were distributed through a national universal mass distribution campaign in Ghana. The campaign included pre-registration of persons and sleeping places, door-to-door distribution of LLINs with 'hang-up' activities by volunteers and post-distribution 'keep-up' behaviour change communication activities. Hang-up activities were included to encourage high and sustained use.

METHODS

The cost and cost-effectiveness of the LLIN Campaign were evaluated using a before-after design in three regions: Brong Ahafo, Central and Western. The incremental cost effectiveness of the 'hang-up' component was estimated using reported variation in the implementation of hang-up activities and LLIN use. Economic costs were estimated from a societal perspective assuming LLINs would be replaced after three years, and included the time of unpaid volunteers and household contributions given to volunteers.

RESULTS

Across the three regions, 3.6 million campaign LLINs were distributed, and 45.5% of households reported the LLINs received were hung-up by a volunteer. The financial cost of the campaign was USD 6.51 per LLIN delivered. The average annual economic cost was USD 2.90 per LLIN delivered and USD 6,619 per additional child death averted by the campaign. The cost-effectiveness of the campaign was sensitive to the price, lifespan and protective efficacy of LLINs.Hang-up activities constituted 7% of the annual economic cost, though the additional financial cost was modest given the use of volunteers. LLIN use was greater in households in which one or more campaign LLINs were hung by a volunteer (OR=1.57; 95% CI=1.09, 2.27; p=0.02). The additional economic cost of the hang-up activities was USD 0.23 per LLIN delivered, and achieved a net saving per LLIN used and per death averted.

CONCLUSION

In this campaign, hang-up activities were estimated to be net saving if hang-up increased LLIN use by 10% or more. This suggests hang-up activities can make a LLIN campaign more cost-effective.

Implementation fidelity of the national malaria control program in Burkina Faso

BACKGROUND

Every year 40,000 people die of malaria in Burkina Faso. In 2010, the Burkinabè authorities implemented a national malaria control program that provides for the distribution of mosquito nets and the home-based treatment of children with fever by community health workers. The objective of this study was to measure the implementation fidelity of this program.

METHODS

We conducted a case study in two comparable districts (Kaya and Zorgho). Data were collected one year after the program's implementation through field observations (10 weeks), documentary analysis, and individual interviews with stakeholders (n = 48) working at different levels of the program. The analysis framework looked at the fidelity of (i) the intervention's content, (ii) its coverage, and (iii) its schedule.

RESULTS

The program's implementation was relatively faithful to what was originally planned and was comparable in the two districts. It encountered certain obstacles in terms of the provision of supplies. Coverage fidelity was better in Kaya than in Zorgho, where many community health workers (CHW) experienced problems with the restocking of artemisinin-based combination therapy and with remuneration for periods of training. In both districts, the community was rarely involved in the process of selecting CHWs. The components affected by scheduling all experienced successive implementation delays that pushed nets distribution and the initial provision of artemisinin-based combination therapies to the CHWs past the 2010 malaria season.

CONCLUSIONS

The activities intended by the program were mostly implemented with good fidelity. However, the implementation was plagued by delays that probably postponed the expected beneficial effects.

Introducing vouchers for malaria prevention in Ghana and Tanzania: context and adoption of innovation in health systems

There are striking similarities in health system and other contexts between Tanzania and Ghana that are relevant to the scaling up of continuous delivery of insecticide treated nets (ITNs) for malaria prevention. However, specific contextual factors of relevance to ITN delivery have led implementation down very different pathways in the two countries. Both countries have made major efforts and investments to address this intervention through integrating consumer discount vouchers into the health system. Discount vouchers require arrangements among the public, private and non-governmental sectors and constitute a complex intervention in both health systems and business systems. In Tanzania, vouchers have moved beyond the planning agenda, had policies and programmes formulated, been sustained in implementation at national scale for many years and have become as of 2012 the main and only publicly supported continuous delivery system for ITNs. In Ghana national-scale implementation of vouchers never progressed beyond consideration on the agenda and piloting towards formulation of policy; and the approach was replaced by mass distribution campaigns with less dependency on or integration with the health system. By 2011, Ghana entered a phase with no publicly supported continuous delivery system for ITNs. To understand the different outcomes, we compared the voucher programme timelines, phases, processes and contexts in both countries in reference to the main health system building blocks (governance, human resources, financing, informatics, technologies and service delivery). Contextual factors which provided an enabling environment for the voucher scheme in Tanzania did not do so in Ghana. The voucher scheme was never seen as an appropriate national strategy, other delivery systems were not complementary and the private sector was under-developed. The extensive time devoted to engagement and consensus building among all stakeholders in Tanzania was an important and clearly enabling difference, as was public sector support of the private sector. This contributed to the alignment of partner action behind a single co-ordinated strategy at service delivery level which in turn gave confidence to the business sector and avoided the 'interference' of competing delivery systems that occurred in Ghana. Principles of systems thinking for intervention design correctly emphasize the importance of enabling contexts and stakeholder management.

The benefits or otherwise of managing malaria cases with or without laboratory diagnosis: the experience in a district hospital in Ghana

Background

This study was conducted at the Kintampo Municipal Hospital in Ghana to determine whether there was any benefit (or otherwise) in basing the management of cases of suspected malaria solely on laboratory confirmation (microscopy or by RDT) as compared with presumptive diagnosis.

Method

Children under five years who reported at the Out-Patient Department of the Hospital with axillary temperature ≥37.5°C or with a 48 hr history of fever were enrolled and had malaria microscopy and RDT performed. The attending clinician was blinded from laboratory results unless a request for these tests had been made earlier. Diagnosis of malaria was based on three main methods: presumptive or microscopy and/or RDT. Cost implication for adopting laboratory diagnosis or not was determined to inform malaria control programmes.

Results

In total, 936 children were enrolled in the study. Proportions of malaria diagnosed presumptively, by RDT and microscopy were 73.6% (689/936), 66.0% (618/936) and 43.2% (404/936) respectively. Over 50% (170/318) of the children who were RDT negative and 60% (321/532) who were microscopy negative were treated for malaria when presumptive diagnoses were used. Comparing the methods of diagnoses, the cost of malaria treatment could have been reduced by 24% and 46% in the RDT and microscopy groups respectively; the reduction was greater in the dry season (43% vs. 50%) compared with the wet season (20% vs. 45%) for the RDT and microscopy confirmed cases respectively.

Discussion/Conclusion

Over-diagnosis of malaria was prevalent in Kintampo during the period of the study. Though the use of RDT for diagnosis of malaria might have improved the quality of care for children, it appeared not to have a cost saving effect on the management of children with suspected malaria. Further research may be needed to confirm this.

PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries

Crick Lund and colleagues describe their plans for the PRogramme for Improving Mental health carE (PRIME), which aims to generate evidence on implementing and scaling up integrated packages of care for priority mental disorders in primary and maternal health care contexts in Ethiopia, India, Nepal, South Africa, and Uganda.

Strategies for delivering insecticide-treated nets at scale for malaria control: a systematic review

OBJECTIVE

To synthesize findings from recent studies of strategies to deliver insecticide-treated nets (ITNs) at scale in malaria-endemic areas.

METHODS

Databases were searched for studies published between January 2000 and December 2010 in which: subjects resided in areas with endemicity for Plasmodium falciparum and Plasmodium vivax malaria; ITN delivery at scale was evaluated; ITN ownership among households, receipt by pregnant women and/or use among children aged < 5 years was evaluated; and the study design was an individual or cluster-randomized controlled design, nonrandomized, quasi-experimental, before-and-after, interrupted time series or cross-sectional without temporal or geographical controls. Papers describing qualitative studies, case studies, process evaluations and cost-effectiveness studies linked to an eligible paper were also included. Study quality was assessed using the Cochrane risk of bias checklist and GRADE criteria. Important influences on scaling up were identified and assessed across delivery strategies.

FINDINGS

A total of 32 papers describing 20 African studies were reviewed. Many delivery strategies involved health sectors and retail outlets (partial subsidy), antenatal care clinics (full subsidy) and campaigns (full subsidy). Strategies achieving high ownership among households and use among children < 5 delivered ITNs free through campaigns. Costs were largely comparable across strategies; ITNs were the main cost. Cost-effectiveness estimates were most sensitive to the assumed net lifespan and leakage. Common barriers to delivery included cost, stock-outs and poor logistics. Common facilitators were staff training and supervision, cooperation across departments or ministries and stakeholder involvement.

CONCLUSION

There is a broad taxonomy of strategies for delivering ITNs at scale.

Who attends antenatal care and expanded programme on immunization services in Chad, Mali and Niger? The implications for insecticide-treated net delivery

Background

Malaria remains one of the largest public health problems facing the developing world. Insecticide-treated nets (ITNs) are an effective intervention against malaria. ITN delivery through routine health services, such as antenatal care (ANC) and childhood vaccination (EPI), is a promising channel of delivery to reach individuals with the highest risk (pregnant women and children under five years old). Decisions on whether to deliver ITNs through both channels depends upon the reach of each of these systems, whether these are independent and the effectiveness and cost effectiveness of each. Predictors of women attending ANC and EPI separately have been studied, but the predictors of those who attend neither service have not been identified.

Methods

Data from Chad, Mali and Niger demographic and health surveys (DHS) were analyzed to determine risk factors for attending neither service. A conceptual framework for preventative health care-seeking behaviour was created to illustrate the hierarchical relationships between the potential risk factors. The independence of attending both ANC and EPI was investigated. A multivariate model of predictors for non-attendance was developed using logistic regression.

Results

ANC and EPI attendance were found to be strongly associated in all three countries. However, 47% of mothers in Chad, 12% in Mali and 36% in Niger did not attend either ANC or EPI. Region, mother's education and partner's education were predictors of non-attendance in all three countries. Wealth index, ethnicity, and occupation were associated with non-attendance in Mali and Niger. Other predictors included religion, healthcare autonomy, household size and number of children under five.

Conclusions

Attendance of ANC and EPI are not independent and therefore the majority of pregnant women in these countries will have the opportunity to receive ITNs through both services. Although attendance at ANC and EPI are not independent, delivery through both systems may still add incrementally to delivery through one alone. Therefore, there is potential to increase the proportion of women and children receiving ITNs by delivering through both of these channels. However, modelling is required to determine the level of attendance and incremental potential at which it's cost effective to deliver through both services.