Creating spaces for dialogue: a cluster-randomized evaluation of CARE's Community Score Card on health governance outcomes

The impact of community and provider-driven social accountability interventions on contraceptive use: findings from a cohort study of new users in Ghana and Tanzania

BACKGROUND

Although contraceptive use has increased over 15 years, discontinuation rates remain high. Contraceptive use is becoming more important when addressing unmet need for family planning. Social accountability, defined here as collective processes for holding duty bearers to account for their actions, is a rights-based participatory process that supports service provision and person-centred care, as well as, informed decision-making among community members regarding their health. A study implemented in Ghana and Tanzania was designed to understand and evaluate how social accountability and participatory processes influences quality of care and client satisfaction and whether this results in increased contraceptive uptake and use. We report here on the relationship between social accountability and the use of modern contraceptives, i.e., contraceptive method discontinuation, contraceptive method switching, and contraceptive discontinuation.

METHODS

As part of Community and Provider driven Social Accountability Intervention (CaPSAI) Project, a cohort of women aged 15 to 49 years who were new users of contraception and accessing family planning and contraceptives services at the study facilities across both intervention and control groups were followed-up over a 12-month period to measure changes contraceptive use.

RESULTS

In this cohort study over a one-year duration, we did not find a statistically significant difference in Ghana and Tanzania in overall method discontinuation, switching, and contraceptive discontinuation after exposure to a social accountability intervention. In Ghana but not in Tanzania, when stratified by the type of facility (district level vs. health centre), there were significantly less method and contraceptive discontinuation in the district level facility and significantly more method and contraceptive discontinuation in the health centres in the intervention group. In Ghana, the most important reasons reported for stopping a method were fear of side-effects, health concerns and wanting to become pregnant in the control group and fear of side-effects wanting a more effective method and infrequent sex in the intervention group. In Tanzania, the most important reasons reported for stopping a method were fear of side-effects, wanting a more effective method, and method not available in the control group compared to wanting a more effective method, fear of side-effects and health concerns in the intervention group.

CONCLUSIONS

We did not demonstrate a statistically significant impact of a six-month CaPSAI intervention on contraceptives use among new users in Tanzania and Ghana. However, since social accountability have important impacts beyond contraceptive use it is important consider results of the intermediate outcomes, cases of change, and process evaluation to fully understand the impact of this intervention.

TRIAL REGISTRATION

The CaPSAI Project has been registered at Australian New Zealand Clinical Trials Registry (ACTRN12619000378123, 11/03/2019).

Barriers to family planning through structural health vulnerabilities: findings from case studies from rural Uganda

The burden of maternal mortality in sub-Saharan Africa is the highest in the world. As high fertility rates present an increased risk to women's health, programmes and services have been put in place to increase access to family planning. Several studies have identified sociocultural factors as limiting access to family planning, including male resistance and women's low decision-making power. However, these studies have often used a behavioural approach that anchors barriers to access in women's attitudes or motivations. In 2019, a qualitative evaluation of a family planning programme using a behavioural approach was conducted in rural Uganda, focusing on women's attitudes and unmet needs. With hindsight, we realised that our evaluation was insufficient to adequately capture barriers that extend beyond women's motivations. This paper shares three case studies that we analysed using a structural health vulnerabilities approach. Our analysis highlights political, economic, social and cultural structures that can impede access to family planning. Using a structural health vulnerabilities approach, we identify structural barriers which have their roots in the local realities women face and propose context-specific structural recommendations. Finally, we contrast our initial results with findings the case studies presented to highlight the relevance of taking structural vulnerabilities into account in future family planning programme evaluation and women's sexual and reproductive health studies.

Reliability and validity of an innovative high performing healthcare system assessment tool

BACKGROUND

Universal Health coverage (UHC) is the mantra of the twenty-first century yet knowing when it has been achieved or how to best influence its progression remains elusive. An innovative framework for High Performing Healthcare (HPHC) attempts to address these issues. It focuses on measuring four constructs of Accountable, Affordable, Accessible, and Reliable (AAAR) healthcare that contribute to better health outcomes and impact. The HPHC tool collects information on the perceived functionality of health system processes and provides real-time data analysis on the AAAR constructs, and on processes for health system resilience, responsiveness, and quality, that include roles of community, private sector, as well as both demand, and supply factors affecting health system performance. The tool attempts to capture the multidimensionality of UHC measurement and evidence that links health system strengthening activities to outcomes. This paper provides evidence on the reliability and validity of the tool.

METHODS

Internet survey with non-probability sampling was used for testing reliability and validity of the HPHC tool. The volunteers were recruited using international networks and listservs. Two hundred and thirteen people from public, private, civil society and international organizations volunteered from 35 low-and-middle-income countries. Analyses involved testing reliability and validity and validation from other international sources of information as well as applicability in different setting and contexts.

RESULTS

The HPHC tool's AAAR constructs, and their sub-domains showed high internal consistency (Cronbach alpha >.80) and construct validity. The tool scores normal distribution displayed variations among respondents. In addition, the tool demonstrated its precision and relevance in different contexts/countries. The triangulation of HPHC findings with other international data sources further confirmed the tool's validity.

CONCLUSIONS

Besides being reliable and valid, the HPHC tool adds value to the state of health system measurement by focusing on linkages between AAAR processes and health outcomes. It ensures that health system stakeholders take responsibility and are accountable for better system performance, and the community is empowered to participate in decision-making process. The HPHC tool collects and analyzes data in real time with minimum costs, supports monitoring, and promotes adaptive management, policy, and program development for better health outcomes.

What's in a name? Unpacking 'Community Blank' terminology in reproductive, maternal, newborn and child health: a scoping review

INTRODUCTION

Engaging the community as actors within reproductive, maternal, newborn and child health (RMNCH) programmes (referred to as 'community blank') has seen increased implementation in recent years. While evidence suggests these approaches are effective, terminology (such as 'community engagement,' 'community participation,' 'community mobilisation,' and 'social accountability') is often used interchangeably across published literature, contributing to a lack of conceptual clarity in practice. The purpose of this review was to describe and clarify varying uses of these terms in the literature by documenting what authors and implementers report they are doing when they use these terms.

METHODS

Seven academic databases (PubMed/MEDLINE, Embase, CINAHL, PsycINFO, Scopus, Web of Science, Global Health), two grey literature databases (OAIster, OpenGrey) and relevant organisation websites were searched for documents that described 'community blank' terms in RMNCH interventions. Eligibility criteria included being published between 1975 and 1 October 2021 and reports or studies detailing the activities used in 'community blank.' RESULTS: A total of 9779 unique documents were retrieved and screened, with 173 included for analysis. Twenty-four distinct 'community blank' terms were used across the documents, falling into 11 broader terms. Use of these terms was distributed across time and all six WHO regions, with 'community mobilisation', 'community engagement' and 'community participation' being the most frequently used terms. While 48 unique activities were described, only 25 activities were mentioned more than twice and 19 of these were attributed to at least three different 'community blank' terms.

CONCLUSION

Across the literature, there is inconsistency in the usage of 'community blank' terms for RMNCH. There is an observed interchangeable use of terms and a lack of descriptions of these terms provided in the literature. There is a need for RMNCH researchers and practitioners to clarify the descriptions reported and improve the documentation of 'community blank' implementation. This can contribute to a better sharing of learning within and across communities and to bringing evidence-based practices to scale. Efforts to improve reporting can be supported with the use of standardised monitoring and evaluation processes and indicators. Therefore, it is recommended that future research endeavours clarify the operational definitions of 'community blank' and improve the documentation of its implementation.

Research and implementation interactions in a social accountability study: utilizing guidance for conducting process evaluations of complex interventions

BACKGROUND

In recent years, researchers and evaluators have made efforts to identify and use appropriate and innovative research designs that account for the complexity in studying social accountability. The relationship between the researchers and those implementing the activities and how this impacts the study have received little attention. In this paper, we reflect on how we managed the relationship between researchers and implementers using the United Kingdom Medical Research Council (MRC) guidance on process evaluation of a complex intervention.

MAIN BODY

The MRC guidance focuses on three areas of interaction between researchers and stakeholders involved in developing and delivering the intervention: (i) working with program developers and implementers; (ii) communication of emerging findings between researchers/evaluators and implementers; and (iii) overlapping roles of the intervention and research/evaluation. We summarize how the recommendations for each of the three areas were operationalized in the Community and Provider driven Social Accountability Intervention (CaPSAI) Project and provide reflections based on experience. We co-developed various tools, including standard operating procedures, contact lists, and manuals. Activities such as training sessions, regular calls, and meetings were also conducted to enable a good working relationship between the different partners.

CONCLUSIONS

Studying social accountability requires the collaboration of multiple partners that need to be planned to ensure a good working relationship while safeguarding both the research and intervention implementation. The MRC guidance is a useful tool for making interaction issues explicit and establishing procedures. Planning procedures for dealing with research and implementers' interactions could be more comprehensive and better adapted to social accountability interventions if both researchers and implementers are involved. There is a need for social accountability research to include clear statements explaining the nature and types of relationships between researchers and implementers involved in the intervention.

The effect of community engagement on healthcare utilization and health insurance enrollment in Ghana: Results from a randomized experiment

Health insurance enrollment in many Sub-Saharan African countries is low, even with highly subsidized premiums and exemptions for vulnerable populations. One possible explanation is low service quality, which results in a low valuation of health insurance. Using a randomized control trial in 64 primary health care facilities in Ghana, this study assesses the impact of a community engagement intervention designed to improve the quality of healthcare and health insurance services on households living nearby the facilities. Although the intervention improved the medical-technical quality of health services, our results show that households' subjective perceptions of the quality of healthcare and insurance services did not increase. Nevertheless, the likelihood of illness and concomitant healthcare utilization reduced, and especially households who were not insured at baseline were more likely to enroll in health insurance. The results show that solely increasing the technical quality of care is not sufficient to increase households' subjective assessments of healthcare quality. Still, improving technical quality can directly contribute to health outcomes and further increase health insurance coverage, especially among the previously uninsured.

The use of legal empowerment to improve access to quality health services: a scoping review

This paper presents the results of a scoping review that examines the extent to which legal empowerment has been used as a strategy in efforts to improve access to quality health services in low- and middle-income countries. The review identifies lessons learned regarding legal empowerment program strategy, as well as impact on health empowerment and health outcomes, research gaps, areas of consensus and tension in the field.

The review included three main sources of data: 1) peer-reviewed literature, 2) grey literature, and 3) interviews with key legal empowerment stakeholders. Peer-reviewed and grey literature were identified via keyword searches, and interviewees were identified by searching an organizational database and snowball sampling.

The key findings were: first, there is very limited documentation on the use of legal empowerment strategies for improving health services. Second, the legal empowerment approach tends to be focussed on issues that communities themselves prioritize, often narrowly defined local challenges. However, legal empowerment as a strategy that pursues collective and individual remedies has the potential to contribute to structural change. Third, for this potential to be realised, legal empowerment entails building capacity of service providers and other duty bearers on health and related rights. Finally, the review also highlights the importance of trust—trust in state institutions, trust in the paralegals who support the process and trust in the channels of engagement with public authorities for grievance redress.

Several gaps also became evident through the review, including lack of work on private health providers, lack of discussion of the ‘empowerment’ effects of legal empowerment programs, and limited exploration of risk and sustainability. The paper concludes with a caution that practitioners need to start with the health challenges they are trying to address, and then assess whether legal empowerment is an appropriate approach, rather than seeing it as a silver bullet.

Collective efficacy measures for women and girls in low- and middle-income countries: a systematic review

Background

Prior research has shown collective efficacy to be a key determinant of women’s well-being. However, much of the work around measuring this construct has been done in high-income geographies, with very little representation from low- and middle-income countries (LMIC). To fill this gap, and guide future research in low resource settings, we aim to summarize best evidence measures of collective efficacy for women and girls from LMICs.

Methods

Following PRISMA guidelines, we systematically searched five databases for English language peer-reviewed literature on measures of collective efficacy, published between 1 January 2009 and 25 August 2020. In addition, we sought expert input for relevant papers in this area. Research staff screened titles, abstracts, and full-text articles in a double-blind review. Inclusion criteria were: (i) original quantitative analysis, and (ii) sample limited to women/girls only (≥ 100), residing in LMICs.

Results

We identified 786 unique articles, 14 of which met inclusion criteria. Eligible studies captured a diversity of population groups, including pregnant women, recent mothers, adolescent girls, and female sex workers, from across national settings. Two broad constructs of collective efficacy were captured by the measures: (i) group dynamics, and (ii) collective action. All 14 studies included items on group dynamics in their measures, whereas seven studies included items on collective action. Four studies validated new measures of collective efficacy, and seven provided evidence supporting the relationship between collective efficacy and outcomes related to women’s well-being. Overall, measures demonstrated good reliability and validity when tested, and those testing for associations or effects found a positive relationship of collective efficacy with women’s health behaviors.

Conclusion

The past decade has resulted in a number of new collective efficacy measures demonstrating good validity in terms of their associations with key health outcomes among women and girls from across LMIC settings, but there remains no standard measure in the field. Those that exist focus on group dynamics, but less often on collective action. A standard measure of collective efficacy inclusive of group dynamics and collective action can support better understanding of the value of women’s collectives across national settings and populations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12905-022-01688-z.

Assessing community health governance for evidence-informed decision-making: a cross-sectional study across nine districts of India

Purpose

The authors intend to assess the village health sanitation and nutrition committees (VHSNC) on six parameters, including their formation, composition, meeting frequencies, activities, supervisory mechanisms and funds receipt and expenditures across nine districts of the three states of India.

Design/methodology/approach

The cross-sectional study, conducted in the states of Uttar Pradesh (five districts), Odisha (two districts) and Rajasthan (two districts), used a quantitative research design. The community health workers of 140 VHSNCs were interviewed using a semi-structured questionnaire. The details about the funds' receipt and expenditures were verified from the VHSNC records (cashbook). Additionally, the authors asked about the role of health workers in the VHSNC meetings, and the issues and challenges faced.

Findings

The average number of members in VHSNCs varied from 10 in Odisha to 15 in Rajasthan. Activities were regularly organized in Rajasthan and Odisha (one per month) compared to Uttar Pradesh (one every alternate month). Most commonly, health promotion activities, cleanliness drives, community monitoring and facilitation of service providers were done by VHSNCs. Funds were received regularly in Odisha compared to Rajasthan and Uttar Pradesh. Funds were received late and less compared to the demands or needs of VHSNCs.

Research limitations/implications

This comprehensive analysis of VHSNCs' functioning in the selected study areas sheds light on the gaps in many components, including the untimely and inadequate receipt of funds, poor documentation of expenditures and involvement of VHSNC heads and inadequate supportive supervision.

Originality/value

VHSNCs assessment has been done for improving community health governance.

From effectiveness to sustainability: understanding the impact of CARE's Community Score Card© social accountability approach in Ntcheu, Malawi

We evaluated the sustainability of CARE’s Community Score Card© (CSC) social accountability approach in Ntcheu, Malawi, approximately 2.5 years after the end of formal intervention activities. Using a cross-sectional, exploratory design, we conducted 41 focus groups with members of Community Health Advisory Groups (CHAGs) and youth groups and 19 semi-structured interviews with local and district government officials, project staff, and national stakeholders to understand how and in what form CSC activities are continuing. Focus groups and interviews were audio-recorded, transcribed and translated into English. Thematic coding was done using Dedoose software. Most groups were continuing to meet and implement the CSC, although some made modifications. CHAGs, youth and local government officials all attributed their continued implementation to the value that they saw in the process that allows marginalized groups within the community, including women and youth, a safe space for sharing their ideas and issues and the initial results this generated. However, lack of access to resources for implementation and challenges in convening and facilitating the interface meeting phase created barriers to continued sustainability. The CSC is sustainable by communities 2.5 years after the end of formal intervention activities. For future interventions, health systems and non-governmental organizations should plan for a transition phase with periodic refresher trainings and a small fund to support implementation, such as refreshments and transportation, to increase the likelihood of community-driven sustainability.

Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation

BACKGROUND

Health services have traditionally been developed to focus on specific diseases or medical specialties. Involving consumers as partners in planning, delivering and evaluating health services may lead to services that are person-centred and so better able to meet the needs of and provide care for individuals. Globally, governments recommend consumer involvement in healthcare decision-making at the systems level, as a strategy for promoting person-centred health services. However, the effects of this 'working in partnership' approach to healthcare decision-making are unclear. Working in partnership is defined here as collaborative relationships between at least one consumer and health provider, meeting jointly and regularly in formal group formats, to equally contribute to and collaborate on health service-related decision-making in real time. In this review, the terms 'consumer' and 'health provider' refer to partnership participants, and 'health service user' and 'health service provider' refer to trial participants. This review of effects of partnership interventions was undertaken concurrently with a Cochrane Qualitative Evidence Synthesis (QES) entitled Consumers and health providers working in partnership for the promotion of person-centred health services: a co-produced qualitative evidence synthesis.

OBJECTIVES

To assess the effects of consumers and health providers working in partnership, as an intervention to promote person-centred health services.

SEARCH METHODS

We searched the CENTRAL, MEDLINE, Embase, PsycINFO and CINAHL databases from 2000 to April 2019; PROQUEST Dissertations and Theses Global from 2016 to April 2019; and grey literature and online trial registries from 2000 until September 2019.

SELECTION CRITERIA

We included randomised controlled trials (RCTs), quasi-RCTs, and cluster-RCTs of 'working in partnership' interventions meeting these three criteria: both consumer and provider participants meet; they meet jointly and regularly in formal group formats; and they make actual decisions that relate to the person-centredness of health service(s).

DATA COLLECTION AND ANALYSIS

Two review authors independently screened most titles and abstracts. One review author screened a subset of titles and abstracts (i.e. those identified through clinical trials registries searches, those classified by the Cochrane RCT Classifier as unlikely to be an RCT, and those identified through other sources). Two review authors independently screened all full texts of potentially eligible articles for inclusion. In case of disagreement, they consulted a third review author to reach consensus. One review author extracted data and assessed risk of bias for all included studies and a second review author independently cross-checked all data and assessments. Any discrepancies were resolved by discussion, or by consulting a third review author to reach consensus. Meta-analysis was not possible due to the small number of included trials and their heterogeneity; we synthesised results descriptively by comparison and outcome. We reported the following outcomes in GRADE 'Summary of findings' tables: health service alterations; the degree to which changed service reflects health service user priorities; health service users' ratings of health service performance; health service users' health service utilisation patterns; resources associated with the decision-making process; resources associated with implementing decisions; and adverse events.

MAIN RESULTS

We included five trials (one RCT and four cluster-RCTs), with 16,257 health service users and more than 469 health service providers as trial participants. For two trials, the aims of the partnerships were to directly improve the person-centredness of health services (via health service planning, and discharge co-ordination). In the remaining trials, the aims were indirect (training first-year medical doctors on patient safety) or broader in focus (which could include person-centredness of health services that targeted the public/community, households or health service delivery to improve maternal and neonatal mortality). Three trials were conducted in high income-countries, one was in a middle-income country and one was in a low-income country. Two studies evaluated working in partnership interventions, compared to usual practice without partnership (Comparison 1); and three studies evaluated working in partnership as part of a multi-component intervention, compared to the same intervention without partnership (Comparison 2). No studies evaluated one form of working in partnership compared to another (Comparison 3). The effects of consumers and health providers working in partnership compared to usual practice without partnership are uncertain: only one of the two studies that assessed this comparison measured health service alteration outcomes, and data were not usable, as only intervention group data were reported. Additionally, none of the included studies evaluating this comparison measured the other primary or secondary outcomes we sought for the 'Summary of findings' table. We are also unsure about the effects of consumers and health providers working in partnership as part of a multi-component intervention compared to the same intervention without partnership. Very low-certainty evidence indicated there may be little or no difference on health service alterations or health service user health service performance ratings (two studies); or on health service user health service utilisation patterns and adverse events (one study each). No studies evaluating this comparison reported the degree to which health service alterations reflect health service user priorities, or resource use. Overall, our confidence in the findings about the effects of working in partnership interventions was very low due to indirectness, imprecision and publication bias, and serious concerns about risk of selection bias; performance bias, detection bias and reporting bias in most studies.

AUTHORS' CONCLUSIONS

The effects of consumers and providers working in partnership as an intervention, or as part of a multi-component intervention, are uncertain, due to a lack of high-quality evidence and/or due to a lack of studies. Further well-designed RCTs with a clear focus on assessing outcomes directly related to partnerships for patient-centred health services are needed in this area, which may also benefit from mixed-methods and qualitative research to build the evidence base.

Evaluating Young People's Ability to Sustain an Evidence-Based Social Accountability Approach to Improve Adolescent Sexual and Reproductive Health in Ntcheu, Malawi

The Community Score Card© (CSC), a social accountability approach, brings together community members, service providers, and local government officials to identify issues, prioritize, and plan actions to improve local health services. In addition, young people in Ntcheu, Malawi have been using the CSC approach to mobilize their communities to bring change across varying issues of importance to them. An earlier cluster randomized trial in Ntcheu showed the CSC effectively increased reproductive health behaviors, improved satisfaction with services, and enhanced the coverage and quality of services. Building upon this evidence of effectiveness, this study aims to evaluate if and how young people were able to sustain implementation of the CSC, and the improvements it brings, approximately 2.5 years after the randomized trial ended. As part of a larger evaluation of CSC sustainability in Ntcheu, we conducted 8 focus groups across 5 health catchment areas with 109 members of mixed-gender youth groups (58 females and 51 males, ages 14–29 years) who continued to engage with the CSC. Audio recordings were transcribed, translated into English, and coded in Dedoose using an a priori codebook augmented with emergent codes and a constant comparative approach. Although the 8 youth groups were still actively using the CSC, they had made some adaptations. While the CSC in Ntcheu initially focused on maternal health, young people adopted the approach for broader sexual and reproductive topics important to them such as child marriages and girls' education. To enable sustainability, young people trained each other in the CSC process; they also requested more formal facilitation training. Young people from Ntcheu recommended nationwide scale-up of the CSC. Young people organically adopted the CSC, which enabled them to highlight issues within their communities that were a priority to them. This diffusion among young people enabled them to elevate their voice and facilitate a process where they hold local government officials, village leaders, and services providers accountable for actions and the quality of healthcare services. Young people organized and sustained the CSC as a social accountability approach to improve adolescent sexual and reproductive health in their communities more than 2.5 years after the initial effectiveness trial ended.

Implementing a Social Accountability Approach for Maternal, Neonatal, and Child Health Service Performances in Ethiopia: A Pre-Post Study Design

BACKGROUND

Social accountability approaches, such as the community scorecard (CSC), can improve the performance of health systems in low-income countries by providing a mechanism for obtaining and incorporating community input. This longitudinal study assessed the effects of CSCs implemented by primary health care units (PHCUs) on health system performance in Ethiopia.

METHODS

This study used a pre-post design and was conducted from October 2018 to September 2019 in 159 PHCUs in 31 districts in Amhara and Southern, Nations, Nationalities and Peoples' regional states. The data were extracted from a routine health information management system database at baseline, midterm, and endline stages over 12 months for statistical analysis. The effects of implementing CSCs on health system performance were evaluated against selected key performance indicators (KPIs).

RESULTS

The CSC measurement results were based on input from 38,556 community representatives. The mean CSC score with standard deviation (±SD) was 60.8%±12.5%, 66.3%±10.8%, and 70.6%±10.0% at baseline, midterm, and endline, respectively. The mean KPI score was 54.9%±17.4%, 61.9%±15.1%, and 67.6%±14.6% at baseline, midterm, and endline, respectively. The average CSC and KPI values were positively correlated (r>0.37). Using a nonparametric Friedman's test, we found a statistically significant difference in CSC and KPI scores at baseline, midterm, and endline (P=.001). Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction and the results showed higher CSC and KPI values from baseline to midterm and from midterm to endline (P<.017).

CONCLUSIONS

The use of CSCs in Ethiopia contributed to the health system's performance in terms of maternal and child health services. The responsiveness of health workers and utilization of basic health services by community members were found to increase significantly with CSC use. We recommend continued implementation of the CSC intervention at PHCUs.

Community-Led Monitoring: When Community Data Drives Implementation Strategies

PURPOSE OF REVIEW

Communities occupy a central position in effective health systems, notably through monitoring of health service quality and by giving recipients of care a voice. Our review identifies community-led monitoring mechanisms and best practices.

RECENT FINDINGS

Implementation of community-led monitoring mechanisms improved service delivery at facility-level, health system-wide infrastructure and health outcomes among recipients of care. Successful models were community-led, collaborative, continuous and systematic, and incorporated advocacy and community education. Identifying and replicating successful community-led monitoring practices is a key pathway to equitable access to HIV and health services overall.

Methods to measure effects of social accountability interventions in reproductive, maternal, newborn, child, and adolescent health programs: systematic review and critique

BACKGROUND

There is no agreed way to measure the effects of social accountability interventions. Studies to examine whether and how social accountability and collective action processes contribute to better health and healthcare services are underway in different areas of health, and health effects are captured using a range of different research designs.

OBJECTIVES

The objective of our review is to help inform evaluation efforts by identifying, summarizing, and critically appraising study designs used to assess and measure social accountability interventions' effects on health, including data collection methods and outcome measures. Specifically, we consider the designs used to assess social accountability interventions for reproductive, maternal, newborn, child, and adolescent health (RMNCAH).

DATA SOURCES

Data were obtained from the Cochrane Library, EMBASE, MEDLINE, SCOPUS, and Social Policy & Practice databases.

ELIGIBILITY CRITERIA

We included papers published on or after 1 January 2009 that described an evaluation of the effects of a social accountability intervention on RMNCAH.

RESULTS

Twenty-two papers met our inclusion criteria. Methods for assessing or reporting health effects of social accountability interventions varied widely and included longitudinal, ethnographic, and experimental designs. Surprisingly, given the topic area, there were no studies that took an explicit systems-orientated approach. Data collection methods ranged from quantitative scorecard data through to in-depth interviews and observations. Analysis of how interventions achieved their effects relied on qualitative data, whereas quantitative data often raised rather than answered questions, and/or seemed likely to be poor quality. Few studies reported on negative effects or harms; studies did not always draw on any particular theoretical framework. None of the studies where there appeared to be financial dependencies between the evaluators and the intervention implementation teams reflected on whether or how these dependencies might have affected the evaluation. The interventions evaluated in the included studies fell into the following categories: aid chain partnership, social audit, community-based monitoring, community-linked maternal death review, community mobilization for improved health, community reporting hotline, evidence for action, report cards, scorecards, and strengthening health communities.

CONCLUSIONS

A wide range of methods are currently being used to attempt to evaluate effects of social accountability interventions. The wider context of interventions including the historical or social context is important, as shown in the few studies to consider these dimensions. While many studies collect useful qualitative data that help illuminate how and whether interventions work, the data and analysis are often limited in scope with little attention to the wider context. Future studies taking into account broader sociopolitical dimensions are likely to help illuminate processes of accountability and inform questions of transferability of interventions. The review protocol was registered with PROSPERO (registration # CRD42018108252).

Can social accountability mechanisms using community scorecards improve quality of pediatric care in rural Cambodia?

Objective

To determine the effect of social accountability strategies on pediatric quality of care.

Design and Setting

A non-randomized quasi experimental study was conducted in four districts in Cambodia and all operational public health facilities were included.

Participants

Five patients under 5 years and their caretakers were randomly selected in each facility.

Interventions

To determine the effect of maternal and child health interventions integrating citizen voice and action using community scorecards on quality of pediatric care.

Outcome Measures

Patient observations were conducted to determine quality of screening and counseling, followed by exit interviews with caretakers.

Results

Results indicated significant differences between intervention and comparison facilities; screening by Integrated Management of Childhood Illness (IMCI) trained providers (100% vs 67%, P &lt; 0.019), screening for danger signs; ability to drink/breastfeed (100% vs 86.7%, P &lt; 0.041), lethargy (86.7% vs 40%, P &lt; 0.004) and convulsions (83.3 vs 46.7%, P &lt; 0.023). Screening was significantly higher for patients in the intervention facilities for edema (56.7% vs 6.7%, P &lt; 0.000), immunization card (90% vs 40%, P &lt; 0.002), child weight (100 vs 86.7, P &lt; 0.041) and checking growth chart (96.7% vs 66.7%, P &lt; 0.035). The IMCI index, constructed from key performance indicators, was significantly higher for patients in the intervention facilities than comparison facilities (screening index 8.8 vs 7.0, P &lt; 0.018, counseling index 2.7 vs 1.5, P &lt; 0.001). Predictors of screening quality were child age, screening by IMCI trained provider, wealthier quintiles and intervention facilities.

Conclusion

The institution of social accountability mechanisms to engage communities and facility providers showed some improvements in quality of care for common pediatric conditions, but socioeconomic disparities were evident.

Effects of the Community Score Card approach on reproductive health service-related outcomes in Malawi

Background

Social accountability approaches are increasingly being employed in low-resource settings to improve government services. In line with the continuous quality improvement (CQI) philosophy that quality is the product of a linked chain, collaborative social accountability approaches like the Community Score Card (CSC) aim to empower clients and frontline service providers to transform their own lives and hold public officials to account for state obligations. Despite being a critical focus of collaborative social accountability approaches, to our knowledge, a quantitative survey of health workers to understand the impact of these approaches on their self-reported responsibilities and service provision has not been conducted. To fill this gap, we carried out a quantitative survey with health workers to assess the CSC’s impact on health worker-reported service responsibilities and provision and complement women’s self-reports.

Methods

We evaluated the effect of the CSC on reproductive health-related outcomes using a cluster-randomized design in Ntcheu district, Malawi. We matched 10 pairs of health facilities and surrounding catchment communities; one from each pair was randomly assigned to the intervention and control arms. The intervention communities and health workers each completed 3–4 cycles of the CSC process by endline. We then surveyed all health workers in the 20 intervention and comparison sites at endline (n = 412) to estimate the intervention’s impact.

Results

Significantly (p < .05) more health workers in the CSC intervention areas compared to control areas reported responsibility for antenatal care, comprehensive antenatal care counseling, recording of the number of pregnant and postpartum women seen each month, and the average age of their last family planning client was younger. In addition, marginally significantly (p < .10) more health workers in treatment versus control areas report visiting women at their home at least once during their pregnancy. However, health worker-reported responsibility for HIV testing was significantly lower in intervention areas than in control.

Conclusions

The CSC aims to empower health workers to collaborate with the community and rest of the health system to identify and overcome the diverse and context-specific range of performance barriers they face. In doing so, it aims to support them to demand and ensure quality care for themselves from the health system so they can, in turn, deliver quality services to clients. Our results contribute to the evidence that the CSC may hold promise at improving service provision. While there is increasing evidence that collaborative social accountability approaches like the CSC are effective means to improving reproductive health-related service provision and outcomes in low-resource settings, additional research is needed.

Promoting women's and children's health through community groups in low-income and middle-income countries: a mixed-methods systematic review of mechanisms, enablers and barriers

INTRODUCTION

Community mobilisation through group activities has been used to improve women's and children's health in a range of low-income and middle-income contexts, but the mechanisms through which it works deserve greater consideration. We did a mixed-methods systematic review of mechanisms, enablers and barriers to the promotion of women's and children's health in community mobilisation interventions.

METHODS

We searched for theoretical and empirical peer-reviewed articles between January 2000 and November 2018. First, we extracted and collated proposed mechanisms, enablers and barriers into categories. Second, we extracted and synthesised evidence for them using narrative synthesis. We assessed risk of bias with adapted Downs and Black and Critical Appraisal Skills Programme checklists. We assigned confidence grades to each proposed mechanism, enabler and barrier.

RESULTS

78 articles met the inclusion criteria, of which 39 described interventions based on a participatory group education model, 19 described community-led structural interventions to promote sexual health in marginalised populations and 20 concerned other types of intervention or multiple interventions at once. We did not have high confidence in any mechanism, enabler or barrier. Two out of 15 proposed mechanisms and 10 out of 12 proposed enablers and barriers reached medium confidence. A few studies provided direct evidence relating proposed mechanisms, enablers or barriers to health behaviours or health outcomes. Only two studies presented mediation or interaction analysis for a proposed mechanism, enabler or barrier.

CONCLUSION

We uncovered multiple proposed mechanisms, enablers and barriers to health promotion through community groups, but much work remains to provide a robust evidence base for proposed mechanisms, enablers and barriers.

PROSPERO REGISTRATION NUMBER

CRD42018093695.