Rapid-Cycle Evaluation in an Early Intervention Program for Children With Developmental Disabilities in South India: Optimizing Service Providers' Quality of Work-Life, Family Program Engagement, and School Enrollment

Monitoring, evaluation, and learning: the key to building effective partnerships with government to improve maternal and child health in the Rakai and Kyotera Districts of Uganda

This article emphasizes the significance of the Monitoring, Evaluation, and Learning (MEL) system within Babies and Mothers Alive (BAMA) Foundation in building effective sustainable interventions at scale. The foundation aims to enhance the availability of high-quality reproductive, maternal, and newborn care services within the government health sector. The distinguishing characteristic of the MEL system is its integration of organizational learning as a strategic approach to inform the development of dynamic program designs. To do this, it has been necessary to identify crucial requirements through open data exchange with all pertinent stakeholders. This paper demonstrates that our approach to evidence-based learning in a diverse population of locally-based actors and stakeholders, gives voice to the community-based health practitioners and patients that is necessary for transformative maternal health delivery systems. The act of sharing data has presented several possibilities for enhancing current initiatives and extending the reach and scale of our partnership model. We trace the development of the core components of learning and decision making, and reflect on the transition of the program to scale using the LADDERS paradigm. The application of our model of practice has been associated with the increased financially viability and the potential for the sustainable scaling of the program intervention.

A capacity-building intervention for parents of children with disabilities in rural South India

BACKGROUND

A disability service organization piloted a parent capacity-building intervention for parents of children with developmental disabilities in rural South India.

AIMS

To examine the impact of this parent capacity-building group intervention on parent empowerment, peer support, social inclusion, advocacy skills, and the factors influencing these outcomes.

METHODS AND PROCEDURES

A qualitative study recruited 37 participants from 17 parent groups through convenience sampling. Six focus groups occurred six months post the start of this intervention and explored these parents' experiences with the groups and their perceived impacts.

OUTCOMES AND RESULTS

Qualitative analysis yielded four main themes and corresponding sub-themes: peer support (feeling peaceful and supported, building self-efficacy), social inclusion (space to share, acceptance and identity), knowledge sharing (increasing awareness, accepting advice), and advocacy (building confidence, making requests). Participation in parent groups provided participants with opportunities to find peer support, utilize knowledge sharing, feel socially included, and build advocacy skills.

CONCLUSIONS AND IMPLICATIONS

This study provides a greater understanding of the impact and benefits of parent capacity-building interventions in low-resource settings. Results will improve this intervention and guide other organizations in creating similar programs.

WHAT THIS PAPER ADDS

This paper adds perspectives of parents who are caregivers of children with developmental disabilities in rural South India. More specifically, this paper highlights the impact of a parent-focused intervention in group settings in both rural and urban areas. Parent support group interventions are not common in India and low- and middle-income countries and there is a paucity of literature describing these interventions and their impact. In this study, parents described a need for multifaceted interventions to support their child in the community. These parent groups provided space for caregivers of children with developmental disabilities to make connections, feel validated, and grow confidence to utilize new knowledge or advice shared by other parents to further enhance their lives. Further, parent-group settings provide opportunities to build advocacy skills through timely discussions. Parents reported an increase in feelings of acceptance within the group itself but not within the wider community. This highlights the need for involving the wider community in interventions to promote integration for families with children with disabilities.

Leveraging monitoring, evaluation, and learning to scale the Enabling Inclusion® program for children with disabilities in India and globally

Introduction

Children with disabilities in low- and middle-income countries face many challenges and lack adequate services, including access to rehabilitation professionals. To address this lack of access, Amar Seva Sangam Ayikudy (ASSA), a non-governmental organization (NGO) in India, designed a technology-leveraged rehabilitation program called Enabling Inclusion® (EI®), and implemented it in one state (Tamil Nadu, India) before scaling it. The model is supported by the EI® app, which enables organizations to screen, assess and monitor progress of children with disabilities via rehabilitation specialists and community rehabilitation workers, and to provide family-centered, goal-based interventions. An extensive monitoring, evaluation, and learning (MEL) framework is embedded into the program. This paper explores how this MEL system supported the scaling of the EI® model, reaching additional beneficiaries nationally and globally.

Methods

This paper describes ASSA's MEL framework and demonstrates its use for decision-making in the process of scaling. It also explores how collaborations with various government departments, NGOs, and private partners contributed to the scaling of the EI® model and technology.

Results

Scaling of the EI® program was achieved by (1) expansion of the program in rural Tamil Nadu (vertical scale-up) in partnership with the Tamil Nadu government and private partners, and (2) by licensing the EI® app and model to other NGOs in various states in India and globally (horizontal scale-up). Systematic examination of key program and performance indicators, as well as stakeholder feedback, informed decisions to modify the EI® app over time. This included further customizing to the needs of children and service providers, covering a greater range of age groups and contexts, and modifying service delivery models. Child functional independence, participation, and inclusion was further strengthened by mobilizing parent empowerment groups, community awareness programs, school advocacy, and entitlements from the government. Flexibility in the implementation model of the EI® app allowed for adaptation to local contexts and organizations, and facilitated its scale-up.

Conclusion

A dynamic, inclusive, and locally grounded MEL system, a flexible and collaborative approach, and an adaptive implementation model increased the accessibility of an early intervention and childhood rehabilitation program for children with disabilities and their families throughout the state of Tamil Nadu, across India, and internationally.

Design-redesign, implementation, and evaluation of effectiveness of maternal nutrition and responsive parenting program on child development at 2 years of age from rural India: a cluster RCT

Background

To promote early childhood development (ECD), we require information not only on what needs to be addressed and on what effects can be achieved but also on effective delivery methods that can be adapted to local context. We describe design, implementation, and evaluation of a complex intervention to strengthen nurturing environment for young children.

Methods

Study participants were pregnant women and their children from birth to 2 years. We used design and redesign, implementation, and evaluation approaches for the study. We co-created curriculum and delivery plan with stakeholders, based on the theoretical framework, findings from formative research, and our preliminary work. We recruited 656 pregnant women and newborns, 326 (49.69%) from intervention and 330 (50.30%) from the control group. We conducted a cluster randomized controlled trial to evaluate the program's effectiveness. The outcomes of children were assessed at 12 and 24 months.

Findings

At recruitment, study participants from both the study arms were similar in sociodemographic characteristics. We conducted 6,665 home visits, 25 toy-making workshops, and 65 caregiver-meetings. The initial examination of program data revealed gaps in quality and coverage of interventions. The intervention was redesigned based on feedback from stakeholders in community meetings. At recruitment, participants in both study groups had similar socio-demographics. We conducted 6,665 home visits, 25 toy workshops, and 65 caregiver meetings. Initial program data showed intervention quality and coverage gaps, leading to a redesign program based on community and stakeholder feedback. Post-re-designing, session quality improved, with program coverage rising from 32 to 98%. Male participation in home visits increased from 4.3 to 32.65%, and data errors reduced from 270 to 140 per month on average. At 24 months, program showed moderate-mild impact on ECD - cognitive (0.31, 95%CI: 0.13-0.48), language (0.2, 95%CI: 0.01-0.39), and socioemotional-development (0.19, 95%CI: 0.01-0.37), moderate effect on home-environment and mother-child interaction. 96% of women initiated breastfeed within one-hour of delivery, and exclusive-breastfeeding rate of 89.80%.

Interpretations

The study provides an evidence-based community centered ECD curriculum and implementation strategies to enhance service providers, and caregivers' knowledge and skills for promoting ECD in low-resource settings with the potential to scale within existing Government Program.

Funding

The trial was funded by the Saving Brains Round 5 Initiative of Grand Challenges Canada (Grant no. SB-1707-05084), and we are grateful for their ongoing support through online sessions and orientation workshops. The trial was also supported by the Indian Council of Medical Research (File No: 5/7/1693/CH/Adhoc/RBMCH-2020).

Measurement for Change: Reflections from innovators' experiences with monitoring, evaluation, and learning systems for Early Childhood Development

In this review paper, we explore how on-the-ground Early Childhood Development (ECD) innovators are using monitoring, evaluation, and learning (MEL) systems to guide the design and implementation of ECD programs, as well as how MEL systems can influence policy and support the achievement of impact at scale. We reflect on articles in the Frontiers series “Effective delivery of integrated interventions in early childhood: innovations in evidence use, monitoring, evaluation, and learning.” The 31 contributions to the series reflect the breadth and depth of complexity that characterizes ECD, including global geographic spread, with studies from Asia, Europe, Africa, and Latin America and the Caribbean. Our synthesis finds that integrating MEL processes and systems into the fabric of a program or policy initiative can broaden the underlying value proposition. Specifically, ECD organizations sought to design their MEL systems to ensure programs fit the values, goals, experiences and conceptual frameworks of diverse stakeholders, so that participating makes sense to all. For example, formative, exploratory research identified the priorities and needs of the target population and frontline service providers, and informed the content and delivery of an intervention. ECD organizations also designed their MEL systems to support a shift of accountability toward broader ownership: They included delivery agents and program participants alike as subjects rather than objects, through active participation in data collection, and by providing opportunities for equitable discussion of results and decision-making. Programs collected data to respond to specialized characteristics, priorities and needs, embedding program activities into existing day-to-day routines. Further, papers pointed to the importance of intentionally involving a variety of stakeholders in national and international dialogues to ensure that diverse ECD data collection efforts are aligned and multiple perspectives are considered in the development of national ECD policies. And, several papers illustrate the value of creative methods and measurement tools to integrate MEL into a program or policy initiative. Finally, our synthesis concludes that these findings align with the five aspirations that were formulated as part of the Measurement for Change dialogue, which motivated the launch of the series.

Rapid-cycle evaluation and learning for the effective delivery of integrated interventions in early childhood in rural India

Background

Effective and real-time data analytics plays an essential role in understanding gaps and improving the quality and coverage of complex public health interventions. Studies of public health information systems identify problems with data quality, such as incomplete records and untimely reporting. Effective data collection and real-time analysis systems for rapid-cycle learning are necessary to monitor public health programs and take timely evidence-based decisions. Early childhood development (ECD) programs are very diverse. Rapid-cycle evaluation and learning (REAL) guides the implementation process of such complex interventions in real time. Stepping stones was one such early childhood development program implemented in Central India.

Objective

The objective was to improve the delivery of complex, integrated public health interventions for early childhood development in remote areas of rural India.

Methodology

The program was developed according to the principles of inclusion and community-centeredness, which can be tested quickly and iteratively. To enhance the decision-making process and improve delivery and coverage, the core team implemented an information system for rapid-cycle learning. We developed performance indicators and a performance measurement matrix after defining the specific needs. Following that, we trained staff to collect complete data using electronic data collection tools and transfer it the same day to the server for quality review and further analysis. A variety of data/information was triangulated to address the gaps in intervention delivery, and those decisions were subsequently implemented.

Results

We observed that the quality of data collection improved, and errors were reduced by 50% in the third quarter. The quality of the narrative was also enhanced; it became more elaborate and reflective. Sharing their field output in meetings and improving the quality of the narrative enhanced the self-reflection skills of field staff and consequently improved the quality of the intervention delivery. Refresher training and mentoring by supervisors helped to improve the data quality over time.

Conclusion

Rapid-cycle evaluation and learning (REAL) can be implemented in resource-limited settings to improve the quality and coverage of integrated intervention in early childhood. It nurtures a reinforcing ecosystem that integrates providers, community, and family perspectives and guides interactions among stakeholders by integrating data from all available sources.