An Impact Mapping Method to Generate Robust Qualitative Evaluation of Community-Based Research Programs for Youth and Adults

Ripple effects mapping: capturing the wider impacts of systems change efforts in public health

BACKGROUND

Systems approaches are currently being advocated and implemented to address complex challenges in Public Health. These approaches work by bringing multi-sectoral stakeholders together to develop a collective understanding of the system, and then to identify places where they can leverage change across the system. Systems approaches are unpredictable, where cause-and-effect cannot always be disentangled, and unintended consequences - positive and negative - frequently arise. Evaluating such approaches is difficult and new methods are warranted.

METHODS

Ripple Effects Mapping (REM) is a qualitative method which can capture the wider impacts, and adaptive nature, of a systems approach. Using a case study example from the evaluation of a physical activity-orientated systems approach in Gloucestershire, we: a) introduce the adapted REM method; b) describe how REM was applied in the example; c) explain how REM outputs were analysed; d) provide examples of how REM outputs were used; and e) describe the strengths, limitations, and future uses of REM based on our reflections.

RESULTS

Ripple Effects Mapping is a participatory method that requires the active input of programme stakeholders in data gathering workshops. It produces visual outputs (i.e., maps) of the programme activities and impacts, which are mapped along a timeline to understand the temporal dimension of systems change efforts. The REM outputs from our example were created over several iterations, with data collected every 3-4 months, to build a picture of activities and impacts that have continued or ceased. Workshops took place both in person and online. An inductive content analysis was undertaken to describe and quantify the patterns within the REM outputs. Detailed guidance related to the preparation, delivery, and analysis of REM are included in this paper.

CONCLUSION

REM may help to advance our understanding and evaluation of complex systems approaches, especially within the field of Public Health. We therefore invite other researchers, practitioners and policymakers to use REM and continuously evolve the method to enhance its application and practical utility.

Supporting Expectant and Parenting Teens: New Evidence to Inform Future Programming and Research

Until recently, federal programs had not explicitly focused on improving the outcomes of highly vulnerable teen parents. Established in 2010, the Pregnancy Assistance Fund (PAF) aims to improve the health, social, educational, and economic outcomes for expectant and parenting teens and young adults, their children, and their families, through providing grants to states and tribes. This article introduces the Maternal and Child Health Journal supplement “Supporting Expectant and Parenting Teens: The Pregnancy Assistance Fund,” which draws together the perspectives of researchers and practitioners to provide insights into serving expectant and parenting teens through the PAF program. The articles in the supplement include examples of programs that use different intervention strategies to support teen parents, with programs based in high school, college, and community settings in both urban and rural locations. Some of the articles provide rigorous evidence of what works to support teen parents. In addition, the articles demonstrate key lessons learned from implementation, including allowing some flexibility in implementation while clearly outlining core programmatic components, using partnerships to meet the multifaceted needs of young parents, hiring the right staff and providing extensive training, using strategies for engaging and recruiting teen parents, and planning for sustainability early. The studies use a range of qualitative and quantitative methods to evaluate programs to support teen parents, and three articles describe how to implement innovative and cost effective methods to evaluate these kinds of programs. By summarizing findings across the supplement, we increase understanding of what is known about serving expectant and parenting teens and point to next steps for future research.

The Use of Ripple Effect Mapping to Understand Successes of the SC Pregnancy Assistance Fund: A Participatory Evaluation Approach

OBJECTIVES

The South Carolina Pregnancy Assistance Fund (SCPAF) funded four counties to increase the amount, quality, and awareness of services for young parents; increase educational attainment among expectant and parenting youth; reduce the number of repeat teen pregnancies among youth; and improve parenting skills. The purpose of this paper is twofold: (1) to describe our application of the Ripple Effect Mapping (REM) technique as an innovative evaluation strategy to gather perspectives from SCPAF stakeholders and (2) to share key findings generated by participants in REM sessions on the perceived success of local SCPAF community collaboratives.

METHODS

REM, an innovative evaluation strategy, was used to gather perspectives from SCPAF stakeholders. Five REM sessions were conducted with 52 participants. REM sessions included partner interviews and collective development of visual maps to illustrate stakeholder perspectives of program successes. Visual maps, as well as transcripts of discussions, were analyzed using an inductive approach.

RESULTS

Stakeholders reported that the connections to resources, supports, and services provided through SCPAF had the potential to alter the life trajectories of expectant and parenting teens (EPT). Stakeholders also described that SCPAF fostered growth in collaboration among partners and reduced duplication of services in funded communities CONCLUSIONS FOR PRACTICE: This paper describes how an innovative evaluation strategy was used to provide a space for stakeholders to dialogue, synthesize their experiences, and construct a collective narrative of key program successes. This paper also illustrates how such approaches can be applied to complex community initiatives.

Development of the iCook 4-H Curriculum for Youth and Adults: Cooking, Eating, and Playing Together for Childhood Obesity Prevention

The objective was to describe the development process of a curriculum (iCook 4-H) targeted to low-income, rural, and/or diverse youths and their adult primary meal preparer to promote cooking, eating, and playing together. Lessons learned highlighted the importance of grounding the curriculum in Social Cognitive Theory and applying the experiential 4-H learning model using a multiphased, community-based participatory approach with cyclical development and evaluation, and key modifications made for dissemination and distribution. Findings across 4 testing phases over 6 years and 5 states demonstrated the time-intensive, cyclical process that required flexibility with fidelity to form a hands-on, interactive curriculum.

Ripple Effect Mapping Outcomes of a Childhood Obesity Prevention Program From Youth and Adult Dyads Using a Qualitative Approach: iCook 4-H

OBJECTIVE

To describe the impact of the iCook 4-H intervention study based on data gathered through ripple effect mapping focus groups through an explorative approach.

DESIGN

Youth-adult dyads responded about ways in which iCook had affected the individual, family, and community. Three questions were asked: (1) What were people doing differently as a result of iCook? (2) Who benefited from iCook and how? (3) Were there changes in the way community groups and institutions did things as a result of iCook?

SETTING

Ripple effect mapping sessions took place across 5 states (Maine, Nebraska, South Dakota, Tennessee, and West Virginia).

PARTICIPANTS

Seventy dyad participants (n = 35 youth, n = 35 adults) from the iCook 4-H intervention.

MAIN OUTCOME MEASURE

Three core themes of iCook 4-H were assessed: cooking, eating, and playing together.

ANALYSIS

Direct content analysis and word frequencies were used.

RESULTS

Seven categories emerged: improved health, increased community involvement, increased knowledge, increased communication, changed motivation, financial mindfulness, and increased appreciation for family. An overarching theme that was determined was that learning new skills together through trying new things (cooking, eating, and playing) leads to positive individual family and community change.

CONCLUSIONS AND IMPLICATIONS

Ripple effect mapping was effective in determining the perceived impact of iCook 4-H on oneself, family, and community.

Dissemination Using Infographic Reports Depicting Program Impact of a Community-Based Research Program: eB4CAST in iCook 4-H

OBJECTIVE

To evaluate barriers to dissemination and implementation and perceptions of the Evidence-Based Forecast Capture, Assemble, Sustain, Timelessness (eB4CAST) dissemination infographic tool as part of iCook 4-H dissemination.

DESIGN

Online surveys and phone interviews.

PARTICIPANTS

Experts (n = 35) in community research completed the survey; 13 completed the interview.

MAIN OUTCOMES MEASURE

Experts' perceptions of eB4CAST reports used for iCook 4-H dissemination.

ANALYSIS

Frequency and thematic analysis.

RESULTS

Survey respondents agreed (85%) that the eB4CAST infographic provided a clear understanding of iCook 4-H and relevant information (83%). Statistics included in the infographic were reported as easily understood (66%). Respondents (83%) stated that the infographic would be helpful to share outcomes with stakeholders. Thematic analysis showed that the majority of interviewees agreed that eB4CAST infographics might aid in overcoming barriers to dissemination and implementation including communication and community ownership.

CONCLUSIONS AND IMPLICATIONS

This study provides perceptions from experts regarding the value of using eB4CAST infographics as a tool to disseminate the impact of a community nutrition program.

eB4CAST: An Evidence-Based Tool to Promote Dissemination and Implementation in Community-Based, Public Health Research

eB4CAST, evidence-Based forecast C-capture, A-assemble, S-sustain, T-timelessness (eB4CAST), framework was developed from existing dissemination and implementation (D & I) constructs as a dissemination tool to promote community-based program usability and future application in targeted populations. eB4CAST captures and transforms research findings into a dissemination report that shows program need and impact to endorse program continuation and expansion. This is achieved through direct and indirect data collection of community factors and program impact that can showcase the need for program sustainability and potential for future dissemination sites. Testimonials, individual feedback, and program process and outcomes contribute to the direct data while data collected from census, county, and state databases and reports allow for indirect information to be captured and analyzed. Capturing data in the two levels allow eB4CAST to forecast program need and highlight program impact through a footprint. eB4CAST framework for dissemination tool creation is organized into four sections: Capture, Assemble, Sustainability, and Timelessness. Capture encompasses the collection of indirect and direct data related to intervention goals. Assemble is the compilation of the data into a visually appealing and easily understood media. Sustainability encourages the use of dissemination tools to provide forecast of program need and footprint of program impact back to community participants, program leaders, and key stakeholders to endorse program sustainability. Lastly, timelessness encourages cyclic movement through these constructs to continue program monitoring and data sharing to ensure timeless program evaluation and conformation to change in needs. The eB4CAST framework provides a systematic method to capture justification of program need and impact of community-based research that can be modified to fit diverse public health interventions providing a necessary D & I tool.