The adaptive nature of implementation practice: case study of a school-based nutrition education intervention

Promoting healthy school food environments and nutrition in Canada: a systematic review of interventions, policies, and programs

CONTEXT

The school food environment is a critical interface for child and adolescent nutrition, and there is a need to understand existing literature on Canadian school food environments to identify equity gaps and opportunities, and empower decision-makers to plan for future action.

OBJECTIVE

Literature on Canadian school food and nutrition interventions, policies, programs, and their effects on diets and nutritional status are synthesized and appraised in this systematic review.

DATA SOURCES

A search strategy was developed for each database used (Medline, Embase, PsycINFO, ERIC, Cochrane Collaboration, Canadian Electronic Library, BiblioMap), with a combination of free text and controlled vocabulary, for articles published from 1990 to 2021. Unpublished data and grey literature were also searched.

DATA EXTRACTION

Quantitative and qualitative studies with an observational or intervention study design, reviews, or program evaluations conducted in Canadian schools with participants aged 5-19.9 years were included. Key study characteristics and risk of bias were extracted independently by 2 investigators using a standardized tool.

DATA ANALYSIS

A total of 298 articles were included (n = 192 peer reviewed and 106 from the grey literature), which were mostly conducted in Ontario (n = 52), British Columbia (n = 43), and Nova Scotia (n = 28). Twenty-four interventions, 5 nonevaluated programs, and 1 policy involved Indigenous populations. Overall, 86 articles measured and reported on effectiveness outcomes, including dietary intake; anthropometry; knowledge, attitudes, and practices; and physical activity. The literature remains largely heterogenous and primarily focused on nutrition education programs that use subjective assessments to infer changes in nutrition. A key facilitator to implementation and sustainability was community engagement, whereas key barriers were staff capacity, access to resources and funding, and consistent leadership.

CONCLUSIONS

This review provides insight into Canadian school food and nutrition interventions, programs, and policies and uncovers important evidence gaps that require careful examination for future evaluations. Governments must create supportive environments that optimize nutrition for children and adolescents through equitable policies and programs.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO registration no. CRD42022303255.

Application of an Implementation Framework Using Mixed Methods in Preschool Settings

Research Findings. Application of mixed methods in a dissemination and implementation framework can give researchers a better understanding of the reach and delivery of early childhood obesity prevention programs in preschool settings, as well as potential facilitators and barriers related to implementation and sustainability. This study utilized a simultaneous, exploratory, mixed-methods design to investigate individual-, organizational-, and intervention-level factors that were related to the implementation and sustainability of policy, system, and environment (PSE) changes as part of a larger obesity prevention program in a randomly selected sample of preschool centers (n = 20). Individual-level factors, and specifically the attitudes and skills of preschool providers, were identified in both the qualitative and quantitative data as important factors related to the sustainability of PSE changes (r = .56, p < .01). Staff and leadership engagement and adaptability of the program were also identified as important factors related to the implementation and sustainability of PSE changes. Practice or Policy. These findings highlight the complexity of implementation success and suggest PSE obesity prevention interventions in preschool centers require the consideration of numerous, multilevel factors to ensure programming is impactful and sustained over time.

To Effectively Address Complex Healthcare Problems, Continuing Professional Development Must Evolve

ABSTRACT

Continuing professional development aims to provide health professionals with the knowledge, skills, and competencies needed to improve care. Physicians and other clinicians increasingly practice within complex health care delivery organizations aiming to improve the care of populations of patients with multiple problems and differing needs. These organizations are composed of local units in different departments and venues; these teams and the patients they care for change over time. Improving outcomes within constantly changing complex organizations delivering population care takes time and persistence. It takes time to equip critical masses of clinicians and other personnel with knowledge and skills to effect change. Although some changes might be simple, those involving new workflows require implementation support. Not all change will be smooth; individuals need opportunities to learn from and adjust their early intervention efforts, measure effectiveness of change, and sustain successful practices. Longitudinal support is necessary to affect change over complex organizations. This essay proposes that to be more supportive and valuable to health care delivery organizations, continuing professional development needs to intentionally participate in longitudinal, collaborative, context-specific, team-based interventions. An expanded menu of evaluation approaches will better describe the role of continuing professional development in helping health care professionals and organizations address increasingly complex health care delivery problems and improve patient and population outcomes. Selected concepts to achieve these ends are introduced at a high level in this article. Readers are invited to explore concepts that resonate with their current situation in further detail.

Realist evaluation of health promotion interventions: a scoping review

This scoping review aims to give a narrative account of existing realist evaluation practices in health promotion. Realist evaluations of health promotion interventions published between 2010 and 2021 were identified by searching five academic databases: Embase, Pubmed, PsycINFO, ScienceDirect and Scopus. A data-charting form was created based on the characteristics of realist evaluation and four core features of an approach appropriate for evaluating health promotion interventions. Seventeen articles met the inclusion criteria. These were classified into two types of studies: those aiming to build an initial program theory and those aiming to test an initial program theory. Our results revealed a great variety of realist evaluation practices and uncovered a growing interest in realist evaluation over the years. Our searches identified a lack of participative practice and capacity-building intention. Our examination of the data collection and analysis methods points to some common practices in using multi-methods. Perspectives on realist evaluation practices and on assessing the effectiveness of health promotion have been identified.

Implementation Factors and Teacher Experience of the Integrated Nutrition Education Program: A Mixed Methods Program Evaluation

BACKGROUND

School-based programs are widely implemented to address childhood obesity. Despite the promise of these programs, evidence on their effectiveness is mixed. Adopting a dissemination and implementation (D&I) science focus utilizing mixed methods can provide a broader understanding and more robust details about these programs. The goal of this evaluation is to understand how implementation factors and teacher experience influence implementation success and outcomes of the Integrated Nutrition Education Program (INEP), an elementary school-based nutrition program, using a mixed-methods design.

METHODS

Reach, effectiveness, adoption, implementation, maintenance (RE-AIM) framework guided the development of the evaluation and multiple methods were deployed. Hierarchical linear regression was used to assess the association between D&I construct variables within levels of influence and teacher perception of INEP impact from a quantitative end-of-year teacher survey. Follow-up qualitative interviews with teachers were analyzed using constant comparison analysis.

RESULTS

Workload and burden emerged as significant factors related to implementation in the quantitative analysis. The follow-up qualitative data collection identified other factors teachers found important to the adoption, implementation, and maintenance of INEP.

CONCLUSION

Results of this evaluation can be used to inform program improvement efforts for INEP and provide information on ways to promote reach, effectiveness, adoption, implementation, and maintenance of similar school-based health promotion programs.

Do the implementation processes of a school-based daily physical activity (DPA) program vary according to the socioeconomic context of the schools? a realist evaluation of the Active at school program

Background

Less than half of Canadian children meet the Canadian Physical Activity (PA) Guidelines, and the proportion is even lower among children living in underprivileged neighbourhoods. Regular PA supports physical, cognitive, and psychological/social health among school-aged children. Successful implementation of school-based daily physical activity (DPA) programs is therefore important for all children and crucial for children who attend schools in lower socioeconomic settings. The purpose of this study is to uncover what worked, for whom, how, and why during the three-year implementation period of a new “flexible” DPA program, while paying particular attention to the socioeconomic setting of the participating schools.

Methods

This study is a realist evaluation using mixed methods for data generation. Longitudinal data were collected in 415 schools once a year during the three-year implementation period of the program using questionnaires. Data analysis was completed in three steps and included qualitative thematic analysis using a mixed inductive and deductive method and chi-square tests to test and refine context-mechanism-outcome (CMO) configurations.

Results

Giving the school teams autonomy in the choice of strategies appropriate to their context have allowed schools to take ownership of program implementation by activating a community empowerment process, which resulted in a cultural shift towards a sustainable DPA provision in most settings. In rural underprivileged settings, the mobilization of local resources seems to have successfully created the conditions necessary for implementing and maintaining changes in practice. In disadvantaged urban settings, implementing local leadership structures (leader, committee, and meetings) provided pivotal assistance to members of the school teams in providing new DPA opportunities. However, without continued external funding, those schools seem unable to support local leadership structures on their own, jeopardizing the sustainability of the program for children living in disadvantaged urban areas.

Conclusion

By exploring CMO configurations, we have been able to better understand what worked, for whom, how and why during the three-year implementation period of the Active at School! program. When implementing DPA policies, decision makers should consider adjusting resource allocations to meet the actual needs of schools from different backgrounds to promote equal PA opportunities for all children.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-12797-7.

Understanding the experience of the implementer: teachers' perspectives on implementing a classroom-based nutrition education program

School-based programs are widely implemented to combat childhood obesity, but these programs have mixed results. Dissemination and implementation science approaches to evaluation using qualitative methods can provide more robust details about program functioning that may be able to help explain the variation in the impact of these programs. Fourteen in-depth interviews were conducted with classroom teachers implementing a school-based program, the Integrated Nutrition Education Program (INEP), to explore their experience. Factors related to organization, individual and intervention levels emerged as facilitators and barriers to program implementation. Key factors were school culture at the organization level, individual perception and belief in the intervention at the individual level and program content, perceived complexity and adaptability at the intervention level. Socioeconomic status of the community and family involvement were contextual factors identified across all levels. Findings from this qualitative evaluation can be used for the quality improvement of INEP, but beyond this these can also be informative for other school-based programs to promote adoption, implementation and maintenance.

Development of a Dissemination and Implementation Framework for an Early Childhood Obesity Prevention Program

Dissemination and implementation (D&I) science addresses the disconnect between evidence-based research and practical application in community settings. Early childhood education settings are ideal for the application of D&I research because of their widespread use for implementing health promotion interventions. A D&I framework was applied to the Culture of Wellness in Preschools program, a comprehensive early childhood obesity prevention program. The development and application of the Culture of Wellness in Preschools D&I framework can lead to a more comprehensive approach to program evaluation and quality improvement and can contribute more broadly to the body of evidence of nutrition-related health promotion programs.

Advancing complexity science in healthcare research: the logic of logic models

Background

Logic models are commonly used in evaluations to represent the causal processes through which interventions produce outcomes, yet significant debate is currently taking place over whether they can describe complex interventions which adapt to context. This paper assesses the logic models used in healthcare research from a complexity perspective. A typology of existing logic models is proposed, as well as a formal methodology for deriving more flexible and dynamic logic models.

Analysis

Various logic model types were tested as part of an evaluation of a complex Patient Experience Toolkit (PET) intervention, developed and implemented through action research across six hospital wards/departments in the English NHS. Three dominant types of logic model were identified, each with certain strengths but ultimately unable to accurately capture the dynamics of PET. Hence, a fourth logic model type was developed to express how success hinges on the adaption of PET to its delivery settings. Aspects of the Promoting Action on Research Implementation in Health Services (PARIHS) model were incorporated into a traditional logic model structure to create a dynamic “type 4” logic model that can accommodate complex interventions taking on a different form in different settings.

Conclusion

Logic models can be used to model complex interventions that adapt to context but more flexible and dynamic models are required. An implication of this is that how logic models are used in healthcare research may have to change. Using logic models to forge consensus among stakeholders and/or provide precise guidance across different settings will be inappropriate in the case of complex interventions that adapt to context. Instead, logic models for complex interventions may be targeted at facilitators to enable them to prospectively assess the settings they will be working in and to develop context-sensitive facilitation strategies. Researchers should be clear as to why they are using a logic model and experiment with different models to ensure they have the correct type.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0701-4) contains supplementary material, which is available to authorized users.

Addressing complexity in population health intervention research: the context/intervention interface

Background

Public health interventions are increasingly being recognised as complex and context dependent. Related to this is the need for a systemic and dynamic conception of interventions that raises the question of delineating the scope and contours of interventions in complex systems. This means identifying which elements belong to the intervention (and therefore participate in its effects and can be transferred), which ones belong to the context and interact with the former to influence results (and therefore must be taken into account when transferring the intervention) and which contextual elements are irrelevant to the intervention.

Discussion

This paper, from which derives criteria based on a network framework, operationalises how the context and intervention systems interact and identify what needs to be replicated as interventions are implemented in different contexts. Representing interventions as networks (composed of human and non-human entities), we introduce the idea that the density of interconnections among the various entities provides a criterion for distinguishing core intervention from intervention context without disconnecting the two systems. This differentiates endogenous and exogenous intervention contexts and the mediators that connect them, which form the fuzzy and constantly changing intervention/context interface.

Conclusion

We propose that a network framework representing intervention/context systems constitutes a promising approach for deriving empirical criteria to delineate the scope and contour of what is replicable in an intervention. This approach should allow better identification and description of the entities that have to be transferred to ensure the potential effectiveness of an intervention in a specific context.

An application of Extended Normalisation Process Theory in a randomised controlled trial of a complex social intervention: Process evaluation of the Strengthening Families Programme (10-14) in Wales, UK

Purpose

Process evaluations generate important data on the extent to which interventions are delivered as intended. However, the tendency to focus only on assessment of pre-specified structural aspects of fidelity has been criticised for paying insufficient attention to implementation processes and how intervention-context interactions influence programme delivery. This paper reports findings from a process evaluation nested within a randomised controlled trial of the Strengthening Families Programme 10–14 (SFP 10–14) in Wales, UK. It uses Extended Normalisation Process Theory to theorise how interaction between SFP 10–14 and local delivery systems - particularly practitioner commitment/capability and organisational capacity - influenced delivery of intended programme activities: fidelity (adherence to SFP 10–14 content and implementation requirements); dose delivered; dose received (participant engagement); participant recruitment and reach (intervention attendance).

Methods

A mixed methods design was utilised. Fidelity assessment sheets (completed by practitioners), structured observation by researchers, and routine data were used to assess: adherence to programme content; staffing numbers and consistency; recruitment/retention; and group size and composition. Interviews with practitioners explored implementation processes and context.

Results

Adherence to programme content was high - with some variation, linked to practitioner commitment to, and understanding of, the intervention’s content and mechanisms. Variation in adherence rates was associated with the extent to which multi-agency delivery team planning meetings were held. Recruitment challenges meant that targets for group size/composition were not always met, but did not affect adherence levels or family engagement. Targets for staffing numbers and consistency were achieved, though capacity within multi-agency networks reduced over time.

Conclusions

Extended Normalisation Process Theory provided a useful framework for assessing implementation and explaining variation by examining intervention-context interactions. Findings highlight the need for process evaluations to consider both the structural and process components of implementation to explain whether programme activities are delivered as intended and why.

•

Uses Extended Normalisation Process Theory to theorise intervention implementation.

•

The Strengthening Families Programme was delivered with good overall fidelity.

•

Adherence to planned content and staffing guidelines was high, with some variation.

•

Recruitment of families was challenging and affected group size/composition

•

Intervention-context interactions shaped variation in fidelity and implementation.

Implementation of a Cooking Bus intervention to support cooking in schools in Wales, UK

Purpose

The teaching of cooking is an important aspect of school-based efforts to promote healthy diets among children, and is frequently done by external agencies. Within a limited evidence base relating to cooking interventions in schools, there are important questions about how interventions are integrated within school settings. The purpose of this paper is to examine how a mobile classroom (Cooking Bus) sought to strengthen connections between schools and cooking, and drawing on the concept of the sociotechnical network, theorise the interactions between the Bus and school contexts.

Design/methodology/approach

Methods comprised a postal questionnaire to 76 schools which had received a Bus visit, and case studies of the Bus’ work in five schools, including a range of school sizes and urban/rural locations. Case studies comprised observation of Cooking Bus sessions, and interviews with school staff.

Findings

The Cooking Bus forged connections with schools through aligning intervention and schools’ goals, focussing on pupils’ cooking skills, training teachers and contributing to schools’ existing cooking-related activities. The Bus expanded its sociotechnical network through post-visit integration of cooking activities within schools, particularly teachers’ use of intervention cooking kits.

Research limitations/implications

The paper highlights the need for research on the long-term impacts of school cooking interventions, and better understanding of the interaction between interventions and school contexts.

Originality/value

This paper adds to the limited evidence base on school-based cooking interventions by theorising how cooking interventions relate to school settings, and how they may achieve integration.

IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis

Background

Exposure to domestic violence and abuse (DVA) during childhood and adolescence increases the risk of negative outcomes across the lifespan.

Objectives

To synthesise evidence on the clinical effectiveness, cost-effectiveness and acceptability of interventions for children exposed to DVA, with the aim of making recommendations for further research.

Design

(1) A systematic review of controlled trials of interventions; (2) a systematic review of qualitative studies of participant and professional experience of interventions; (3) a network meta-analysis (NMA) of controlled trials and cost-effectiveness analysis; (4) an overview of current UK provision of interventions; and (5) consultations with young people, parents, service providers and commissioners.

Settings

North America (11), the Netherlands (1) and Israel (1) for the systematic review of controlled trials of interventions; the USA (4) and the UK (1) for the systematic review of qualitative studies of participant and professional experience of interventions; and the UK for the overview of current UK provision of interventions and consultations with young people, parents, service providers and commissioners.

Participants

A total of 1345 children for the systematic review of controlled trials of interventions; 100 children, 202 parents and 39 professionals for the systematic review of qualitative studies of participant and professional experience of interventions; and 16 young people, six parents and 20 service providers and commissioners for the consultation with young people, parents, service providers and commissioners.

Interventions

Psychotherapeutic, advocacy, parenting skills and advocacy, psychoeducation, psychoeducation and advocacy, guided self-help.

Main outcome measures

Internalising symptoms and externalising behaviour, mood, depression symptoms and diagnosis, post-traumatic stress disorder symptoms and self-esteem for the systematic review of controlled trials of interventions and NMA; views about and experience of interventions for the systematic review of qualitative studies of participant and professional experience of interventions and consultations.

Data sources

MEDLINE, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, EMBASE, Cochrane Central Register of Controlled Trials, Science Citation Index, Applied Social Sciences Index and Abstracts, International Bibliography of the Social Sciences, Social Services Abstracts, Social Care Online, Sociological Abstracts, Social Science Citation Index, World Health Organization trials portal and clinicaltrials.gov.

Review methods

A narrative review; a NMA and incremental cost-effectiveness analysis; and a qualitative synthesis.

Results

The evidence base on targeted interventions was small, with limited settings and types of interventions; children were mostly < 14 years of age, and there was an absence of comparative studies. The interventions evaluated in trials were mostly psychotherapeutic and psychoeducational interventions delivered to the non-abusive parent and child, usually based on the child’s exposure to DVA (not specific clinical or broader social needs). Qualitative studies largely focused on psychoeducational interventions, some of which included the abusive parent. The evidence for clinical effectiveness was as follows: 11 trials reported improvements in behavioural or mental health outcomes, with modest effect sizes but significant heterogeneity and high or unclear risk of bias. Psychoeducational group-based interventions delivered to the child were found to be more effective for improving mental health outcomes than other types of intervention. Interventions delivered to (non-abusive) parents and to children were most likely to be effective for improving behavioural outcomes. However, there is a large degree of uncertainty around comparisons, particularly with regard to mental health outcomes. In terms of evidence of cost-effectiveness, there were no economic studies of interventions. Cost-effectiveness was modelled on the basis of the NMA, estimating differences between types of interventions. The outcomes measured in trials were largely confined to children’s mental health and behavioural symptoms and disorders, although stakeholders’ concepts of success were broader, suggesting that a broader range of outcomes should be measured in trials. Group-based psychoeducational interventions delivered to children and non-abusive parents in parallel were largely acceptable to all stakeholders. There is limited evidence for the acceptability of other types of intervention. In terms of the UK evidence base and service delivery landscape, there were no UK-based trials, few qualitative studies and little widespread service evaluation. Most programmes are group-based psychoeducational interventions. However, the funding crisis in the DVA sector is significantly undermining programme delivery.

Conclusions

The evidence base regarding the acceptability, clinical effectiveness and cost-effectiveness of interventions to improve outcomes for children exposed to DVA is underdeveloped. There is an urgent need for more high-quality studies, particularly trials, that are designed to produce actionable, generalisable findings that can be implemented in real-world settings and that can inform decisions about which interventions to commission and scale. We suggest that there is a need to pause the development of new interventions and to focus on the systematic evaluation of existing programmes. With regard to the UK, we have identified three types of programme that could be justifiably prioritised for further study: psycho-education delivered to mothers and children, or children alone; parent skills training in combination with advocacy: and interventions involving the abusive parent/caregiver. We also suggest that there is need for key stakeholders to come together to explicitly identify and address the structural, practical and cultural barriers that may have hampered the development of the UK evidence base to date.

Future work recommendations

There is a need for well-designed, well-conducted and well-reported UK-based randomised controlled trials with cost-effectiveness analyses and nested qualitative studies. Development of consensus in the field about core outcome data sets is required. There is a need for further exploration of the acceptability and effectiveness of interventions for specific groups of children and young people (i.e. based on ethnicity, age, trauma exposure and clinical profile). There is also a need for an investigation of the context in which interventions are delivered, including organisational setting and the broader community context, and the evaluation of qualities, qualifications and disciplines of personnel delivering interventions. We recommend prioritisation of psychoeducational interventions and parent skills training delivered in combination with advocacy in the next phase of trials, and exploratory trials of interventions that engage both the abusive and the non-abusive parent.

Study registration

This study is registered as PROSPERO CRD42013004348 and PROSPERO CRD420130043489.

Funding

The National Institute for Health Research Public Health Research programme.

Local policies to tackle a national problem: Comparative qualitative case studies of an English local authority alcohol availability intervention

Cumulative impact policies (CIPs) are widely used in UK local government to help regulate alcohol markets in localities characterised by high density of outlets and high rates of alcohol related harms. CIPs have been advocated as a means of protecting health by controlling or limiting alcohol availability. We use a comparative qualitative case study approach (n=5 English local government authorities, 48 participants) to assess how CIPs vary across different localities, what they are intended to achieve, and the implications for local-level alcohol availability. We found that the case study CIPs varied greatly in terms of aims, health focus and scale of implementation. However, they shared some common functions around influencing the types and managerial practices of alcohol outlets in specific neighbourhoods without reducing outlet density. The assumption that this will lead to alcohol harm-reduction needs to be quantitatively tested.

A comparison of the effectiveness of an adult nutrition education program for Aboriginal and non-Aboriginal Australians

BACKGROUND

Adult nutrition education is an important component of broader societal efforts to address the high prevalence of nutrition-related diseases. In Australia, Aboriginal people are a critical target group for such programs because of their substantially higher rates of these diseases.

OBJECTIVE

The aim of this study was to assess the relative effectiveness of an adult nutrition education program for Aboriginal and non-Aboriginal participants.

METHODS

Pre-and post-course evaluation data were used to assess changes in confidence in ability to buy healthy foods on a budget, nutrition knowledge, and dietary behaviours among individuals attending FOODcents nutrition education courses. The total sample of 875 Western Australians included 169 who self-identified as Aboriginal or Torres Strait Islander.

RESULTS

Perceptions of course usefulness were very high and comparable between Aboriginal and non-Aboriginal participants. Significantly larger improvements in confidence, nutrition knowledge, and reported consumption behaviours were evident among Aboriginal participants.

CONCLUSION

The findings suggest that adult nutrition education programs that address specific knowledge and skill deficits that are common among disadvantaged groups can be effective for multiple target groups, and may also assist in reducing nutrition-related inequalities.

Lessons from complex interventions to improve health

Complexity-resulting from interactions among many component parts-is a property of both the intervention and the context (or system) into which it is placed. Complexity increases the unpredictability of effects. Complexity invites new approaches to logic modeling, definitions of integrity and means of standardization, and evaluation. New metaphors and terminology are needed to capture the recognition that knowledge generation comes from the hands of practitioners/implementers as much as it comes from those usually playing the role of intervention researcher. Failure to acknowledge this may blind us to the very mechanisms we seek to understand. Researchers in clinical settings are documenting health improvement gains made as a consequence of complex systems thinking. Improvement science in clinical settings has much to offer researchers in population health.