Process evaluation for a school-based physical activity intervention for 6th- and 7th-grade boys: reach, dose, and fidelity

Theory-guided process evaluation of a multicomponent, technology-based 'SMART Eating' trial among Indian adults: an embedded mixed-methods study

We report process evaluation findings from the 'SMART Eating' intervention trial, which significantly improved fat, sugar and salt (FSS), and fruits and vegetables (FVs) intake among adults. Intervention used information technology [short message service (SMS), WhatsApp and website] and interpersonal communication (distribution of SMART Eating kit) and pamphlet for comparison group. Guided by UK Medical Research Council's framework, using embedded mixed-methods design, continuous process evaluation documented fidelity, dose, reach, acceptability and mechanisms. Intervention was implemented as intended, with high reach (91%) in both groups: 'comparison group' (n = 366): inadequate use of pamphlets (46%); 'intervention group' (n = 366): with timely remedial measures to remove implementation/usage barriers, dose of SMS (93%), WhatsApp (89%) and 'SMART Eating' kit (100%) was adequate, but website usage was low (50%); compliance was evident from participants' interactions with the implementor and observations on kit usage. All these might have improved attitude, social influence, self-efficacy and household behaviours, which, in turn, mediated intervention's effect on improving FSS and FV intake. Among poor performers, lack of effect on FV intake was perceived to be related to high cost/pesticides use and FSS intake was related to lack of family support. Low website usage, challenges with WhatsApp messaging and contextual factors (cost, pesticides abuse and family support) need to be considered while designing similar future interventions.

Fuel for Fun Process Evaluation Reveals Strong Implementation and Approval with Varied Parent Engagement

OBJECTIVE

To describe the implementation and process characteristics of Fuel for Fun: Cooking with Kids Plus Parents and Play (FFF).

DESIGN

Mixed methods.

SETTING

Elementary schools in 2 Northern Colorado school districts.

PARTICIPANTS

Fourth graders (aged 9-11 years), parents, school staff, and implementation researchers; measured over 2 consecutive years in 8 schools (851 students, 45 classrooms).

INTERVENTION(S)

Social Cognitive and active learning theory-based classroom cooking with tasting lessons, active recess games, cooking with tasting food promotion during school lunch, family nights, and take-home intervention reinforcements. A 7-month program delivered by a trained intervention team.

MAIN OUTCOME MEASURE(S)

Implementation measures (observations and debriefings) assessed context, reach, dose delivered, fidelity, and dose received; process measures (surveys) assessed student and parent perceptions and intervention participation.

ANALYSIS

Descriptive statistics for quantitative and themes for qualitative data.

RESULTS

Minor adjustments in program delivery plans were required to accommodate changes in school schedules and policies. Process measures demonstrated > 90% achievement of goals for nearly all child-centered activities. One-quarter of eligible families participated in evening events, with strong parent and student approval. Fifty out of 116 parents (43%) completing an online survey reported preparing ≥ 1 of 5 recipes with their child. Fifty-nine percent of eligible students completed >1 of 10 take-home activity sheets with their parents.

CONCLUSIONS AND IMPLICATIONS

Engagement and commitment of the intervention team and school staff supported strong implementation. Participant responses were positive, but improvement in parent engagement requires investigation.

Effect of a Three-Month HOPSports Brain Breaks® Intervention Program on the Physical Fitness Levels of Grade 6-Learners in South Africa

Despite the numerous health benefits of being physically active, children are not active enough. Various researchers have indicated that intervention programs improve physical fitness levels. Still, only a few have focused on improving physical fitness levels by incorporating technology. HOPSports Brain Breaks® are designed and presented as physical activity solutions with online videos requiring the participants to imitate the movements. These videos are 2-5-min classroom activity breaks. This study determined the effect of a three-month HOPSports Brain Breaks® intervention program on the physical fitness levels of Grade 6-learners. Physical fitness was measured with the EUROFIT test battery. The experimental group consisted of 79 children (26 boys and 47 girls) and the control group of 47 children (16 boys and 33 girls). The mean age for the entire group was 11.92 (±0.36) years. The results indicated that there was a statistically (p ≤ 0.05) and practically (d ≥ 0.20) significant difference between the experimental and control group for percentage body fat; stork balance; plate tapping; sit-and-reach; standing jump; sit-ups; and 10 × 5 m shuttle run and 20 m shuttle run between the pre-and post-test. Therefore, considering the results mentioned above, the HOPSports Brain Breaks® intervention program can indeed contribute to the improvement of physical fitness, and motor skills of children. Therefore, future studies should be conducted to determine the effect of HOPSports Brain Breaks® between genders as well as what impact it will have on academic performance.

Process and Outcome Evaluations of Interventions to Promote Voluntary Exercise Training Among South Korean Firefighters

The main aim of the present study is to evaluate reach, dose, fidelity, and outcomes of the interventions for promoting voluntary exercise training among South Korean firefighters. Four interventions for promoting voluntary exercise training among firefighters (i.e., virtual reality exercise system intervention, poster intervention, monitor intervention, and wearable health device intervention) were performed in a fire station located in Seoul, South Korea. To evaluate reach and dose received related to each intervention, participants were asked to answer several simple questions. Three process evaluators completed a 20-item survey to share their impressions related to the quality of intervention delivery. Paired t test was used to examine mean changes in primary (i.e., mean minutes of exercise training per week) and secondary outcomes (i.e., beliefs and intention) between pre- and postinterventions. More than 60% of participants experienced monitor and wearable health device interventions. Process evaluators tend not to agree with a statement saying that the number of the virtual reality exercise equipment was appropriate. Among firefighters who participated in exercise training less than 150 min per week at 1-month follow-up, mean minutes of exercise training per week increased by 67.95 min after interventions. Future studies need to examine whether the monitor and wearable health device interventions effectively increase exercise training participation among firefighters in other fire stations located in Seoul, South Korea.

School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18

BACKGROUND

Physical activity among children and adolescents is associated with lower adiposity, improved cardio-metabolic health, and improved fitness. Worldwide, fewer than 30% of children and adolescents meet global physical activity recommendations of at least 60 minutes of moderate to vigorous physical activity per day. Schools may be ideal sites for interventions given that children and adolescents in most parts of the world spend a substantial amount of time in transit to and from school or attending school.

OBJECTIVES

The purpose of this review update is to summarise the evidence on effectiveness of school-based interventions in increasing moderate to vigorous physical activity and improving fitness among children and adolescents 6 to 18 years of age. Specific objectives are: • to evaluate the effects of school-based interventions on increasing physical activity and improving fitness among children and adolescents; • to evaluate the effects of school-based interventions on improving body composition; and • to determine whether certain combinations or components (or both) of school-based interventions are more effective than others in promoting physical activity and fitness in this target population.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, CINAHL, PsycINFO, BIOSIS, SPORTDiscus, and Sociological Abstracts to 1 June 2020, without language restrictions. We screened reference lists of included articles and relevant systematic reviews. We contacted primary authors of studies to ask for additional information.

SELECTION CRITERIA

Eligible interventions were relevant to public health practice (i.e. were not delivered by a clinician), were implemented in the school setting, and aimed to increase physical activity among all school-attending children and adolescents (aged 6 to 18) for at least 12 weeks. The review was limited to randomised controlled trials. For this update, we have added two new criteria: the primary aim of the study was to increase physical activity or fitness, and the study used an objective measure of physical activity or fitness. Primary outcomes included proportion of participants meeting physical activity guidelines and duration of moderate to vigorous physical activity and sedentary time (new to this update). Secondary outcomes included measured body mass index (BMI), physical fitness, health-related quality of life (new to this update), and adverse events (new to this update). Television viewing time, blood cholesterol, and blood pressure have been removed from this update.  DATA COLLECTION AND ANALYSIS: Two independent review authors used standardised forms to assess each study for relevance, to extract data, and to assess risk of bias. When discrepancies existed, discussion occurred until consensus was reached. Certainty of evidence was assessed according to GRADE. A random-effects meta-analysis based on the inverse variance method was conducted with participants stratified by age (children versus adolescents) when sufficient data were reported. Subgroup analyses explored effects by intervention type.

MAIN RESULTS

Based on the three new inclusion criteria, we excluded 16 of the 44 studies included in the previous version of this review. We screened an additional 9968 titles (search October 2011 to June 2020), of which 978 unique studies were potentially relevant and 61 met all criteria for this update. We included a total of 89 studies representing complete data for 66,752 study participants. Most studies included children only (n = 56), followed by adolescents only (n = 22), and both (n = 10); one study did not report student age. Multi-component interventions were most common (n = 40), followed by schooltime physical activity (n = 19), enhanced physical education (n = 15), and before and after school programmes (n = 14); one study explored both enhanced physical education and an after school programme. Lack of blinding of participants, personnel, and outcome assessors and loss to follow-up were the most common sources of bias.  Results show that school-based physical activity interventions probably result in little to no increase in time engaged in moderate to vigorous physical activity (mean difference (MD) 0.73 minutes/d, 95% confidence interval (CI) 0.16 to 1.30; 33 studies; moderate-certainty evidence) and may lead to little to no decrease in sedentary time (MD -3.78 minutes/d, 95% CI -7.80 to 0.24; 16 studies; low-certainty evidence). School-based physical activity interventions may improve physical fitness reported as maximal oxygen uptake (VO₂max) (MD 1.19 mL/kg/min, 95% CI 0.57 to 1.82; 13 studies; low-certainty evidence). School-based physical activity interventions may result in a very small decrease in BMI z-scores (MD -0.06, 95% CI -0.09 to -0.02; 21 studies; low-certainty evidence) and may not impact BMI expressed as kg/m² (MD -0.07, 95% CI -0.15 to 0.01; 50 studies; low-certainty evidence). We are very uncertain whether school-based physical activity interventions impact health-related quality of life or adverse events.

AUTHORS' CONCLUSIONS

Given the variability of results and the overall small effects, school staff and public health professionals must give the matter considerable thought before implementing school-based physical activity interventions. Given the heterogeneity of effects, the risk of bias, and findings that the magnitude of effect is generally small, results should be interpreted cautiously.

Reach, Recruitment, Dose, and Intervention Fidelity of the GoActive School-Based Physical Activity Intervention in the UK: A Mixed-Methods Process Evaluation

School-based multi-component physical activity (PA) promotion is advocated; however, research has indicated that a multi-component approach may not always be effective at increasing adolescent PA. Evaluation of the GoActive 12-week multi-component school-based intervention showed no effect on adolescent PA. A mixed-methods process evaluation was embedded to facilitate greater understanding of the results, to elicit subgroup perceptions, and to provide insight into contextual factors influencing intervention implementation. This paper presents the reach, recruitment, dose, and fidelity of GoActive, and identifies challenges to implementation. The process evaluation employed questionnaires (1543 Year 9s), individual interviews (16 Year 9s; 7 facilitators; 9 contact teachers), focus groups (48 Year 9s; 58 mentors), alongside GoActive website analytics and researcher observations. GoActive sessions reached 39.4% of Year 9s. Intervention satisfaction was relatively high for mentors (87.3%) and facilitators (85.7%), but lower for Year 9s (59.5%) and teachers (50%). Intervention fidelity was mixed within and between schools. Mentorship was the most implemented component. Factors potentially contributing to low implementation included ambiguity of the roles subgroups played within intervention delivery, Year 9 engagement, institutional support, and further school-level constraints. Multiple challenges and varying contextual considerations hindered the implementation of GoActive in multiple school sites. Methods to overcome contextual challenges to implementation warrant in-depth consideration and innovative approaches.

Process Evaluation of a School-Based Program Aimed at Preventing Obesity in Adolescents from Lima and Callao, Peru

The study aims to describe process evaluation measures of the three-year Health Promoting Schools’ obesity prevention program in Lima and Callao, Peru, and to assess factors that influenced the implementation of the intervention leading to the mentioned process outcomes results. The program was implemented in four public high schools located in low-income areas of Lima and Callao. Embedded in a Health Promoting School Intervention, the program consisted of two main components—an education program and several environmental activities. Quantitative data were collected and analyzed based on dose delivered and reach for each specific activity. Dose received was analyzed by satisfaction scores related to six specific activities. Furthermore, qualitative data including documentation of activities and transcriptions from individual in-depth interviews were qualitatively analyzed to identify factors influencing the implementation. The education component of the Health Promoting Schools’ program achieved a 78.4% average nutrition sessions delivery in 2015 and 88.0% in 2017; while for PA sessions, the average delivery in 2015 was of 79.7% and 93.8% in 2017. In the case of reach, at least 75% of total students participated in all sessions per year. Nevertheless, there were differences in terms of delivery and participation in a number of environmental activities within and between schools during the program period. Differences in delivery included education sessions for parents, teachers, Junior Health Promoters, and school food kiosk staff, besides execution of physical activity events every year. Impeding factors included the complexity of the overall intervention, limited strategies to facilitate program implementation and those to maintain the participation of subjects, and related contextual factors.

Impact of Combined Theory-Based Intervention on Psychological Effects and Physical Activity among Chinese Adolescents

PURPOSE

The current study was intended to assess the effect of a facilitated behavioral intervention based on the extended theory of planned behavior (TPB) on psychological constructs and physical activity among adolescents over a period of eight weeks.

METHODS

Students (n = 51, 12 ± 0.3 years of age) in the seventh grade at a junior middle school in China were randomly assigned to two groups: the intervention group (n = 24) and the control group (n = 27). Both groups were pre- and post-tested with the related psychological constructs of the extended TPB, along with behavioral measures of the Physical Activity Scale and ActiGraph accelerometer (model wGT3X-BT). The intervention group took part in 45 min classes once per week for 8 weeks, including five indoor theoretical courses and three outdoor basketball matches. The control group was not required to make any change to their normal school day. Also, 2 × 2 repeated measures analysis of variance (ANOVA) was conducted to compare the differences between the two groups, and then t-test was employed to compare the independent and paired differences.

RESULTS

Significant increases in pre-post subjective norms (SN) (p = 0.041, Cohen's d = 0.62), perceived behavior control (PBC) (p = 0.023, Cohen's d = 0.72), exercise intention (EI) (p = 0.043, Cohen's d = 0.61), and self-efficacy (SE) (p = 0.035, Cohen's d = 1.36) were observed in the intervention group. In addition, participants in the intervention group increased their exercise frequency (p < 0.001, Cohen's d = 1.25) and intensity (p = 0.028, Cohen's d = 0.68), especially their time spent on light intensity physical activity (light-PA%; p = 0.031, Cohen's d = 0.68), and their percentage of sedentary time (SB%) was also reduced (from 68% ± 10% to 58% ± 7%, p < 0.001, Cohen's d = 1.17). Furthermore, the intervention group showed significantly better performance in PBC (p = 0.032, Cohen's d = 0.62), EI (p < 0.001, Cohen's d = 1.32), SE (p < 0.001, Cohen's d = 1.15), SB% (p < 0.001, Cohen's d = 1.22), light-PA% (p < 0.001, Cohen's d = 1.12), and total physical activity (TPA) (p = 0.015, Cohen's d = 0.72) compared to the control group at the post-test. No significant pre post differences were observed for any psychological or behavioral variables in the control group, except for exercise frequency, but the values were still lower than those in the intervention group after the 8-week intervention (3.70 ± 0.72 versus 3.92 ± 0.83).

CONCLUSION

The combined theory-based intervention was effective at improving psychological constructs and physical activity among seventh-grade adolescents in 8 weeks.

Variation in the operationalisation of dose in implementation of health promotion interventions: insights and recommendations from a scoping review

BACKGROUND

While 'dose' is broadly understood as the 'amount' of an intervention, there is considerable variation in how this concept is defined. How we conceptualise, and subsequently measure, the dose of interventions has important implications for understanding how interventions produce their effects and are subsequently resourced and scaled up. This paper aims to explore the degree to which dose is currently understood as a distinct and well-defined implementation concept outside of clinical settings.

METHODS

We searched four databases (MEDLINE, PsycINFO, EBM Reviews and Global Health) to identify original research articles published between 2000 and 2015 on health promotion interventions that contained the word 'dose' or 'dosage' in the title, abstract or keywords. We identified 130 articles meeting inclusion criteria and extracted data on how dose/dosage was defined and operationalised, which we then synthesised to reveal key themes in the use of this concept across health promotion interventions.

RESULTS

Dose was defined in a variety of ways, including in relation to the amount of intervention delivered and/or received, the level of participation in the intervention and, in some instances, the quality of intervention delivery. We also observed some conflation of concepts that are traditionally kept separate (such as fidelity) either as slippage or as part of composite measures (such as 'intervention dose').

DISCUSSION

Dose is not a well-defined or consistently applied concept in evaluations of health promotion interventions. While current approaches to conceptualisation and measurement of dose are suitable for interventions in organisational settings, they are less well suited to policies delivered at a population level. Dose often accompanies a traditional monotonic linear view of causality (e.g. dose response) which may or may not fully represent the intervention's theory of how change is brought about. Finally, we found dose and dosage to be used interchangeably. We recommend a distinction between these terms, with 'dosage' having the advantage of capturing change to amount 'dispensed' over time (in response to effects achieved). Dosage therefore acknowledges the inevitable dynamic and complexity of implementation.

Measuring implementation fidelity of school-based obesity prevention programmes: a systematic review

Background

Until now, there is no clear overview of how fidelity is assessed in school-based obesity prevention programmes. In order to move the field of obesity prevention programmes forward, the current review aimed to 1) identify which fidelity components have been measured in school-based obesity prevention programmes; 2) identify how fidelity components have been measured; and 3) score the quality of these methods.

Methods

Studies published between January 2001–October 2017 were selected from searches in PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Library and ERIC. We included studies examining the fidelity of obesity prevention programmes (nutrition and/or physical activity and/or sitting) at school (children aged 4–18 year) measuring at least one component of implementation fidelity. A data extraction was performed to identify which and how fidelity components were measured. Thereafter, a quality assessment was performed to score the quality of these methods. We scored each fidelity component on 7 quality criteria. Each fidelity component was rated high (> 75% positive), moderate (50–75%) or low (< 50%).

Results

Of the 26,294 retrieved articles, 73 articles reporting on 63 different studies were included in this review. In 17 studies a process evaluation was based on a theoretical framework. In total, 120 fidelity components were measured across studies: dose was measured most often (N = 50), followed by responsiveness (N = 36), adherence (N = 26) and quality of delivery (N = 8). There was substantial variability in how fidelity components were defined as well as how they were measured. Most common methods were observations, logbooks and questionnaires targeting teachers. The quality assessment scores ranged from 0 to 86%; most fidelity components scored low quality (n = 77).

Conclusions

There is no consensus on the operationalisation of concepts and methods used for assessing fidelity in school-based obesity prevention programmes and the quality of methods used is weak. As a result, we call for more consensus on the concepts and clear reporting on the methods employed for measurements of fidelity to increase the quality of fidelity measurements. Moreover, researchers should focus on the relation between fidelity and programme outcomes and determine to what extent adaptations to programmes have been made, whilst still being effective.

Electronic supplementary material

The online version of this article (10.1186/s12966-018-0709-x) contains supplementary material, which is available to authorized users.

Strategies for enhancing the implementation of school-based policies or practices targeting risk factors for chronic disease

BACKGROUND

A number of school-based policies or practices have been found to be effective in improving child diet and physical activity, and preventing excessive weight gain, tobacco or harmful alcohol use. Schools, however, frequently fail to implement such evidence-based interventions.

OBJECTIVES

The primary aims of the review are to examine the effectiveness of strategies aiming to improve the implementation of school-based policies, programs or practices to address child diet, physical activity, obesity, tobacco or alcohol use.Secondary objectives of the review are to: Examine the effectiveness of implementation strategies on health behaviour (e.g. fruit and vegetable consumption) and anthropometric outcomes (e.g. BMI, weight); describe the impact of such strategies on the knowledge, skills or attitudes of school staff involved in implementing health-promoting policies, programs or practices; describe the cost or cost-effectiveness of such strategies; and describe any unintended adverse effects of strategies on schools, school staff or children.

SEARCH METHODS

All electronic databases were searched on 16 July 2017 for studies published up to 31 August 2016. We searched the following electronic databases: Cochrane Library including the Cochrane Central Register of Controlled Trials (CENTRAL); MEDLINE; MEDLINE In-Process & Other Non-Indexed Citations; Embase Classic and Embase; PsycINFO; Education Resource Information Center (ERIC); Cumulative Index to Nursing and Allied Health Literature (CINAHL); Dissertations and Theses; and SCOPUS. We screened reference lists of all included trials for citations of other potentially relevant trials. We handsearched all publications between 2011 and 2016 in two specialty journals (Implementation Science and Journal of Translational Behavioral Medicine) and conducted searches of the WHO International Clinical Trials Registry Platform (ICTRP) (http://apps.who.int/trialsearch/) as well as the US National Institutes of Health registry (https://clinicaltrials.gov). We consulted with experts in the field to identify other relevant research.

SELECTION CRITERIA

'Implementation' was defined as the use of strategies to adopt and integrate evidence-based health interventions and to change practice patterns within specific settings. We included any trial (randomised or non-randomised) conducted at any scale, with a parallel control group that compared a strategy to implement policies or practices to address diet, physical activity, overweight or obesity, tobacco or alcohol use by school staff to 'no intervention', 'usual' practice or a different implementation strategy.

DATA COLLECTION AND ANALYSIS

Citation screening, data extraction and assessment of risk of bias was performed by review authors in pairs. Disagreements between review authors were resolved via consensus, or if required, by a third author. Considerable trial heterogeneity precluded meta-analysis. We narratively synthesised trial findings by describing the effect size of the primary outcome measure for policy or practice implementation (or the median of such measures where a single primary outcome was not stated).

MAIN RESULTS

We included 27 trials, 18 of which were conducted in the USA. Nineteen studies employed randomised controlled trial (RCT) designs. Fifteen trials tested strategies to implement healthy eating policies, practice or programs; six trials tested strategies targeting physical activity policies or practices; and three trials targeted tobacco policies or practices. Three trials targeted a combination of risk factors. None of the included trials sought to increase the implementation of interventions to delay initiation or reduce the consumption of alcohol. All trials examined multi-strategic implementation strategies and no two trials examined the same combinations of implementation strategies. The most common implementation strategies included educational materials, educational outreach and educational meetings. For all outcomes, the overall quality of evidence was very low and the risk of bias was high for the majority of trials for detection and performance bias.Among 13 trials reporting dichotomous implementation outcomes-the proportion of schools or school staff (e.g. classes) implementing a targeted policy or practice-the median unadjusted (improvement) effect sizes ranged from 8.5% to 66.6%. Of seven trials reporting the percentage of a practice, program or policy that had been implemented, the median unadjusted effect (improvement), relative to the control ranged from -8% to 43%. The effect, relative to control, reported in two trials assessing the impact of implementation strategies on the time per week teachers spent delivering targeted policies or practices ranged from 26.6 to 54.9 minutes per week. Among trials reporting other continuous implementation outcomes, findings were mixed. Four trials were conducted of strategies that sought to achieve implementation 'at scale', that is, across samples of at least 50 schools, of which improvements in implementation were reported in three trials.The impact of interventions on student health behaviour or weight status were mixed. Three of the eight trials with physical activity outcomes reported no significant improvements. Two trials reported reductions in tobacco use among intervention relative to control. Seven of nine trials reported no between-group differences on student overweight, obesity or adiposity. Positive improvements in child dietary intake were generally reported among trials reporting these outcomes. Three trials assessed the impact of implementation strategies on the attitudes of school staff and found mixed effects. Two trials specified in the study methods an assessment of potential unintended adverse effects, of which, they reported none. One trial reported implementation support did not significantly increase school revenue or expenses and another, conducted a formal economic evaluation, reporting the intervention to be cost-effective. Trial heterogeneity, and the lack of consistent terminology describing implementation strategies, were important limitations of the review.

AUTHORS' CONCLUSIONS

Given the very low quality of the available evidence, it is uncertain whether the strategies tested improve implementation of the targeted school-based policies or practices, student health behaviours, or the knowledge or attitudes of school staff. It is also uncertain if strategies to improve implementation are cost-effective or if they result in unintended adverse consequences. Further research is required to guide efforts to facilitate the translation of evidence into practice in this setting.

Process evaluation results of a cluster randomised controlled childhood obesity prevention trial: the WAVES study

BACKGROUND

Increasing prevalence of childhood obesity and its related consequences emphasises the importance of developing and evaluating interventions aimed at prevention. The importance of process evaluation in health intervention research is increasingly recognised, assessing implementation and participant response, and how these may relate to intervention success or failure. A comprehensive process evaluation was designed and undertaken for the West Midlands ActiVe lifestyle and healthy Eating in School children (WAVES) study that tested the effectiveness of an obesity prevention programme for children aged 6-7 years, delivered in 24 UK schools. The four intervention components were: additional daily school-time physical activity (PA); cooking workshops for children and parents; Villa Vitality (VV), a 6-week healthy lifestyle promotion programme run by a local football club; and signposting to local PA opportunities.

METHODS

Data relating to six dimensions (Fidelity, Reach, Recruitment, Quality, Participant Responsiveness, Context) were collected via questionnaires, logbooks, direct observations, focus groups and interviews. Multiple data collection methods allowed for data triangulation and validation of methods, comparing research observations with teacher records. The 6-stage WAVES study model ((i) Data collection, (ii) Collation, (iii) Tabulation, (iv) Score allocation and discussion, (v) Consultation, (vi) Final score allocation) was developed to guide the collection, assimilation and analysis of process evaluation data. Two researchers independently allocated school scores on a 5-point Likert scale for each process evaluation dimension. Researchers then discussed school score allocations and reached a consensus. Schools were ranked by total score, and grouped to reflect low, medium or high intervention implementation.

RESULTS

The intervention was predominantly well-implemented and well-received by teachers, parents and children. The PA component was identified as the most challenging, VV the least. Median implementation score across schools was 56/75 (IQR, 51.0 - 60.8). Agreement between teacher logbooks and researcher observations was generally high, the main discrepancies occurred in session duration reporting where in some cases teachers' estimations tended to be higher than researchers'.

CONCLUSIONS

The WAVES study model provides a rigorous and replicable approach to undertaking and analysing a multi-component process evaluation. Challenges to implementing school-based obesity prevention interventions have been identified which can be used to inform future trials.

TRIAL REGISTRATION

ISRCTN97000586 . 19 May 2010.

Examining reach, dose, and fidelity of the "Girls on the Move" after-school physical activity club: a process evaluation

Background

After-school programs represent a promising opportunity to assist adolescent girls’ in attaining adequate physical activity. Although evaluating the process of intervention implementation is important for determining if an intervention was delivered and received as intended, comprehensive information about process evaluation methods and results are rarely reported. The purpose of this article was to evaluate the reach, dose, and fidelity of a 90-minute after-school physical activity club offered 3 days a week. The club is 1 of 3 components included in a 17-week intervention designed for 5th-8th grade girls, the majority of whom were of minority and/or low socioeconomic status.

Methods

A total of 24 schools (12 intervention; 12 control) and 56–67 girls per school (total N = 1519 girls) were included in the Girls on the Move group randomized controlled trial. At the beginning of each of 3 academic years (2012–2015), 8 schools per year were randomized to receive either the intervention (n = 4) or control condition (n = 4). To evaluate the club, data collected via surveys from girls, club coaches and managers, and process evaluators were analyzed. To evaluate the opportunity for physical activity provided by the coaches and managers, process evaluators used an observation tool based on the System for Observing Fitness Instruction Time and Academic Learning Time - Physical Education. Girls wore accelerometers every other week during the club time.

Results

Mean attendance was 41 % with the average attendance in year 3 being higher than rates for years 1 or 2. Mean moderate-to-vigorous physical activity time was 21.85 minutes measured via accelerometry and 21.81 minutes observed by process evaluators. Satisfaction with the intervention was high. For the most part, process evaluators perceived the club was delivered as planned and reflected constructs of the Health Promotion Model and Self-Determination Theory. Areas contributing to success included using incentives and offering a variety of activities. Issues negatively impacting implementation included managing behavioral problems, having limited space for moderate-to-vigorous physical activity, dealing with inclement weather, and getting coaches to actively participate in all physical activities with the girls.

Conclusions

This process evaluation provides important information to guide future school-based physical activity intervention delivery. Barriers to implementation have been identified. Ways to overcome them warrant consideration when designing physical activity interventions. Research is needed to test innovative approaches for enhancing attendance and increasing girls’ moderate-to-vigorous physical activity in after-school programs.

Trial registration

ClinicalTrials.gov Identifier NCT01503333. Registered 23 December, 2011.

Bristol Girls Dance Project: a cluster randomised controlled trial of an after-school dance programme to increase physical activity among 11- to 12-year-old girls

Background

Many children do not meet UK physical activity (PA) guidelines. Girls are less active than boys, and the age-related decline in activity is steeper for girls. Dance is the favourite form of PA among adolescent girls in the UK. Participation in after-school dance classes could significantly contribute to girls’ PA. Therefore, after-school dance may be effective for increasing PA levels.

Objectives

To determine the effectiveness and cost-effectiveness of a dance-based intervention to increase the objectively assessed mean weekday minutes of moderate- to vigorous-intensity physical activity (MVPA) of Year 7 girls (11- and 12-year olds) 1 year after baseline measurement.

Design

Two-arm cluster randomised controlled trial and economic evaluation. Year 7 girls in participant schools received a ‘taster’ session and were invited to participate. Up to 33 girls per school were able to participate. Schools were randomly assigned (equal numbers) to intervention or control arms.

Setting

A total of 18 mainstream secondary schools across greater Bristol.

Participants

Year 7 girls in participating schools who could participate in physical education.

Intervention

Nine intervention schools received an after-school dance intervention (40 × 75-minute sessions) underpinned by self-determination theory, which attempts to improve intrinsic motivation for being active, and delivered by external dance instructors. Control schools continued as normal.

Main outcome measures

The main outcome was accelerometer-assessed mean minutes of MVPA at T2. Measures were assessed at baseline (T0), the end of the intervention (T1) and at T0 + 52 weeks (T2).

Results

Baseline MVPA levels were high. A total of 508 girls were included in the primary analysis, which found no difference in weekday MVPA between trial arms. There was no effect on secondary accelerometer outcomes. Data were subjected to a per-protocol analysis and no effect was found. However, at T1, girls who attended dance classes had 4.61 minutes more of MVPA and 14.27 minutes more of light-intensity activity between 15.00 and 17.00 on the days on which they attended intervention sessions. The intervention was inexpensive at £73 per participant (£63 when excluding dance instructor travel) but was not cost-effective owing to the ineffectiveness of the intervention. The European Quality of Life-5 Dimensions Youth survey data were unresponsive to changes in the sample. The process evaluation reported that girls in attendance enjoyed the sessions, that exertion levels were low during sessions and that attendance was low and declined. Fidelity to the session-plan manual was low but theoretical fidelity (to self-determination theory) was good. Qualitative information provides information for improving future interventions.

Conclusions

The intervention was enjoyed by participants. However, there was no difference in the MVPA levels (which were high at baseline) of girls allocated to receive dance compared with girls receiving the control. High baseline MVPA levels indicate that the study appealed to an already active cohort and, therefore, may not have targeted those most in need of an intervention. Dance is an enjoyable activity for adolescent girls and could be further trialled as a means by which to increase PA. Research might consider the impact of dividing the intervention period into smaller blocks.

Trial Registration

Current Controlled Trials ISRCTN52882523.

Funding

The National Institute for Health Research Public Health Research programme. The work was also undertaken with the support of the Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), a UK Clinical Research Collaboration (UKCRC) PHR Centre of Excellence. Joint funding (MR/KO232331/1) from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council, the Welsh Government and the Wellcome Trust, under the auspices of the UKCRC, is gratefully acknowledged. This study was designed and delivered in collaboration with the Bristol Randomised Trials Collaboration, a UKCRC Registered Clinical Trials Unit in receipt of NIHR Clinical Trials Unit support funding. All intervention costs were funded by the respective councils to which the participant schools were affiliated, namely North Somerset Council, Bristol City Council, and Bath and North East Somerset Council.

Process evaluation of the Bristol girls dance project

Background

The Bristol Girls Dance Project was a cluster randomised controlled trial that aimed to increase objectively measured moderate-to-vigorous physical activity (MVPA) levels of Year 7 (age 11–12) girls through a dance-based after-school intervention. The intervention was delivered in nine schools and consisted of up to forty after-school dance sessions. This paper reports on the main findings from the detailed process evaluation that was conducted.

Methods

Quantitative and qualitative data were collected from intervention schools. Dose and fidelity were reported by dance instructors at every session. Intervention dose was defined as attending two thirds of sessions and was measured by attendance registers. Fidelity to the intervention manual was reported by dance instructors. On four randomly-selected occasions, participants reported their perceived level of exertion and enjoyment. Reasons for non-attendance were self-reported at the end of the intervention. Semi-structured interviews were conducted with all dance instructors who delivered the intervention (n = 10) and school contacts (n = 9) in intervention schools. A focus group was conducted with girls who participated in each intervention school (n = 9).

Results

The study did not affect girls’ MVPA. An average of 31.7 girls participated in each school, with 9.1 per school receiving the intervention dose. Mean attendance and instructors’ fidelity to the intervention manual decreased over time. The decline in attendance was largely attributed to extraneous factors common to after-school activities. Qualitative data suggest that the training and intervention manual were helpful to most instructors. Participant ratings of session enjoyment were high but perceived exertion was low, however, girls found parts of the intervention challenging.

Conclusions

The intervention was enjoyed by participants. Attendance at the intervention sessions was low but typical of after-school activities. Participants reported that the intervention brought about numerous health and social benefits and improved their dance-based knowledge and skills. The intervention could be improved by reducing the number of girls allowed to participate in each school and providing longer and more in-depth training to those delivering the intervention.

Trial registration

ISRCTN52882523 Registered 25th April 2013.

Process Evaluation of the Project SHINE Intervention for African American Families: An Integrated Positive Parenting and Peer Monitoring Approach to Health Promotion

This study describes the process evaluation of Project SHINE, a randomized family-based health promotion intervention that integrated parenting and peer monitoring for improving sedentary behavior, physical activity, and diet in African American families. Adolescent-parent dyads (n = 89) were randomized to a 6-week behavioral, positive parenting, and peer monitoring skills intervention or a general health education comparison condition. Process evaluation included observational ratings of fidelity, attendance records, psychosocial measures, and qualitative interviews. Results indicated that the intervention was delivered with high fidelity based on facilitator adherence (>98% of content delivered) and competent use of theoretically based behavior change and positive parenting skills (100% of ratings >3 on a 1-4 scale). Although only 43% of peers attended the "bring a friend" session, overall attendance was high (4.39 ± 1.51 sessions) as was the retention rate (88%). Parents in the intervention condition reported significant improvements in communication related to adolescents' engagement in health behaviors both on their own and with peers. These findings were supported by qualitative themes related to improvements in family communication and connectedness. This study provides an innovative example of how future family-based health promotion trials can expand their process evaluation approaches by assessing theoretically relevant positive parenting variables as part of ongoing monitoring.

Development of an innovative process evaluation approach for the Families Improving Together (FIT) for weight loss trial in African American adolescents

This study demonstrates how a multi-theoretical, multilevel process evaluation was used to assess implementation of the Families Improving Together (FIT) for weight loss intervention. FIT is a randomized controlled trial evaluating a culturally tailored, motivational plus family-based program on weight loss in African American adolescents and their parents. Social Cognitive, Self Determination, Family Systems theories and cultural tailoring principles guided the conceptualization of essential elements across individual/family, facilitator, and group levels. Data collection included an observational rating tool, attendance records, and a validated psychosocial measure.

RESULTS

Attendance records (0=absent, 1=present, criteria=≥70%) indicated that 71.5% of families attended each session. The survey (1=false, 6=true, criteria=≥4.5) indicated that participants perceived a positive group climate (M=5.16, SD=0.69). A trained evaluator reported that facilitator dose delivered (0=no, 1=yes, criteria=≥75%) was high (99.6%), and fidelity (1=none to 4=all, criteria=≥3) was adequate at facilitator (M=3.63, SD=0.41) and group levels (M=3.35, SD=0.49). Five cultural topics were raised by participants related to eating (n=3) and physical activity (n=2) behaviors and were integrated as part of the final curriculum.

DISCUSSION

Results identify areas for program improvement related to delivery of multi-theoretical and cultural tailoring elements. Findings may inform future strategies for implementing effective weight loss programs for ethnic minority families.

Process evaluation of an exercise counseling intervention using motivational interviewing

AIM

To describe the results of the process evaluation of an exercise counseling intervention using motivational interviewing (MI).

BACKGROUND

Exercise can safely be incorporated into heart failure self-care, but many lack access to cardiac rehabilitation. One alternative is to provide exercise counseling in the clinical setting.

METHODS

This process evaluation was conducted according to previously established guidelines for health promotion programs. This includes an assessment of recruitment and retention, implementation, and reach.

RESULTS

Desired number of subjects were recruited, but 25% dropped out during study. Good fidelity to the intervention was achieved; the use of MI was evaluated with improvement in adherence over time. Dose included initial session plus 12 weekly phone calls. Subjects varied in participation of daily diary usage. Setting was conducive to recruitment and data collection.

CONCLUSIONS

Evaluating the process of an intervention provides valuable feedback on content, delivery and fidelity.

Process evaluation of "Girls on the Run": exploring implementation in a physical activity-based positive youth development program

Many positive youth development programs rely on physical activity as a primary program component. Referred to as physical activity-based youth development programs, these program designs have great potential for promoting healthy youth development. This study examined how one such physical activity-based positive youth development program was implemented in order to identify design features critical to maximizing positive youth outcomes. This mixed method, multi-site process evaluation of Girls on the Run (GOTR) utilized focus groups, site visits, and self-report implementation checklists. Implementation scores were calculated to assess implementation fidelity across twenty-nine sites, and qualitative data were inductively analyzed to identify factors influential for implementation. Results reveal variability in how GOTR was implemented. Five themes emerged from the data that represented factors serving as facilitators or barriers to programmatic implementation. These included contextual/environmental factors (e.g., parental involvement, relationships with school personnel), organizational factors (e.g., implementation support and responsiveness of staff), program-specific factors (e.g., curriculum design), coach factors (e.g., existing relationships with participants, responsiveness to participant's needs), and youth factors (e.g., behavioral and discipline issues). Study findings have implications for improving the design of physical activity-based and other positive youth development programs, with relevance to evaluators, program planners, youth development leaders, and others working with children and youth.

Process evaluation design in a cluster randomised controlled childhood obesity prevention trial: the WAVES study

Background

The implementation of a complex intervention is heavily influenced by individual context. Variation in implementation and tailoring of the intervention to the particular context will occur, even in a trial setting. It is recognised that in trials, evaluating the process of implementation of a complex intervention is important, yet process evaluation methods are rarely reported. The WAVES study is a cluster randomised controlled trial to evaluate the effectiveness of an obesity prevention intervention programme targeting children aged 6–7 years, delivered by teachers in primary schools across the West Midlands, UK. The intervention promoted activities encouraging physical activity and healthy eating. This paper presents the methods used to assess implementation of the intervention.

Methods

Previous literature was used to identify the dimensions of intervention process and implementation to be assessed, including adherence, exposure, quality of delivery, participant responsiveness, context, and programme differentiation.

Results

Multiple methods and tools were developed to capture information on all these dimensions. These included observations, logbooks, qualitative evaluation, questionnaires and research team reflection.

Discussion

Data collection posed several challenges, predominantly when relying on teachers to complete paperwork, which they saw as burdensome on top of their teaching responsibilities. However, the use of multiple methods helped to ensure data on each dimension, where possible, was collected using more than one method. This also allowed for triangulation of the findings when several data sources on any one dimension were available.

Conclusions

We have reported a comprehensive approach to the assessment of the implementation and processes of a complex childhood obesity prevention intervention within a cluster randomised controlled trial. These approaches can be transferred and adapted for use in other complex intervention trials.

Trial registration number

ISRCTN97000586