Examining Contextual Influences on Classroom-Based Implementation of Positive Behavior Support Strategies: Findings from a Randomized Controlled Effectiveness Trial

Facilitators and barriers for implementing the PALS school-wide positive behavior support model in a Swedish municipality: A focus group study

A research-supported model to support schools' prevention of behavior problems and promotion of a positive school climate is school-wide positive behavior support (SW-PBS), in Scandinavia adapted into "positive behavior, interactions and learning environment in school" (PALS). Facilitators and barriers for achieving and sustaining a full implementation of PALS in a Swedish primary school context have not previously been studied. The purpose of this study was to explore school staff and administrators' experiences of implementing PALS in a municipal school district in the western region of Sweden. Using a qualitative focus group design, staff with roles in the implementation and school administrators (N = 22) representing 12 schools were asked about their experiences regarding facilitators and barriers for implementing PALS. Data were analyzed using qualitative content analyses involving a combination of inductive (data-driven) and deductive (theory-driven) approaches. In the deductive, final step, the inductively derived categories were associated with the Normalization process theory core constructs coherence, cognitive participation, collective action and reflexive monitoring. The analysis resulted in nine categories in which both facilitators and barriers were identified: alignment with school mission and values; building and strengthening relationships; shared staff engagement; administrators' commitment and leadership; support and resources for adopting PALS; changed experiences of PALS work over time; positive feedback sustains motivation; learning for quality improvement; and staff continuity throughout implementation. Findings indicate that staff and administrators experienced the implementation as predominantly positive, that PALS provides a structure for relationship-based work, and fits particularly well in primary school contexts. The facilitators and barriers identified provide valuable knowledge that can inform sustainable implementations of PALS in similar contexts.

Role of early childhood educators' demographic characteristics and perceived work environment in implementation of a preschool health promotion intervention

BACKGROUND

Research has indicated that the effectiveness of intervention programs is affected by how well these programs are implemented, but key gaps remain in our understanding of the factors that promote or inhibit implementation. This study examined how demographic characteristics and perceived work environment among early childhood educators were associated with implementation outcomes of the Increased Health and Wellbeing in Preschools (DAGIS) intervention, which was conducted as a cluster randomized trial.

METHODS

Participants included 101 educators from 32 intervention preschool classrooms. Data were analyzed at the classroom level, as the DAGIS intervention was delivered in preschool classrooms consisting of several educators instead of individual implementers. Linear regression was used to estimate the associations of educators' demographic characteristics and perceived work environment with different aspects of implementation (i.e., dose delivered; dose received - exposure; dose received - satisfaction; and perceived quality, as well as a total sum score based on these four dimensions). Municipality was controlled in the adjusted models.

RESULTS

Findings indicated that having a higher proportion of educators with a Bachelor's or Master's degree in education within the classroom was associated with higher dose received - exposure and higher total degree of implementation, and the significance of the models was unaffected by adjustment for municipality. Moreover, having a higher proportion of educators younger than 35 years within the classroom was associated with higher dose received - exposure. However, the association was non-significant when adjusted for municipality. No other educator factor (i.e., work experience in years and perceived support from coworkers, group work, and innovative climate) predicted implementation outcomes.

CONCLUSIONS

Higher educational attainment and younger age among educators at the classroom level were associated with higher scores for some of the implementation outcomes. Educators' work experience in years at the current preschool and in early childhood education, support from coworkers, group work, and innovative climate were not significantly associated with any implementation outcomes. Future research should explore ways to improve educators' implementation of interventions aimed at promoting children's health behaviors.

Influence of Classroom-Level Factors on Implementation Fidelity During Scale-up of Evidence-Based Interventions

As evidence-based interventions (EBIs) become more widely disseminated, fidelity of implementation (FOI) often wanes. This study explores the association between FOI and malleable variables within classrooms that could be targeted to optimize resources without compromising FOI as school-based EBIs are disseminated across real-world settings. We utilized process evaluation data from a national dissemination project of the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention shown to reduce substance use. The sample included 1,626 teachers in 371 schools across 14 states. Hierarchical linear models examined the relationship between observational measures of implementation factors and three domains of fidelity (e.g., adherence, student responsiveness, and quality of delivery). Findings suggest that curriculum modifications, student misbehavior, and shortage of time to implement the LST middle school program were factors most associated with lower FOI. Class size, access to program materials, and whether LST was delivered in a traditional classroom setting that is well-suited for instruction (versus in a less structured environment such as the school cafeteria) are less predictive. In scale-up of classroom-based universal interventions targeting behavioral health outcomes, our findings indicate that carefully vetting modifications, supporting classroom management strategies, and ensuring sufficient class time for implementation of highly interactive EBIs such as LST are important considerations. Since changes to EBIs are inevitable, efforts are needed to guide facilitators in making adjustments that improve program fit without compromising the essential intervention activities deemed necessary to produce desired outcomes.

Does Training Modality Predict Fidelity of an Evidence-based Intervention Delivered in Schools?

Training prior to implementing evidence-based interventions (EBIs) is essential to reach high levels of fidelity. However, the time and cost of in-person training are often barriers to implementation. Online learning offers a potential solution, though few studies examine the relationship between online training and fidelity of implementation. This study explored whether teachers trained online have similar levels of adherence, dosage, quality of delivery, and student responsiveness compared to teachers trained in-person on the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention proven to reduce substance use and violence, as part of a national dissemination project. This study involved a sample of 989 LST teachers across 114 school districts, representing 296 schools in 14 states. All teachers were first trained in LST implementation between 2016 and 2019. Hierarchical linear models were used to assess relationships between training modality and the four fidelity outcomes. Online training was associated with lower ratings of quality of delivery compared to in-person training, but no significant associations existed between online training and adherence to the curriculum, dosage, or student responsiveness. Findings from this study generally indicate that online training builds competencies important for school-based EBI implementation, while also highlighting potential shortcomings related to quality of delivery. Ensuring the inclusion of experiential learning activities (e.g., practice delivering content, receiving feedback on delivery) may be key to quality of delivery as online trainings for facilitators of school-based EBIs evolve.

Knowledge and Will: An Explorative Study on the Implementation of School-Wide Positive Behavior Support in Sweden

School-wide positive behavior support (SWPBIS) is a well-evaluated school approach to promoting a positive school climate and decreasing problem behaviors. Initial implementation is one of the most critical stages of program implementation. In this qualitative study, the initial implementation of SWPBIS in Swedish schools was studied using an implementation model of behavior change as guidance for interviews and analyses. The study makes significant contributions to previous research as little is known of the implementation of SWPBIS in Swedish context. Focus-group interviews were conducted with 59 professionals on implementation teams from nine schools. Themes were extracted according to implementation team members' perceptions and descriptions of how the initial implementation was carried out. The results of this study revealed relevant themes within the three domains of Capability, Opportunity, and Motivation. Core features were found under the themes of knowledge and experience of similar evidence-based programs, process or result orientation, time, manual content, organizational prerequisites, team functioning, implementation leadership, program as a unifying factor, program aligning with staff beliefs, plausible expectations, and emotional reinforcement. Results are discussed in terms of how they can be used in continuing to develop the Swedish model of SWPBIS. Implications regarding implementation in Swedish schools are discussed, as is the applicability of the model of behavior change for studying implementation in schools.

Do student social background and school context affect implementation of a school-based physical activity program?

This study examined the relationship between student- and school- characteristics and different implementation parameters of "Active All Year Round", a nationwide Danish school-based physical activity program. The study is based on data from multiple sources: questionnaire data from students from 16 fifth-grade school classes (n = 276) and their teachers (n = 16), in-class observations (n = 15) and register data. Predictors included the student-level characteristics of gender, family affluence, immigration background, and perception of school connectedness as well as the school-level characteristics of school size, school parental education level, school physical activity policy and school's prioritization of health promotion. Implementation was assessed by creating an implementation score from data on program psychological reach, dose delivered, dose received and fidelity. Data were analyzed by multilevel linear regression analysis. Findings indicated that the program was more easily implemented for students with higher school connectedness. To conclude, school-based health promotion programs can be improved by knowing, before the program is implemented, which factors increase the odds of better implementation.

Implementation of targeted mental health interventions in urban schools: Preliminary findings for impact of training strategy on program fidelity

School-based mental health programs are increasingly recognized as methods by which to improve children's access to evidence-based practices (EBPs), particularly in urban under resourced communities. School-wide positive behavior interventions and supports (PBIS) is one approach to integrating mental health services into school-based programming; however, school providers require training and support to implement programs as intended. We have conducted a randomized controlled trial to compare two models for training school-based personnel to deliver group EBPs to children at high risk of developing internalizing or externalizing problems. School personnel (N = 24) from 6 schools in a large urban school district were trained with either a basic training and consultation strategy, or an enhanced training and consultation strategy. Preliminary findings show that the enhanced strategy resulted in 9% higher content fidelity than the basic strategy. School personnel who were switched to the basic strategy had slightly lower content fidelity for the last two years of the trial and school personnel who continued to receive basic consultation during the step-down phase saw their fidelity decline. The two conditions did not differ with regard to process fidelity.

Game On-Complier Average Causal Effect Estimation Reveals Sleeper Effects on Academic Attainment in a Randomized Trial of the Good Behavior Game

To examine the efficacy of the Good Behavior Game (GBG) in improving children's reading attainment, and the extent to which this varies as a function of cumulative intervention intensity (dosage) and timing of outcome measurement. A 2-year cluster-randomized controlled trial was conducted. Seventy-seven primary schools from three regions in England were randomly assigned to intervention and control groups. Children (N = 3084) aged 67 at baseline were the target cohort. The GBG is an interdependent group-contingency behavior management strategy used by teachers in elementary schools. Reading attainment was assessed via national teacher assessment scores at baseline, and the Hodder Group Reading Test at post-test and 1-year post-intervention follow-up. Dosage was assessed using a bespoke online GBG scoreboard system. Multi-level intent-to-treat (ITT) and complier average causal effect (CACE) estimation were utilized. At post-test, no effects of the GBG on children's reading attainment were found in either the ITT or CACE models. At 1-year follow-up, results remained null in the ITT model, but a significant intervention effect was found among moderate compliers (Δ = 0.10) in the CACE model. The GBG can produce measurable improvements in children's academic attainment, but these effects may take time to become apparent and are contingent upon implementation dosage falling within an optimal range. The project was supported by funding from the Education Endowment Foundation and the National Institute for Health Research. ISRCTN: 64152096.

Minimizing Health-Compromising Behaviors via School-Based Programs: An Optimization Approach

School health programs are united by their desire to promote health and health-related outcomes among youth. They are also united by the fact that their expected effects are contingent on successful program implementation, which is often impeded by a multitude of real-world barriers. Techniques used in management science may help optimize school-based programs by accounting for implementation barriers. In this exploratory study, we present a detailed example of the first known application of linear programming (LP), which is an optimization technique, to Positive Action (PA). PA is a social emotional and character development program that includes a six-unit, teacher-delivered, classroom curriculum. We specify how we used LP to calculate the optimal levels of program implementation needed to minimize substance use, subject to known levels of implementation barriers (e.g., disruptive behavior, teacher education, teacher attitudes towards character development, school resources, and school safety). We found that LP is a technique that can be applied to data from a school health program. Specifically, we were able to develop a model that calculated the number of lessons that should be taught to minimize a specific health-compromising behavior, given expected levels of predetermined implementation barriers. Our findings from this exploratory study support the utility of applying LP during the program planning and implementation processes of school health programs.

Individual and organizational factors associated with teacher self-reported implementation of the PATHS curriculum

The current study examined a comprehensive set of individual and organizational factors as potential predictors of how the Promoting Alternative THinking (PATHS) Curriculum was implemented by teachers in an urban Midwestern school district. The study used data from a randomized trial of an implementation support model conducted in 28 urban elementary schools. All schools implemented PATHS in grades K-3. Program fidelity was assessed with teacher self-reported ratings of lesson dosage, frequency of supplemental material use, and quality of delivery over repeated time points. The initial levels of implementation and change over time were examined in a series of three level longitudinal hierarchical liner models (HLM). Over time, teachers reported somewhat lower levels of program use but maintained their quality of delivery. Teachers with more positive attitudes towards PATHS reported more frequent lesson delivery at the intercept (winter Year 1). Teachers from schools with a more positive culture for social-emotional learning (SEL) rated the quality of their PATHS delivery more positively than teachers from schools lacking SEL culture and reported more frequent supplemental materials use. Early career teachers initially described themselves as delivering PATHS with lower quality but improved similarly over time compared to more experienced teachers. Implications of the findings and recommendations for future research are discussed.

How do collective student behavior and other classroom contextual factors relate to teachers' implementation of an evidence-based intervention? A multilevel structural equation model

Building on prior work regarding the potential for peer contagion or deviance training in group delivered interventions (Dishion & Dodge, 2005, 2006; Dodge, Dishion, & Lansford, 2006), we leveraged data from a randomized trial, testing the integration of two preventive interventions (Promoting Alternative THinking Strategies and PAX Good Behavior Game), to explore the extent to which classroom contextual factors served as either a barrier to or a motivator for teachers to implement the evidence-based PAX Good Behavior Game with high frequency or dosage. We included students' baseline levels of behavior, measured with regard to both positive (i.e., engagement and social emotional skills) and negative (i.e., hyperactive and aggressive-disruptive) behaviors. Data were collected from 204 teachers in 18 urban elementary schools. A series of multilevel structural equation models were fit to the data. The analyses indicated that classrooms with higher classroom levels of aggressive behavior, on average, at baseline had teachers with lower implementation dosage (i.e., played fewer games) across the school year. In addition, teachers who reported higher baseline levels of emotional exhaustion, regardless of student behavior, also reported lower implementation dosage. Taken together, the results indicated that negative, but not positive, contextual factors at baseline were related to lower implementation dosage; this, in turn, suggests that negative contextual factors may serve as a barrier, rather than a motivator, of teachers' implementation dosage of classroom-based preventive interventions.

Promoting Afterschool Quality and Positive Youth Development: Cluster Randomized Trial of the Pax Good Behavior Game

This randomized trial tested a strategy originally developed for school settings, the Pax Good Behavior Game (PAX GBG), in the new context of afterschool programs. We examined this approach in afterschool since 70% of all juvenile crime occurs between the hours of 3-6 pm, making afterschool an important setting for prevention and promotion. Dual-career and working families need monitoring and supervision for their children in quality settings that are safe and appropriately structured. While substantial work has identified important features of afterschool programs, increasing attention is being given to how to foster quality. PAX GBG, with its focus on shared norms, cooperative teams, contingent activity rewards, and liberal praise, could potentially enhance not only appropriate structure and supportive relationships, but also youth self-regulation, co-regulation, and socio-emotional development. This study examined the PAX GBG among 76 afterschool programs, serving 811 youth ages 5-12, who were diverse in race-ethnicity, socio-economic status, and geographic locale. Demographically matched pairs of afterschool programs were randomized to PAX GBG or treatment-as-usual. Independent observers conducted ratings of implementation fidelity and program quality across time; along with surveys of children's problem and prosocial behavior. Interaction effects were found using hierarchical linear models such that experimental programs evidencing higher implementation fidelity demonstrated better program quality than controls, (i.e., less harshness, increased appropriate structure, support, and engagement), as well as reduced child-reported hyperactivity and intent-to-treat effects on prosocial behavior. This study demonstrates that best practices fostered by PAX GBG and implemented with fidelity in afterschool result in higher quality contexts for positive youth development.

Percepções da comunidade escolar sobre a implantação do programa unplugged em escolas

Resumo O objetivo do presente estudo foi investigar as percepções da comunidade escolar sobre o programa Unplugged, implantado pelo Ministério da Saúde em escolas públicas brasileiras de ensino fundamental II. A metodologia utilizada foi a qualitativa e envolveu coleta de dados através de 11 grupos focais de professores (n=13), alunos (n=54) e administradores escolares (n=11) em 8 escolas de 3 cidades brasileiras. Análise de conteúdo permitiu a identificação de 3 eixos temáticos. As facilidades identificadas estiveram relacionadas com o suporte técnico para implantação da metodologia, como a formação e supervisão recebida, além do suporte dos administradores das escolas. As dificuldades apontadas estiveram relacionadas ao tempo das aulas, a obtenção de materiais de suporte para as aulas e ao cumprimento adequado do currículo regular. Como potencialidade, identificou-se a melhoria das relações de convivência. Os resultados aqui alcançados permitem o aprimoramento da implantação do programa Unplugged nas escolas brasileiras.

Challenges, Opportunities, and Methods for Large-Scale Evaluations

The logic, theory, and practice of large-scale evaluation were once limited to large federal initiatives. However, with the advent of regularly collected performance measures and the often multisite implementation of quality improvement efforts, there is an opportunity to adopt large-scale evaluation methods in local and regional evaluation efforts. While ineffective programs show little variation in their ineffectiveness, effective programs generally show a range of effects. A central task of large-scale evaluation is to describe and ascribe why the same program, implemented in multiple settings, produces different effects. By its attention to variation attributable to setting, activities, outputs, and by participants and documenting the conditions in which programs achieve greater and lesser success, large-scale evaluation supports the needs of decision-makers when choosing to implement an evidence-based program. In addition to knowing a program is effective, decision-makers want to know whether it is appropriate for their situation and what facilitates or impedes effective implementation and bears on the program's ultimate effectiveness. This article presents the different methods and approaches appropriate for effectively and efficiently constructing and executing a large-scale evaluation that will provide decision-makers the evidence they need for evidence-informed adoption of effective programs.

Why Research Design and Methods Is So Crucial to Understanding Drug Use/Abuse: Introduction to the Special Issue

The collection of articles in this special issue both raise the bar and inspire new thinking with regard to both design and methodology concerns that influence drug use/abuse research. Thematically speaking, the articles focus on issues related to missing data, response formats, strategies for data harmonization, propensity scoring methods as an alternative to randomized control trials, integrative data analysis, statistical corrections to reduce bias from attrition, challenges faced from conducting large-scale evaluations, and employing abductive theory of method as an alternative to the more traditional hypothetico-deductive reasoning. Collectively, these issues are of paramount importance as they provide specific means to improve our investigative tools and refine the logical framework we employ to examine the problem of drug use/abuse. Each of the authors addresses a specific challenge outlining how it affects our current research efforts and then outlines remedies that can advance the field. To their credit, they have included issues that affect both etiology and prevention, thus broadening our horizons as we learn more about developmental processes causally related to drug use/abuse and intervention strategies that can mitigate developmental vulnerability. This is the essential dialogue required to advance our intellectual tool kit and improve the research skills we bring to bear on the important questions facing the field of drug use/abuse. Ultimately, the goal is to increase our ability to identify the causes and consequences of drug use/abuse and find ways to ameliorate these problems as we engage the public health agenda.

Factors Associated with Teacher Delivery of a Classroom-Based Tier 2 Prevention Program

Teachers sometimes struggle to deliver evidence-based programs designed to prevent and ameliorate chronic problem behaviors of young children with integrity. Identifying factors associated with variations in the quantity and quality of delivery is thus an important goal for the field. This study investigated factors associated with teacher treatment integrity of BEST in CLASS, a tier-2 prevention program designed for young children at risk for developing emotional/behavioral disorders. Ninety-two early childhood teachers and 231 young children at-risk for emotional/behavioral disorders participated in the study. Latent growth curve analyses indicated that both adherence and competence of delivery increased across six observed time points. Results suggest that teacher education and initial levels of classroom quality may be important factors to consider when teachers deliver tier-2 (i.e., targeted to children who are not responsive to universal or tier-1 programming) prevention programs in early childhood settings. Teachers with higher levels of education delivered the program with more adherence and competence initially. Teachers with higher initial scores on the Emotional Support subscale of the Classroom Assessment Scoring System (CLASS) delivered the program with more competence initially and exhibited higher growth in both adherence and competence of delivery across time. Teachers with higher initial scores on the Classroom Organization subscale of the CLASS exhibited lower growth in adherence across time. Contrary to hypotheses, teacher self-efficacy did not predict adherence, and teachers who reported higher initial levels of Student Engagement self-efficacy exhibited lower growth in competence of delivery. Results are discussed in relation to teacher delivery of evidence-based programs in early childhood classrooms.

Schooling and Children's Mental Health: Realigning Resources to Reduce Disparities and Advance Public Health

Schools have long been the primary setting for children's mental health services but have neither the resources nor the expertise to manage these services independently. The critical importance of school success for children's adjustment provides a strong rationale for schooling as an essential component of children's mental health services. In this article, we review evidence for how schooling and mental health coalesce, suggesting an alignment of school and community mental health resources that prioritizes successful schooling as a key mental health outcome. We describe collaborative principles and ecological practices that advance a public health focus on children's mental health while also reducing the burden on schools to maintain mental health services. We close with a model of mental health services illustrating these principles and practices in high-poverty urban schools and propose future directions for research and practice to promote positive mental health for all children and youth.

Studying Program Implementation Is Not Easy but It Is Essential

This study offers a commentary on the articles contained in the special issue of Prevention Science, "Readiness to implement Social- Emotional Learning interventions." The commentary also puts these articles into current context by summarizing important findings in implementation research and listing some priorities for future work.