Developing Treatment Integrity Measures for Teacher-Delivered Interventions: Progress, Recommendations and Future Directions

Using Adherence and Competence Measures Based on Practice Elements to Evaluate Treatment Fidelity for Two CBT Programs for Youth Anxiety

Measures designed to assess the quantity and quality of practices found across treatment programs for specific youth emotional or behavioral problems may be a good fit for evaluating treatment fidelity in effectiveness and implementation research. Treatment fidelity measures must demonstrate certain reliability and validity characteristics to realize this potential. This study examines the extent to which two observational measures, the Cognitive-Behavioral Treatment for Anxiety in Youth Adherence Scale (CBAY-A) and the CBAY Competence Scale (CBAY-C), can assess the quantity (the degree to which prescribed therapeutic techniques are delivered as intended) or quality (the competence with which prescribed techniques are delivered) of practices found in two distinct treatment programs for youth anxiety. Treatment sessions (N = 796) from 55 youth participants (M age = 9.89 years, SD = 1.71; 46% female; 55% White) with primary anxiety problems who participated in an effectiveness study were independently coded by raters who coded quantity, quality, and the youth-clinician alliance. Youth received one of three treatments: (a) standard (i.e., cognitive-behavioral therapy program), (b) modular (i.e., a cognitive-behavioral and parent-training program), and (c) usual clinical care. Interrater reliability for the CBAY-A items was good across the standard and modular conditions but mixed for the CBAY-C items. Across the standard and modular conditions, the CBAY-A Model subscale scores demonstrated evidence of construct validity, but the CBAY-C Model subscale scores showed mixed evidence. The results provide preliminary evidence that the CBAY-A can be used across different treatment programs but raise concerns about the generalizability of the CBAY-C.

Examining the Role of Student Responsiveness in Treatment Effects of a Tier 2 Program Targeting Reductions in Problem Behavior

Student responsiveness's role in promoting intervention outcomes for students who exhibit problem behavior is understudied. Due to the relational nature of many interventions delivered by teachers that target social, emotional, or behavioral outcomes of students in classrooms, it is essential to assess how responsive students are to teachers' attempts to engage them in the intervention, particularly for students with problem behaviors that may impede teachers' attempts to engage these students in intervention effectively. In the current study, we combine samples from four randomized controlled trials to examine the relationship between student outcomes and teacher attempts to deliver BEST in CLASS, a Tier 2 intervention, via student responsiveness. Delivery of BEST in CLASS and student responsiveness were assessed through direct observations and teachers' reported measures. Results suggest that teacher adherence and competence in delivering BEST in CLASS practices was associated with reductions in problem behavior from pretest to post-test via student responsiveness. Limitations of the current study and implications for future research are discussed.

Children’s Perspectives on Using Serious Games as a Complement to Promoting Their Social–Emotional Skills

The use of serious games may be an appealing and complementary way to motivate curriculum-based social and emotional learning (SEL); still, investigation into this potential usefulness is scarce. This study aims to address the usefulness of serious games within the program ‘Me and Us of Emotions’. Specifically, we analyzed the differences in children’s satisfaction in sessions that did or did not use serious games as a complement to the intervention, explored the contribution of using serious games to the global satisfaction with the program, and explored children’s qualitative feedback regarding the sessions. The participants were 232 children (122 boys and 110 girls) aged between 8 and 12 years old (M = 9.09, SD = 0.80). The measures were based on the subjective appraisals of the sessions made by the participating children, including quantitative and qualitative assessments of the degree of satisfaction of the participants. The results showed that there were similar levels of satisfaction with the sessions that did or did not use serious games as a complement to the program. However, only satisfaction with the sessions that used serious games (and not satisfaction with the sessions that did not use them) contributed significantly to explaining both the enjoyment of the activities and the interest in the subjects. Satisfaction with serious games was significantly and positively associated with fun, easiness, ability to understand the session, and ability to cope with emotions. Qualitative analysis showed three main themes, namely: positive aspects, negative aspects, and opportunities for improvement of the program. Overall, these results indicate that children’s satisfaction with the ‘Me and Us of Emotions’ program is related to serious games, suggesting the relevance of using this complementary tool more often when intervening with younger generations.