Overcoming challenges in outcome evaluations of school mental health programs

Three-Year Outcomes of a School-Hospital Partnership Providing Multitiered Mental Health Services in Urban Schools

BACKGROUND

From the perspective of schools, children's mental health is essential to academic success. Multitiered systems of support (MTSS) that provide prevention/promotion, early identification/intervention, and clinical treatment services have been embraced as a promising means to deliver mental health services in the school setting.

METHODS

A school-hospital partnership tested the implementation over 3 consecutive academic years of an MTSS model for elementary through high-school students enrolled in 6 urban public schools. The model was designed to embed whole-school and classroom-level prevention/promotion programs alongside small group and individual services for selected youth. Utilization, satisfaction, and student- and school-level outcomes were assessed.

RESULTS

Students participating in early intervention had significantly improved social-emotional competencies and coping skills (particularly those students at highest risk), and students with mental health disorders who participated in clinical treatment had significantly improved symptoms and functioning. Schools significantly enhanced their capacity to provide mental health services across prevention/promotion, early identification/intervention, and crisis management domains. These findings were replicated across 3 consecutive years of program implementation.

CONCLUSIONS

A school-hospital partnership using an MTSS model was found to be feasible, valued, and effective in improving students' mental health status and schools' capacities to provide mental health services.

Measuring Success: Evaluation Designs and Approaches to Assessing the Impact of School-Based Health Centers

Since the founding of the first school-based health centers (SBHCs) >45 years ago, researchers have attempted to measure their impact on child and adolescent physical and mental health and academic outcomes. A review of the literature finds that SBHC evaluation studies have been diverse, encompassing different outcomes and varying target populations, study periods, methodological designs, and scales. A complex picture emerges of the impact of SBHCs on health outcomes, which may be a function of the specific health outcomes examined, the health needs of specific communities and schools, the characteristics of the individuals assessed, and/or the specific constellation of SBHC services. SBHC evaluations face numerous challenges that affect the interpretation of evaluation findings, including maturation, self-selection, low statistical power, and displacement effects. Using novel approaches such as implementing a multipronged approach to maximize participation, entering-class proxy-baseline design, propensity score methods, data set linkage, and multisite collaboration may mitigate documented challenges in SBHC evaluation.

School-related and social-emotional outcomes of providing mental health services in schools

This study evaluated student outcomes of an expanded school mental health (ESMH) model that placed community mental health clinicians on-site in schools to identify and treat children with mental health needs. The first aim of this study was to consider school-related outcomes (suspension rates and attendance rates) for those students who received ESMH treatment (n = 159) were compared to a matched high-risk sample that did not receive such services (n = 148). Results demonstrated differences between groups over time on measures of suspensions and attendance but not academic achievement. The second aim of this study was to evaluate change in social-emotional functioning (Strengths and Difficulties Questionnaire Scores) over time for the treatment group. Results indicated significant improvements on several parent and teacher ratings. Despite limitations of the ESMH framework examined in this study, the overall results suggest some promising advantages for students who received ESMH services.

A cluster randomized trial to evaluate external support for the implementation of positive behavioral interventions and supports by school personnel

Background

Urban schools lag behind non-urban schools in attending to the behavioral health needs of their students. This is especially evident with regard to the level of use of evidence-based interventions with school children. Increased used of evidence-based interventions in urban schools would contribute to reducing mental health services disparities in low-income communities. School-wide positive behavioral interventions and supports (SWPBIS) is a service delivery framework that can be used to deliver universal preventive interventions and evidence-based behavioral health treatments, such as group cognitive behavioral therapy. In this article, we describe our ongoing research on creating internal capacity for program implementation. We also examine the cost-effectiveness and resulting school climate when two different levels of external support are provided to personnel as they implement a two-tier SWPBIS program.

Methods/Design

The study follows six K – 8 schools in the School District of Philadelphia randomly assigned to consultation support or consultation-plus-coaching support. Participants are: approximately 48 leadership team members, 180 school staff and 3,900 students in Tier 1, and 12 counselors, and 306 child participants in Tier 2. Children who meet inclusion criteria for Tier 2 will participate in group cognitive behavioral therapy for externalizing or anxiety disorders. The study has three phases, baseline/training, implementation, and sustainability. We will measure implementation outcomes, service outcomes, child outcomes, and cost.

Discussion

Findings from this study will provide evidence as to the appropriateness of school-wide prevention and treatment service delivery models for addressing services disparities in schools. The effectiveness and cost-effectiveness analyses of the two levels of training and consultation should help urban school districts and policymakers with the planning and deployment of cost-effective strategies for the implementation of evidence-based interventions for some of the most common behavioral health problems in school children.

Trial registration

ClinicalTrials.gov identifier: NCT01941069

Cognitive-behavior therapy for disaster-exposed youth with posttraumatic stress: results from a multiple-baseline examination

Youth traumatized by natural disasters report high levels of posttraumatic stress such as symptoms of posttraumatic stress disorder, other anxiety disorders, and depression. Research suggests that cognitive behavioral therapies are promising interventions for symptom reduction; however, few cognitive behavioral treatments have been systematically tested in youth hurricane survivors. The current study provides an examination of the efficacy of an intervention manual designed specifically for hurricane-exposed youth (i.e., the StArT manual) using a partially nonconcurrent multiple baseline design. Youth ages 8-13 (n = 6) who met diagnostic criteria for posttraumatic stress disorder were provided the individual StArT treatment in their school. Youth were assessed at pretreatment, weekly during treatment, and at posttreatment. Results provide initial evidence for the efficacy of the StArT manual and suggest the feasibility of conducting the StArT manual in a school setting. The importance of large-scale tests of effectiveness and implementation of cognitive behavioral treatments in the wake of disaster among youth are discussed.

Intervening to improve communication between parents, teachers, and primary care providers of children with ADHD or at high risk for ADHD

This study examines interventions designed to improve communication between individuals who take care of children with ADHD. A teacher rating of 6,171 elementary school children identifies 1,573 children with ADHD or with high risk for ADHD. Parent interviews and information from teachers are collected on 243 children who are randomized into treatment and control conditions and followed for 39 months. The interventions consisted of group workshops and single one-on-one tutorials with parents, teachers, and providers about the evaluation and treatment of ADHD that stressed the need for communication between the three parties. There are few significant effects on communication that are short lived. Results suggest that the interventions are insufficient to cause significant increase in communication. Future attempts to improve parent-teacher-provider coordination should be continuing rather than single-session interventions.