Psychometrics of the Symptoms and Functioning Severity Scale for High-Risk Youth

A semi-structured interview for the dimensional assessment of internalizing and externalizing symptoms in children and adolescents: Interview Version of the Symptoms and Functioning Severity Scale (SFSS-I)

BACKGROUND

This study evaluates the psychometric properties of the newly developed semi-structured interview, Interview Version of the Symptoms and Functioning Severity Scale (SFSS-I), which is designed to provide a dimensional assessment of internalizing and externalizing symptoms.

METHODS

Multi-informant baseline data from the OPTIE study was used, involving 358 children and adolescents aged 6 to 17 years (M = 11.54, SD = 3.4, n = 140 [39.1%] were female). Participants were screened for internalizing and externalizing symptoms. For validity analyses, caregiver (Child Behavior Checklist), youth (Youth Self Report), and teacher ratings (Teacher Report Form) were used. We performed Receiver Operating Characteristic (ROC) analyses to evaluate the effectiveness of the SFSS-I subscales in distinguishing between children and adolescents diagnosed with internalizing and externalizing disorders, as determined by clinical judgement in routine care.

RESULTS

Confirmatory factor analyses supported a correlated two-factor model for internalizing and externalizing symptoms. Acceptable to good internal consistencies (α = 0.76 to 0.89; ω = 0.76 to 0.90) and excellent interrater reliability on the scale level (ICC ≥ 0.91) was found. The ROC analyses showed an acceptable accuracy in identifying internalizing diagnoses (AUC = 0.76) and excellent accuracy for externalizing diagnoses (AUC = 0.84).

CONCLUSION

The SFSS-I demonstrates potential as a clinically-rated instrument for screening and routine outcome monitoring, offering utility in both clinical practice and research settings for the dimensional assessment of broad psychopathological dimensions.

TRIAL REGISTRATION

German Clinical Trials Register (DRKS) DRKS00016737 ( https://www.drks.de/DRKS00016737 ). Registered 17 September, 2019.

Examining Change in Therapeutic Alliance to Predict Youth Mental Health Outcomes

OBJECTIVE

To examine the link between therapeutic alliance and youth outcomes.

METHOD

The study was conducted at a group-home with 112 youth with a disruptive-behavior diagnosis. Therapeutic alliance was collected routinely via youth and staff report. Outcome data were collected using youth and staff reports of externalizing behavior as well as behavioral incidents occurring during care. Outcome data were collected following intake into services and at 6 and 12 months of care. Data were analyzed to examine (1) if youth behavior problems at intake were predictive of therapeutic alliance and (2) if changes in alliance were predictive of subsequent youth outcomes. These were conducted with a 6-month service-delivery model and replicated with a 12-month model.

RESULTS

There was some support for the first hypothesis, that initial levels of youth externalizing behavior would be related to alliance ratings; however, most of the effects were marginally significant. The second hypothesis, that changes in therapeutic alliance would be related to subsequent youth outcomes, was supported for the 6-month model, but not the 12-month model.

CONCLUSIONS

Changes in therapeutic alliance may be predictive of youth outcomes during care. Additional research into examining therapeutic alliance trajectories is warranted to improve mental health services for youth.

Psychometric Evaluation of the Symptoms and Functioning Severity Scale (SFSS) Short Forms with Out-of-Home Care Youth

BACKGROUND

There is a need for brief progress monitoring measures of behavioral and emotional symptoms for youth in out-of-home care. The Symptoms and Functioning Severity Scale (SFSS; Bickman et al., 2010) is one measure that has clinician and youth short forms (SFSS-SFs); however, the psychometric soundness of the SFSS-SFs with youth in out-of-home care has yet to be examined.

OBJECTIVE

The objective was to determine if the psychometric characteristics of the clinician and youth SFSS-SFs are viable for use in out-of-home care programs.

METHODS

The participants included 143 youth receiving residential treatment and 52 direct care residential staff. The current study assessed internal consistency and alternate forms reliability for SFSS-SFs for youth in a residential care setting. Further, a binary classification test was completed to determine if the SFSS-SFs similarly classified youth as the SFSS full version for low- and elevated-severity.

RESULTS

The internal consistency for the clinician and youth SFSS-SFs was adequate (α = .75 to .82) as was the parallel forms reliability (r = .85 to .97). The sensitivity (0.80 to 0.95), specificity (0.88 to 0.97), and overall accuracy (0.89 to 0.93) for differentiating low and elevated symptom severity was acceptable.

CONCLUSIONS

The clinician and youth SFSS-SFs have acceptable psychometrics and may be beneficial for progress monitoring and additional research should clarify their potential for progress monitoring of youth in out-of-home programs.

Validation of the symptoms and functioning severity scale in residential group care

Tests that measure the emotional and behavioral problems of children and youth are typically not normed and standardized on youth diagnosed with disruptive behavior, particularly those youth in residential care. Yet professional standards mandate that before instruments are used with a specific population the psychometric properties need to be studied and re-established: specifically, psychometric properties, including validity, need to be evaluated (AERA, APA, and NCME, The standards for educational and psychological testing. AERA, Washington, DC, 1999). The purpose of the present study was to assess the validity characteristics of the Symptoms and Functioning Severity Scale (SFSS; Bickman et al., Manual of the Peabody Treatment Progress Battery, Vanderbilt University, Nashville, TN, 2010), a widely used test developed for use in outpatient clinics, with youth in a residential care program. The convergent validity of the SFSS was established with the large correlations (0.78-0.86) with the CBCL. Several binary classification analyses including specificity, area under the receiver operating characteristic curve, positive and negative likelihood ratios, and the Youden Index supported the validity of the SFSS. However, the sensitivity index was somewhat low indicating the test may produce a high level of false negatives. Limitations, future research and implications are discussed.