Description and psychometric evaluation of the Youth Counseling Impact Scale

Examining concurrent validity and item selection of the Session Wants and Needs Outcome Measure (SWAN-OM) in a children and young people web-based therapy service

BACKGROUND

Single-session mental health interventions are frequently attended by children and young people (CYP) in both web-based and face-to-face therapy settings. The Session "Wants" and "Needs" Outcome Measure (SWAN-OM) is an instrument developed in a web-based therapy service to overcome the challenges of collecting outcomes and experiences of single-session therapies (SSTs). It provides pre-defined goals for the session, selected by the young person prior to the intervention, on which progress toward achievement is scored at the end of the session.

OBJECTIVE

The objective of this study was to evaluate the instrument's psychometric properties, including concurrent validity against three other frequently used outcome and experience measures, at a web-based and text-based mental health service.

METHODS

The SWAN-OM was administered for a period of 6 months to 1,401 CYP (aged 10-32 years; 79.3% white; 77.59% female) accessing SST on a web-based service. Item correlations with comparator measures and hierarchical logistic regressions to predict item selection were calculated for concurrent validity and psychometric exploration.

RESULTS

The most frequently selected items were "Feel better" (N = 431; 11.61%) and "Find ways I can help myself" (N = 411; 11.07%); unpopular items were "Feel safe in my relationships" (N = 53; 1.43%) and "Learn the steps to achieve something I want" (N = 58; 1.56%). The SWAN-OM was significantly correlated with the Experience of Service Questionnaire, particularly the item "Feel better" [rs₍₁₀₉₎ = 0.48, p < 0.001], the Youth Counseling Impact Scale, particularly the item "Learn the steps to achieve something I want" [rs₍₂₂₎ = 0.76, p < 0.001], and the Positive and Negative Affect Schedule, particularly the items "Learn how to feel better" [rs₍₂₂₎ = 0.72, p < 0.001] and "Explore how I feel" [rs₍₇₀₎ = -0.44, p < 0.001].

CONCLUSION

The SWAN-OM demonstrates good concurrent validity with common measures of outcome and experience. Analysis suggests that lesser-endorsed items may be removed in future iterations of the measure to improve functionality. Future research is required to explore SWAN-OM's potential to measure meaningful change in a range of therapeutic settings.

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support

Single-session, brief interventions in therapy for young people make up a large proportion of service provision, including in digital mental health settings. Current nomothetic mental health measures are not specifically designed to capture the benefit or ‘change’ directly related to these brief interventions. As a consequence, we set out to design an outcome measure to concretely demonstrate the value of single-session interventions. The Session Wants and Needs Outcome Measure (SWAN-OM) aims to capture in-session goals and focuses on being user-centric, elements critical to the success of single-session and brief interventions which typically are asset-based and solution-focused. We describe the 4-stage process that was followed to develop this measure: (I) classical item generation and development, (II) content and (III) face validity pilot testing, and (IV) a user-experience approach with young people using framework analysis. This final stage was critical to ensure the integration of this outcome tool into a web-based digital therapy setting, a context which adds another layer of design complexity to item and measure development. This iterative methodology was used to overcome the challenges encountered and to place the needs of the young people and service practitioners at the centre of the design process, thus ensuring measure usability. To end, we highlight the main lessons learnt from engaging in this design process. Specifically, the needs of a measure for single-session interventions are considered, before outlining the learning associated with integrating the measure into a digital mental health platform. Both of these areas are emerging fields and, as such, this study contributes to our understanding of how an idiographic patient outcome theory driven measure can be created for use in a web-based digital mental health therapy service.

Texting at a Child Helpline: How Text Volume, Session Length and Duration, Response Latency, and Waiting Time Are Associated with Counseling Impact

This study investigated the association between formal features, such as text volume, session length and duration, response latency, and waiting time, and the impact of counseling. The analysis was based on 603 text message counseling sessions at a child helpline and connected information about the formal features of the sessions and the effects on clients. The results showed that sessions characterized by more text volume from the counselor in each message, but with fewer messages from beginning to end, were more effective than sessions without these characteristics. Furthermore, session duration was associated with a positive impact, whereas counselor response latency was not. This indicates that clients might benefit from the asynchronous affordance of texting as long as the counselor responds promptly and with dense messages. We also found that the impact measured at end of session predicted the impact measured 2 weeks after the child or young person received counseling.

Texting for Help: Processes and Impact of Text Counseling with Children and Youth with Suicide Ideation

OBJECTIVE

To explore: (1) how children contacting a child helpline with suicide ideation differ from children discussing other topics, (2) whether text messaging effectively helps, and (3) which counselor behaviors are most effective.

METHOD

Of 6,060 text sessions at the Danish national child helpline, 444 concerned suicidality, of which the 102 sessions that included self-rated, end Session ratings were selected for content analysis.

RESULTS

Twenty-six percentage of suicidal children had severe suicidality. The suicide sample had significantly more girls, was older than the nonsuicide sample, and more often recontacted the helpline in the 2 weeks prior to follow-up. 35.9% of suicidal children felt better immediately and over half ended the session with a plan of action. At follow-up, 23.9% of suicidal children reported feeling better; however, 37.0% reported feeling worse. Talking about emotions, expressing empathy, and encouraging the child to talk to someone were associated with positive impacts. Setting boundaries was associated with negative impacts.

CONCLUSIONS

Texting with suicidal children can be helpful, but should be considered a first step toward obtaining more sustainable help. Research is needed to determine how to better help children who felt worse or did not improve in the 2 weeks after contacting the helpline. Suggestions to further training of counselors are discussed.

Free, brief, and validated: Standardized instruments for low-resource mental health settings

Evidence-based assessment has received little attention despite its critical importance to the evidence-based practice movement. Given the limited resources in the public sector, it is necessary for evidence-based assessment to utilize tools with established reliability and validity metrics that are free, easily accessible, and brief. We review tools that meet these criteria for youth and adult mental health for the most prevalent mental health disorders to provide a clinical guide and reference for the selection of assessment tools for public sector settings. We also discuss recommendations for how to move forward the evidence-based assessment agenda.

Therapeutic Alliance Between Youth and Staff in Residential Group Care: Psychometrics of the Therapeutic Alliance Quality Scale

Therapeutic alliance has been frequently studied in individual counseling sessions; however, research on therapeutic alliance in residential settings for youth with mental health diagnoses has been limited. This may be due, in part, to the presence of multiple service providers often in caregiving roles. The purpose of this study was to examine the psychometric quality of a widely utilized measure of therapeutic alliance used in psychotherapy with youth in residential care where the treatment is provided by a trained married couple. We also compared the relationship between youth ratings of their male and female service provider, as well as examined correlations in ratings between youth and staff on therapeutic alliance. Finally, we investigated the direction, magnitude, and trajectory of change in therapeutic alliance over a 12-month period following admission into residential care. The method was a longitudinal assessment of 135 youth and 124 staff regarding therapeutic alliance over the course of 12 months or discharge from services. Results indicated strong psychometric properties and high correlations for youth ratings of both their male and female service providers. However, the correlation was low between youth and service provider ratings of alliance. Longitudinal analyses indicated that rates of therapeutic alliance changed over time.

Measuring youths' perceptions of counseling impact: description, psychometric evaluation, and longitudinal examination of the Youth Counseling Impact Scale v.2

The Youth Counseling Impact Scale (YCIS) is an empirically validated treatment progress measure that assesses youths' perceptions of the short term effectiveness of therapy. Since its initial publication, the original 10-item measure has been shortened to ease measurement burden and revised to include a question about a youth's insight into his or her strengths. The current study describes the development of the revised YCIS (v.2) and evaluates its psychometric properties. Additionally, this study examines whether the YCIS (v.2) total score or subscale scores change over time and investigates whether there are gender or age differences for youths' perceptions of the impact of therapy. Results found the revised version obtained comparable information to that of the original measure, and that the revised version retained the factor structure of the original model with one primary general factor of Counseling Impact and two secondary factors (Insight and Change). Results also suggested that while the YCIS (v.2) total score and Change subscale score did not change linearly over the course of treatment, the Insight subscale score showed a small but significant linear increase over time. No significant differences in YCIS scores based on youth age or gender were found. The implication of these findings, the clinical and empirical utility of this measure, and its limitations are discussed.

The Peabody Treatment Progress Battery: history and methods for developing a comprehensive measurement battery for youth mental health

There is increased need for comprehensive, flexible, and evidence-based approaches to measuring the process and outcomes of youth mental health treatment. This paper introduces a special issue dedicated to the Peabody Treatment Progress Battery (PTPB), a battery of measures created to meet this need. The PTPB is an integrated set of brief, reliable, and valid instruments that can be administered efficiently at low cost and can provide systematic feedback for use in treatment planning. It includes eleven measures completed by youth, caregivers, and/or clinicians that assess clinically-relevant constructs such as symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction. This introductory article describes the rationale for the PTPB and its development and evaluation, detailing the specific analytic approaches utilized by the different papers in the special issue and a description of the study and samples from which the participants were taken.

The symptoms and functioning severity scale (SFSS): psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time

This paper describes the development and psychometric evaluation of the symptoms and functioning severity scale (SFSS), which includes three parallel forms to systematically capture clinician, youth, and caregiver perspectives of youth symptoms on a frequent basis. While there is widespread consensus that different raters of youth psychopathology vary significantly in their assessment, this is the first paper that specifically investigated the discrepancies among clinician, youth, and caregiver ratings throughout the treatment process within a community mental health setting. Results for all three respondent versions indicated the SFSS is a psychometrically sound instrument for use in this population. Significant discrepancies in scores existed at baseline among the three respondents. Longitudinal analyses reveal the youth-clinician and caregiver-clinician score discrepancies decreased significantly over time. Differences by youth gender existed for caregiver-clinician discrepancies. The average youth-caregiver score discrepancy remained consistent throughout treatment. Implications for future research and clinical practice are discussed.