Digital mental health outcome monitoring for a structured text-based youth counselling intervention: Demographic profile and outcome change

BACKGROUND

Digital mental health interventions comprise a potentially effective and accessible form of support for young people, particularly at times when traditional face-to-face service delivery is reduced, as in the COVID-19 pandemic.

AIMS

This study assessed the demographic profile of young people using a digital mental health support service and evaluated outcome change over the course of a structured online counselling intervention (synchronous text-chat sessions with a practitioner).

MATERIALS AND METHODS

The data were collected from 23,260 young people aged between 10 and 25 years engaging with the intervention between April 2019 and June 2021.

RESULTS

Young people accessing these services had high levels of mental health needs, particularly those identifying with non-binary gender identity. Service users were mostly female, with equitable rates of access for young people from racialised communities. Overall outcome change demonstrated small effect sizes according to the YP-CORE (0.19) and CORE-10 (0.38), which increased to a moderate level when young people remained engaged with a dedicated practitioner for at least seven sessions (0.38, 0.58). Regression analysis illustrated the effect of the number of sessions on outcome change, but this can be also influenced by other variables such as age and gender.

DISCUSSION AND CONCLUSION

Further research is required to explore ways to engage with young people using digital web-based services for a longer period and to collect and analyse single-session outcome data.

Examining concurrent validity and item selection of the Session Wants and Needs Outcome Measure (SWAN-OM) in a children and young people web-based therapy service

BACKGROUND

Single-session mental health interventions are frequently attended by children and young people (CYP) in both web-based and face-to-face therapy settings. The Session "Wants" and "Needs" Outcome Measure (SWAN-OM) is an instrument developed in a web-based therapy service to overcome the challenges of collecting outcomes and experiences of single-session therapies (SSTs). It provides pre-defined goals for the session, selected by the young person prior to the intervention, on which progress toward achievement is scored at the end of the session.

OBJECTIVE

The objective of this study was to evaluate the instrument's psychometric properties, including concurrent validity against three other frequently used outcome and experience measures, at a web-based and text-based mental health service.

METHODS

The SWAN-OM was administered for a period of 6 months to 1,401 CYP (aged 10-32 years; 79.3% white; 77.59% female) accessing SST on a web-based service. Item correlations with comparator measures and hierarchical logistic regressions to predict item selection were calculated for concurrent validity and psychometric exploration.

RESULTS

The most frequently selected items were "Feel better" (N = 431; 11.61%) and "Find ways I can help myself" (N = 411; 11.07%); unpopular items were "Feel safe in my relationships" (N = 53; 1.43%) and "Learn the steps to achieve something I want" (N = 58; 1.56%). The SWAN-OM was significantly correlated with the Experience of Service Questionnaire, particularly the item "Feel better" [rs₍₁₀₉₎ = 0.48, p < 0.001], the Youth Counseling Impact Scale, particularly the item "Learn the steps to achieve something I want" [rs₍₂₂₎ = 0.76, p < 0.001], and the Positive and Negative Affect Schedule, particularly the items "Learn how to feel better" [rs₍₂₂₎ = 0.72, p < 0.001] and "Explore how I feel" [rs₍₇₀₎ = -0.44, p < 0.001].

CONCLUSION

The SWAN-OM demonstrates good concurrent validity with common measures of outcome and experience. Analysis suggests that lesser-endorsed items may be removed in future iterations of the measure to improve functionality. Future research is required to explore SWAN-OM's potential to measure meaningful change in a range of therapeutic settings.

The design and development of an experience measure for a peer community moderated forum in a digital mental health service

Online digital mental health communities can contribute to users' mental health positively and negatively. Yet the measurement of experience, outcomes and impact mechanisms relating to digital mental health communities is difficult to capture. In this paper we demonstrate the development of an online experience measure for a specific children and young people's community forum inside a digital mental health service. The development of the Peer Online Community Experience Measure (POCEM) is informed by a multi-phased design: (i) item reduction through Estimate-Talk-Estimate modified Delphi methods, (ii) user testing with think-aloud protocols and (iii) a pilot study within the digital service community to explore observational data within the platform. Experts in the field were consulted to help reduce the items in the pool and to check their theoretical coherence. User testing workshops helped to inform the usability appearance, wording, and purpose of the measure. Finally, the pilot results highlight completion rates, differences in scores for age and roles and "relate to others", as the most frequent domain mechanism of support for this community. Outcomes frequently selected show the importance of certain aspects of the community, such as safety, connection, and non-judgment previously highlighted in the literature. Experience measures like this one could be used as indicators of active therapeutic engagement within the forum community and its content but further research is required to ascertain its acceptability and validity. Multi-phased approaches involving stakeholders and user-centred design activities enhances the development of digitally enabled measurement tools.

The Helpfulness of Web-Based Mental Health and Well-being Forums for Providing Peer Support for Young People: Cross-sectional Exploration

Background

Young people are increasingly seeking out web-based support for their mental health and well-being. Peer support forums are popular with this age group, with young individuals valuing the fact that the forums are available 24/7, providing a safe and anonymous space for exploration. Currently, little systematic evaluation of the helpfulness of such forums in providing support has been conducted.

Objective

This study examined the helpfulness of the support offered within web-based mental health and well-being peer support forums for young people. It specifically investigated the self-reported user ratings of helpfulness reported through the completion of a developing experience measure. The ratings will be used to consider further development of the measure and reflect upon the overall helpfulness of the forums as indicated by the reported scores.

Methods

The study used routinely collected practice-based outcome data from web-based mental health forums for young people. These forums are hosted by the UK-based web-based therapy and support service, Kooth. A cross-sectional design was used to explore—using a range of inferential statistical measures—the outcomes reported by those accessing the forums using a Peer Online Community Experience Measure (POCEM). To consider the helpfulness in general, 23,443 POCEMs completed in 2020 were used. A second data set of 17,137 completed POCEMs from the same year was used to consider whether various engagement indicators had an impact upon the helpfulness rating.

Results

Female users aged between 11 and 16 years predominantly completed the POCEM. This is in keeping with the majority of those using the service. In total, 74.6% (8240/11,045) of the scores on the POCEM indicated that the individuals found the posts helpful. An ANOVA indicated that male users were more likely to report obtaining intrapersonal support, whereas female users obtained interpersonal support. Furthermore, the POCEM scores reflected the internal consistency of the measure and provided an insight into the way that young people made use of the peer support resource; for instance, posts that were rated more helpful were correlated with spending longer time reading them, and the topics discussed varied throughout the day with more mental health issues being discussed later at night.

Conclusions

The results seem to demonstrate that, overall, the young people involved in this study found web-based peer support helpful. They indicate that peer support can provide an important strand of care within a supportive mental health ecosystem, particularly during time periods when in-person support is typically closed. However, limitations were noted, suggesting that caution is needed when interpreting the results of this study. Although such services are incredibly well used, they have received little research attention to date. As such, further investigation into what constitutes helpful and unhelpful peer support is needed.

The Therapeutic Goals Set by University Students in an Anonymous Web-Based Therapy and Support Setting

The interest in student mental health and wellbeing has increased in recent years. Additionally, there is a rising volume of students seeking support. Numerous online resources have been developed to meet this need, including anonymous web-based therapy. To date, there has been little focus upon how students may utilise such a service, and this study examines routine evaluation data (solicited from a Goal-based Outcome Measure) from a United Kingdom based service provider. Over the course of one academic year (2018–2019), 211 students articulated therapeutic goals within Kooth Student, a web-based therapy and support service for individuals in higher education. These goals were examined for key trends. The students identified a total of 625 goals to work on in therapy, with individuals setting an average of three goals each. The most common goals focused upon obtaining additional support within the service and exploring their emotions. The results suggested that female students were more likely to move towards achieving their goals, with goals that did move shifting an average of 7.74 on a 10-point scale. Practical goals that focused upon getting more help, both inside and outside the service, were most likely to be achieved. In contrast, self-help/self-care goals were less likely to be achieved. These results provide a helpful insight into how students made use of therapy and highlight the importance of the interaction that web-based services have with other provision (web-based and in-person). They also demonstrate the challenge of capturing meaningful outcome data in anonymous services.

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support

Single-session, brief interventions in therapy for young people make up a large proportion of service provision, including in digital mental health settings. Current nomothetic mental health measures are not specifically designed to capture the benefit or ‘change’ directly related to these brief interventions. As a consequence, we set out to design an outcome measure to concretely demonstrate the value of single-session interventions. The Session Wants and Needs Outcome Measure (SWAN-OM) aims to capture in-session goals and focuses on being user-centric, elements critical to the success of single-session and brief interventions which typically are asset-based and solution-focused. We describe the 4-stage process that was followed to develop this measure: (I) classical item generation and development, (II) content and (III) face validity pilot testing, and (IV) a user-experience approach with young people using framework analysis. This final stage was critical to ensure the integration of this outcome tool into a web-based digital therapy setting, a context which adds another layer of design complexity to item and measure development. This iterative methodology was used to overcome the challenges encountered and to place the needs of the young people and service practitioners at the centre of the design process, thus ensuring measure usability. To end, we highlight the main lessons learnt from engaging in this design process. Specifically, the needs of a measure for single-session interventions are considered, before outlining the learning associated with integrating the measure into a digital mental health platform. Both of these areas are emerging fields and, as such, this study contributes to our understanding of how an idiographic patient outcome theory driven measure can be created for use in a web-based digital mental health therapy service.

Assessing the need of young people using online counselling services: how useful are standardised measures?

BACKGROUND

Clinical assessments for children and young people entering a mental health service help to identify the prevalence of need within that population, support intervention recommendations, and enable service evaluation. Evidence related to the use of standardised measures in an ever-expanding online environment, for the purpose of identifying need, is limited.

METHODS

This study explores the reliability of using a standardised measure to detect clinical need in an online therapeutic environment, and the measures assessed are as follows: Strengths and Difficulties Questionnaire (SDQ), Young Person's CORE (YP-CORE) and the Short Warwick and Edinburgh Wellbeing Scale (SWEMWBS). A deep-dive approach is used to inform practitioner assessment of young people, followed by a Weighted Cohen's Kappa (Κw) to measure the interrater reliability between this and the individuals' self-rated outcome. Composite case studies represent the complexities of presentation among the sample population.

RESULTS

The interrater reliability between self-rated and practitioner rated assessment varied between Κw = .222 and Κw = 0.446 depending on the measure. High levels of need and low levels of well-being were found among the sample (YP-CORE Avg. = 26.9, SDQ Avg. = 19.56, SWEMWBS Avg. = 18.1).

CONCLUSIONS

The findings demonstrate a fair to moderate reliability when assessing concordance between service users and practitioners, which suggests standardised measures are a reliable indicator of need. Higher levels of need were present than those seen previously in general or face-to-face clinical populations, which suggests using such measures in an online therapeutic environment influences the way in which assessments are responded to.

A Theory of Change for Web-Based Therapy and Support Services for Children and Young People: Collaborative Qualitative Exploration

BACKGROUND

Web-based counseling and support has become increasingly commonplace for children and young people (CYP). Currently, there is limited research that focuses on the mechanisms of change within complex telepsychology platforms, a factor that makes designing and implementing outcome measures challenging.

OBJECTIVE

This project aims to articulate a theory of change (ToC) for Kooth, a web-based therapy and support platform for CYP.

METHODS

A collaborative qualitative research design involving professional staff, academic partners, and young people was used to develop the ToC. The following three major reflective phases were engaged: a scoping workshop involving professional staff and academic partners, a series of explorative projects were completed to inform the development of the ToC, and the draft ToC was reviewed for coherence by key stakeholders (young people, online professionals, and service managers).

RESULTS

A collaboratively developed ToC was presented. This was divided into the conditions that lead to individuals wanting to access web-based therapy and support (eg, individuals wanting support there and then or quickly), the mode of service delivery (eg, skilled and experienced professionals able to build empathetic relationships with CYP), and the observed and reported changes that occur as a consequence of using the service (eg, individuals being better able to manage current and future situations).

CONCLUSIONS

Developing the ToC helps to shed light on how web-based therapy and support services aid the mental health and well-being of CYP. Furthermore, it helps to understand the development of positive virtual ecosystems and can be used to devise evaluative tools for CYP telepsychology providers.