Development and psychometric evaluation of the youth and caregiver Service Satisfaction Scale

A unique model of care for youth in crisis: A pilot open trial

Suicide is the second leading cause of death for those ages 10-24 years in the United States, and emergency department (ED) visits due to youth self-injurious thoughts and behaviors (SITB) and increased substantially between 2016 and 2021. Although ED services are essential for an effective system of care, the ED setting is typically not well-suited for the comprehensive, collaborative, and therapeutic evaluation of SITB; treatment planning; and care coordination that youth in a suicidal crisis need. As a result, an urgent care model for mental health designed to provide comprehensive crisis triage and intervention services is needed within outpatient psychiatry. This pilot trial examined the feasibility, acceptability, and preliminary clinical outcomes of a brief, urgent care model, the Behavioral Health Crisis Care Clinic (CCC), designed to provide comprehensive outpatient triage and intervention services aimed at reducing suicide risk for youth in crisis. Participants were 189 youth (ages 10-20; 62.4% females; 58% Caucasian) who had past-week suicidal ideation or behavior and their caregivers. The results demonstrated the CCC model exceeded feasibility and acceptability benchmarks based on the Service Satisfaction Scale (M score > 3.00). CCC care was associated with significant decreases in self-reported suicide risk based on the Collaborative Assessment and Management of Suicidality Suicide Status Form with low levels of ED usage during CCC care (7.7%) and 1-month posttreatment (11.8%). Over 88% of patients without established outpatient care at the time of referral were connected to care during CCC treatment, almost all of whom (95%) continued with ongoing mental health care 1 month after ending CCC care. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Service user satisfaction with care in a specialist service for young people with attention deficit hyperactivity disorder

OBJECTIVES

Consumer satisfaction is considered one of the most important measures of service quality in child mental health; however, there is limited understanding of factors that influence satisfaction. The objective of this study was to investigate key factors influencing satisfaction with care (SWC) in ADMiRE, a specialist service for young people (YP) with attention deficit hyperactivity disorder (ADHD).

METHODS

Parents/carers (n = 67) and YP > 9 years (n = 44) attending ADMiRE completed an anonymous Experience of Service Questionnaire (ESQ), a quantitative/qualitative measure of service user satisfaction. Parents/carers also completed symptom severity rating scales. Data were analysed to determine (i) overall SWC, (ii) the relationship between parent- and youth-reported SWC and (iii) the impact of symptom severity on SWC. Thematic analysis of qualitative ESQ data was completed.

RESULTS

Parents/carers were significantly more satisfied than YP (p = 0.028). Symptom severity did not impact significantly on parent/carer satisfaction. YP with severe hyperactive/impulsive and inattentive ADHD symptoms were significantly less satisfied with care than those with less severe ADHD symptoms (p = 0.022 and p = 0.017 respectively). Factors related to the therapeutic alliance were identified as being particularly important to both parents/carers and YP.

CONCLUSIONS

This is the first Irish study that has investigated the impact of symptom severity on service user satisfaction in a child mental health service. The results highlight the different perspectives of YP and parents and provide novel insights into the impact of symptom severity on service user satisfaction. The importance of the therapeutic alliance should not be underestimated in future development of services.

Examining the psychometric properties of the headspace Youth (mental health) Service Satisfaction Scale in a mental health service in Ireland

INTRODUCTION

Evaluating service quality and satisfaction is central to the provision of accessible and developmentally appropriate youth mental health services. However, there are limited suitable measures and a lack of published evidence on the psychometric properties of measures to assess young people's satisfaction with youth mental health services. The headspace Youth (Mental Health) Service Satisfaction Scale (YSSS) was designed and implemented to assess young people's satisfaction with headspace mental health services in Australia. This study examined the reliability and factor structure of the YSSS in a youth mental health service in Ireland.

METHODS

The sample comprised 1449 young people (66.2% female) aged 12-25 years (M = 16.48, SD = 2.97). Participants completed the YSSS after their final brief intervention session through Jigsaw-The National Centre for Youth Mental Health. Confirmatory factor analysis (CFA) was performed on one- and four-factor models to test findings from previous studies. Reliability was also examined.

RESULTS

CFA supported a single-factor structure of the YSSS, and all items were suitable for inclusion. The internal consistency of the measure was deemed acceptable (α = 0.89).

CONCLUSIONS

Findings suggest that the YSSS is a reliable measure for monitoring satisfaction with youth mental health services in an Irish context. The measure demonstrated a unidimensional construct of satisfaction. These findings support the broader application of the YSSS and add to existing knowledge on measuring satisfaction within youth mental health services.

Better together? A randomized controlled microtrial comparing different levels of therapist and parental involvement in exposure-based treatment of childhood specific phobia

INTRODUCTION

Exposure is often limited to homework assignments in routine clinical care. The current study compares minimally-guided (MGE) and parent-guided (PGE) out-session homework formats to the 'golden standard' of therapist-guided in-session exposure with minimally-guided exposure at home (TGE).

METHODS

Children with specific phobia (N = 55, age 8-12, 56% girls) participated in a single-blind, randomized controlled microtrial with a four-week baseline-treatment period design. Clinical interviews, behavioral avoidance tests, and self-report measures were assessed at pre-treatment, post-treatment, and at one-month follow-up.

RESULTS

TGE resulted in a larger decline of specific phobia severity from baseline to post-treatment compared to MGE but not compared to PGE. Parental anxiety was found to be a moderator of less treatment efficacy of PGE from baseline to post-treatment. Overall, there was no meaningful difference in efficacy of TGE versus MGE or PGE from baseline to follow-up.

CONCLUSIONS

These findings suggest that for improving short-term treatment gains, exposure exercises can best be conducted with the help of a therapist within the therapy session before they are conducted as homework assignments outside the therapy session. However, for long-term treatment gains exposure exercises can be handled by the child itself or with help of its parents.

The Feasibility and Efficacy of a Group-Based, Brief Transdiagnostic Cognitive-Behavioral Treatment for Adolescents with Internalizing Problems

The present study aimed to assess the efficacy of a group-based, brief transdiagnostic cognitive-behavioral therapy (CBT) program for adolescents with internalizing problems, such as anxiety and depressive disorders, seeking help in a primary health care clinic in Iceland. The group-based CBT program consisted of eight weekly 110-min sessions covering psychoeducation, cognitive restructuring, behavioral activation, exposure, problem-solving, social skills, and mindfulness. The study recruited 53 participants, who were randomly assigned to either receive the group treatment or be placed on a wait-list for monitoring purposes. Measures were taken at baseline, during treatment (week 4), at posttreatment (week 8), and at 2-, 4-month, and 1-year follow-ups. The primary outcome measures were the self-reported total scores of total anxiety and depression using the Revised Children's Anxiety and Depression Scale (RCADS). The study found a significant effect of time and time * treatment interaction on the depression and anxiety total scores. The secondary outcome measures, RCADS parent-rated depression and anxiety total scores, did not show significant effects of time * treatment interaction. However, during naturalistic follow-up, a significant reduction in parent-reported depression and anxiety total scores was observed. The study also observed good treatment adherence, as well as high satisfaction among parents and youth. The results suggest that this group-based, brief transdiagnostic CBT group treatment is feasible and effective in reducing depressive and anxiety symptoms in adolescents with internalizing problems and highlights the importance of addressing comorbidity in treatment.

An evaluation of satisfaction with emergency department care in children and adolescents with mental health concerns

OBJECTIVES

We hypothesized that an association exists between satisfaction with ED mental health care delivery and patient and system characteristics. Primary: To evaluate overall satisfaction with ED mental health care delivery. Secondary: To explore aspects of ED mental health care delivery associated with general satisfaction, and patient and ED visit characteristic associated with total satisfaction scores and reported care experience themes.

METHODS

We enrolled patients < 18 years of age presenting with a mental health concern between February 1, 2020 and January 31, 2021, to two pediatric EDs in Alberta, Canada. Satisfaction data were collected using the Service Satisfaction Scale, a measure of global satisfaction with mental health services. Association of general satisfaction with ED mental health care was evaluated using Pearson's correlation coefficient and variables associated with total satisfaction score was assessed using multivariable regression analyses. Inductive thematic analysis of qualitative feedback identified satisfaction and patient experience themes.

RESULTS

646 participants were enrolled. 71.2% were Caucasian and 56.3% female. Median age was 13 years (IQR 11-15). Parents/caregivers (n = 606) and adolescents (n = 40) were most satisfied with confidentiality and respect in the ED and least satisfied with how ED services helped reduce symptoms and/or problems. General satisfaction was associated with perceived amount of help received in the ED (r = 0.85) and total satisfaction with evaluation by a mental health team member (p = 0.004) and psychiatrist consultation (p = 0.05). Comments demonstrated satisfaction with ED provider attitudes and interpersonal skills and dissatisfaction with access to mental health and addictions care, wait time, and the impact of COVID-19.

CONCLUSIONS

There is a need to improve ED mental health care delivery, with a focus on timely access to ED mental health providers. Access to outpatient/community-based mental health care is needed to complement care received in the ED and to provide continuity of care for youth with mental health concerns.

Treating specific phobia in youth: A randomized controlled microtrial comparing gradual exposure in large steps to exposure in small steps

INTRODUCTION

Exposure may be especially effective when within exercises, there is a strong violation of threat expectancies and much opportunity for fear reduction. Outcomes of exposure may therefore improve when exposure is conducted in large steps (LargeSE) relative to small steps (SmallSE).

METHODS

Children and young people with a specific phobia (SP) (N = 50, age 8-17, 64 % girls) participated in a preregistered single-blind, randomized controlled microtrial comparing LargeSE and SmallSE in a four-week baseline-treatment design. Clinical interviews, behavioral avoidance tests, and self-report measures were assessed at pre-treatment, post-treatment, and at one-month follow-up.

RESULTS

Within exercises, LargeSE resulted in higher initial fear levels and more within-session expectancy violation. Nevertheless, SmallSE resulted in a larger decline of SP severity from baseline to post-treatment and follow-up, and a larger decline of anxiety and avoidance towards one's individual goal from baseline to follow-up. There were no differences between LargeSE and SmallSE regarding changes in general self-efficacy or behavioral avoidance. Although session duration was standardized and similar for both conditions, participants in SmallSE received more (shorter) exercises.

DISCUSSION

SmallSE might be more effective in reducing SP severity because children in SmallSE were exposed to a larger number and variety of exercises than children in LargeSE.

Personalised app-based relapse prevention of depressive and anxiety disorders in remitted adolescents and young adults: a protocol of the StayFine RCT

INTRODUCTION

Youth in remission of depression or anxiety have high risks of relapse. Relapse prevention interventions may prevent chronicity. Aim of the study is therefore to (1) examine efficacy of the personalised StayFine app for remitted youth and (2) identify high-risk groups for relapse and resilience.

METHOD AND ANALYSIS

In this Dutch single-blind parallel-group randomised controlled trial, efficacy of app-based monitoring combined with guided app-based personalised StayFine intervention modules is assessed compared with monitoring only. In both conditions, care as usual is allowed. StayFine modules plus monitoring is hypothesised to be superior to monitoring only in preventing relapse over 36 months. Participants (N=254) are 13-21 years and in remission of depression or anxiety for >2 months. Randomisation (1:1) is stratified by previous treatment (no treatment vs treatment) and previous episodes (1, 2 or >3 episodes). Assessments include diagnostic interviews, online questionnaires and monitoring (ecological momentary assessment with optional wearable) after 0, 4, 12, 24 and 36 months. The StayFine modules are guided by certified experts by experience and based on preventive cognitive therapy and ingredients of cognitive behavioural therapy. Personalisation is based on shared decision-making informed by baseline assessments and individual symptom networks. Time to relapse (primary outcome) is assessed by the Kiddie Schedule for Affective Disorders and Schizophrenia-lifetime version diagnostic interview. Intention-to-treat survival analyses will be used to examine the data. Secondary outcomes are symptoms of depression and anxiety, number and duration of relapses, global functioning, and quality of life. Mediators and moderators will be explored. Exploratory endpoints are monitoring and wearable outcomes.

ETHICS, FUNDING AND DISSEMINATION

The study was approved by METC Utrecht and is funded by the Netherlands Organisation for Health Research and Development (636310007). Results will be submitted to peer-reviewed scientific journals and presented at (inter)national conferences.

TRIAL REGISTRATION NUMBER

NCT05551468; NL8237.

Integrating Animal-Assisted Therapy Into TF-CBT for Abused Youth With PTSD: A Randomized Controlled Feasibility Trial

This clinical trial examined animal-assisted therapy (AAT) as an adjunct to Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) for abused youth with posttraumatic stress disorder (PTSD). Youth between the ages of 6 and 17 (M = 11.79, SD = 3.08) were randomized to receive standard TF-CBT or TF-CBT with adjunctive AAT (TF-CBT+AAT) employing retired service dogs. Feasibility metrics evaluating the addition of AAT were collected in addition to common clinical outcomes evaluated in TF-CBT trials. The inclusion of AAT increased the number of potential participants who declined participation and there were no noted benefits for treatment retention or satisfaction with services. Analyses showed that the inclusion of AAT did not enhance improvement of PTSD symptom severity (β = .90, t = .94, p = .351) or a number of other outcomes. On the contrary, there were indications from analyses and clinician feedback that AAT may have attenuated improvement in many cases. This study identified a number of important feasibility considerations in the design of studies testing AAT. However, the results examining clinical outcomes suggest that the inclusion of AAT with TF-CBT in the treatment of maltreated youth with PTSD is not warranted at this time.

Establishing a Working Definition of User Experience for eHealth Interventions of Self-reported User Experience Measures With eHealth Researchers and Adolescents: Scoping Review

BACKGROUND

Across eHealth intervention studies involving children, adolescents, and their parents, researchers have measured user experience to assist with intervention development, refinement, and evaluation. To date, no widely accepted definitions or measures of user experience exist to support a standardized approach for evaluation and comparison within or across interventions.

OBJECTIVE

We conduct a scoping review with subsequent Delphi consultation to identify how user experience is defined and measured in eHealth research studies, characterize the measurement tools used, and establish working definitions for domains of user experience that could be used in future eHealth evaluations.

METHODS

We systematically searched electronic databases for published and gray literature available from January 1, 2005, to April 11, 2019. We included studies assessing an eHealth intervention that targeted any health condition and was designed for use by children, adolescents, and their parents. eHealth interventions needed to be web-, computer-, or mobile-based, mediated by the internet with some degree of interactivity. We required studies to report the measurement of user experience as first-person experiences, involving cognitive and behavioral factors reported by intervention users. We appraised the quality of user experience measures in included studies using published criteria: well-established, approaching well-established, promising, or not yet established. We conducted a descriptive analysis of how user experience was defined and measured in each study. Review findings subsequently informed the survey questions used in the Delphi consultations with eHealth researchers and adolescent users for how user experience should be defined and measured.

RESULTS

Of the 8634 articles screened for eligibility, 129 articles and 1 erratum were included in the review. A total of 30 eHealth researchers and 27 adolescents participated in the Delphi consultations. On the basis of the literature and consultations, we proposed working definitions for 6 main user experience domains: acceptability, satisfaction, credibility, usability, user-reported adherence, and perceived impact. Although most studies incorporated a study-specific measure, we identified 10 well-established measures to quantify 5 of the 6 domains of user experience (all except for self-reported adherence). Our adolescent and researcher participants ranked perceived impact as one of the most important domains of user experience and usability as one of the least important domains. Rankings between adolescents and researchers diverged for other domains.

CONCLUSIONS

Findings highlight the various ways in which user experience has been defined and measured across studies and what aspects are most valued by researchers and adolescent users. We propose incorporating the working definitions and available measures of user experience to support consistent evaluation and reporting of outcomes across studies. Future studies can refine the definitions and measurement of user experience, explore how user experience relates to other eHealth outcomes, and inform the design and use of human-centered eHealth interventions.

Treating Speech Anxiety in Youth: A Randomized Controlled Microtrial Testing the Efficacy of Exposure Only Versus Exposure Combined With Anxiety Management Strategies

CBT for anxious youth usually combines anxiety management strategies (AMS) with exposure, with exposure assumed to be critical for treatment success. To limit therapy time while retaining effectiveness, one might optimize CBT by restricting treatment to necessary components. This study tested whether devoting all sessions to exposure is more effective in reducing speech anxiety in youth than devoting half to AMS including cognitive or relaxation strategies and half to exposure. After a 6-week waitlist period, adolescents with speech anxiety (N = 65; age 12-15; 42 girls) were randomized to a 5-session in-school group-based CBT training consisting of either (1) exposure-only (EXP+EXP) or (2) cognitive strategies followed by exposure (COG+EXP) or (3) relaxation strategies followed by exposure (REL+EXP). Clinical interviews, speech tests, and self-report measures were assessed at pretest, posttest, and follow-up. For all conditions (a) the intervention period resulted in a stronger decline of speech anxiety than waitlist period; (b) there was a large sized reduction of speech anxiety that was maintained at six-week follow-up; (c) there was no meaningful difference in the efficacy of EXP+EXP versus COG+EXP or REL+EXP. These findings suggest that devoting all sessions to exposure is not more effective than combining exposure with AMS. AMS appeared neither necessary for CBT to be effective, nor necessary for youth to tolerate exposure. This indicates that CBT can be optimized by restricting treatment to exposure.

Satisfaction with in-patient child and adolescent psychiatric treatment: development and psychometric properties of the BEST questionnaires for adolescents and for parents

BACKGROUND

Interest in the assessment of patient satisfaction with in-patient psychiatric treatment has steadily increased and several measurement tools are available for the quantification of patients' experience. However, they are often uni-dimensional or focus mainly on therapeutic relationship and environment, and neglect other important issues such as information about treatment and participation. The BEST questionnaires were developed as comprehensive instruments that include items on all of the mentioned topics. The present study evaluates the psychometric properties of the BEST in a version for adolescents and for parents. Furthermore, the dimensionality of the satisfaction ratings is analyzed.

METHOD

Descriptive statistics were applied to data of 1582 adolescents (mean age = 15.0 years, SD = 1.65; 62.4% female) and 1998 parents/guardians assessed in seven in-patient units across Germany. The factorial structure of the BEST questionnaires was determined by exploratory and confirmatory factor analyses, including a bifactor model.

RESULTS

The psychometric quality of the scales was strong. Correlations with another assessment instrument of patient satisfaction were good to high, indicating good convergent validity. Exploratory factor analyses revealed three factors in adolescents that were labelled as: Therapeutic relationship, environment, and general satisfaction and treatment success. For parents, the same three factors could be distinguished. Confirmatory bifactor models suggested that the vast majority of variance was accounted for by the general factor; the three specific factors provided some additional information. Agreement between the subscales of adolescents and their parents was only moderate. Parents were usually more satisfied.

CONCLUSIONS

The BEST questionnaires can be considered as reliable and valid instruments to not only assess the "classical" aspects of patient satisfaction, but to also assess newer fundamental topics such as children's rights and treatment participation. For scientific usage, the total score seems superior because of the high explained variance by the general factor, but the subscale scores provide further information. The use of single items seems advantageous for quality management purposes.

Parent- and therapist-rated treatment satisfaction following routine child cognitive-behavioral therapy

This observational study examined treatment satisfaction (TS) following routine outpatient cognitive-behavioral therapy (CBT) in a large sample of children (n = 795; aged 6 to 10 years). TS was investigated in parent and therapist rating. Means, standard deviations and inter-rater correlations were calculated to investigate TS. Regression analysis was conducted to examine potential correlates of TS (patient-related variables, mental disorder characteristics, socio-demographic factors and treatment variables). High TS in parent and therapist rating was found, with therapists showing a lower degree of TS than parents (completely or predominantly satisfied: parent rating 94.1%, therapist rating 69.5%). A statistically significant, moderate inter-rater correlation was found. Regression analysis explained 21.8% of the variance in parent rating and 57.2% in therapist rating. Most of the TS variance was explained by mental disorder characteristics (parent-rated symptoms and therapist-rated global impairment at treatment end) and by treatment variables (especially the therapist-rated cooperation of parents and patients), whereas socio-demographic and patient-related variables did not show any relevant associations with TS. Based on these results, to optimize TS, therapists should concentrate on establishing a sustainable cooperation of parents and children during therapy, and work to achieve a low global impairment at treatment end.

Implementing risk stratification to the treatment of adolescent substance use among youth involved in the juvenile justice system: protocol of a hybrid type I trial

BACKGROUND

Youth involved in the juvenile justice system (YIJJ) have high rates of substance use problems; however, rates of YIJJ engagement in substance use services is low. Barriers to service engagement include lack of appropriate screening and connection to services by the juvenile justice system, as well as lack of resources for delivering evidence-based treatment in community-based settings. To address these barriers, this paper describes a protocol for a type 1 hybrid design to (1) implement universal substance use screening for YIJJ; (2) implement and evaluate the feasibility and effectiveness of a brief, three-session substance use interventions based in motivational interviewing for youth with mild/moderate substance use: Teen Intervene (an individual-based intervention); (3) implement ENCOMPASS, an evidence-based substance use intervention based in motivational enhancement and cognitive behavioral therapy for youth with severe substance use; and (4) evaluate facilitators and barriers to implementing these interventions for mild to severe substance use among YIJJ in community mental health centers (CMHC).

METHODS/DESIGN

Using a hybrid type 1 clinical effectiveness-implementation design, we will collaborate with CMHCs and juvenile justice in two rural Indiana counties. Guided by the EPIS (exploration, preparation, implementation, sustainability) framework, we will measure factors that affect implementation of substance use screening in juvenile justice and implementation of substance use interventions in CMHCs utilizing self-reports and qualitative interviews with juvenile justice and CMHC staff pre- and post-implementation. YIJJ with mild/moderate substance use will receive a brief interventions and YIJJ with severe substance use will receive ENCOMPASS. We will measure the effectiveness of a brief and comprehensive intervention by assessing changes in substance use across treatment. We anticipate recruiting 160 YIJJ and their caregivers into the study. We will assess intervention outcomes utilizing baseline, 3-, and 6-month assessments.

DISCUSSION

Findings have the potential to improve screening and intervention services for YIJJ.

Young People's Satisfaction With the Online Mental Health Service eheadspace: Development and Implementation of a Service Satisfaction Measure

BACKGROUND

Online youth mental health services are an expanding approach to meeting service need and can be used as the first step in a stepped-care approach. However, limited evidence exists regarding satisfaction with online services, and there is no standardized service satisfaction measure.

OBJECTIVE

This study implemented an online youth mental health service satisfaction questionnaire within eheadspace, an online youth mental health service. The aims were to test the questionnaire's psychometric properties and identify current levels of satisfaction among service users, as well as to identify client and service contact characteristics that affect satisfaction.

METHODS

Data were collected from 2280 eheadspace clients via an online questionnaire advertised and accessed through the eheadspace service platform between September 2016 and February 2018. Client and service contact characteristics, potential outcomes, and session and service feedback data were collected.

RESULTS

The service satisfaction questionnaire demonstrated high internal consistency for the overall satisfaction scale (alpha=.95) and its three subscales: session satisfaction, potential outcomes, and service satisfaction. A three-factor model was the best fit to the data, although including a higher order unidimensional construct of overall satisfaction was also a reasonable fit. Overall, young people were very satisfied with eheadspace (mean 3.60, SD 0.83). Service characteristics, but not client characteristics, were significantly associated with satisfaction. Young people were more satisfied with eheadspace when they had greater engagement as evident through receiving esupport rather than briefer service provision, having a longer session and greater interaction with the clinician, and not previously attending a face-to-face headspace center.

CONCLUSIONS

The online youth mental health service satisfaction questionnaire developed for and implemented in eheadspace showed good psychometric properties. The measure is brief, has good internal consistency, and has a clear factor structure. The measure could be adapted for use in other online youth mental health services. The young people using eheadspace and completing the feedback survey were highly satisfied. Greater engagement with the online service was shown to be associated with greater satisfaction. No specific client demographic groups were shown to be more or less satisfied.

Treatment satisfaction following routine outpatient cognitive-behavioral therapy of adolescents with mental disorders: a triple perspective of patients, parents and therapists

The present study investigates treatment satisfaction (TS) rated by multiple informants (patient, parent, therapist) following routine outpatient cognitive-behavioral therapy (CBT) within a large sample (n = 965) of clinically referred adolescents aged 11-20 years. Moreover, potential predictors of TS were analyzed (patient-related variables, mental disorder characteristics, socio-demographic factors and treatment variables). Overall, our results show a high treatment satisfaction in patient, parent and therapist ratings, with the therapists being the most critical raters (completely/predominantly satisfied: 87.8% in patient, 92.0% in parent, and 64.0% in therapist ratings). Correlations between the three raters were only small to moderate, but statistically significant. Regression analysis examining differential effects found that mental disorder characteristics (parent- and patient-reported symptoms at post) and treatment variables (especially cooperation of patients and parents as rated by therapists) explained most of the variance in TS, whereas patient-related or socio-demographic variables did not emerge as relevant predictors of TS. The amounts of explained variance were R _adj. ²  = 0.594 in therapist rating, R _adj. ²  = 0.322 in patient rating and R _adj. ²  = 0.203 in parent rating.

Development of the headspace Family and Friends Satisfaction Scale: Findings from a pilot study

AIMS

The primary aim of this pilot study was to determine the psychometric properties of the 18-item headspace Family and Friends Satisfaction Scale (hFAFSS).

METHODS

During August 2015, staff from 22 headspace centres approached family members and friends of young people attending headspace to complete the hFAFSS. Principal components analysis with oblique Promin rotation and polychoric correlations were used to assess the factor structure of the hFAFSS.

RESULTS

There were 277 usable responses. Satisfaction was high, resulting in little variance. Parallel analyses suggested that the scale items tapped a single factor (68% of variance).

CONCLUSIONS

This study is one of the first attempts to measure the satisfaction of family and friends with primary care-based youth mental health services. Satisfaction of family members and friends was shown to be high, but limited variance restricts the usability of the hFAFSS as an evaluation measure, and revision and further testing is needed.

Identifying the determinants of perceived quality in outpatient child and adolescent mental health services from the perspectives of parents and patients

This cross-sectional survey adopting a multiple-informant perspective explores the factors that influence perceived quality (i.e., therapeutic alliance and satisfaction) in an outpatient setting within child and adolescent mental health services (CAMHS). A total of 1433 participants (parents, n = 770, and patients, n = 663) attending or having attended (drop-out) outpatient units participated in the study. The outcome measures were satisfaction (Client Satisfaction Questionnaire) and the therapeutic alliance (Helping Alliance Questionnaire). The determinants of these quality indicators were socio-demographic variables (e.g., age, gender, and mother's socio-economic status), factors related to the extent of difficulties (number of reasons for the consultation, number of people who referred the child to the CAMHS), the approach to treatment at outset (agreeing to the consultation, feeling reassured at the first appointment), the organizational friendliness (secretary, waiting room, waiting time for the first appointment) and the organization of the therapy (frequency of sessions, time for questions, change of therapist). The approach to treatment at outset, accessibility by phone, satisfaction with the frequency of the sessions and having enough time for questions were the factors that consistently explain the quality indicators from both perspectives (patients and parents). In contrast, the socio-demographic variables as well as the extent of difficulties and factors related to the organizational friendliness and the organization of the therapy (frequency of sessions, change of therapist) were not related to the quality indicators. This study identifies key determinants of the quality indicators from the perspective of patients and parents that should be considered to improve CAMHS care quality. First appointments should be carefully prepared, and clinicians should centre care on the needs and expectations of patients and parents.

Psychometric properties and relations with coping and family strain of the Health Services and Caregiver Experience questionnaire (HSCE): an outcome measure of informal caregivers' experience for inpatient care in Italy

Background

In the last decade, the number of patients supported by informal caregivers has substantially increased. In the Italian healthcare context, informal caregivers’ experience of care is a new under-recognized construct, and no assessment tool is available. Measuring caregivers’ experience is important since in Italy the relationship between doctors and patients/relatives is still considered asymmetrical. The current study presented development and initial psychometric properties of the Health Services and Caregiver Experience questionnaire (HSCE), a self-report tool of caregivers’ global experience for inpatient clinical care, including factor structure, reliability and its relations with measures of coping strategies and family strain.

Methods

The HSCE was administered to a total of 503 informal caregivers of inpatients admitted at an Italian University Hospital (mean age = 48.08 years, SD = 14.82, females = 61.40%). Family Strain Questionnaire-Short Form (FSQ-SF) and Coping Orientations to Problems Experience-New Italian Version (COPE-NVI) were administered to a subgroup of participants. First-grade relatives were 73.10%, whereas 13.20% were second-grade relatives and 13.70% were home-watch caregivers.

Results

Exploratory and confirmatory factor analyses showed a structure with a single factor, which explained 64.80% of the total variance. All the items had salient loadings. In the two subsamples, HSCE had excellent internal consistency (Cronbach’s alpha = 0.95–0.97). Positive moderate correlations were found between HSCE and FSQ-SF scores (r = 0.45, p < .05), between HSCE and COPE-NVI scale scores, including COPE-NVI positive attitude and COPE-NVI problem solving scores (rs’ range = 0.51–0.57, p < .05). Moreover, a positive large correlation between HSCE and COPE-NVI social support scores emerged (r = 0.72, p < .05). Correlations were not significant between HSCE scores and COPE-NVI turning to religion and avoidance strategies.

Conclusions

The HSCE resulted to have good psychometric properties. Better caregivers’ experience correlated with stronger family strain but also with better problem solving and social support. The study expanded knowledge on caregiver’s experience in Italy and indicated that HSCE is a valid and reliable tool to measure this under-recognized construct in Italy.

Feedback mechanisms of change: How problem alerts reported by youth clients and their caregivers impact clinician-reported session content

OBJECTIVE

This study explored how clinician-reported content addressed in treatment sessions was predicted by clinician feedback group and multi-informant cumulative problem alerts that appeared in computerized feedback reports for 299 clients aged 11-18 years receiving home-based community mental health treatment.

METHOD

Measures included a clinician report of content addressed in sessions and additional measures of treatment progress and process (e.g., therapeutic alliance) completed by clinicians, clients, and their caregivers. Item responses in the top 25th percentile in severity from these measures appeared as "problem alerts" on corresponding computerized feedback reports. Clinicians randomized to the feedback group received feedback weekly while the control group did not. Analyses were conducted using the Cox proportional hazards regression for recurrent events.

RESULTS

For all content domains, the results of the survival analyses indicated a robust effect of the feedback group on addressing specific content in sessions, with feedback associated with shorter duration to first occurrence and increased likelihood of addressing or focusing on a topic compared to the non-feedback group.

CONCLUSION

There appears to be an important relationship between feedback and cumulative problem alerts reported by multiple informants as they influence session content.

Consumer satisfaction with the Child and Adolescent Mental Health Service and its association with treatment outcome: a 3-4-year follow-up study

BACKGROUND

Consumer satisfaction studies with the Child and Adolescent Mental Health Service (CAMHS) have mainly assessed evaluations in a short-term follow-up perspective. Adolescent reports with CAMHS have not been included nationally.

AIMS

The purposes of this study were to explore adolescent and parental satisfaction with the CAMHS in a 3-4-year follow-up perspective, and to examine the relationships between reported consumer satisfaction and clinical parameters such as reason for adolescent referral, emotional/behavioral symptoms and treatment outcome.

METHODS

Of 190 adolescent-parent pairs in a sample of CAMHS outpatients, 120 completed a Consumer Satisfaction Questionnaire. Parents assessed adolescent emotional/behavior problems both at baseline and at follow-up by completing the Child Behavior Checklist (CBCL). Correlations were examined between adolescent and parental evaluations. The relationships between service satisfaction and symptom load at baseline and follow-up and treatment outcome at follow-up were explored.

RESULTS

Overall, adolescents and parents were satisfied with the services received from the CAMHS. The correlations between adolescent and parent consumer satisfaction ratings were low to moderate. Consumer satisfaction was significantly and negatively correlated with symptom load on the CBCL Total Problems scores at baseline, but not at follow-up. There was no difference in satisfaction levels between those who improved after treatment and those who did not.

CONCLUSIONS

Given the differences in informant ratings of consumer satisfaction, it is important to include both adolescent and parental perceptions in evaluations of CAMHS services and treatment outcomes. Consumer satisfaction should serve as a supplement to established standardized outcome measures.

Using Measurement-Based Care to Enhance Any Treatment

Measurement-based care (MBC) can be defined as the practice of basing clinical care on client data collected throughout treatment. MBC is considered a core component of numerous evidence-based practices (e.g., Beck & Beck, 2011; Klerman, Weissman, Rounsaville, & Chevron, 1984) and has emerging empirical support as an evidence-based framework that can be added to any treatment (Lambert et al., 2003, Trivedi et al., 2007). The observed benefits of MBC are numerous. MBC provides insight into treatment progress, highlights ongoing treatment targets, reduces symptom deterioration, and improves client outcomes (Lambert et al., 2005). Moreover, as a framework to guide treatment, MBC has transtheoretical and transdiagnostic relevance with broad reach across clinical settings. Although MBC has primarily focused on assessing symptoms (e.g., depression, anxiety), MBC can also be used to assess valuable information about (a) symptoms, (b) functioning and satisfaction with life, (c) putative mechanisms of change (e.g., readiness to change), and (d) the treatment process (e.g., session feedback, working alliance). This paper provides an overview of the benefits and challenges of MBC implementation when conceptualized as a transtheoretical and transdiagnostic framework for evaluating client therapy progress and outcomes across these four domains. The empirical support for MBC use is briefly reviewed, an adult case example is presented to serve as a guide for successful implementation of MBC in clinical practice, and future directions to maximize MBC utility are discussed.

Perceived program qualities and outcomes of a youth program in Hong Kong based on the views of the workers

STUDY OBJECTIVES

Based on the data collected in the extension phase of the Project P.A.T.H.S. in Hong Kong, this study examined the views of 9,765 program implementers on the universal curricula-based program (ie, Tier 1 Program).

DESIGN

After the Tier 1 Program was completed, workers responded to a client satisfaction scale (Form B). Utilizing the data supplied by the participating schools, the profiles and correlates of client satisfaction data were examined.

RESULTS

Program attributes, implementer attributes, and benefits of the program were viewed positively by the program implementers, with high proportion of the instructors perceiving the program contributing to the development of the students. Small grade differences on client satisfaction levels were found. Regarding predictors of perceived program effectiveness, perceived program and instructor attributes predicted perceived effectiveness of the program.

CONCLUSION

Consistent with the data collected from the students, the present findings suggest that the Tier 1 Program was well-received by the major stakeholders, particularly with respect to its ability to promote positive development in Chinese junior high school students. The present finding replicated the previous observation that perceived program and implementer qualities were significant determinants of perceived program effectiveness.

The Peabody Treatment Progress Battery: history and methods for developing a comprehensive measurement battery for youth mental health

There is increased need for comprehensive, flexible, and evidence-based approaches to measuring the process and outcomes of youth mental health treatment. This paper introduces a special issue dedicated to the Peabody Treatment Progress Battery (PTPB), a battery of measures created to meet this need. The PTPB is an integrated set of brief, reliable, and valid instruments that can be administered efficiently at low cost and can provide systematic feedback for use in treatment planning. It includes eleven measures completed by youth, caregivers, and/or clinicians that assess clinically-relevant constructs such as symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction. This introductory article describes the rationale for the PTPB and its development and evaluation, detailing the specific analytic approaches utilized by the different papers in the special issue and a description of the study and samples from which the participants were taken.

The Session Report Form (SRF): are clinicians addressing concerns reported by youth and caregivers?

This study explores the relationship between clinician-reported content addressed in sessions, measured with the Session Report Form (SRF), and multi-informant problem alerts stemming from a larger battery of treatment process and progress measures. Multilevel Multinomial Logit Models were conducted with 133 clinicians and 299 youths receiving home-based treatment (N = 3,143 sessions). Results indicate a strong relationship between session content and problems related to youth symptoms and functioning as reported by clinicians in the same session. Session content was related to emotional, family, and friend/peer problems reported by youth and youth behavioral problems reported by caregivers. High-risk problems (alcohol/substance use, harm to self or others) were strongly related to session content regardless of informant. Session content was not related to problem alerts associated with the treatment process, caregiver strain, or client/caregiver strengths. The SRF appears to be a useful measure for assessing common themes addressed in routine mental health settings.