Measuring treatment outcome and client satisfaction among children and families

User satisfaction with child and adolescent mental health services: the association between user satisfaction and clinical outcomes

BACKGROUND

This study aimed to examine the association between user satisfaction and clinical outcomes with child and adolescent mental health services (CAMHS) from the perspective of young people and their parents. The evidence bases for CAMHS user satisfaction measures are limited, with few studies investigating the link between user satisfaction and clinical outcomes. In particular, the perspectives of young people are missing.

METHODS

The parent and youth versions of the Experience of Service Questionnaire (ESQ), which evaluates the factors of general satisfaction (GS), satisfaction with care (SWC) and satisfaction with environment (SWE), were used to measure user satisfaction. The outcome measures were scores on the Strengths and Difficulties Questionnaire (SDQ), Children's Global Assessment Scale (CGAS), and Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). Hierarchical regression analysis was conducted on data collected from 233 young people and 495 parents who utilized CAMHS services.

RESULTS

GS and SWC predicted outcomes for both young people (ΔR2 = 0.08, p <.05) and parents (ΔR2 = 0.01, p <.05), indicating that user satisfaction had a significant impact on clinical outcomes for CAMHS users. In addition, GS and SWC significantly predicted young people-reported outcomes in the interaction model (ΔR2 = 0.10, p <.05), while no significant association was found with parent-reported outcomes (ΔR2 = 0.02, p =.09).

CONCLUSION

User satisfaction, particularly for young people, has a significant impact on clinical outcomes. The causal relationship between user satisfaction and mental health outcomes requires further study.

Magnitude of Satisfaction and Associated Factors Among Caregivers of Children with Disabilities Towards Community-Based Rehabilitation Service in Central and North Gondar Zone, Northwest Ethiopia: A Community-Based Cross-Sectional Study

Introduction

Community-based rehabilitation service (CBR) is a community development strategy set for persons with disabilities and their families. Measuring caregivers’ satisfaction is a recommended approach to know the overall satisfaction of rehabilitation service. Therefore, this study aimed to assess the magnitude of satisfaction and associated factors among caregivers of children with disabilities towards community-based rehabilitation in Central and North Gondar zone, North West Ethiopia.

Methods

A community-based cross-sectional study was conducted from March to September 2020. The data were collected by socio-demographic variables, types of services, types of disabilities, and Client Satisfaction Questionnaire (CSQ-8). Data analysis was done by SPSS Version-20 statistical software. Descriptive statistics, bi-variable, and multivariable logistic regression analysis were computed. The multivariable logistic regression analysis model a P-value <0.05 was used to identify the associated factors with caregivers’ satisfaction.

Results

The study showed that the overall caregivers’ satisfaction with community-based rehabilitation service among caregivers of children with disabilities was 60.9% (95% CI; 55–66%). Female gender (AOR: 0.19, 95% CI: 0.05–0.68), older age (AOR: 0.160, 95% CI: 0.03–0.85), being farmer (AOR: 0.14, 95% CI: 0.05–0.42), North Gondar zone residence (AOR: 7.02, 95% CI: 6.78–8.59), high monthly income (AOR: 2.48, 95% CI: 1.09–5.61), got two or more type of service (AOR: 3.65, 95% CI: 2.64–12.39), service duration 8–14 years (AOR: 4.36, 95% CI: 1.44–7.04) and service frequency 5–8 times per month (AOR: 3.91, 95% CI 2.18–7.01) were factors associated with the caregivers’ satisfaction on community based-rehabilitation service.

Conclusion and Recommendation

The magnitude of caregivers’ satisfaction with community-based rehabilitation services in the study area was more than half. Female gender, older age, being farmer, high monthly income, North Gondar zone residence, the high number of services, duration of services started and frequency of follow-up were factors significantly associated with caregiver’s satisfaction.

Using a Brief Parent-Report Measure to Track Outcomes for Children and Teens with ADHD

The Pediatric Symptom Checklist (PSC) is a widely used, parent-completed measure of children's emotional and behavioral functioning. Previous research has shown that the PSC and its subscales are responsive to patient progress over the course of psychiatric treatment. In this naturalistic study, parents and clinicians of 1736 patients aged 17 or younger completed standardized measures at intake and 3-month follow-up appointments. We assessed the 5-item PSC Attention Subscale (PSC-AS) as a longitudinal measure of attention-related symptoms in routine outpatient psychiatry treatment. Secondarily, we compared PSC-AS scores with clinician-reported diagnoses, psychomotor excitation symptoms, and overall functioning. Change scores on the PSC-AS were larger among patients with ADHD diagnoses than those with non-ADHD diagnoses. Comparisons between PSC-AS scores and clinician reports also showed acceptable levels of agreement. Given its effectiveness in tracking attention-related symptoms, the PSC may be particularly useful as a quality assurance or treatment outcome measure for clinicians treating ADHD.

Using a Brief Parent-Report Measure to Track Outcomes for Children and Teens with Internalizing Disorders

The Pediatric Symptom Checklist (PSC) is a widely-used, parent-completed measure of children's emotional and behavioral functioning. Previous research has shown that the PSC and its subscales are generally responsive to patient progress over the course of psychiatric treatment. In this naturalistic study, we examined the performance and utility of the five-item PSC Internalizing Subscale (PSC-IS) as an assessment of routine treatment in outpatient pediatric psychiatry. Parents and clinicians of 1,593 patients aged 17 or younger completed standardized measures at intake and three-month follow-up appointments. Comparisons between PSC-IS scores and clinician-reported diagnoses, internalizing symptoms, and overall functioning showed acceptable levels of agreement. Change scores on the PSC-IS were also larger among patients with internalizing diagnoses than those with non-internalizing diagnoses. As a brief measure of internalizing symptoms, the PSC may be particularly useful to mental health clinicians treating youth with depression and anxiety as a quality assurance or treatment outcome measure.

Improvement in symptoms versus functioning: how do our best treatments measure up?

We examined the effects of redefining standards of evidence for treatments targeting childhood mental health problems by expanding outcomes beyond symptom reduction to include functioning. Over 750 treatment protocols from 435 randomized controlled trials were rated based on empirical evidence. Nearly two-thirds (63.9%) demonstrated at least a minimum level of evidence for reducing symptoms; however, only 18.8% of treatments demonstrated evidence for reducing functional impairment. Of those treatments with empirical support for symptom reduction, the majority did not demonstrate empirical support for improvement in functioning because measures of functioning were not included in the studies in which these treatments were tested. However, even when measures of functioning were included, it was much more difficult for treatments to achieve improvement. Among treatments that achieved improvement in functioning, the most notable were Collaborative Problem Solving for disruptive behavior and Cognitive Behavioral Therapy plus Medication for traumatic stress because they demonstrated no support for symptom reduction but good support for improvement in functioning. Results are discussed within the context of evaluating the standards of evidence for treatments and the opportunity to move towards a multidimensional framework whose utility has the potential to exceed the sum of its parts.

A comparison of two types of social support for mothers of mentally ill children

PROBLEM

The purpose of this analysis was to compare social support offered by two telehealth nursing interventions for mothers of children with serious mental illnesses.

METHODS

A randomized, controlled, quantitative investigation is underway to test two support interventions, using the telephone (TSS) or Internet (WEB). Qualitative description was used to analyze data generated during telehealth interventions.

FINDINGS

The behaviors and attitudes of children were challenging for the mothers to manage. Mothers' emotional reactions included fear, frustration, concern, and guilt. They sought to be advocates for their children. The nurses provided emotional, informational, and appraisal support. TSS mothers were passive recipients, while WEB mothers had to choose to participate.

CONCLUSIONS

Mothers in both interventions shared similar concerns and sought support related to their child's problems.

Model testing: examining parent satisfaction

The purpose of this study was to test a model predicting social-psychological determinants of parent satisfaction with children's mental health services. Model concepts were parents' characteristics, definition of situation, desired services, expectations, met desires, met expectations, and satisfaction. The sample included 120 parents of children, ages 3-18 years, admitted to one of five treatment programs. With minor changes, results of structural equation modeling indicated a good fit between the data and the proposed relationships. The strongest predictor of parent satisfaction was met expectations; however, inclusion of the desire concepts was essential in obtaining a strong, acceptable model fit.

Development and testing of a scale measuring parent satisfaction with staff interactions

This study tested the psychometric properties of the Parent Satisfaction Scale (PSS), a new scale measuring parents' satisfaction with their child's mental health services, primarily focusing on staff's interpersonal interactions. Data were collected by postal surveys 5 to 10 months after a child's admission to 1 of 5 mental health treatment programs. Completed surveys were received from parents of 120 of a potential 232 children (52% response rate). Internal consistency reliability of the PSS was strong (alpha = .96). Construct validity was supported by significant relationships between the PSS and constructs considered to be antecedent variables influencing parent satisfaction, including met expectations (r = 0.62, P < .001), met desires (r = 0.32, P < .001), and met needs (r = 0.55, P < .001). Convergent validity was supported, with the PSS having Pearson correlations of .86 (P < .001) with the Client Satisfaction Questionnaire-8 and .76 (P < .001) with parents' ratings of satisfaction with specific services received.

The brief psychiatric rating scale for children (BPRS-C): validity and reliability of an anchored version

OBJECTIVE

Because the accuracy of problems reported by referred children may be compromised by their academic, cognitive, or motivational limitations, clinician rating forms may contribute to the accurate assessment of youth adjustment. One such measure, the 21-item Brief Psychiatric Rating Scale for Children (BPRS-C), received psychometric study to estimate its potential contribution to the measurement of symptom dimensions. BPRS-C reliability and concurrent validity were calculated for youths who were receiving psychiatric services within a medical school department.

METHOD

Five hundred forty-seven children aged 3 to 18 years were rated by faculty or trainees; a subsample of 90 was concurrently rated by two observers. BPRS-C psychometric performance was demonstrated through interrater agreement, factor analysis, and multivariate analyses of variance across seven diagnosis-based groups.

RESULTS

Although items and scales demonstrated substantial reliability and concurrent validity, item factor analysis revealed a few apparent errors in item-to-scale assignment. These errors were minimized by the use of three new second-order factor-derived scales: Internalization, Developmental Maladjustment, and Externalization.

CONCLUSIONS

The BPRS-C can be easily integrated into academic clinical practice and is a reliable and valid method of child description. Additional study of three new BPRS-C factor scales and the application of the BPRS-C to the quantification of clinician observation of child symptomatic status are warranted.

Measuring treatment outcomes and client satisfaction in a partial hospitalization program

Mental health practitioners are increasingly being called on to evaluate the effectiveness of the treatment they provide. The partial hospitalization component of the mental health industry also has felt this call for outcome research. This article presents the results of one program's attempts to answer that call through an assessment of treatment outcome and client satisfaction at a partial hospitalization program. Participants (N = 287) were evaluated at admission, discharge, and 3-month follow-up. The article outlines the procedures used for assessment and uses the results as an example of the type of data that can be obtained through outcomes measurement. The article is intended to provide an example of program evaluation that is easy and inexpensive to administer.

Self-regulating service delivery systems: a model for children and youth at risk

A conceptual model of self-regulating service delivery is proposed for use by governments, planners, and policymakers to help children reach optimal adult functioning. It addresses most problems of present service delivery and is applicable to children in the general population and registered consumers of mental health, child welfare, special education, drug dependency, and juvenile justice (young offender) services. It has four essential components: (1) outcome measurements that reflect the mental health statuses of children in services and in the general population; (2) regular feedback of these measurements to governments, the public, and service delivery organizations; (3) powerful and significant rewards and incentives for the most desired outcomes to increase their salience and to motivate provider behavior; and (4) decision making that can affect all entities that contribute to the health of children. The evaluable expected benefits are improved mental health for children and greater efficiency within the "system."

Using client satisfaction surveys to evaluate and improve services in locked and unlocked adult inpatient facilities

This article describes the implementation, in five inpatient subacute treatment facilities, of a satisfaction survey designed especially for adults with serious and persistent mental illnesses. The survey measures not only global satisfaction but also client perceptions about different treatment modalities and services, important treatment goals, and the philosophy of treatment. Data are presented from 770 completed surveys, illustrating patterns of satisfaction across facilities and services and patterns over time of stability and change in satisfaction. Data are also presented showing how the surveys were used to facilitate and measure improvements in clinical services. Finally, the implications for mental health services delivery are summarized.

Measuring treatment outcome for adults and adolescents: reliability and validity of BASIS-32

This article examines the reliability and validity for adults and adolescents of Behavioral and Symptom Identification Scale (BASIS-32), a 32-item patient-report instrument designed to facilitate psychiatric outcome assessment of adult inpatient populations. This study extends the original analysis of the psychometric properties of BASIS-32 to a different site, using a self-report rather than interview format and samples of both adult and adolescent patients. Responses to the BASIS-32 are reported for two groups of patients consecutively admitted between 1991 and 1994: adults over 18 years old (n = 462) and adolescents between 12 and 18 (n = 244). Results of this investigation confirm the utility of BASIS-32 as an easily administered and scored self-report outcome assessment tool for adult psychiatric patients. Its utility for adolescent patients was not as clearly demonstrated.

Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model

OBJECTIVE

Accountability for mental health care has become a standard of clinical practice. With the expansion of managed care as a corporate response to health reform, attention to outcomes will intensity. Assessment of clinical treatment has typically focused on symptom reduction at an individual level, whereas assessment of service effectiveness has more often targeted service-level change.

METHOD

A dynamic and interactional model of outcomes is presented that broadens the range of intended consequences of care. The model comprises five domains: symptoms, functioning, consumer perspectives, environmental contexts, and systems.

RESULTS

The model reflects the changeable interaction between children's evolving capacities and their primary environments (home, school, and community).

CONCLUSIONS

As health care practices shift, attention to improved care is likely to depend increasingly on scientifically credible evidence of its impact. Greater integration between research and standard practice will be needed. Such a partnership can be strengthened by a more comprehensive view of the impact of care.