Routine outcome measurement in youth mental health: a comparison of two clinician rated measures, HoNOSCA and HoNOS

User satisfaction with child and adolescent mental health services: the association between user satisfaction and clinical outcomes

BACKGROUND

This study aimed to examine the association between user satisfaction and clinical outcomes with child and adolescent mental health services (CAMHS) from the perspective of young people and their parents. The evidence bases for CAMHS user satisfaction measures are limited, with few studies investigating the link between user satisfaction and clinical outcomes. In particular, the perspectives of young people are missing.

METHODS

The parent and youth versions of the Experience of Service Questionnaire (ESQ), which evaluates the factors of general satisfaction (GS), satisfaction with care (SWC) and satisfaction with environment (SWE), were used to measure user satisfaction. The outcome measures were scores on the Strengths and Difficulties Questionnaire (SDQ), Children's Global Assessment Scale (CGAS), and Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). Hierarchical regression analysis was conducted on data collected from 233 young people and 495 parents who utilized CAMHS services.

RESULTS

GS and SWC predicted outcomes for both young people (ΔR2 = 0.08, p <.05) and parents (ΔR2 = 0.01, p <.05), indicating that user satisfaction had a significant impact on clinical outcomes for CAMHS users. In addition, GS and SWC significantly predicted young people-reported outcomes in the interaction model (ΔR2 = 0.10, p <.05), while no significant association was found with parent-reported outcomes (ΔR2 = 0.02, p =.09).

CONCLUSION

User satisfaction, particularly for young people, has a significant impact on clinical outcomes. The causal relationship between user satisfaction and mental health outcomes requires further study.

Health of the Nation Outcome Scales for Infants field trial: concurrent validity

BACKGROUND

A review of Australian mental health services identified a gap in routine outcome measures addressing social, emotional and behavioural domains for pre-schoolers and infants. A Child and Adolescent Mental Health Information Development Expert Advisory Panel working group developed the Health of the Nation Outcome Scales for Infants (HoNOSI), a clinician-reported routine outcome measure for infants 0-47 months. Prior face validity testing showed that the HoNOSI was considered useful in measuring mental health outcomes.

AIMS

To examine the concurrent validity of the HoNOSI.

METHOD

Mental health clinicians providing assessment and treatment to infants in routine clinical practice participated in the study. The mental health status of 108 infants were rated by a minimum of 26 clinicians with the HoNOSI, the Parent-Infant Relationship Global Assessment Scale (PIR-GAS) and measures of symptom severity and distress.

RESULTS

The HoNOSI was statistically significantly correlated with the PIR-;GAS, rs = -0.73; Clinical Worry, rs = 0.77; and Severity Judgement ratings, rs = 0.85; P < 0.001. A good level of internal consistency was found. Using the COsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria for judging instrument acceptability, the HoNOSI meets the standard for both concurrent validity and internal consistency.

CONCLUSIONS

There has been a clear need for a routine outcome measure for use with infants. This study provides positive evidence of aspects of validity. These findings, along with those from the prior face validity study, support a controlled release of the HoNOSI accompanied by further research and development.

The interrater reliability of a routine outcome measure for infants and pre-schoolers aged under 48 months: Health of the Nation Outcome Scales for Infants

BACKGROUND

A review of Australian mental health services identified a gap in routine outcome measures addressing social, emotional and behavioural domains for pre-schoolers and infants. The Child and Adolescent Mental Health Information Development Expert Advisory Panel Working Group developed the Health of the Nation Outcome Scales for Infants (HoNOSI), a clinician-reported routine outcome measure for use with those aged under 4 years. Prior psychometric testing showed that the HoNOSI was considered to show face validity, and that it met the standards for concurrent validity and internal consistency.

AIMS

We aimed to investigate the interrater reliability of the HoNOSI.

METHOD

Forty-five infant mental health clinicians completed HoNOSI ratings on a set of five case vignettes.

RESULTS

Quadratic weighted kappa interrater reliability estimates showed the HoNOSI to have Almost Perfect interrater reliability for the HoNOSI total score. Of the 15 scales, one had Moderate, seven had Substantial and seven had Almost Perfect interrater reliability. Ten of the fifteen scales and the total score exceeded the COnsensus-based Standards for the Selection of Health Measurement INstruments criteria for interrater reliability (κw ≥ 0.7).

CONCLUSIONS

There has been a clear need for a routine outcome measure for use with infants and pre-schoolers. This study provides evidence of interrater reliability. The current findings, combined with the face and concurrent validity studies, support further examination of HoNOSI in real-world settings.

Measure of clinical improvement in children and adolescents with psychiatric disorders: an evaluation of multiple perspectives with HoNOSCA

Introduction

Routine outcome assessment is helpful to inform decision-making, resource allocation, and health policy design. Routine outcome assessment in the hospital setting for children and adolescents with psychiatric disorders remains limited. The clinical instrument HoNOSCA (Health of the Nation Outcome Scales for Children and Adolescents), which has recently become available in Spanish and Catalan, allows outcome assessment in this population from the perspective of patients, their parents or legal guardians, and clinicians. HoNOSCA measures 13 areas of health and psychosocial functioning.

Objectives

The aim of this study was to assess mental health outcomes in psychiatric day hospital pediatric patients from three perspectives (patient, par-ent/legal guardian, clinician), using the Spanish and Catalan versions of HoNOSCA.

Methods

We recruited patients up to 18 years old with any psychiatric disorder at the day unit of the Salut Mental Parc Taulí Hospital Universitari (Sabadell, Catalonia, Spain). We obtained admission and discharge HoNOSCA scores for the patients, their parents or legal guardians, and their clinicians.

Results

We recruited 99 patients over the study period (January 2015 to December 2017), 11 of which were lost to follow-up. Among the remaining 88, we found significant improvement in HoNOSCA scores from admission to discharge. Agreement between the HoNOSCA scores for the three different groups of evaluators (patients, parents/legal guardians, and clinicians) was weak at admission but better at discharge. In general, evaluations from patients and their parents or legal guardians had lower HoNOSCA scores (indicating a better mental health status) at admis-sion compared to those from clinicians. At discharge, however, the scores were more homogenous across the three groups of stakeholders.

Conclusions

Use of HoNOSCA allows for routine evaluation of mental health outcomes in the psychiatric day hospital setting from the perspective of pa-tients, their parents or legal guardians, and clinicians.

Development and validation of MyLifeTracker: a routine outcome measure for youth mental health

PURPOSE

Routine outcome measures are now being designed for session-by-session use, with emphasis on clinically meaningful items and sensitivity to change. Despite an increasing mental health service focus for young people aged 12-25 years, there is a lack of outcome measures that are designed to be used across this age group. Consequently, MyLifeTracker (MLT) was developed as a brief mental health outcome measure designed for young people for routine use. It consists of the following five items targeting areas of importance to young people: general well-being, day-to-day activities, relationships with friends, relationships with family, and general coping.

PARTICIPANTS AND METHODS

The measure was tested with 75,893 young people aged 12-25 years attending headspace centers across Australia for mental health-related issues.

RESULTS

MLT showed a robust unidimensional factor structure and appropriate reliability. It exhibited good concurrent validity against well-validated measures of psychological distress, well-being, functioning, and life satisfaction. The measure was further demonstrated to be sensitive to change.

CONCLUSION

MLT provides a psychometrically sound mental health outcome measure for young people. The measure taps into items that are meaningful to young people and provides an additional clinical support tool for clinicians and clients during therapy. The measure is brief and easy to use and has been incorporated into an electronic system that routinely tracks session-by-session change and produces time-series charts for the ease of use and interpretation.

Validation of the Spanish and Catalan versions of the Health of the Nation Outcome Scale for Children and Adolescents (HoNOSCA)

Outcome measurement in outpatient and day-care mental health facilities for children and adolescents in Spain remains limited, in part due to a lack of validated scales. To address this issue, we translated HoNOSCA (glossary, score sheet, self-rated questionnaire, and parent/legal guardian questionnaire) into Spanish and Catalan using a reverse translation approach. We ascertained the validity and psychometric quality of the HoNOSCA in Spanish by assessing correlation with the Children's Global Assessment Scale (C-GAS). We recruited 64 participants 7-17 years of age in five day-care Psychiatry hospitals in Catalonia (Spain). Two evaluators administered both instruments twice, two weeks apart. Patients and parents/legal guardians completed the corresponding HoNOSCA versions. We calculated Cronbach's alpha for assessing internal consistency, intra-class correlation coefficients (ICC) for inter-rater and test-retest reliability, and Pearson's correlation coefficients for validity. We found that all HoNOSCA versions in Spanish presented satisfactory internal consistency, inter-rater and test-retest reliability. Concurrent validity for HoNOSCA-Glossary was also acceptable, with Pearson's coefficients of -0.543 and -0.519 for evaluators in the first administration, and of -0.675 and -0.685 in the second administration. HoNOSCA was also successfully translated into Catalan; acceptability was determined using cognitive interviews.

Young Australians with moderate to severe mental health problems: client data and outcomes at Children and Young People's Mental Health

AIM

Almost a quarter of young Australians experience a mental health issue that may become chronic if left untreated. Children and Young People's Mental Health (CYPMH) is a specialist tertiary service for young people with moderate to severe mental health problems on the Central Coast in Australia. This paper presents an overview of client data and service use collected over a 1 year period specific to the Youth Mental Health (YMH) component of the service.

METHOD

Client data, including demographic characteristics, service usage, presenting issues and standardized outcome measures, were analysed using SPSS. Clinicians routinely collect MH-OAT (Mental Health Outcomes and Assessment Tools) measures at different points in a client's episode of care, and each of these measures was analysed separately. Wilcoxon Z and a series of McNemar's tests were used to report on the difference between admission and discharge scores.

RESULTS

During a designated 1 year period, 830 referrals to YMH were received. The most prevalent presenting issue was suicidal ideation followed by deliberate self-harm and depression. A comparison of admission and discharge outcome scores shows significant improvement by discharge on a range of measures. Specifically, analysis identified significant differences between admission and discharge HoNOSCA (Health of the Nation Outcomes Scales for Children and Adolescents) and CGAS (Children's Global Assessment Scale) scores for young people aged 12-17 and HONOS (Health of the Nation Outcomes Scale) scores for young people aged 18-24.

CONCLUSION

The clinical outcomes for young people are positive with improvements seen on a range of measures.

Mental health screening in immigration detention: A fresh look at Australian government data

OBJECTIVES

The poor mental health of asylum seekers and refugees in immigration detention has consistently been reported in peer-reviewed literature internationally; however, data on the mental health of asylum seekers and refugees detained in Australian immigration has been very limited.

METHODS

We re-analysed mental health screening data obtained by the Human Rights Commission.

RESULTS

Longer time in detention was associated with higher self-reported depression scores, with female individuals being more vulnerable to time in detention than those of male gender. Approximately one-half of the refugee group who agreed to complete the Harvard Trauma Questionnaire had post-traumatic stress disorder symptoms. On clinician-rated measures, one-third of the children, adolescents and adults suffered with clinical symptoms requiring tertiary outpatient assessment.

CONCLUSIONS

This paper consolidates the findings of the 2014 Australian Human Rights Commission report and it provides an argument for public reporting of refugee data.