Outcome measurement in chronic mental illness

The effects of a computerized clinical decision aid on clinical decision-making in psychosis care

OBJECTIVE

Clinicians in mental healthcare have few objective tools to identify and analyze their patient's care needs. Clinical decision aids are tools that support this process. This study examines whether 1) clinicians working with a clinical decision aid (TREAT) discuss more of their patient's care needs compared to usual treatment, and 2) agree on more evidence-based treatment decisions.

METHODS

Clinicians participated in consultations (n = 166) with patients diagnosed with psychotic disorders from four Dutch mental healthcare institutions (research registration number 201700763). Primary outcomes were measured with the modified Clinical Decision-making in Routine Care questionnaire and combined with psychiatric, physical and social wellbeing related care needs. A multilevel analysis compared discussed care needs and evidence-based treatment decisions between treatment as usual (TAU) before, TAU after and the TREAT condition.

RESULTS

First, a significant increase in discussed care needs for TREAT compared to both TAU conditions (β = 20.2, SE = 5.2, p = 0.00 and β = 15.8, SE = 5.4, p = 0.01) was found. Next, a significant increase in evidence-based treatments decisions for care needs was observed for TREAT compared to both TAU conditions (β = 16.7, SE = 4.8, p = 0.00 and β = 16.0, SE = 5.1, p = 0.01).

CONCLUSION

TREAT improved the discussion about physical health issues and social wellbeing related topics. It also increased evidence-based treatment decisions for care needs which are sometimes overlooked and difficult to treat. Our findings suggest that TREAT makes sense of routine outcome monitoring data and improves guideline-informed care.

First-episode psychosis in the Ferrara Mental Health Department: Incidence and clinical course within the first 2 years

AIM

To examine the incidence of with first-episode psychosis (FEP) in the Integrated Department of Mental Health and Pathological Addictions in Ferrara, Italy, and to examine the association between the Duration of Untreated Psychosis (DUP) and the clinical course.

METHODS

Participants recruited in 2013-2019 were assessed with the Health of the Nation Outcome Scale (HoNOS) every 6 months for 24 months. Hierarchical growth models analysed changes of global severity (HoNOS total scores) and symptom dimensions. Regression modelled factors associated with remission (HoNOS < 8) and clinical improvement (<12).

RESULTS

The incidence of FEP was 21.5 (95%CI: 21.2-21.9) cases per 100 000 person year. Among participants (n = 86, mean age 23, 76% males), baseline HoNOS scores were higher for those with a longer DUP. More than half subjects reached clinical remission (61.6%) or improvement (82.6%), while very few (2.3%) were re-hospitalized. HoNOS total scores decayed with a mixed linear/quadratic trend, with a slower decay among migrants. A longer DUP was associated with reduced improvements of positive symptoms and lower likelihood of clinical improvement (OR: 0.84; 95%CI: 0.73-0.96).

CONCLUSIONS

Patients from the FEP program of Ferrara reached good clinical outcomes. Nonetheless, individuals with a longer DUP may need additional clinical attention. Systematic monitoring of clinical outcomes may be an optimal strategy to improve the outcomes of FEP in the real world.

Effectiveness of treatment for 6813 patients with mental health conditions in Cambridgeshire: a cross-sectional study

BACKGROUND

The Health of the Nation Outcomes Scales (HoNOS) has been widely used as an outcome measure in UK mental health settings for the past decade. The data-set gathered provides a unique opportunity to evaluate the effectiveness of the totality of mental healthcare in 'real-world' conditions; much of our clinical evidence currently comes from highly parameterised clinical trials investigating single interventions in highly selected patients.

AIMS

To examine all outcomes measured by HoNOS for a range of diagnostic groups, evaluate the influence of patient demographics on those outcomes, and observe changes in patient groups over time.

METHOD

Here we show the data from 6813 adult patients treated in Cambridgeshire between 2012 and 2017. Patients were split into three diagnostic groups: psychosis, non-psychosis and organic. Changes in HoNOS scores from initial assessment to discharge were tested and regressions were used to evaluate the influence of age, gender and ethnicity on the changes, as well as to model changes in the severity of initial presenting symptoms with time.

RESULTS

HoNOS scores significantly improve after treatment for psychotic, non-psychotic and organic conditions in adults and older adults. Age, but not gender or ethnicity, influenced change in HoNOS scores. Patients entering secondary mental health services had increased initial HoNOS scores over time.

CONCLUSIONS

The UK repository of HoNOS scores provides a significant and relatively underutilised resource that can be exploited to gain insights into mental illness and treatment effectiveness. This is likely to have many applications, including influencing the commissioning of services.

"The Italian Study on Recovery 2" Phase 1: Psychometric Properties of the Recovery Assessment Scale (RAS), Italian Validation of the Recovery Assessment Scale

BACKGROUND

The achievement of recovery is related to the notion of developing personal potential and restoring a legitimate social role, even against the backdrop of mental illness limitations. It is still difficult to fully understand this highly subjective and dynamic process. Therefore, in order to test the recovery process, specific tools, still only marginally used in our country, are needed.

AIMS

The Italian Study on Recovery is the first study aimed at confirming the validity of the Italian version of the Recovery Assessment Scale (RAS), an instrument developed with the goal of detecting recovery among patients.

METHOD

This multicentric research involved several Mental Health Services from various parts of Italy. The first phase of the study consisted in the administration of the Italian translation of RAS, previously used in a pilot study conducted in 2009. RAS was administered to 219 patients diagnosed with psychosis, whose mental disorder lasted for at least 5 years.

RESULTS

Findings supported the good psychometric properties of the Italian version of RAS, demonstrating its capability of identifying patients matching the "in recovery" operational criteria.

CONCLUSIONS

In consideration of the results highlighting the good psychometric properties of RAS, the present study may contribute to the diffusion of instruments to be included in Mental Health Service planning in the Italian context, in order to start a recovery-oriented transformation.

Longitudinal analysis of statistical and clinically significant psychosocial change following mental health rehabilitation

Purpose: With appropriate mental health rehabilitation, schizophrenia is increasingly associated with reports of recovery and stability. However, there is little empirical evidence evaluating the efficacy of services delivering this care. This study evaluated the effectiveness of rehabilitation for improving psychosocial function in consumers with schizophrenia.Methods: An electronic database of standardized assessment instruments mandated and maintained by the health service was retrospectively reviewed to extract ratings of psychosocial function, daily living skills, and mood state from consecutive admissions to an inpatient rehabilitation service. Outcomes were compared at admission, discharge, and one-year follow-up to identify statistically significant change. Individual reliable and clinically significant change was also assessed by comparison with a normative group of clients functioning independently in the community.Results: From admission to discharge the rehabilitation group made statistically significant gains in psychosocial function and daily living skills. Improvements were reliable and clinically significant in one-quarter to one-third of individual consumers. Approximately half sustained their improvements at follow-up, although this represented only a small fraction of the overall cohort. Consumers not demonstrating gains exhibited psychometric floor effects at admission.Conclusions: Rehabilitation can produce statistically and clinically significant immediate improvement in psychosocial function for a sub-set of consumers with elevated scores at admission. The durability of any gains is less clear, and strategies promoting longer-term maintenance are encouraged. Furthermore, currently mandated outcome measures are confounded by issues of sensitivity and reporting compliance, and exploration of alternative instruments for assessing recovery is recommended.Implications for RehabilitationRoutinely collected standardized outcome measures can be used to investigate the effectiveness of mental health rehabilitationIn addition to statistical significance, the clinical significance of outcomes should be evaluated to identify change that is individually meaningfulCurrently mandated outcomes instruments do not adequately evaluate many individuals' recovery journeyMental health service evaluation and quality improvement processes would likely benefit from adoption of recovery-oriented measures.

The development and evaluation of a computerized decision aid for the treatment of psychotic disorders

BACKGROUND

Routinely monitoring of symptoms and medical needs can improve the diagnostics and treatment of medical problems, including psychiatric. However, several studies show that few clinicians use Routine Outcome Monitoring (ROM) in their daily work. We describe the development and first evaluation of a ROM based computerized clinical decision aid, Treatment-E-Assist (TREAT) for the treatment of psychotic disorders. The goal is to generate personalized treatment recommendations, based on international guidelines combined with outcomes of mental and physical health acquired through ROM. We present a pilot study aimed to assess the feasibility of this computerized clinical decision aid in daily clinical practice by evaluating clinicians' experiences with the system.

METHODS

Clinical decision algorithms were developed based on international schizophrenia treatment guidelines and the input of multidisciplinary expert panels from multiple psychiatric institutes. Yearly obtained diagnostic (ROM) information of patients was presented to treating clinicians combined with treatment suggestions generated by the algorithms of TREAT. In this pilot study 6 clinicians and 16 patients of Lentis Psychiatric Institute used the application. Clinicians were interviewed and asked to fill out self-report questionnaires evaluating their opinions about ROM and the effectiveness of TREAT.

RESULTS

Six clinicians and 16 patients with psychotic disorders participated in the pilot study. The clinicians were psychiatrists, physicians and nurse-practitioners which all worked at least 8 years in mental health care of which at least 3 years treating patients with psychotic illnesses. All Clinicians found TREAT easy to use and would like to continue using the application. They reported that TREAT offered support in using diagnostic ROM information when drafting the treatment plans, by creating more awareness of current treatment options.

CONCLUSION

This article presents a pilot study on the implementation of a computerized clinical decision aid linking routine outcome monitoring to clinical guidelines in order to generate personalized treatment advice. TREAT was found to be feasible for daily clinical practice and effective based on this first evaluation by clinicians. However, adjustments have to be made to the system and algorithms of the application. The ultimate goal is to provide appropriate evidence based care for patients with severe mental illnesses.

The Individual Recovery Outcomes Counter: preliminary validation of a personal recovery measure

Aims and method

The Individual Recovery Outcomes Counter (I.ROC) is to date the only recovery outcomes instrument developed in Scotland. This paper describes the steps taken to initially assess its validity and reliability, including factorial analysis, internal consistency and a correlation benchmarking analysis.

Results

The I.ROC tool showed high internal consistency. Exploratory factor analysis indicated a two-factor structure comprising intrapersonal recovery (factor 1) and interpersonal recovery (factor 2), explaining between them over 50% of the variance in I.ROC scores. There were no redundant items and all loaded on at least one of the factors. The I.ROC significantly correlated with widely used existing instruments assessing both personal recovery and clinical outcomes.

Clinical implications

I.ROC is a valid and reliable measure of recovery in mental health, preferred by service users when compared with well-established instruments. It could be used in clinical settings to map individual recovery, providing feedback for service users and helping to assess service outcomes.

Early Intervention in Psychosis: Health of the Nation Outcome Scales (HoNOS) Outcomes From a Five-Year Prospective Study

INTRODUCTION

Over the last two decades, mental health services internationally have shifted towards intervening early in psychosis. The critical period for intervention is estimated to be five-years and many specialised programs target early psychosis.

AIM/QUESTION

This prospective cohort study aimed to evaluate five-year outcomes from an early psychosis program (EPP) that adopted an integrated model, providing nursing and multidisciplinary community mental healthcare to clients aged 16-65years, beyond the typical age range of 16-25years.

METHOD

We examined one routine outcome measure, the Health of the Nation Outcome Scales (HoNOS) across episodes of care for clients receiving EPP over a 5year period (n=239), comparing these results with HoNOS outcomes in an Australian national dataset for all public mental health clients.

RESULTS

HoNOS improvements were highly significant from intake to discharge and from review to discharge for EPP clients, and these compared well with national outcome performance.

CONCLUSION

There is potential for mental health nurses and other clinicians to significantly improve client symptoms and functioning, in a model of early psychosis treatment beyond a youth focus.

Exploring the use of Routine Outcome Monitoring in the treatment of patients with a psychotic disorder

BACKGROUND

Routine Outcome Monitoring (ROM) has become part of the treatment process in mental health care. However, studies have indicated that few clinicians in psychiatry use the outcome of ROM in their daily work. The aim of this study was to explore the degree of ROM use in clinical practice as well as the explanatory factors of this use.

METHODS

In the Northern Netherlands, a ROM-protocol (ROM-Phamous) for patients with a psychotic disorder has been implemented. To establish the degree of ROM-Phamous use in clinical practice, the ROM results of patients (n=204) were compared to the treatment goals formulated in their treatment plans. To investigate factors that might influence ROM use, clinicians (n=32) were asked to fill out a questionnaire about ROM-Phamous.

RESULTS

Care domains that were problematic according to the ROM-Phamous results were mentioned in the treatment plan in 28% of cases on average (range 5-45%). The use of ROM-Phamous in the treatment process varies considerably among clinicians. Most of the clinicians find ROM-Phamous both useful and important for good clinical practice. In contrast, the perceived ease-of-use is low and most clinicians report insufficient time to use ROM-Phamous.

CONCLUSIONS

More frequent ROM use should be facilitated in clinicians. This could be achieved by improving the fit with clinical routines and the ease-of-use of ROM systems. It is important for all stakeholders to invest in integrating ROM in clinical practice. Eventually, this might improve the diagnostics and treatment of patients in mental health care.

A critical interpretive synthesis of the most commonly used self-report measures in Australian mental health research

OBJECTIVE

To critically examine the self-report measures most commonly used in Australian mental health research in the last 10 years.

METHOD

A critical interpretive synthesis was conducted using seven outcome measures that were identified as most popular in 43 studies from three mental health journals.

RESULTS

Results suggest that the amount and type of language used in outcome measures is important in both increasing the accuracy of the data collected and fostering positive experiences of data collection for participants.

CONCLUSIONS

Results indicate that many of the measures most often used in Australian mental health research may not align with the current contemporary philosophy of mental health clinical practice in Australia.

Longitudinal comparative evaluation of the equivalence of an integrated peer-support and clinical staffing model for residential mental health rehabilitation: a mixed methods protocol incorporating multiple stakeholder perspectives

BACKGROUND

A novel staffing model integrating peer support workers and clinical staff within a unified team is being trialled at community based residential rehabilitation units in Australia. A mixed-methods protocol for the longitudinal evaluation of the outcomes, expectations and experiences of care by consumers and staff under this staffing model in two units will be compared to one unit operating a traditional clinical staffing. The study is unique with regards to the context, the longitudinal approach and consideration of multiple stakeholder perspectives.

METHODS/DESIGN

The longitudinal mixed methods design integrates a quantitative evaluation of the outcomes of care for consumers at three residential rehabilitation units with an applied qualitative research methodology. The quantitative component utilizes a prospective cohort design to explore whether equivalent outcomes are achieved through engagement at residential rehabilitation units operating integrated and clinical staffing models. Comparative data will be available from the time of admission, discharge and 12-month period post-discharge from the units. Additionally, retrospective data for the 12-month period prior to admission will be utilized to consider changes in functioning pre and post engagement with residential rehabilitation care. The primary outcome will be change in psychosocial functioning, assessed using the total score on the Health of the Nation Outcome Scales (HoNOS). Planned secondary outcomes will include changes in symptomatology, disability, recovery orientation, carer quality of life, emergency department presentations, psychiatric inpatient bed days, and psychological distress and wellbeing. Planned analyses will include: cohort description; hierarchical linear regression modelling of the predictors of change in HoNOS following CCU care; and descriptive comparisons of the costs associated with the two staffing models. The qualitative component utilizes a pragmatic approach to grounded theory, with collection of data from consumers and staff at multiple time points exploring their expectations, experiences and reflections on the care provided by these services.

DISCUSSION

It is expected that the new knowledge gained through this study will guide the adaptation of these and similar services. For example, if differential outcomes are achieved for consumers under the integrated and clinical staffing models this may inform staffing guidelines.

Consistency of the Health of the Nation Outcome Scales (HoNOS) at inpatient-to-community transition

OBJECTIVES

The Health of the Nation Outcome Scales (HoNOS) are mandated outcome-measures in many mental-health jurisdictions. When HoNOS are used in different care settings, it is important to assess if setting specific bias exists. This article examines the consistency of HoNOS in a sample of psychiatric patients transitioned from acute inpatient care and community centres.

SETTING

A regional mental health service with both acute and community facilities.

PARTICIPANTS

111 psychiatric patients were transferred from inpatient care to community care from 2012 to 2014. Their HoNOS scores were extracted from a clinical database; Each inpatient-discharge assessment was followed by a community-intake assessment, with the median period between assessments being 4 days (range 0-14). Assessor experience and professional background were recorded.

PRIMARY AND SECONDARY OUTCOME MEASURES

The difference of HoNOS at inpatient-discharge and community-intake were assessed with Pearson correlation, Cohen's κ and effect size.

RESULTS

Inpatient-discharge HoNOS was on average lower than community-intake HoNOS. The average HoNOS was 8.05 at discharge (median 7, range 1-22), and 12.16 at intake (median 12, range 1-25), an average increase of 4.11 (SD 6.97). Pearson correlation between two total scores was 0.073 (95% CI -0.095 to 0.238) and Cohen's κ was 0.02 (95% CI -0.02 to 0.06). Differences did not appear to depend on assessor experience or professional background.

CONCLUSIONS

Systematic change in the HoNOS occurs at inpatient-to-community transition. Some caution should be exercised in making direct comparisons between inpatient HoNOS and community HoNOS scores.

Clinical use of the Kessler psychological distress scales with culturally diverse groups

The Kessler 10 (K10) and embedded Kessler 6 (K6) was developed to screen for non-specific psychological distress and serious mental illness in mental health surveys of English-speaking populations, but has been adopted in Western and non-Western countries as a screening and outcome measure in primary care and mental health settings. This review examines whether the original K6/K10's validity for culturally diverse populations was established, and whether the cultural equivalence, and sensitivity to change of translated or culturally adapted K6/K10s, has been demonstrated with culturally diverse client groups. Evidence for the original K6/K10's validity for culturally diverse populations is limited. Questions about the conceptual and linguistic equivalence of translated/adapted K6/K10s arise from reports of changes in item connotation and differential item functioning. Evidence for structural equivalence is inconsistent, as is support for criterion equivalence, with the majority of studies compromising on accuracy in case prediction. Research demonstrating sensitivity to change with culturally diverse groups is lacking. Inconsistent evidence for the K6/K10's cultural appropriateness in clinical settings, and a lack of clinical norms for either majority or culturally diverse groups, indicate the importance of further research into the psychological distress construct with culturally diverse clients, and the need for caution in interpreting K6/K10 scores.

Development of a short measure of psychological recovery in serious mental illness: the STORI-30

OBJECTIVE

To develop a brief measure of stage of psychological recovery from mental illness by identifying the best-performing items of the 50-item Stages of Recovery Instrument (STORI).

METHOD

Item response modelling was used to identify a short form of the full-length STORI. The resulting items were subjected to factor analysis to further refine the subscales. A second data set was used to confirm the construct validity of the new measure. A correlational analysis was conducted to examine relationships among the five subscale scores.

RESULTS

Analyses identified 30 items that represented the five stages of the full STORI. The five stage subscale scores of the shorter measure, the STORI-30, showed a pattern of correlations that demonstrated an ordinal relationship between the stages.

CONCLUSIONS

There is a need for recovery-oriented measures to augment established clinical assessment tools. The shorter version of the STORI, the STORI-30, shows promise as a brief measure of stage of recovery, more feasible for routine clinical use. Further psychometric and longitudinal testing is recommended. Qualitative research would be valuable in establishing acceptability to consumers and the clinical usefulness of the STORI-30.

An evaluation of the implementation of the FACE Health and Social Assessment in routine psychiatric practice

BACKGROUND

Systematic assessment of patients is seen as an important aspect of mental health care. As yet few UK services have successfully implemented outcome measures.

AIMS

To examine the feasibility of using FACE Health and Social Assessment in everyday practice and explore issues raised for staff.

METHOD

This comprised a feasibility questionnaire completed by 52 staff, 3 focus groups involving 18 staff and an analysis of patient scores on routinely collected FACE data.

RESULT

The questionnaire findings were positive yet only just over half of the patients on Assertive Outreach (AO) teams had some part of FACE completed in the previous year. The focus groups drew attention to many other calls on staff time and the importance of ongoing support. Staff were uncertain about the benefits of FACE for measuring change but believed that it promoted reflective practice. FACE subscores showed some ability to discriminate between patients on Rehabilitation and Recovery and AO teams but there were no significant differences for AO patients over time.

CONCLUSIONS

Although competing demands and low priority may underpin the failure to implement FACE it is perhaps clinical staffs lack of conviction in its contribution to improving mental health care that is the fundamental issue.

The concurrent validity and sensitivity of change of the German version of the health of the nation outcome scales in a psychiatric inpatient setting

BACKGROUND

The Health of the Nation Outcome Scales (HoNOS) were developed to assess the severity of a mental illness. They are used as outcome measures in different countries, and are meanwhile translated from the original English version into many languages, among others into German (HoNOS-D). We conducted a study in order to estimate the concurrent validity and sensitivity to change using clinical parameters as ICD-10 diagnoses, as well as the Clinical Global Impression Scale (CGI), and the Association for Methodology and Documentation in Psychiatry (AMDP) psychopathology scale, a frequently used psychopathological rating system, in a representative clinical sample.

SAMPLING AND METHODS

Data on the three instruments (CGI, AMDP, HoNOS-D) were collected at admission and discharge of 100 psychiatric inpatients using a representative clinical sample. Experienced clinicians completed the CGI, AMDP and HoNOS-D. Descriptive and comparative data analyses were performed. We estimated the concurrent validity by calculating correlations between the HoNOS and other scales. Secondly, we examined the differences between HoNOS scores related to diagnoses and demographic parameters. Thirdly we calculated change criteria and outcome effect size for the HoNOS.

RESULTS

Even in a small clinical sample (n = 100), the HoNOS-D items are highly correlated with the corresponding AMDP syndromes (p < 0.003). The HoNOS-D score is associated with the CGI score (p < 0.01). Correlations of HoNOS symptoms, behavior and impairment items with AMDP syndromes as well as differences in diagnoses were appropriate and comprehensible as regards clinical content, and change on the HoNOS total score is statistically significant (t = 6.57, d.f. = 89, p < 0.0001).

CONCLUSION

This study is the first to investigate the concurrent validity of HoNOS-D concerning psychopathology using the AMDP rating system in a clinical sample of patients with mental disorders in an inpatient setting. HoNOS-D can be recommended for routinely screening outcomes in inpatient psychiatric settings. Our analysis showed that HoNOS-D covers psychopathology corresponding to the AMDP rating system. A limitation of the study is that the study sample comprised only an inpatient population; there may well be differences compared to an outpatient sample.