Routine mental health outcome measurement in the UK

Development and implementation of quality indicators in a group of nine psychiatric hospitals

Quality indicators (QI) are becoming increasingly important in mental healthcare in Germany. QI can be used for various purposes, such as for creating transparency as well as for benchmarking between hospitals. QI themselves are subject to high quality standards. The aim of this report is to describe the development and implementation of QI in a group of psychiatric hospitals. Since 2015, the LVR hospital group has developed and gradually implemented QI for the purposes of quality measurement, quality assurance and internal benchmarking in its nine psychiatric hospitals in a comprehensive, multidisciplinary, scientifically accompanied process. The full LVR-QI set, consisting of eight structure-, twelve process- and four outcome indicators as well as one patient satisfaction questionnaire, was implemented by 2019. In order to create high documentation quality and acceptance by clinicians, various implementation and dissemination strategies were used, such as written documentation manuals, staff training as well as regular face-to-face communication between the LVR hospitals, the LVR Institute for Health Services Research as the central coordinating body and the headquarters of the LVR hospital group. The QI led to a quality-oriented dialogue within and between the LVR hospitals.

Mental health outcome measures in the Australian context: what is the problem represented to be?

BACKGROUND

There is growing interest in the use of routine outcome measures (ROM) in mental health services worldwide. Australia has been at the forefront of introducing ROM in public mental health services, with the aim of improving services and consumer outcomes.

METHODS

An in-depth policy and document analysis was conducted using Carol Bacchi's 'What is the problem represented to be?' approach to critically analyse the use of ROM. This approach was used to identify and analyse the problem representations relating to the need for, and the choice of, outcome measures in Australian public mental health services, and the potential consequences of policy and practice. Data included in the analysis were seven policy documents, four reports on the introduction of outcome measures in Australia, the Australian Mental Health Outcomes and Classifications Network website, and the content of the outcome measures themselves.

RESULTS

Two dominant representations of the 'problem' were identified: 1) the 'problem' of mental health service quality and accountability, relating to the need for mental health outcome measures; and 2) the 'problem' of addressing deficits in biopsychosocial functioning of mental health consumers, which relates to the choice of outcome measures. Framing the 'problem' of mental health outcomes in these ways locates the problem within individual health providers, services, and consumers, ignoring the broader socioeconomic conditions underpinning mental health and effective service provision.

CONCLUSIONS

This critical analysis of the introduction and use of ROM in public mental health services in Australia highlights the need to consider the role of the social determinants of mental health, mental health service funding, and recovery-oriented care in ensuring services are meeting consumer needs and expectations. Broader governmental engagement is central to genuine change and opportunities.

Progress toward a performance measure for mental health based on a generic patient- reported outcome measure: Findings from the Veterans Outcome Assessment survey

We report on studies conducted to develop outcome-based performance measures (PROM-PMs) based on generic patient-reported outcome measures (PROMs) that could support strategies for quality improvement applicable to all patients in a mental health system. Data were from the Veterans Outcome Assessment Survey at baseline and three months for the Mental Component Score (MCS-12), a widely used measure of mental health-related quality of life, for 15,540 outpatients beginning treatment in General Mental Health clinics in 140 Veterans Affairs (VA) facilities. Mental health diagnoses from medical records were coded using hierarchical categories. Mental health staffing levels and quality measures were from administrative data. Changes in MCS-12 scores were associated with demographics, baseline scores, and diagnostic categories; in fully adjusted models, differences between facilities accounted for only 0.5% of the total variance between patients. There were small but significant associations of both baseline and changes in MCS-12 scores with staffing levels and administrative measures of the quality of care that support the potential value of adjusted measures of changes in MCS-12 as a PROM-PM. Remaining issues include the low proportion of variability that can be attributed to differences between facilities and the associations of staffing and quality with possible case-mix adjustment variables.

Assessing the Content Validity of the Revised Health of the Nation Outcome Scales (HoNOS 2018)

The Health of the Nation Outcome Scales (HoNOS) comprises 12 scales that cover the kinds of problems that may be experienced by working-age adults in contact with specialised mental health services. Drawing on 20 years' experience in clinical practice, a collaborative, international review of the HoNOS was undertaken and a revised measure (known as the HoNOS 2018) was published. In this study, 32 experts from Australia, England and New Zealand completed an anonymous web-based survey to assess the relevance, comprehensiveness and comprehensibility (aspects of content validity) of the HoNOS 2018. The experts rated 11 of the 12 HoNOS 2018 scales as 'important' or 'very important' for determining the overall clinical severity (item-level content validity index or I-CVI ≥ 0.75). Evaluations of the scales' ability to capture change, comprehensiveness and comprehensibility were more variable, but generally positive. Experts' comments provided further insights into this variability; for example, they noted that some scales combine multiple phenomena, which can result in ambiguity in item wording and assessment challenges. Results from this study suggest that the revisions have not altered the importance of the scales. Given the measure's breadth of content, training remains important for ensuring rating fidelity. Inter-rater reliability and utility testing are indicated.

A very short Symptom Checklist-90-R version for routine outcome monitoring in psychotherapy; The SCL-3/7

OBJECTIVE

Routine outcome monitoring (ROM) is applied in many physical and mental health treatments. The treatment course is monitored with patient reported outcome measures (PROMs). A potential problem with PROM is response burden. This can be decreased by presenting such measures with less and better selected items. The SCL-90-R is an often used PROM for psychotherapies and a number of very short forms have been developed; the SCL-5, SCL-8, SCL-9 and SCL-10. This study aims to develop a new very short form, the symptom checklist 3 out of 7 (SCL-3/7) and to evaluate the effectiveness of these PROM with the precision relative to the complete SCL-90-R score.

METHODS

Item Response Theory analysis was applied to select the 7 best discriminating items, evenly distributed over the latent trait. A routing serves that patients only need to administer 3 items.

RESULTS

In a sample of 15,055 cases, the relative precisions of the SCL-3/7 were best for outpatients (122.7%), day care patients (111.8%) and inpatients (108.3). The SCL-5 was best for juvenile patients (110.0%), and the SCL-9 was best for addicted patients (107.2%).

CONCLUSION

The SCL-3/7 decreases patient burden in ROM and has a better precision in adult therapies than other SCL-90 short forms.

Quality indicators for mental healthcare in the Danube region: results from a pilot feasibility study

Quality indicators are vital for monitoring the transformation of institution-based mental health services towards the provision of person-centered mental healthcare. While several mental healthcare quality indicators have been identified as relevant and valid, their actual usability and utility for routine monitoring healthcare quality over time is significantly determined by the availability and trustworthiness of the underlying data. In this feasibility study, quality indicators that have been systematically identified for use in the Danube region countries of Bulgaria, the Czech Republic, Hungary, and Serbia were measured on the basis of existing mental healthcare data in the four countries. Data were collected retrospectively by means of the best available, most standardized, trustworthy, and up-to-date data in each country. Out of 21 proposed quality indicators, 18 could be measured in Hungary, 17 could be measured in Bulgaria and in the Czech Republic, and 8 could be measured in Serbia. The results demonstrate that a majority of quality indicators can be measured in most of the countries by means of already existing data, thereby demonstrating the feasibility of quality measurement and regular quality monitoring. However, data availability and usability are scattered across countries and care sectors, which leads to variations in the quality of the quality indicators themselves. Making the planning and outputs of national mental healthcare reforms more transparent and evidence-based requires (trans-)national standardization of healthcare quality data, their routine availability and standardized assessment, and the regular reporting of quality indicators.

A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers

PURPOSE

To review and integrate the literature on mental-health-related patient-reported outcome measures (PROMs) and routine outcome measures (ROMs), namely in the domains of goals, characteristics, implementation, settings, measurements and barriers. PROM/ROM aims mainly to ascertain treatment impact in routine clinical practice through systematic service users' health assessment using standardized self-report, caretaker and/or provider assessment.

DATA SOURCES

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science's Direct.

STUDY SELECTION

Systemized review of literature (2000-2018) on implementation and sustainability of PROMs/ROMs in adult mental health settings (MHS).

DATA EXTRACTION AND SYNTHESIS

Systemized review of literature (2000-2018) on numerous aspects of PROM/ROM implementation and sustainability in adult MHS worldwide.

RESULTS

Based on 103 articles, PROMs/ROMs were implemented mostly in outpatient settings for people with assorted mental health disorders receiving a diversity of services. Frequency of assessments and completion rates varied: one-third of projects had provider assessments; about half had both provider and self-assessments. Barriers to implementation: perceptions that PROM/ROM is intrusive to clinical practice, lack of infrastructure, fear that results may be used for cost containment and service eligibility instead of service quality improvement, difficulties with measures, ethical and confidentiality regulations and web security data management regulations.

CONCLUSION

Improving data input systems, sufficient training, regular feedback, measures to increase administrative and logistic support to improve implementation, acceptability, feasibility and sustainability, follow-up assessments and client attrition rate reduction efforts are only some measures needed to enhance PROM/ROM efficiency and efficacy.

Implementing Measurement-Based Care for Depression: Practical Solutions for Psychiatrists and Primary Care Physicians

Measurement-based care (MBC) can be defined as the clinical practice in which care providers collect patient data through validated outcome scales and use the results to guide their decision-making processes. Despite growing evidence supporting the effectiveness of MBC for depression and other mental health conditions, many physicians and mental health clinicians have yet to adopt MBC practice. In part, this is due to individual and organizational barriers to implementing MBC in busy clinical settings. In this paper, we briefly review the evidence for the efficacy of MBC focusing on pharmacological management of depression and provide example clinical scenarios to illustrate its potential clinical utility in psychiatric settings. We discuss the barriers and challenges for MBC adoption and then address these by suggesting simple solutions to implement MBC for depression care, including recommended outcome scales, monitoring tools, and technology solutions such as cloud-based MBC services and mobile health apps for mood tracking. The availability of MBC tools, ranging from paper-pencil questionnaires to mobile health technology, can allow psychiatrists and clinicians in all types of practice settings to easily incorporate MBC into their practices and improve outcomes for their patients with depression.

Young Minds: Mental Health and Transitional Care in Adolescent and Young Adult Rheumatology

Consideration of the mental health and emotional wellbeing is an important component of health care for all young people, irrespective of setting. Mental health disorders are common during adolescence and young adulthood and young people with rheumatic musculoskeletal diseases (RMD) are not exempt. For such young people, risks of poor outcomes are related to both mental health as well as their RMD. Times of change during adolescence and young adulthood-transitions-are potentially vulnerable life stages for young people with RMD and warrant specific attention in health care provision. Such transitions include those occurring at puberty, during education, training, and employment, socially with moves away from the parental home, as well as from child to adult-centered health services. There is great potential for rheumatology professionals to support young people with RMD at these transitions in view of their frequent encounters and ongoing therapeutic relationships. In this review, we aim to assess the impact of mental health on RMD during adolescence and young adulthood with particular reference to transitional care provision and how rheumatology professionals can be involved in addressing mental health issues during this time of change.

Routine measurement of satisfaction with life and treatment aspects in mental health patients - the DIALOG scale in East London

AIMS

The DIALOG scale has been implemented as a routine patient outcome and experience measure (PROM/PREM) in a mental health trust in East London since 2017. The resulting healthcare dataset was used to estimate satisfaction with life and treatment aspects over time and factors associated with it.

METHODS

Variables available from the Trust were DIALOG items, service level, clinical and basic demographic data. Data was extracted in February 2019. Data is described using a range of descriptive statistics and looking at the subgroups: treatment stage, diagnosis, service type. Predictors for average DIALOG scores across patients was explored with clustered linear regression models. A fixed effect model was chosen to estimate the impact of clinical and service related variables on patient's average DIALOG scores over time. Sensitivity analyses with the whole data set and complete cases were carried out.

RESULTS

Of the original 18,481 DIALOG records 12, 592 were kept after data cleaning (5646 patients). The average DIALOG score was 4.8 (SD 1.0) on the 7-point scale. Average satisfaction with life aspects (PROM) was 4.65 (SD 1.1) and with treatment aspects (PREM) was 5.25 (SD 1.17). Across all 11 items, "job situation" scored lowest (mean 4.05) and "meetings with professionals" highest (mean 5.5). Satisfaction for all items increased over time (average increase 0.47). The largest increase was in "mental health" (0.94) and the smallest in "family relationships" (0.34).

CONCLUSIONS

Patients in mental healthcare services were "fairly satisfied" in both life and treatment aspects with improvements seen over time. These results will act as a benchmark for clinical services currently implementing DIALOG across the UK and inform local service developments.

Improving measurement feedback systems for measurement-based care

Abstract Objectives: Measurement feedback systems (MFS) are a class of health information technologies developed to facilitate measurement-based care. The individual clinical decision support features within MFS are diverse and their influence on clinicians is largely unknown. This study tested the impact of MFS features on clinicians' progress assessments and treatment decisions in different scenarios. Method: Clinicians (n = 299) were randomly assigned to view one of six combinations of the following MFS features: graph, expected change trajectory line, alert, and treatment suggestions. The assigned feature combination was paired with three vignettes and clinical data representing three clinical scenarios: patient deterioration, no progress, and approaching remission. Clinicians answered questions after each vignette, and at the conclusion. Results: MFS features differentially impacted clinicians' progress assessment accuracy, their likelihood of making a treatment change, and their treatment choices. Which feature was most impactful varied depending on the clinical scenario. Clinicians reported graphs influenced their assessments and choices significantly more than the other features, and the majority stated they would prefer to use all of the features. Conclusions: Specific MFS features impact clinicians' assessments and choices to greater degrees, and the impact of those features can be influenced by the clinical state of the patient.

Value-based healthcare in mental health services

In value-based healthcare (VBHC) value is defined as outcomes that matter to patients divided by the cost of achieving these outcomes. Value is measured for discrete medical conditions across the whole cycle of care. Data on the value achieved by different providers is openly shared. Providers increase value using quality improvement (QI) techniques to improve outcomes, reduce costs or both. Patients or commissioners choose the provider achieving the greatest value. Units should compete regionally or nationally. There are challenges to implementing such ideas in the mental health services in the UK. However, measuring outcomes, understanding costs and using QI to drive up value may be possible without adopting the complete model that has developed in the context of a North American and acute hospital healthcare system.

A qualitative exploration of SMART Recovery meetings in Australia and the role of a digital platform to support routine outcome monitoring

Previous research has reported on the benefits of mutual support groups. However, such groups do not routinely collect data on participant outcomes. Moreover, the effect of collecting outcomes measures on these groups is unknown. The objective of this mixed methods study was to elicit participant views on using a novel, purpose built digital platform for routine outcome monitoring (ROM) as a standard component of a mutual support group. SMART Recovery, or the Self-Management and Recovery Training program, is group-based and uses professional clinicians to facilitate discussion and foster mutual support for a range of addictive behaviours, alongside Cognitive Behavioural Therapy and Motivational Interviewing techniques. This paper reports on the qualitative component of this study and how participants perceive ROMs, and the potential shift to technological resources. Twenty semi-structured telephone interviews were conducted with participants from SMART Recovery groups across New South Wales, Australia. Participants discussed their use of mutual support within group meetings to manage their recovery, including: naming their goals in front of peers; learning from clinicians and group discussion; and developing reciprocal and caring relationships. They also described any previous experience with routine outcomes measures and how digital technologies might enhance or hinder group function. Participants valued mutual support groups and reported that digital technologies could be complementary to physical, weekly group meetings. They were also concerned that the introduction of technological resources might pose a threat to physical meetings, thereby risking their access to mutual support. Findings have implications for the implementation of ROM when delivered via digital mechanisms, and indicate threats and opportunities that warrant consideration for future initiatives.

Patient-reported outcome measurements (PROMs) and provider assessment in mental health: a systematic review of the context of implementation

Purpose

To review and integrate the vast amount of literature yielded by recent growing interest in patient-reported outcome measurement and routine outcome measures (PROMs/ROMs), in order to suggest options and improvements for implementation. PROMs are the systematic assessment of service users’ health using standardized self-report measures. Specifically, for ROMs, it includes routine provider or caretaker assessment measures. Both are administered to ascertain routinely, the impact of treatment in mental health settings and to improve care. A review is needed because of the large differences in setting, conceptualization, practice and implementation. Here, we examine the different major projects worldwide.

Data sources

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science’s Direct.

Study selection

We conducted a systematized review of the literature published from 2000 to 2018 on the implementation and sustainability of PROMs and ROMs in mental health services for adults.

Data extraction, synthesis and Results

We described and characterized the programs in different countries worldwide. We identified 103 articles that met the inclusion criteria, representing over 80 PROMs/ROMs initiatives in 15 countries. National policy and structure of mental health services were found to be major factors in implementation. We discuss the great variability in PROMs/ROMs models in different countries, making suggestions for their streamlining and improvement.

Conclusion

We extracted valuable information on the different characteristics of the numerous PROMs/ROMs initiatives worldwide. However, in the absence of a strong nationwide policy effort and support, implementation seems scattered and irregular. Thus, development of the implementation of PROMs/ROMs is left to groups of enthusiastic clinicians and researchers, making sustainability problematic.

Implementation of outcome measurement (HoNOS) in an outpatient psychiatric clinic in Sligo/Leitrim mental health service

BACKGROUND

Routine clinical outcome monitoring (RCOM) is the standardised gathering of measures of clinical outcomes in everyday practice. HoNOS (Health of the Nation Outcome Scales) is a tool used in RCOM.

AIMS

To examine (a) agreement between HoNOS and Global Assessment of Functioning (GAF), (b) HoNOS changes over time/attendance and (c) clinical parameters affecting HoNOS scores.

METHODS

Data from outpatient clinics were collected at each contact over 2 years until June 2016 including: gender, age, diagnosis (ICD-10) and HoNOS scores. In a subsample, the GAF also were completed by community psychiatric nurses blind to HoNOS scores.

RESULTS

A number of 470 outpatients have undergone 1125 HoNOS assessments during the study period. Mean age of the attendants was 43.12; SD 14.6. Male = 220 (46.8%). Longitudinal analysis demonstrated that lower HoNOS scores are independently significantly associated to number of assessments and diagnosis in ICD-10 categories of F20-F29 (Schizophrenia, schizotypal and delusional disorders) F30-F39 (mood disorders) F40-F48 (neurotic, stress-related and somatoform disorders) and F50-F59 (behavioural disorders associated with physiological disturbances). Gender and age were not significantly associated with decline of HoNOS scores. Neither were other diagnostic categories. Agreement between HoNOS and GAF was excellent (N = 261, rho = - 0.919, p < 0.001).

CONCLUSIONS

This study shows that HoNOS is a feasible instrument which can be potentially used in ROCM in mental health services in Ireland and supports further the need for implementation of routine measurements in Mental Health Services. It adds longitudinal data which is lacking in similar previous studies.