Current practices for measuring mental health outcomes in the USA: International overview of routine outcome measures in mental health

Progress toward a performance measure for mental health based on a generic patient- reported outcome measure: Findings from the Veterans Outcome Assessment survey

We report on studies conducted to develop outcome-based performance measures (PROM-PMs) based on generic patient-reported outcome measures (PROMs) that could support strategies for quality improvement applicable to all patients in a mental health system. Data were from the Veterans Outcome Assessment Survey at baseline and three months for the Mental Component Score (MCS-12), a widely used measure of mental health-related quality of life, for 15,540 outpatients beginning treatment in General Mental Health clinics in 140 Veterans Affairs (VA) facilities. Mental health diagnoses from medical records were coded using hierarchical categories. Mental health staffing levels and quality measures were from administrative data. Changes in MCS-12 scores were associated with demographics, baseline scores, and diagnostic categories; in fully adjusted models, differences between facilities accounted for only 0.5% of the total variance between patients. There were small but significant associations of both baseline and changes in MCS-12 scores with staffing levels and administrative measures of the quality of care that support the potential value of adjusted measures of changes in MCS-12 as a PROM-PM. Remaining issues include the low proportion of variability that can be attributed to differences between facilities and the associations of staffing and quality with possible case-mix adjustment variables.

A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers

PURPOSE

To review and integrate the literature on mental-health-related patient-reported outcome measures (PROMs) and routine outcome measures (ROMs), namely in the domains of goals, characteristics, implementation, settings, measurements and barriers. PROM/ROM aims mainly to ascertain treatment impact in routine clinical practice through systematic service users' health assessment using standardized self-report, caretaker and/or provider assessment.

DATA SOURCES

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science's Direct.

STUDY SELECTION

Systemized review of literature (2000-2018) on implementation and sustainability of PROMs/ROMs in adult mental health settings (MHS).

DATA EXTRACTION AND SYNTHESIS

Systemized review of literature (2000-2018) on numerous aspects of PROM/ROM implementation and sustainability in adult MHS worldwide.

RESULTS

Based on 103 articles, PROMs/ROMs were implemented mostly in outpatient settings for people with assorted mental health disorders receiving a diversity of services. Frequency of assessments and completion rates varied: one-third of projects had provider assessments; about half had both provider and self-assessments. Barriers to implementation: perceptions that PROM/ROM is intrusive to clinical practice, lack of infrastructure, fear that results may be used for cost containment and service eligibility instead of service quality improvement, difficulties with measures, ethical and confidentiality regulations and web security data management regulations.

CONCLUSION

Improving data input systems, sufficient training, regular feedback, measures to increase administrative and logistic support to improve implementation, acceptability, feasibility and sustainability, follow-up assessments and client attrition rate reduction efforts are only some measures needed to enhance PROM/ROM efficiency and efficacy.

Prospectively identifying adults with serious mental illness at risk for poor physical health: The role of person reported outcomes

OBJECTIVE

Persons with serious mental illnesses are at increased risk for co-occurring physical comorbidities. Patient-reported outcome measures are increasingly used in routine assessments of persons with serious mental illnesses, yet the relation of patient-reported outcome measures to physical health outcomes has not been comprehensively investigated. We examined the association between patient-reported outcome measures and self-reported physical health at 1-year follow-up.

DESIGN

A retrospective cohort study.

SETTING

Data were collected as part of the Israeli Psychiatric Rehabilitation Patient-Reported Outcome Measurement program in Israel.

PARTICIPANTS

A total of 2581 psychiatric rehabilitation service users assessed between April 2013 and January 2016.

MAIN OUTCOME MEASURES

Self-reports on two consecutive years of physical health dichotomized as poor versus good.

RESULTS

More than one-third of participants reported having poor physical health. Multivariate regression analysis showed that quality of life (odds ratio [OR] = 0.71; 95% confidence interval [CI]: 0.60-0.84) and lack of effect of symptoms on functioning (OR = 0.81; 95%CI: 0.74-0.89) predict subsequent physical health, controlling for all other factors. Compared to a multivariate model with personal characteristics and self-reports on physical health at baseline (Model A), the model which also included patient-reported outcome measures (Model B) showed slightly better discrimination (c-statistic: 0.74 vs. 0.76, respectively).

CONCLUSIONS

These results suggest that patient-reported outcome measures contribute to the prediction of poor physical health and thus can be useful as an early screening tool for people with serious mental illnesses living in the community, who are at risk of physical health problems.

Developing medical record-based, healthcare quality indicators for psychiatric hospitals in China: a modified Delphi-Analytic Hierarchy Process study

OBJECTIVES

To develop a medical record-based, comprehensive system of healthcare quality indicators for psychiatric hospitals in China.

DESIGN

A modified Delphi process with analytic hierarchy process (AHP) was used.

PARTICIPANTS

Twenty nationally-recognized experts were invited to participate in two rounds of Delphi expert consultation and AHP.

METHODS

Fifty potential indicators were included based on literature review, and 20 experts were asked to rate the importance of each indicator using two rounds of email surveys. The AHP was used to determine the relative importance of the finalized quality indicators.

RESULTS

The average authoritative coefficient was 0.92 ± 0.07. After two rounds of Delphi consultation, 47 healthcare quality indicators were identified for Chinese psychiatric hospitals. The mean importance ratings ranged from 7.06 to 8.84 on a nine-point scale, with variation coefficients ranging from 0.04 to 0.22. The percentage of full score for potential indicators ranged from 16% to 74%. In two rounds, the Kendall's W coefficients ranged from 0.423 to 0.535. The weights of structure, process and outcome were 0.175, 0.211 and 0.614, respectively.

CONCLUSION

We developed the first set of healthcare quality indicators for psychiatric hospitals in mainland China, and it will provide a standardized and meaningful guide to evaluate the healthcare quality of psychiatric hospitals across the country.

An Examination of the Association Between Patient Experience and Quality of Mental Health Care Among Women Veterans

Improving patient experience is one strategy that may increase the quality of mental health care if better experience is linked to the likelihood of a potentially therapeutic dose (PTD) of treatment. This study sought to examine: (1) the proportion of women veterans who obtained a PTD of mental health treatment; and (2) the association between women's experiences with Veterans Health Administration (VHA) mental health services and obtaining a PTD of mental health treatment. We assessed patient experience via a survey that measured experiences with gender-sensitive care, ease of getting care, perceived quality of care, and extent to which care met needs. We used VHA administrative data to determine mental health utilization across a national sample of 2109 women veterans with episodes of mental health care that included psychotherapy or pharmacotherapy. Results indicated that 71% of women received a PTD. Positive ratings regarding perceived quality of care and whether care met needs were associated with higher odds of receiving a PTD of treatment. Findings provide supporting evidence for the continued necessity  of offering patient-centered mental health care to women veterans. Careful consideration of women veterans' mental health care experiences may be crucial in promoting high value mental health care for this population in VHA.

Patient-reported outcome measurements (PROMs) and provider assessment in mental health: a systematic review of the context of implementation

Purpose

To review and integrate the vast amount of literature yielded by recent growing interest in patient-reported outcome measurement and routine outcome measures (PROMs/ROMs), in order to suggest options and improvements for implementation. PROMs are the systematic assessment of service users’ health using standardized self-report measures. Specifically, for ROMs, it includes routine provider or caretaker assessment measures. Both are administered to ascertain routinely, the impact of treatment in mental health settings and to improve care. A review is needed because of the large differences in setting, conceptualization, practice and implementation. Here, we examine the different major projects worldwide.

Data sources

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science’s Direct.

Study selection

We conducted a systematized review of the literature published from 2000 to 2018 on the implementation and sustainability of PROMs and ROMs in mental health services for adults.

Data extraction, synthesis and Results

We described and characterized the programs in different countries worldwide. We identified 103 articles that met the inclusion criteria, representing over 80 PROMs/ROMs initiatives in 15 countries. National policy and structure of mental health services were found to be major factors in implementation. We discuss the great variability in PROMs/ROMs models in different countries, making suggestions for their streamlining and improvement.

Conclusion

We extracted valuable information on the different characteristics of the numerous PROMs/ROMs initiatives worldwide. However, in the absence of a strong nationwide policy effort and support, implementation seems scattered and irregular. Thus, development of the implementation of PROMs/ROMs is left to groups of enthusiastic clinicians and researchers, making sustainability problematic.

Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients

OBJECTIVES

If patients change their perspective due to treatment, this may alter the way they conceptualize, prioritize, or calibrate questionnaire items. These psychological changes, also called "response shifts," may pose a threat to the measurement of therapeutic change in patients. Therefore, it is important to test the occurrence of response shift in patients across their treatment.

METHODS

This study focused on self-reported psychological distress/psychopathology in a naturalistic sample of 206 psychiatric outpatients. Longitudinal measurement invariance tests were computed across treatment in order to detect response shifts.

RESULTS

Compared with before treatment, post-treatment psychopathology scores showed an increase in model fit and factor loading, suggesting that symptoms became more coherently interrelated within their psychopathology domains. Reconceptualization (depression/mood) and reprioritization (somatic and cognitive problems) response shift types were found in several items. We found no recalibration response shift.

CONCLUSION

This study provides further evidence that response shift can occur in adult psychiatric patients across their mental health treatment. Future research is needed to determine whether response shift implies an unwanted potential bias in treatment evaluation or a desired cognitive change intended by treatment.

Depression Quality of Care: Measuring Quality over Time Using VA Electronic Medical Record Data

BACKGROUND

The Veterans Health Administration (VA) has invested substantially in evidence-based mental health care. Yet no electronic performance measures for assessing the level at which the population of Veterans with depression receive appropriate care have proven robust enough to support rigorous evaluation of the VA's depression initiatives.

OBJECTIVE

Our objectives were to develop prototype longitudinal electronic population-based measures of depression care quality, validate the measures using expert panel judgment by VA and non-VA experts, and examine detection, follow-up and treatment rates over a decade (2000-2010). We describe our development methodology and the challenges to creating measures that capture the longitudinal course of clinical care from detection to treatment.

DESIGN AND PARTICIPANTS

Data come from the National Patient Care Database and Pharmacy Benefits Management Database for primary care patients from 1999 to 2011, from nine Veteran Integrated Service Networks.

MEASURES

We developed four population-based quality metrics for depression care that incorporate a 6-month look back and 1-year follow-up: detection of a new episode of depression, 84 and 180 day follow-up, and minimum appropriate treatment 1-year post detection. Expert panel techniques were used to evaluate the measure development methodology and results. Key challenges to creating valid longitudinal measures are discussed.

KEY RESULTS

Over the decade, the rates for detection of new episodes of depression remained stable at 7-8 %. Follow-up at 84 and 180 days were 37 % and 45 % in 2000 and increased to 56 % and 63 % by 2010. Minimum appropriate treatment remained relatively stable over the decade (82-84 %).

CONCLUSIONS

The development of valid longitudinal, population-based quality measures for depression care is a complex process with numerous challenges. If the full spectrum of care from detection to follow-up and treatment is not captured, performance measures could actually mask the clinical areas in need of quality improvement efforts.