1
|
Katz IR, Liebmann EP, Resnick SG, Hoff RA, Schmidt EM. Progress toward a performance measure for mental health based on a generic patient- reported outcome measure: Findings from the Veterans Outcome Assessment survey. Psychiatry Res 2022; 317:114797. [PMID: 36030700 DOI: 10.1016/j.psychres.2022.114797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 08/15/2022] [Accepted: 08/16/2022] [Indexed: 11/29/2022]
Abstract
We report on studies conducted to develop outcome-based performance measures (PROM-PMs) based on generic patient-reported outcome measures (PROMs) that could support strategies for quality improvement applicable to all patients in a mental health system. Data were from the Veterans Outcome Assessment Survey at baseline and three months for the Mental Component Score (MCS-12), a widely used measure of mental health-related quality of life, for 15,540 outpatients beginning treatment in General Mental Health clinics in 140 Veterans Affairs (VA) facilities. Mental health diagnoses from medical records were coded using hierarchical categories. Mental health staffing levels and quality measures were from administrative data. Changes in MCS-12 scores were associated with demographics, baseline scores, and diagnostic categories; in fully adjusted models, differences between facilities accounted for only 0.5% of the total variance between patients. There were small but significant associations of both baseline and changes in MCS-12 scores with staffing levels and administrative measures of the quality of care that support the potential value of adjusted measures of changes in MCS-12 as a PROM-PM. Remaining issues include the low proportion of variability that can be attributed to differences between facilities and the associations of staffing and quality with possible case-mix adjustment variables.
Collapse
Affiliation(s)
- Ira R Katz
- VA Office of Mental Health and Suicide Prevention, Philadelphia, PA 19104, United States.
| | - Edward P Liebmann
- Northeast Program Evaluation Center, Office of Mental Health and Suicide Prevention, West Haven, CT 06516, United States; Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06510, United States
| | - Sandra G Resnick
- Northeast Program Evaluation Center, Office of Mental Health and Suicide Prevention, West Haven, CT 06516, United States; Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06510, United States
| | - Rani A Hoff
- Northeast Program Evaluation Center, Office of Mental Health and Suicide Prevention, West Haven, CT 06516, United States; Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06510, United States
| | - Eric M Schmidt
- VA Office of Performance Measurement, Analytics and Performance Integration, Quality and Patient Safety, Veterans Health Administration, Menlo Park, CA 94025, United States; Center for Innovation to Implementation (Ci2i), VA Medical Center, Menlo Park, CA 94025, United States
| |
Collapse
|
2
|
Gelkopf M, Mazor Y, Roe D. A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers. Int J Qual Health Care 2021; 34:ii13–ii27. [PMID: 32159763 DOI: 10.1093/intqhc/mzz133] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Revised: 11/08/2019] [Accepted: 12/03/2019] [Indexed: 01/27/2023] Open
Abstract
PURPOSE To review and integrate the literature on mental-health-related patient-reported outcome measures (PROMs) and routine outcome measures (ROMs), namely in the domains of goals, characteristics, implementation, settings, measurements and barriers. PROM/ROM aims mainly to ascertain treatment impact in routine clinical practice through systematic service users' health assessment using standardized self-report, caretaker and/or provider assessment. DATA SOURCES Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science's Direct. STUDY SELECTION Systemized review of literature (2000-2018) on implementation and sustainability of PROMs/ROMs in adult mental health settings (MHS). DATA EXTRACTION AND SYNTHESIS Systemized review of literature (2000-2018) on numerous aspects of PROM/ROM implementation and sustainability in adult MHS worldwide. RESULTS Based on 103 articles, PROMs/ROMs were implemented mostly in outpatient settings for people with assorted mental health disorders receiving a diversity of services. Frequency of assessments and completion rates varied: one-third of projects had provider assessments; about half had both provider and self-assessments. Barriers to implementation: perceptions that PROM/ROM is intrusive to clinical practice, lack of infrastructure, fear that results may be used for cost containment and service eligibility instead of service quality improvement, difficulties with measures, ethical and confidentiality regulations and web security data management regulations. CONCLUSION Improving data input systems, sufficient training, regular feedback, measures to increase administrative and logistic support to improve implementation, acceptability, feasibility and sustainability, follow-up assessments and client attrition rate reduction efforts are only some measures needed to enhance PROM/ROM efficiency and efficacy.
Collapse
Affiliation(s)
- Marc Gelkopf
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel.,Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel
| | - Yael Mazor
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel.,Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel
| | - David Roe
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel.,Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel.,Department of Clinical Medicine, Psychiatry, Aalborg University, Denmark
| |
Collapse
|
3
|
Hochman L, Moran GS, Gelkopf M, Roe D, Shadmi E. Prospectively identifying adults with serious mental illness at risk for poor physical health: The role of person reported outcomes. Int J Qual Health Care 2021; 34:ii65–ii69. [PMID: 32296822 DOI: 10.1093/intqhc/mzaa033] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Revised: 02/27/2020] [Accepted: 03/16/2020] [Indexed: 01/06/2023] Open
Abstract
OBJECTIVE Persons with serious mental illnesses are at increased risk for co-occurring physical comorbidities. Patient-reported outcome measures are increasingly used in routine assessments of persons with serious mental illnesses, yet the relation of patient-reported outcome measures to physical health outcomes has not been comprehensively investigated. We examined the association between patient-reported outcome measures and self-reported physical health at 1-year follow-up. DESIGN A retrospective cohort study. SETTING Data were collected as part of the Israeli Psychiatric Rehabilitation Patient-Reported Outcome Measurement program in Israel. PARTICIPANTS A total of 2581 psychiatric rehabilitation service users assessed between April 2013 and January 2016. MAIN OUTCOME MEASURES Self-reports on two consecutive years of physical health dichotomized as poor versus good. RESULTS More than one-third of participants reported having poor physical health. Multivariate regression analysis showed that quality of life (odds ratio [OR] = 0.71; 95% confidence interval [CI]: 0.60-0.84) and lack of effect of symptoms on functioning (OR = 0.81; 95%CI: 0.74-0.89) predict subsequent physical health, controlling for all other factors. Compared to a multivariate model with personal characteristics and self-reports on physical health at baseline (Model A), the model which also included patient-reported outcome measures (Model B) showed slightly better discrimination (c-statistic: 0.74 vs. 0.76, respectively). CONCLUSIONS These results suggest that patient-reported outcome measures contribute to the prediction of poor physical health and thus can be useful as an early screening tool for people with serious mental illnesses living in the community, who are at risk of physical health problems.
Collapse
Affiliation(s)
- Limor Hochman
- The Spitzer Department of Social Work, Ben-Gurion University of the Negev, Beersheba, Israel, 8410501
| | - Galia S Moran
- The Spitzer Department of Social Work, Ben-Gurion University of the Negev, Beersheba, Israel, 8410501
| | - Marc Gelkopf
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel.,Faculty of Social Welfare and Health Sciences, Center for Community Mental Health, Research, Practice and Policy, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel
| | - David Roe
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel.,Faculty of Social Welfare and Health Sciences, Center for Community Mental Health, Research, Practice and Policy, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel.,Department of Clinical Medicine, Psychiatry, Aalborg University, Fredrik Bajers Vej 5, 9100 Aalborg, Denmark
| | - Efrat Shadmi
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel.,Faculty of Social Welfare and Health Sciences, Center for Community Mental Health, Research, Practice and Policy, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel.,The Cheryl Spencer Department of Nursing, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 AbaKhoushy Ave. Mount Carmel, Haifa, 3498838 Israel
| |
Collapse
|
4
|
Jiang F, Liu T, Zhou H, Rakofsky JJ, Liu H, Liu Y, Tang YL. Developing medical record-based, healthcare quality indicators for psychiatric hospitals in China: a modified Delphi-Analytic Hierarchy Process study. Int J Qual Health Care 2020; 31:733-740. [PMID: 30753601 DOI: 10.1093/intqhc/mzz005] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Revised: 11/26/2018] [Accepted: 01/16/2019] [Indexed: 12/18/2022] Open
Abstract
OBJECTIVES To develop a medical record-based, comprehensive system of healthcare quality indicators for psychiatric hospitals in China. DESIGN A modified Delphi process with analytic hierarchy process (AHP) was used. PARTICIPANTS Twenty nationally-recognized experts were invited to participate in two rounds of Delphi expert consultation and AHP. METHODS Fifty potential indicators were included based on literature review, and 20 experts were asked to rate the importance of each indicator using two rounds of email surveys. The AHP was used to determine the relative importance of the finalized quality indicators. RESULTS The average authoritative coefficient was 0.92 ± 0.07. After two rounds of Delphi consultation, 47 healthcare quality indicators were identified for Chinese psychiatric hospitals. The mean importance ratings ranged from 7.06 to 8.84 on a nine-point scale, with variation coefficients ranging from 0.04 to 0.22. The percentage of full score for potential indicators ranged from 16% to 74%. In two rounds, the Kendall's W coefficients ranged from 0.423 to 0.535. The weights of structure, process and outcome were 0.175, 0.211 and 0.614, respectively. CONCLUSION We developed the first set of healthcare quality indicators for psychiatric hospitals in mainland China, and it will provide a standardized and meaningful guide to evaluate the healthcare quality of psychiatric hospitals across the country.
Collapse
Affiliation(s)
- Feng Jiang
- School of Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Tingfang Liu
- Institute for Hospital Management of Tsinghua University, Beijing, China
| | - Huixuan Zhou
- School of Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jeffrey J Rakofsky
- Department of Psychiatry and Behavioral Sciences, Emory University, Atlanta, GA, USA
| | - Huanzhong Liu
- Department of Psychiatry, Chaohu Hospital of Anhui Medical University, Hefei, China
| | - Yuanli Liu
- School of Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yi-Lang Tang
- Department of Psychiatry and Behavioral Sciences, Emory University, Atlanta, GA, USA.,Atlanta VA Medical Center, Decatur, Georgia, USA
| |
Collapse
|
5
|
Rossi FS, Javier SJ, Kimerling R. An Examination of the Association Between Patient Experience and Quality of Mental Health Care Among Women Veterans. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2020; 48:61-69. [PMID: 32415345 DOI: 10.1007/s10488-020-01046-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
Improving patient experience is one strategy that may increase the quality of mental health care if better experience is linked to the likelihood of a potentially therapeutic dose (PTD) of treatment. This study sought to examine: (1) the proportion of women veterans who obtained a PTD of mental health treatment; and (2) the association between women's experiences with Veterans Health Administration (VHA) mental health services and obtaining a PTD of mental health treatment. We assessed patient experience via a survey that measured experiences with gender-sensitive care, ease of getting care, perceived quality of care, and extent to which care met needs. We used VHA administrative data to determine mental health utilization across a national sample of 2109 women veterans with episodes of mental health care that included psychotherapy or pharmacotherapy. Results indicated that 71% of women received a PTD. Positive ratings regarding perceived quality of care and whether care met needs were associated with higher odds of receiving a PTD of treatment. Findings provide supporting evidence for the continued necessity of offering patient-centered mental health care to women veterans. Careful consideration of women veterans' mental health care experiences may be crucial in promoting high value mental health care for this population in VHA.
Collapse
Affiliation(s)
- Fernanda S Rossi
- Department of Veterans Affairs, Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, CA, USA. .,Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA.
| | - Sarah J Javier
- Department of Veterans Affairs, Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, CA, USA.,Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA
| | - Rachel Kimerling
- Department of Veterans Affairs, Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, CA, USA.,National Center for PTSD, VA Palo Alto Health Care System, Menlo Park, CA, USA
| |
Collapse
|
6
|
Roe D, Mazor Y, Gelkopf M. Patient-reported outcome measurements (PROMs) and provider assessment in mental health: a systematic review of the context of implementation. Int J Qual Health Care 2019; 34:ii28–ii39. [DOI: 10.1093/intqhc/mzz084] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Revised: 07/25/2019] [Accepted: 08/02/2019] [Indexed: 02/06/2023] Open
Abstract
Abstract
Purpose
To review and integrate the vast amount of literature yielded by recent growing interest in patient-reported outcome measurement and routine outcome measures (PROMs/ROMs), in order to suggest options and improvements for implementation. PROMs are the systematic assessment of service users’ health using standardized self-report measures. Specifically, for ROMs, it includes routine provider or caretaker assessment measures. Both are administered to ascertain routinely, the impact of treatment in mental health settings and to improve care. A review is needed because of the large differences in setting, conceptualization, practice and implementation. Here, we examine the different major projects worldwide.
Data sources
Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science’s Direct.
Study selection
We conducted a systematized review of the literature published from 2000 to 2018 on the implementation and sustainability of PROMs and ROMs in mental health services for adults.
Data extraction, synthesis and Results
We described and characterized the programs in different countries worldwide. We identified 103 articles that met the inclusion criteria, representing over 80 PROMs/ROMs initiatives in 15 countries. National policy and structure of mental health services were found to be major factors in implementation. We discuss the great variability in PROMs/ROMs models in different countries, making suggestions for their streamlining and improvement.
Conclusion
We extracted valuable information on the different characteristics of the numerous PROMs/ROMs initiatives worldwide. However, in the absence of a strong nationwide policy effort and support, implementation seems scattered and irregular. Thus, development of the implementation of PROMs/ROMs is left to groups of enthusiastic clinicians and researchers, making sustainability problematic.
Collapse
Affiliation(s)
- David Roe
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 Aba Khoushy Ave. Mount Carmel, Haifa, 3498838 Israe
- Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel
- Department of Clinical Medicine, Psychiatry, Aalborg University, Denmark
| | - Yael Mazor
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 Aba Khoushy Ave. Mount Carmel, Haifa, 3498838 Israe
- Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel
| | - Marc Gelkopf
- Department of Community Mental Health, Faculty of Social Welfare and Health Sciences, University of Haifa, 199 Aba Khoushy Ave. Mount Carmel, Haifa, 3498838 Israe
- Center for Community Mental Health, Research, Practice and Policy, Faculty of Social Welfare and Health Sciences, University of Haifa, Israel
| |
Collapse
|
7
|
Carlier IVE, van Eeden WA, de Jong K, Giltay EJ, van Noorden MS, van der Feltz-Cornelis C, Zitman FG, Kelderman H, van Hemert AM. Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients. Int J Methods Psychiatr Res 2019; 28:e1785. [PMID: 31206911 PMCID: PMC6852603 DOI: 10.1002/mpr.1785] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/18/2018] [Revised: 03/18/2019] [Accepted: 04/01/2019] [Indexed: 12/17/2022] Open
Abstract
OBJECTIVES If patients change their perspective due to treatment, this may alter the way they conceptualize, prioritize, or calibrate questionnaire items. These psychological changes, also called "response shifts," may pose a threat to the measurement of therapeutic change in patients. Therefore, it is important to test the occurrence of response shift in patients across their treatment. METHODS This study focused on self-reported psychological distress/psychopathology in a naturalistic sample of 206 psychiatric outpatients. Longitudinal measurement invariance tests were computed across treatment in order to detect response shifts. RESULTS Compared with before treatment, post-treatment psychopathology scores showed an increase in model fit and factor loading, suggesting that symptoms became more coherently interrelated within their psychopathology domains. Reconceptualization (depression/mood) and reprioritization (somatic and cognitive problems) response shift types were found in several items. We found no recalibration response shift. CONCLUSION This study provides further evidence that response shift can occur in adult psychiatric patients across their mental health treatment. Future research is needed to determine whether response shift implies an unwanted potential bias in treatment evaluation or a desired cognitive change intended by treatment.
Collapse
Affiliation(s)
- Ingrid V E Carlier
- Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands
| | - Wessel A van Eeden
- Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands
| | - Kim de Jong
- Clinical Psychology Unit, Institute of Psychology, Leiden University, Leiden, The Netherlands
| | - Erik J Giltay
- Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands
| | | | | | - Frans G Zitman
- Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands
| | - Henk Kelderman
- Department of Methodology and Statistics, Institute of Psychology, Leiden University, Leiden, The Netherlands
| | - Albert M van Hemert
- Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands
| |
Collapse
|
8
|
Alegría M, Nakash O, NeMoyer A. Increasing equity in access to mental health care: a critical first step in improving service quality. World Psychiatry 2018; 17:43-44. [PMID: 29352534 PMCID: PMC5775117 DOI: 10.1002/wps.20486] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Affiliation(s)
- Margarita Alegría
- Disparities Research Unit, Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
- Departments of Medicine and Psychiatry, Harvard Medical School, Boston, MA, USA
| | - Ora Nakash
- Disparities Research Unit, Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
- Baruch Ivcher School of Psychiatry, Interdisciplinary Center Herzliya, Herzliya, Israel
| | - Amanda NeMoyer
- Disparities Research Unit, Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
9
|
|
10
|
Abstract
BACKGROUND The Veterans Health Administration (VA) has invested substantially in evidence-based mental health care. Yet no electronic performance measures for assessing the level at which the population of Veterans with depression receive appropriate care have proven robust enough to support rigorous evaluation of the VA's depression initiatives. OBJECTIVE Our objectives were to develop prototype longitudinal electronic population-based measures of depression care quality, validate the measures using expert panel judgment by VA and non-VA experts, and examine detection, follow-up and treatment rates over a decade (2000-2010). We describe our development methodology and the challenges to creating measures that capture the longitudinal course of clinical care from detection to treatment. DESIGN AND PARTICIPANTS Data come from the National Patient Care Database and Pharmacy Benefits Management Database for primary care patients from 1999 to 2011, from nine Veteran Integrated Service Networks. MEASURES We developed four population-based quality metrics for depression care that incorporate a 6-month look back and 1-year follow-up: detection of a new episode of depression, 84 and 180 day follow-up, and minimum appropriate treatment 1-year post detection. Expert panel techniques were used to evaluate the measure development methodology and results. Key challenges to creating valid longitudinal measures are discussed. KEY RESULTS Over the decade, the rates for detection of new episodes of depression remained stable at 7-8 %. Follow-up at 84 and 180 days were 37 % and 45 % in 2000 and increased to 56 % and 63 % by 2010. Minimum appropriate treatment remained relatively stable over the decade (82-84 %). CONCLUSIONS The development of valid longitudinal, population-based quality measures for depression care is a complex process with numerous challenges. If the full spectrum of care from detection to follow-up and treatment is not captured, performance measures could actually mask the clinical areas in need of quality improvement efforts.
Collapse
|