Routine outcome monitoring: An international endeavour

Recovery at 30: Integrating Lived Experience Expertise into Mental Health Research in Israel

A major component of recovery is the inclusion of lived experience to transform the culture of Mental Health (MH) services. In Israel lived experience has been increasingly integrated into services through peer roles. However, lived experience knowledge and expertise has not been sufficiently nor systematically integrated into the design of mental health research. This paper documents an attempt to initiate change by convening multiple stakeholders (with and without lived experience) in a specialized workshop aiming to learn and discuss the potential role of lived experience for mental health research in Israel. Participants raised ideas and core questions on how lived experience can shape research and augment mental health practices and policies. They highlighted current challenges regarding self-disclosure facing lived experience researchers, as well as challenges for developing participatory research collaborations among consumers, family members and practitioners. By bringing to the fore-front the 'insider perspective' of MH system as experienced among service users and families, we expect a development of a research culture with reduced paternalism, increased coproduction and recovery-orientation. We hope this endeavor will inspire others and help develop a lived experience expertise-based research network of interested stakeholders.

Effects and experiences of idiographic patient-reported outcome measures for feedback in psychotherapy: A systematic review and secondary analysis of the empirical literature

Objective: There is a growing interest in idiographic patient-reported outcome measures (I-PROMs) for routine outcome monitoring (ROM) and clinical feedback (CF) in psychotherapy, but to our knowledge, no systematic reviews of the empirical literature. Method: We conducted a systematic literature search for empirical papers investigating effects and experiences of I-PROMs for ROM/CF and found a total of 13 papers. There was only one experimental controlled effectiveness study. Results: We formulated a narrative summary of the data set as a whole. We conducted a secondary analysis of nine papers containing qualitative data on stakeholder experiences with I-PROMs and found three superordinate themes and eight subthemes, summarized as "I-PROMs can facilitate self-reflection for patients, assist in identifying and tracking therapeutic topics, and make patients more committed to therapy by giving them a greater sense of responsibility and empowerment. Formulating goals and problems can be difficult, and lack of progress can be demotivating. Time in therapy is precious and must be spent wisely. Effective use of I-PROMs is facilitated by flexibility and therapists' use of clinical skills." Conclusion: We discuss the limitations of the study and provide recommendations for future research and clinical practice.

The Patient-Perceived Helpfulness of Measures Scale: Development and Validation of a Scale to Assess the Helpfulness of Using Measures in Psychological Treatment

In response to the increase in Routine Outcome Monitoring and Clinical Feedback, the Patient-Perceived Helpfulness of Measures Scale (ppHMS) was developed to assess the helpfulness-as perceived by patients-of using measures in psychological treatment. Study 1: The construct of patient-perceived helpfulness of measures was explored using thematic analysis with 15 patients. Six helpful and three unhelpful themes were identified and informed item development. Study 2: 28 items were formulated and rated by experts. Ten items were taken forward for psychometric shortening in a sample of 76 patients. Confirmatory factor analysis (CFA) led to an adequately fitting six-item model with excellent internal consistency, and convergence with the Delighted-Terrible single item of product satisfaction and a single item of measure helpfulness. Study 3: In a stratified online sample of 514 U.K. psychotherapy patients, a five-item model constituted the best fit. The final ppHMS had excellent internal consistency (McDonald's ω = .90), convergent validity with psychotherapy satisfaction (r = .5; p < .001), divergence from social desirability (r = .1), and metric and scalar invariance across measures. Study 4: Analyses were replicated and confirmed in a stratified U.S. sample (n = 602). The ppHMS is a reliable and valid scale that can be used to assess and compare patients' perceptions of the helpfulness of different measures as part of their psychological treatment.

Examining Predictors of Psychological Distress Among Youth Engaging with Jigsaw for a Brief Intervention

Risk factors for psychological distress among help-seeking youth are poorly understood. Addressing this gap is important for informing mental health service provision. This study aimed to identify risk factors among youth attending Jigsaw, a youth mental health service in Ireland. Routine data were collected from N = 9,673 youth who engaged with Jigsaw (Mean age = 16.9 years, SD = 3.14), including presenting issues, levels of psychological distress, age, and gender. Confirmatory Factor Analysis identified thirteen factors of clustering issues. Several factors, including Self-criticism and Negative Thoughts, were strongly associated with items clustering as psychological distress, however these factors were poorly predictive of distress as measured by the CORE (YP-CORE: R2 = 14.7%, CORE-10: R2 = 6.9%). The findings provide insight into associations between young people's identified presenting issues and self-identified distress. Implications include applying appropriate therapeutic modalities to focus on risk factors and informing routine outcome measurement in integrated youth mental health services.

Feasibility of Recovery Assessment Scale - Domains and Stages (RAS-DS) for everyday mental health practice

Introduction

Routine use of self-rated measures of mental health recovery can support recovery-oriented practice. However, to be widely adopted, outcome measures must be feasible. This study examined the feasibility of Recovery Assessment Scale - Domains and Stages (RAS-DS) from the perspectives of mental health workers.

Method

Mental health workers who had previously sought permission to use RAS-DS (n=58) completed an online survey that explored three aspects of feasibility: practicality, acceptability and applicability.

Results

The highest-rated feasibility items related to applicability, or usefulness in practice, with over 90% of participants reporting that RAS-DS helps "promote discussion" and covers areas that are "meaningful to consumers". Acceptability items indicated that the purpose of RAS-DS is clear but length was an issue for some participants. At a practical level, RAS-DS was seen as easy to access but training was seen by many as necessary to ensure optimal use.

Conclusion

Results suggest potential usefulness of RAS-DS as a routine outcome measure and identify aspects that can be addressed to further enhance feasibility including provision of training materials and opportunities, wide-reaching promotion of its use as a collaborative tool, and further investigation of issues around instrument length.

Preferred outcome measures in treatments for challenging behaviour in individuals with intellectual disabilities: Results of an inclusive Delphi method

BACKGROUND

Interventions for challenging behaviours in individuals with intellectual disabilities benefit from outcome monitoring that takes clients' preferences into account. We determined clients' and representatives' preferred outcome domains and measures to secure their involvement in treatment decisions for challenging behaviours.

METHOD

We used an inclusive Delphi method. A focus group of individuals with mild intellectual disabilities and representatives of those with moderate and severe intellectual disabilities prepared the first round by assisting us in collecting possible outcomes. Panels of individuals with intellectual disabilities and representatives were composed to achieve consensus on instruments for preferred outcome domains.

RESULTS

Preferred outcome domains were behaviour, side-effects of psychotropic drugs, quality of life, daily functioning, caregiver burden and family quality of life. Corresponding outcome measures included self-report, interview and proxy-scales, including spoken versions.

CONCLUSION

Including the preferred domains on outcomes of interventions for challenging behaviours is recommended. Research on corresponding outcome measures is necessary.

Mental health outcome measures in the Australian context: what is the problem represented to be?

BACKGROUND

There is growing interest in the use of routine outcome measures (ROM) in mental health services worldwide. Australia has been at the forefront of introducing ROM in public mental health services, with the aim of improving services and consumer outcomes.

METHODS

An in-depth policy and document analysis was conducted using Carol Bacchi's 'What is the problem represented to be?' approach to critically analyse the use of ROM. This approach was used to identify and analyse the problem representations relating to the need for, and the choice of, outcome measures in Australian public mental health services, and the potential consequences of policy and practice. Data included in the analysis were seven policy documents, four reports on the introduction of outcome measures in Australia, the Australian Mental Health Outcomes and Classifications Network website, and the content of the outcome measures themselves.

RESULTS

Two dominant representations of the 'problem' were identified: 1) the 'problem' of mental health service quality and accountability, relating to the need for mental health outcome measures; and 2) the 'problem' of addressing deficits in biopsychosocial functioning of mental health consumers, which relates to the choice of outcome measures. Framing the 'problem' of mental health outcomes in these ways locates the problem within individual health providers, services, and consumers, ignoring the broader socioeconomic conditions underpinning mental health and effective service provision.

CONCLUSIONS

This critical analysis of the introduction and use of ROM in public mental health services in Australia highlights the need to consider the role of the social determinants of mental health, mental health service funding, and recovery-oriented care in ensuring services are meeting consumer needs and expectations. Broader governmental engagement is central to genuine change and opportunities.

A digital patient-reported outcome (electronic patient-reported outcome) system for patients with severe psychiatric disorders: User-centered development study and study protocol of a multicenter-controlled trial

Background

The effective treatment of patients with severe psychiatric disorders primarily relies on subjective reporting of symptoms and side-effects. This information is crucial for a clinician's decision regarding medication adjustment. Treatment adjustment usually happens at a low frequency (∼4-8 weeks). In between points of care, patients are left alone with their symptoms and side-effects. This leads to uncertainty regarding the treatment, non-adherence, possible relapse, and rehospitalization.

Objectives

We aim to design a flexible electronic patient-reported outcome (ePRO) system, which allows patients with severe psychiatric disorders to: (a) record their symptoms using an app; (b) share the data with the clinical team at points of care; and (c) utilize the data to support therapy decisions.

Methods

In this article, we describe the development process which included the following steps: (a) formation of a co-design team; (b) stakeholder interviews with patients, practitioners, and digital health experts to access needs, requirements, and barriers; (c) prototype conceptualization and design; (d) user acceptance testing and refinement; and (e) finalization of the system for testing in a pilottrial.

Results

We included input from patients with lived experience of psychiatric disorders, clinical team members, software engineers, and researchers. A prototype system was refined, and iterative changes were made before finalization during a series of operational meetings. The system allows patients to digitally self-report their symptoms and provides longitudinal ePRO symptom data for export into the electronic health record.

Conclusions

Routine ePRO collection has the potential to improve outcomes and hereby also reduce health service costs. We have successfully developed a trial-ready ePRO system for severe psychiatric disorders. The findings were incorporated in the planning of a feasibility pilot trial. Assuming feasibility will be established, the system might be subjected to a certification process evaluation of safety and efficacy including a randomized controlled trial.

Shared decision-making interventions for people with mental health conditions

BACKGROUND

One person in every four will suffer from a diagnosable mental health condition during their life. Such conditions can have a devastating impact on the lives of the individual and their family, as well as society.  International healthcare policy makers have increasingly advocated and enshrined partnership models of mental health care. Shared decision-making (SDM) is one such partnership approach. Shared decision-making is a form of service user-provider communication where both parties are acknowledged to bring expertise to the process and work in partnership to make a decision.  This review assesses whether SDM interventions improve a range of outcomes. This is the first update of this Cochrane Review, first published in 2010.

OBJECTIVES

To assess the effects of SDM interventions for people of all ages with mental health conditions, directed at people with mental health conditions, carers, or healthcare professionals, on a range of outcomes including: clinical outcomes, participation/involvement in decision-making process (observations on the process of SDM; user-reported, SDM-specific outcomes of encounters), recovery, satisfaction, knowledge, treatment/medication continuation, health service outcomes, and adverse outcomes.

SEARCH METHODS

We ran searches in January 2020 in CENTRAL, MEDLINE, Embase, and PsycINFO (2009 to January 2020). We also searched trial registers and the bibliographies of relevant papers, and contacted authors of included studies.  We updated the searches in February 2022. When we identified studies as potentially relevant, we labelled these as studies awaiting classification.

SELECTION CRITERIA

Randomised controlled trials (RCTs), including cluster-randomised controlled trials, of SDM interventions in people with mental health conditions (by Diagnostic and Statistical Manual of Mental Disorders (DSM) or International Classification of Diseases (ICD) criteria).

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures expected by Cochrane. Two review authors independently screened citations for inclusion, extracted data, and assessed risk of bias. We used GRADE to assess the certainty of the evidence.

MAIN RESULTS

This updated review included 13 new studies, for a total of 15 RCTs. Most participants were adults with severe mental illnesses such as schizophrenia, depression, and bipolar disorder, in higher-income countries. None of the studies included children or adolescents.  Primary outcomes We are uncertain whether SDM interventions improve clinical outcomes, such as psychiatric symptoms, depression, anxiety, and readmission, compared with control due to very low-certainty evidence. For readmission, we conducted subgroup analysis between studies that used usual care and those that used cognitive training in the control group. There were no subgroup differences. Regarding participation (by the person with the mental health condition) or level of involvement in the decision-making process, we are uncertain if SDM interventions improve observations on the process of SDM compared with no intervention due to very low-certainty evidence. On the other hand, SDM interventions may improve SDM-specific user-reported outcomes from encounters immediately after intervention compared with no intervention (standardised mean difference (SMD) 0.63, 95% confidence interval (CI) 0.26 to 1.01; 3 studies, 534 participants; low-certainty evidence). However, there was insufficient evidence for sustained participation or involvement in the decision-making processes. Secondary outcomes We are uncertain whether SDM interventions improve recovery compared with no intervention due to very low-certainty evidence. We are uncertain if SDM interventions improve users' overall satisfaction. However, one study (241 participants) showed that SDM interventions probably improve some aspects of users' satisfaction with received information compared with no intervention: information given was rated as helpful (risk ratio (RR) 1.33, 95% CI 1.08 to 1.65); participants expressed a strong desire to receive information this way for other treatment decisions (RR 1.35, 95% CI 1.08 to 1.68); and strongly recommended the information be shared with others in this way (RR 1.32, 95% CI 1.11 to 1.58). The evidence was of moderate certainty for these outcomes. However, this same study reported there may be little or no effect on amount or clarity of information, while another small study reported there may be little or no change in carer satisfaction with the SDM intervention. The effects of healthcare professional satisfaction were mixed: SDM interventions may have little or no effect on healthcare professional satisfaction when measured continuously, but probably improve healthcare professional satisfaction when assessed categorically. We are uncertain whether SDM interventions improve knowledge, treatment continuation assessed through clinic visits, medication continuation, carer participation, and the relationship between users and healthcare professionals because of very low-certainty evidence. Regarding length of consultation, SDM interventions probably have little or no effect compared with no intervention (SDM 0.09, 95% CI -0.24 to 0.41; 2 studies, 282 participants; moderate-certainty evidence). On the other hand, we are uncertain whether SDM interventions improve length of hospital stay due to very low-certainty evidence. There were no adverse effects on health outcomes and no other adverse events reported.

AUTHORS' CONCLUSIONS

This review update suggests that people exposed to SDM interventions may perceive greater levels of involvement immediately after an encounter compared with those in control groups. Moreover, SDM interventions probably have little or no effect on the length of consultations.  Overall we found that most evidence was of low or very low certainty, meaning there is a generally low level of certainty about the effects of SDM interventions based on the studies assembled thus far. There is a need for further research in this area.

A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers

PURPOSE

To review and integrate the literature on mental-health-related patient-reported outcome measures (PROMs) and routine outcome measures (ROMs), namely in the domains of goals, characteristics, implementation, settings, measurements and barriers. PROM/ROM aims mainly to ascertain treatment impact in routine clinical practice through systematic service users' health assessment using standardized self-report, caretaker and/or provider assessment.

DATA SOURCES

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science's Direct.

STUDY SELECTION

Systemized review of literature (2000-2018) on implementation and sustainability of PROMs/ROMs in adult mental health settings (MHS).

DATA EXTRACTION AND SYNTHESIS

Systemized review of literature (2000-2018) on numerous aspects of PROM/ROM implementation and sustainability in adult MHS worldwide.

RESULTS

Based on 103 articles, PROMs/ROMs were implemented mostly in outpatient settings for people with assorted mental health disorders receiving a diversity of services. Frequency of assessments and completion rates varied: one-third of projects had provider assessments; about half had both provider and self-assessments. Barriers to implementation: perceptions that PROM/ROM is intrusive to clinical practice, lack of infrastructure, fear that results may be used for cost containment and service eligibility instead of service quality improvement, difficulties with measures, ethical and confidentiality regulations and web security data management regulations.

CONCLUSION

Improving data input systems, sufficient training, regular feedback, measures to increase administrative and logistic support to improve implementation, acceptability, feasibility and sustainability, follow-up assessments and client attrition rate reduction efforts are only some measures needed to enhance PROM/ROM efficiency and efficacy.

Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences

ABSTRACT Purpose: Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming prevalent in mental health services, but there are several challenges to successful implementation. ROM/CFS seem to be helpful for some patients, but not for others. To investigate this, we explored patients' experiences with ROM/CFS as an interpersonal and psychotherapeutic process, in naturalistic settings. Method: We used video-assisted interpersonal process recall interviews to investigate the experiences of 12 patients using ROM/CFS in a Norwegian mental health outpatient clinic. Data were analyzed through systematic text condensation. Results: Our analysis resulted in three pairs of experiences with ROM/CFS: (1) Explicit vs. implicit use of information, (2) Directing focus towards or away from therapeutic topics, and (3) Giving and receiving feedback. These experiences could be helpful or hindering, depending on participants' needs and preferences. All participants needed to know that the CFS was used in a meaningful way. If not, it could be detrimental to the therapeutic process. Conclusion: In order to be helpful for patients, ROM/CFS should be used in a way that is flexible, meaningful to patients, and sensitive to individual needs and preferences. Future research should further explore this how-to aspect of ROM/CFS with different CFS and populations.

Complexity and potentials of clinical feedback in mental health: an in-depth study of patient processes

Purpose

Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming increasingly prevalent in mental health services. Their overall efficacy is unclear, but quantitative evidence suggests they can be useful tools for preventing treatment failure and enhancing therapeutic outcomes, especially for patients who are not progressing in therapy. The body of qualitative material, however, is smaller and less refined. We need to know more about how ROM/CFS is used in psychotherapy, and why it is helpful for some patients, but not others.

Methods

We recorded therapy sessions of 12 patients who were using a CFS as part of their therapies at an outpatient clinic in Norway. We then conducted video-assisted interviews and follow-up interviews with patients. Data were analyzed with systematic text condensation.

Results

Analysis revealed three themes: (1) triggering reflections, emotions, and self-awareness, (2) Ambivalent and ambiguous self-presentation, and (3) potential for feeling understood and talking about what matters.

Conclusion

Answering questions in a CFS is an interpretative and intentional process of self-presentation and the results from ROM/CFS must be interpreted and explored in conversation to be clinically useful. When they are, they have potential for enhancing the therapeutic process by stimulating self-awareness, reflexivity, and allowing access to new therapeutic topics. Further research should explore this how-to aspect of ROM/CFS with different CFS and different types of patients. Integrating clinical feedback in therapeutic practice can be conceptualized as a clinical skill, which should be a part of training programs for therapists.

Electronic supplementary material

The online version of this article (10.1007/s11136-020-02550-1) contains supplementary material, which is available to authorized users.

Redefining Therapeutic Outcomes of Depression Treatment

Responses to evidence-based interventions for depression are divergent: Some patients benefit more than others during treatment and some do not benefit at all or even deteriorate. Tailoring interventions to the individual may improve outcomes. However, such personalization of evidence-based treatment in depression requires investigation of individual outcomes and the individual trajectories towards these outcomes. This theoretical paper provides a critical reflection on individual outcomes of depression treatment. First, it is argued that outcomes should be broadened, from a focus on mainly depressive symptomatology to recovery in different domains. It is acknowledged that recovery from depression reflects a personal journey that differs from person to person. Second, outcome measures should be lengthened beyond the acute treatment phase, taking a lifetime perspective on depression. The challenge then is to discover which trajectories of what measures during what interventions result in personalized sustainable recovery and for whom. Routine outcome monitoring systems may be used to inform this quest towards assessment of personalized sustainable therapeutic outcomes. Adaptations to broaden and lengthen measurements in routine outcome monitoring systems are proposed to identify predictors of personalized sustainable recovery. Routine outcome monitoring systems may eventually be used to implement personalized treatments for depression that result in personalized sustainable recovery.

Patient-reported outcome measurements (PROMs) and provider assessment in mental health: a systematic review of the context of implementation

Purpose

To review and integrate the vast amount of literature yielded by recent growing interest in patient-reported outcome measurement and routine outcome measures (PROMs/ROMs), in order to suggest options and improvements for implementation. PROMs are the systematic assessment of service users’ health using standardized self-report measures. Specifically, for ROMs, it includes routine provider or caretaker assessment measures. Both are administered to ascertain routinely, the impact of treatment in mental health settings and to improve care. A review is needed because of the large differences in setting, conceptualization, practice and implementation. Here, we examine the different major projects worldwide.

Data sources

Psych INFO and PubMed including Medline, Biomed Central, EMBASE Psychiatry and Elsevier Science’s Direct.

Study selection

We conducted a systematized review of the literature published from 2000 to 2018 on the implementation and sustainability of PROMs and ROMs in mental health services for adults.

Data extraction, synthesis and Results

We described and characterized the programs in different countries worldwide. We identified 103 articles that met the inclusion criteria, representing over 80 PROMs/ROMs initiatives in 15 countries. National policy and structure of mental health services were found to be major factors in implementation. We discuss the great variability in PROMs/ROMs models in different countries, making suggestions for their streamlining and improvement.

Conclusion

We extracted valuable information on the different characteristics of the numerous PROMs/ROMs initiatives worldwide. However, in the absence of a strong nationwide policy effort and support, implementation seems scattered and irregular. Thus, development of the implementation of PROMs/ROMs is left to groups of enthusiastic clinicians and researchers, making sustainability problematic.

Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients

OBJECTIVES

If patients change their perspective due to treatment, this may alter the way they conceptualize, prioritize, or calibrate questionnaire items. These psychological changes, also called "response shifts," may pose a threat to the measurement of therapeutic change in patients. Therefore, it is important to test the occurrence of response shift in patients across their treatment.

METHODS

This study focused on self-reported psychological distress/psychopathology in a naturalistic sample of 206 psychiatric outpatients. Longitudinal measurement invariance tests were computed across treatment in order to detect response shifts.

RESULTS

Compared with before treatment, post-treatment psychopathology scores showed an increase in model fit and factor loading, suggesting that symptoms became more coherently interrelated within their psychopathology domains. Reconceptualization (depression/mood) and reprioritization (somatic and cognitive problems) response shift types were found in several items. We found no recalibration response shift.

CONCLUSION

This study provides further evidence that response shift can occur in adult psychiatric patients across their mental health treatment. Future research is needed to determine whether response shift implies an unwanted potential bias in treatment evaluation or a desired cognitive change intended by treatment.

Trajectories of patients with severe mental illness in two-year contact with Flexible Assertive Community Treatment teams using Routine Outcome Monitoring data: An observational study

OBJECTIVE

Using outcome data collected routinely over a continuous two-year treatment period, we wished to distinguish homogeneous subgroups of patients with a severe mental illness whose psychosocial problems followed a similar pattern over time. By identifying the effectiveness of health services for different patient groups, this approach allowed us to identify patients at risk of deterioration and those recovering from their symptoms.

METHODS

In total we included 2,660 patients who were in two-year continuous contact with a Flexible Assertive Community Treatment team (FACT). We collected outcome data on psychosocial functioning, needs for care and quality of life. We performed a latent class growth analysis (LCGA).

RESULTS

The LCGA identified six homogenous patient subgroups using trajectories of HoNOS scores. On the basis of the patterns of patients' psychosocial problems over time, we labelled these as follows: 1) stable at a low problem-severity level (N = 709; 27%); 2) stable at a low medium problem-severity level (N = 1,208; 45%); 3) stable at a high medium problem-severity level (N = 528; 20%); 4) stable at a high problem-severity level (N = 116; 4%); 5) amelioration of problems (N = 42; 2%); and 6) deterioration of problems (N = 57; 2%). Patients with stable and a high severity of psychosocial problems had more practical and somatic unmet needs than those in other subgroups, and also had the fewest decrease in the number of unmet needs.

DISCUSSION

After linking patient subgroups with clinical features such as the need for care, we found that, over two years, most patients remained relatively stable in terms of psychosocial functioning, but that their unmet needs decreased over time. However, in terms of needs for treatment during two years of contact with a FACT team, patients in the subgroup with a stable and high problem-severity level tended to derive little or no benefit.

Behavior Problems Inventory (BPI): Psychometric characteristics of an instrument for routine assessment of persons with psychoses and related disorders

This study explores the psychometric characteristics of the Behavior Problems Inventory (BPI), an instrument for routine clinical assessment of behavior problems in outpatients with psychosis based on information provided by key family informants. Six hundred and twenty-one patients diagnosed with psychosis and bipolar affective disorder (ICD-10 F20-F31) attended at Community Mental Health Units were evaluated in routine reviews using the BPI and the Social Functioning Scale (SFS). Twenty-five subjects were simultaneously administered the Social Behavior Schedule (SBS) and 28 were again administered the BPI eight weeks later. The instrument shows adequate psychometric characteristics with high internal consistency and robust temporal reliability, as well as satisfactory concurrent and construct validity. Factor analysis identified three factors: Underactivity/Social Withdrawal, Active Problems and Lack of Impulse Control, with adequate saturation of the items on each of the factors. The BPI is easy to apply, reliable and valid, takes up little of valuable clinical time, allowing routine assessment in public service contexts for persons diagnosed with psychosis and bipolar affective disorder for whom key family informants are available.

Personal recovery within positive psychiatry

BACKGROUND

One goal within positive psychiatry is to support the personal recovery of persons with mental illness and providing opportunities for well-being.

AIM

The current article aims to introduce readers to the concept of personal recovery and the potential and importance of recovery-oriented services and measures.

METHODS

A literature review was conducted to help consider the domains of 'personal recovery', 'recovery-oriented services/interventions', and 'measures'. A database search was complemented with a web-based search. Both medical subject heading (MESH) terms and free-text search terms were used.

RESULTS

Literature from research journals, grey literature, and websites were included. Within this context, recovery does not refer to a cure but involves a process in which a person acts as an agent to develop new goals and meaning in life, despite and beyond limitations posed by the illness and its consequences. A positive focus on recovery is in sharp contrast to historical deterministic and pessimistic concepts of mental illnesses. Recovery-oriented services such as peer support, assertive community treatment, supported employment/education/housing, illness self-management, and decreasing self-stigma are highlighted. A review of 27 measures that focus on personal recovery and promotion of well-being are also discussed.

CONCLUSIONS

The literature overview presents perspectives and knowledge of how to develop positive psychiatry, how mental health services and their partner organizations may become more recovery oriented and help persons reach well-being and a better quality of life. This study is limited to a narrative review and may precede future systematic reviews.

Development and validation of MyLifeTracker: a routine outcome measure for youth mental health

PURPOSE

Routine outcome measures are now being designed for session-by-session use, with emphasis on clinically meaningful items and sensitivity to change. Despite an increasing mental health service focus for young people aged 12-25 years, there is a lack of outcome measures that are designed to be used across this age group. Consequently, MyLifeTracker (MLT) was developed as a brief mental health outcome measure designed for young people for routine use. It consists of the following five items targeting areas of importance to young people: general well-being, day-to-day activities, relationships with friends, relationships with family, and general coping.

PARTICIPANTS AND METHODS

The measure was tested with 75,893 young people aged 12-25 years attending headspace centers across Australia for mental health-related issues.

RESULTS

MLT showed a robust unidimensional factor structure and appropriate reliability. It exhibited good concurrent validity against well-validated measures of psychological distress, well-being, functioning, and life satisfaction. The measure was further demonstrated to be sensitive to change.

CONCLUSION

MLT provides a psychometrically sound mental health outcome measure for young people. The measure taps into items that are meaningful to young people and provides an additional clinical support tool for clinicians and clients during therapy. The measure is brief and easy to use and has been incorporated into an electronic system that routinely tracks session-by-session change and produces time-series charts for the ease of use and interpretation.

Provider-Associated Measurement Error in Routine Outcome Monitoring in Community Mental Health

Despite the strengths of routine outcome monitoring (ROM) in community mental health settings, there are a number of barriers to effective implementation of ROM, including measurement error due to provider factors (e.g., training level) and non-target client factors (i.e., client characteristics which have no meaningful relationship to the outcome of interest). In this study, ROM data from 80 client-provider dyads were examined for sources of variance due to provider factors and non-target client factors. Results indicated that provider factors and non-target client factors accounted for between 9.6 and 54% of the variance in the ROM measures. Our findings supported past research that provider characteristics impact ROM, and added the novel finding that client gender, age, diagnosis, and cognition also impact ROM. Methods to increase accuracy and utility of ROM in community mental health are discussed.

Psychosocial Services for Individuals With Serious Mental Illness/Severe Emotional Disturbance: Clinical Practice Guideline Toolkit

This article presents a short, easy-to-use guideline for clinicians working with people with serious mental illness, particularly those with schizophrenia, schizoaffective disorder, delusional disorders, other psychotic disorders, or depression with significant impact on functioning, and may be of use for those working with individuals with bipolar disorder. This guideline is not meant to supplant the more detailed, excellent clinical practice guidelines that have been developed and published. Rather, the charts and explanation presented here are designed as a step-by-step tool that clinicians can use to ensure they are following the most appropriate course of action and providing the best psychosocial rehabilitation services possible for persons with these disorders. Program managers can use this tool to evaluate the processes used in their system to ensure that appropriate services are offered for the individuals in their care. This guideline is also not meant as a substitute for comprehensive training in the practice of assessment and treatment for individuals with serious mental illness. Very little detail is provided about the disorders, appropriate assessments, and interventions because clinicians providing services to people with these disorders should have received the specialized training necessary to appropriately deliver the services needed by these individuals.

Demonstrating the reliability of transdiagnostic mHealth Routine Outcome Monitoring in mental health services using experience sampling technology

Background

Routine Outcome Monitoring (ROM) should provide a dynamic, within-treatment forward feedback loop to guide individual treatment decisions across diagnostic categories. It has been suggested that the Experience Sampling Method (ESM), capturing the film of daily life adaptive processes, offers a flexible, personalised and transdiagnostic feedback system for monitoring and adapting treatment strategies. This is the first study that uses an ESM application (the PsyMate™) as a routine mobile-ROM (mROM) tool in an ambulatory mental health setting.

Objective

To demonstrate adequate psychometric properties of the PsyMate™ app assessing both symptom severity levels as well as daily life functioning.

Method

In a transdiagnostic sample of 64 outpatients, an mROM protocol (ESM for 6 days, at 10 semi-random moments a day) and a standard ROM instrument (HADS) were administered at baseline and at three-month follow-up. We measured positive affect (PA), negative affect (NA), quality of sleep, positive social interaction, activity-related stress, tiredness, and feeling unwell.

Results

Subjects completed 53% of the measurements at baseline (N = 64) and 48% at follow-up (N = 29). Factor analysis and subsequent reliability analysis of PA and NA confirmed the two constructs. Significant and meaningful correlations were found between PA, NA and HADS scores (ranging from r = .4 to r = .7). Multilevel analyses yielded significant change scores for all measures.

Conclusion

The ESM-based, transdiagnostic mROM tool can be used reliably in clinical settings: it shows adequate psychometric properties, as well as concurrent validity and sensitivity to change over time with respect to relevant ROM constructs. Person-tailored items can be added. In addition, mROM offers added value over standard symptom-based ROM, as it provides information on adaptive functioning in the daily environment of patients.

Patients' experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature

Routine outcome monitoring (ROM) and clinical feedback (CF) systems have become important tools for psychological therapies, but there are challenges for their successful implementation.

OBJECTIVE

To overcome these challenges, a greater understanding is needed about how patients experience the use of ROM/CF.

METHOD

We conducted a systematic literature search of qualitative studies on patient experiences with the use of ROM/CF in mental health services.

RESULTS

The findings from 16 studies were synthesized, resulting in four meta-themes: (1) Suspicion towards service providers, (2) Flexibility and support to capture complexity, (3) Empowering patients, and (4) Developing collaborative practice.

CONCLUSIONS

We discuss the implications of these meta-themes for further development and implementation of ROM/CF into clinical practice, acknowledging the limitations of our review and suggesting avenues for further research. Clinical or methodological significance of this article: This article provides useful and actionable knowledge about the patient perspective on ROM/CF, an important discussion on the current state of research in this area, and useful and concrete suggestions for further avenues of research.

Using Routine Outcome Measures to Provide Feedback at the Service Agency Level

The use of routine outcome measurement (ROM) to assess service effectiveness has been on the rise in mental health settings. However, there is a scarcity of information on the use of ROM data to provide feedback to teams of service providers. In this paper we review the existing literature to identify the principles that can guide the use of ROM data as feedback with the aim to improve quality of service provision in mental health settings. We present a pilot trial of 12 agencies participating in group feedback sessions. The guiding principles and core processes, the procedure and implementation in a pilot trial, lessons learned and future directions are discussed. Based on this experience we conclude that using ROM to implement group feedback among mental health stakeholders is feasible and can generate discussions and directions for improvement.

Health Information Technologies-Academic and Commercial Evaluation (HIT-ACE) methodology: description and application to clinical feedback systems

Background

Health information technologies (HIT) have become nearly ubiquitous in the contemporary healthcare landscape, but information about HIT development, functionality, and implementation readiness is frequently siloed. Theory-driven methods of compiling, evaluating, and integrating information from the academic and commercial sectors are necessary to guide stakeholder decision-making surrounding HIT adoption and to develop pragmatic HIT research agendas. This article presents the Health Information Technologies—Academic and Commercial Evaluation (HIT-ACE) methodology, a structured, theory-driven method for compiling and evaluating information from multiple sectors. As an example demonstration of the methodology, we apply HIT-ACE to mental and behavioral health measurement feedback systems (MFS). MFS are a specific class of HIT that support the implementation of routine outcome monitoring, an evidence-based practice.

Results

HIT-ACE is guided by theories and frameworks related to user-centered design and implementation science. The methodology involves four phases: (1) coding academic and commercial materials, (2) developer/purveyor interviews, (3) linking putative implementation mechanisms to hit capabilities, and (4) experimental testing of capabilities and mechanisms. In the current demonstration, phase 1 included a systematic process to identify MFS in mental and behavioral health using academic literature and commercial websites. Using user-centered design, implementation science, and feedback frameworks, the HIT-ACE coding system was developed, piloted, and used to review each identified system for the presence of 38 capabilities and 18 additional characteristics via a consensus coding process. Bibliometic data were also collected to examine the representation of the systems in the scientific literature. As an example, results are presented for the application of HIT-ACE phase 1 to MFS wherein 49 separate MFS were identified, reflecting a diverse array of characteristics and capabilities.

Conclusions

Preliminary findings demonstrate the utility of HIT-ACE to represent the scope and diversity of a given class of HIT beyond what can be identified in the academic literature. Phase 2 data collection is expected to confirm and expand the information presented and phases 3 and 4 will provide more nuanced information about the impact of specific HIT capabilities. In all, HIT-ACE is expected to support adoption decisions and additional HIT development and implementation research.

Planning Treatment and Assessing Recovery in Participants With Dual Diagnosis: Preliminary Evaluation of a New Clinical Tool

OBJECTIVE

Integrated treatment for people with co-occurring mental illness and substance use disorder would be enhanced by a simple, recovery-oriented instrument to plan treatment and monitor progress toward dual recovery. This paper describes the development of a clinical instrument, the WestBridge Dual Recovery Inventory, and presents a preliminary evaluation of its usability.

METHODS

In collaboration with participants in treatment, family members, clinicians, and program leaders, we developed an inventory on dual recovery and then examined its utility through a series of iterative steps. First, we tested the inventory for inter-rater reliability among 10 program participants (rated independently by the first and last author). Second, we examined concordance by having a separate group of 10 program participants and their care managers complete the inventory independently. Third, we observed 3 care managers and 10 participants working together to complete the inventory as part of routine care during the quarterly assessment; we evaluated shared decision making based on the process they used to resolve differences and on a brief survey completed by program participants as the end of the session. Finally, to measure the inventory's capacity to detect change over time, we analyzed the ratings from admission to one year for 43 participants with quarterly assessments available for that time period.

RESULTS

The WestBridge Dual Recovery Inventory assesses 14 domains of recovery, each rated on 5-point scales. Inter-rater reliability was high (Kappa = .82 to 1.00); agreement between independent ratings by care managers and participants varied considerably (Kendall's tau = -.83 to +.87); and collaborative ratings demonstrated high scores on shared decision making. Participants improved significantly on 11 of 14 domains during the initial residential treatment phase (admission to six months) and sustained gains during outpatient assertive community treatment (6 to 12 months).

CONCLUSIONS

This preliminary assessment of the WestBridge Dual Recovery Inventory suggests that it reliably assesses dual recovery, facilitates shared decision making, and captures changes over time. The inventory appears to be usable, well received by participants and care managers, specific for program goals, and sensitive to changes in the participants. Recovery measures may need to be program-specific.