A model for developing outcome measures from the perspectives of mental health service users

Reliability and validity of the Arabic version of the modified falls efficacy scale

PURPOSE

The purpose of this study was to adapt Modified Falls Efficacy Scale (MFES) into Arabic and determine the reliability and validity of the instrument.

MATERIALS AND METHODS

The study was conducted in two phases: (i) translation and adaptation by the systematic approach of the 'forward-back' translation method and (ii) psychometric testing of the Arabic version of the Modified Falls Efficacy Scale among 207 community-dwelling older adults (≥ 60 years).

RESULTS

The Arabic version of the Modified Falls Efficacy Scale demonstrated excellent internal consistency (Cronbach's alpha = 0.98) and test-retest reliability scores (ICC = 0.96, 95% CI; 0.95-0.97). And also showed strong correlations with both the Falls Efficacy International (r = -0.82) and the activities-specific Balance Confidence Scale (r = 0.87). Sampling adequacy for factor analysis was proven by a Kaiser-Meyer-Olkin value of 0.962. Goodness-of-fit (GFI) statistics for the model were in the acceptable range (Chi-Square/Degree of Freedom (CMIN/DF) = 2.59, Goodness-of-fit index (GFI) = 0.9, Comparative Fit Index (CFI) = 0.97, Root Mean Square Error of Approximation (RMSEA) = 0.79).

CONCLUSION

The Arabic version of the Modified Falls Efficacy Scale has demonstrated excellent psychometric qualities to measure the level of fear of falling.

Mental health and access to care among the Roma population in Europe: A scoping review

The Roma are Europe's largest ethnic minority group, and often face discrimination and social exclusion. Social strife and lack of access to healthcare are associated with increased symptoms of psychopathology. We aimed to review evidence on mental health outcomes and on access to mental healthcare among the Roma population in Europe. We systematically searched five databases (PsycINFO, Global Health, Social Policy and Practice, Web of Science and PubMed) and conducted a grey literature search in August 2020. We identified 133 studies, 26 of which were included for final analysis. We present the results using a narrative synthesis. The available literature indicates a relatively high prevalence of anxiety, depression and substance abuse among Roma, and females seem to be more affected than males. Roma children exhibit more externalizing and internalizing disorders when compared with non-Roma children. Mental health and perceived well-being among the Roma population are strongly linked to social determinants of health such as housing or economic income. Access to mental healthcare is limited for Roma people because of several barriers pertaining to language, lack of information regarding available services, and the insurance and economic status of Roma people. Roma people report mainly negative experiences with mental health services, including a lack of understanding from healthcare providers, and instances of racism and discrimination. There is a need for more research on mental health and access to healthcare in Roma people. Future studies should be participatory in order to provide guidelines for mental healthcare that addresses the needs of the Roma population.

Understanding parent engagement in advanced allied health single session therapy for children with developmental and behavioural disabilities

BACKGROUND

Increasingly, health services are adopting short-term consultative therapy models, such as single session therapy, to cope with increased service demands, finite budgets and changing consumer expectations. Within the paediatric sub-specialty of child development and behaviour, allied health clinicians are central to supporting families to understand their children, provide therapeutic interventions and help achieve optimal developmental and health outcomes. This study aimed to describe parents'/caregivers' engagement during advanced allied health consultations utilizing a single session therapy framework for managing neurodevelopmental and behavioural concerns in children.

METHODS

Between March to June 2021, 14 parents participated in a qualitative study involving focus groups. Reflexive thematic analysis was used to code and interpret the data to understand and explore families' engagement in single session therapy.

RESULTS

Three overarching themes relating to parents' engagement during consultations emerged from the data: connection with the clinician, parental attributes and organizational influences. Clinician connection was the prominent construct that enhanced parent attributes and the positive perception of the organization in an ongoing manner. There was a complex interplay between the behavioural and affective components of the clinician, parents and the organization to create mutual presence, valued time, understanding and action.

CONCLUSIONS

This study demonstrates the importance of clinician connection, parental attributes and organizational influences to enhance engagement in therapeutic consultations. These factors are interrelated and are a complex blend of behaviour and internal states and highly individualized for each family. Understanding how services can continue to engage families is integral to ongoing sustainability and overall health and well-being.

Involvement of people who use alcohol and other drug services in the development of patient-reported measures of experience: A scoping review

INTRODUCTION

Patient-reported measures that assess satisfaction and experience are increasingly utilised in healthcare sectors, including the alcohol and other drug (AOD) sector. This scoping review identifies how and to what extent people accessing AOD services have been involved in the development of satisfaction and experience measures to date.

METHODS

PubMed, EMBASE, CINAHL, Scopus, ProQuest, Google and Google Scholar were searched. Included papers described the development and/or implementation of a multiple-item measure of patient-reported experience or satisfaction specifically for people accessing AOD treatment and/or harm reduction programmes. If there was more than one paper, key papers were chosen that described each measure. The method of development, including service user involvement, was assessed against a framework generated for this review. Two reviewers were involved at each stage.

RESULTS

Thirty measures-23 satisfaction and 7 experience-were identified. Sixteen measures reported some level of involvement by people accessing AOD services in their development, although, for most measures, at a relatively low level. This involvement increased over the time span of the review becoming more frequent in later years. Only four measures were developed for use in harm reduction-specific settings, and fewer than half reported undertaking analysis of underlying scale structure and constructs.

CONCLUSION

Several gaps could be addressed to enhance the measurement of patient-centred care in the AOD sector, including: developing experience measures for use in harm reduction settings and across various AOD settings in a service system; improved reporting of psychometric properties of these measures and increasing commitment to the meaningful involvement of AOD service users in measure development.

PATIENT OR PUBLIC CONTRIBUTION

This scoping review is part of a broader codesign project that involves a partnership between the peak organisation for AOD services and the peer-based AOD consumer organisation in the Australian Capital Territory, Australia. These organisations are working closely together to engage with AOD service users, service providers and policy makers in this codesign project. As such, the Executive Director of the peer-based AOD consumer organisation is involved as a co-author of this scoping review.

Development and Validation of the Lewy Body Disease Caregiver Activities Scale

Background and Purpose: Caring for someone with Lewy body disease (LBD) is difficult. This study describes the development and validity testing of the LBD Caregiver Activities Scale (LBD-CAS). Methods: Caregiver interviews informed the development of the LBD-CAS. Experts estimated the content validity of items (I-CVI) and provided feedback about the scale. Family caregivers evaluated items for face validity. Results: Expert I-CVI ratings yielded 49 items with scores of 0.83 or higher. Four items with I-CVI scores <.83 were retained due to conceptual significance. The overall scale CVI was 0.86. Items evaluated by caregivers for face validity showed excellent variability in responses, with no major ceiling or floor effects. Conclusions: LBD-CAS showed evidence of content and face validity for the assessment of activities performed by LBD caregivers. Further psychometric testing is recommended.

Satisfaction with cognitive remediation therapy: its effects on implementation and outcomes using the cognitive remediation satisfaction scale

Cognitive Remediation (CR) improves cognition and functioning but is implemented in a variety of ways (independent, group and one-to-one). There is no information on whether service users find these implementation methods acceptable or if their satisfaction influences CR outcomes. We used mixed participatory methods, including focus groups, to co-develop a CR satisfaction scale. This was refined using three psychometric criteria (Cronbach's alpha, item discrimination, test-retest agreement) to select items. Factor analysis explored potential substructures. The refined measure was used in structural equation joint modelling to evaluate whether satisfaction with CR is affected by implementation method and treatment engagement or influences recovery outcome, using data from a randomised controlled trial. Four themes (therapy hours, therapist, treatment effects, computer use) generated a 31-item Cognitive Remediation Satisfaction scale (CRS) that reduced to 18 Likert items, 2 binary and 2 open-ended questions following psychometric assessment. CRS had good internal consistency (Alpha = 0.814), test-retest reliability (r= 0.763), and concurrent validity using the Working Alliance Inventory (r = 0.56). A 2-factor solution divided items into therapy engagement and therapy effects. Satisfaction was not related to implementation method but was significantly associated with CR engagement. Therapy hours were significantly associated with recovery, but there was no direct effect of satisfaction on outcome. Although satisfaction is important to therapy engagement, it has no direct effect on outcome. CR therapy hours directly affect outcome irrespective of which implementation model is used, so measuring satisfaction early might help to identify those who are likely to disengage. The study has mixed methods design.

Development of an addiction recovery patient-reported outcome measure: Response to Addiction Recovery (R2AR)

BACKGROUND

Recovery, a primary goal of addiction treatment, goes beyond abstinence. Incorporating broad domains with key elements that vary across individuals, recovery is a difficult concept to measure. Most addiction-related quality measurement has emphasized process measures, which limits evaluation of treatment quality and long-term outcomes, whereas patient-reported outcomes are richer and nuanced. To address these gaps, this study developed and tested a patient-reported outcome measure for addiction recovery, named Response to Addiction Recovery (R2AR).

METHODS

A multi-stage mixed methods approach followed the Patient-Reported Outcomes Measurement Information System (PROMIS) measure development standard. People with lived experience (PWLE) of addiction, treatment providers, and other experts contributed to item distillation and iterative measure refinement. From an item bank of 356 unique items, 57 items were tested via survey and interviews, followed by focus groups and cognitive interviews.

RESULTS

Face validity was demonstrated throughout. PWLE rated item importance higher and with greater variance than providers, yet both agreed that "There are more important things to me in my life than using substances" was the most important item. The final R2AR instrument has 19 items across 8 recovery domains, spanning early, active, and long-term recovery phases. Respondents assess agreement for each item as (1) a strength, and (2) importance to ongoing recovery.

CONCLUSION

R2AR allows PWLE to define what is important to their recovery. It is designed to support treatment planning as part of clinical workflows and to track recovery progress. Inclusion of PWLE and providers in the development process enhances its face validity. Including PWLE in the development of R2AR and using the tool to guide recovery planning emphasizes the importance of patient-centeredness in designing clinical tools and involving patients in their own care.

[Experience-based items of quality in psychiatric treatment: A first multivariate construct]

In the context of psychiatric care, user-generated measurement instruments may contribute to quality development and assurance. An explorative construct of experience-related quality components was developed in participative-collaborative cooperation that grasps the users' experiences of psychiatric care. After developing the components using a grounded theory methodology, they were quantified, and their interrelations were investigated using a multidimensional scaling method to explore their internal cohesion. The construct makes it possible to separate structural from interpersonal requirements of the quality components. It further indicated which components are more feasible for the home treatment setting, and which ones for an institutional setting. The components and the construct may be perceived as first steps towards the development of user-generated quality indicators; however, further validation steps are necessary.

Experience of Inpatient Mental Health Care Assessed With Service User-Developed and Conventional Patient-Reported Outcome Measures

OBJECTIVE

The goal of this study was to examine and compare the psychometric properties of a patient-reported outcome measure (PROM) generated with patients’ input (Views on Inpatient Care [VOICE]) and a PROM conventionally generated without patients’ input (Service Satisfaction Scale: Residential Services Evaluation [SSS-Res]) for assessing a patient’s perception of psychiatric ward care.

METHODS

In a stepped-wedge cluster-randomized trial conducted in the United Kingdom, 1,058 participants admitted to 16 wards reported on their perceptions of care via VOICE and SSS-Res before or up to 2 years after the staff training. Exploratory and confirmatory factor analyses were used to investigate the structure of the PROMs and to assess reliability and convergent validity as well as sensitivity to change; the analyses also considered whether study participants had been admitted voluntarily to the ward.

RESULTS

Two factors emerged from VOICE, labeled “trust” and “involvement,” and from SSS-Res, labeled “environment” and “care,” at baseline. All subscales had high internal consistency and good convergent validity. An ability to detect change in care due to the staff training was observed on the trust subscale of VOICE (N=1,058, mean difference=−0.25, 95% CI=−0.48 to −0.02), but no change was detected on any of the SSS-Res subscales. Patients admitted involuntarily benefited the most from the staff training.

CONCLUSIONS

VOICE captured patients’ perceptions of ward care better than SSS-Res and was sensitive to changes in aspects of trust, suggesting that participatory approaches for developing PROMs improve patients’ self-reports on the care they received.

Survivor-led guidelines for conducting trauma-informed psychological therapy assessments: Development and modified Delphi study

BACKGROUND

Psychological therapy assessments are a key point at which a person is accepted into a service or referred on. There is evidence of service users experiencing harm, dropping out of services and potentially experiencing poor outcomes because of inadequate assessment practices. Approaches to assessment tend to be developed by individual services, with a lack of research identifying what makes a good assessment.

METHODS

This survivor-led study, based in England, aimed to generate guidelines for conducting trauma-informed psychological therapy assessments. The study was guided by a Service User Advisory Group and a Clinician Advisory Group. The study was conducted in three key stages: (i) identifying, modelling and drafting guideline content (ii) modified Delphi study and (iii) guideline finalization. Stage 1 was informed by literature reviews, qualitative research, data workshops with Advisory Groups and an expert consultation. Fifty-nine people with relevant experiences then participated in a single-stage modified Delphi (Stage 2). The guidelines were finalized through an analysis of Delphi open comments and a final expert consultation (Stage 3).

RESULTS

The guidelines evolved through each stage of the process, and all items were deemed important by >90% of Delphi participants. The final trauma-informed guidelines contain eight principles, including 'focus on relationships', 'from systems to people' and 'healing environments'.

CONCLUSIONS

Experiential knowledge was key in generating the guidelines and conceptualizing content, with a consequent focus on areas, such as recognizing power differentials, understanding oppression as trauma and the relational aspects of assessments. Future research should focus on guideline implementation and investigate whether this impacts service user dropout, engagement with therapy, and outcomes.

PATIENT OR PUBLIC CONTRIBUTION

This study is an example of survivor research, with several authors, including the study lead, identifying as survivors. We consider the ways in which our identities as survivor researchers impacted the study findings.

Assessing the Perception of Family and Caregivers' Experience with Mental Health and Substance Use Services

Standardized client feedback surveys encourage a culture of continuous quality improvement, allow for comparison of results over time and across similar types of service providers, and encourage use of evidence-based practices. Recognizing the importance of family and other caregivers in supporting people accessing services for mental health and substance use challenges, a standardized perception-of-care tool (the Ontario Perception of Care Tool for Mental Health and Addictions, OPOC-MHA) was adapted to collect feedback specific to the caregiver experience with these services. A collaborative process engaged a broad range of mental health and/or addiction providers, family advisory networks, and family members and caregivers to identify themes, specific items, and implementation approaches. The final version of the tool evolved through an iterative process of pilot testing and stakeholder feedback. Family member and caregiver perceptions of care will identify service areas in need of improvement, contribute to quality improvement initiatives, and facilitate the comparison of findings over time.

The role of patient and public involvement and engagement (PPIE) within the development of the EQ Health and Wellbeing (EQ-HWB)

Objectives

The value of patient and public involvement and engagement (PPIE) within the development and refinement of outcome measures is becoming increasingly recognized. The aim of this paper is to provide an overview of how PPIE was integrated within the development of a new measure designed for use in economic evaluations across health and social care, the EQ Health and Wellbeing (EQ-HWB™).

Methods

Four PPIE sessions were held at key stages. Discussions from each session and the outcome of any tasks were shared with the wider research team and used to help inform decision-making.

Results and discussion

PPIE covered several components of outcome measure development including; review of conceptual model; discussion on sub-domain inclusion; item refinement and reduction; pre-testing of items; selection of items for the measure; and design of the measure. Key learning points for future projects were highlighted including; consideration of practicalities, resources and logistics of PPIE activities; how sessions and activities are managed effectively; and how to managing expectations and communication from both researcher and PPIE perspectives.

Conclusions

The PPIE group provided invaluable insight into perspectives of future patients and carers. Their input was fed into a number of developmental stages. The formal involvement from the PPIE group meant that the voice of the general public was heard. This helped ensure the appropriateness of the design of the final measure.

Needs and Experiences in Psychiatric Treatment (NEPT)- Piloting a Collaboratively Generated, Initial Research Tool to Evaluate Cross-Sectoral Mental Health Services

BACKGROUND

Research tools to evaluate institutions or interventions in the field of mental health have rarely been constructed by researchers with personal experience of using the mental health system ("experiential expertise"). This paper presents a preliminary tool that has been developed within a participatory-collaborative process evaluation as part of a controlled, multi-center, prospective cohort study (PsychCare) to evaluate psychiatric flexible and integrative treatment, FIT for short, models in Germany.

METHOD

The collaborative research team consisting of researchers with and without experiential expertise developed 12 experiential program components of FIT models by an iterative research process based on the Grounded Theory Methodology. These components were transformed into a preliminary research tool that was evaluated by a participatory expert panel, and during a pilot and validation study, the latter using a random sample of 327 users from 14 mental health departments. Internal consistency of the tool was tested using Cronbach's alpha. Construct validity was evaluated using a Principal Components Analysis (PCA) and a Jonckheere Terpstra test in relation to different implementation levels of the FIT model. Concurrent validity was tested against a German version of the Client Satisfaction Questionnaire (ZUF-8) using correlation analysis and a linear regression model.

RESULTS

The evaluation of the expert panel reduced 29 initial items to 16 that were further reduced to 11 items during the pilot study, resulting into a research tool (Needs and Experiences in Psychiatric Treatment-NEPT) that demonstrated good internal consistency (Cronbach's alpha of 0.89). PCA yielded a 1-component structure, which accounted for 49% of the total variance supporting the unidimensional structure of the tool. The total NEPT score increased alongside the increasing implementation of the FIT model (p < 0.05). There was evidence (p < 0.001) for convergent validity assessed against the ZUF-8 as criterion measure.

CONCLUSIONS

The NEPT tool seems to be promising for further development to assess the experiences with and fulfillment of needs of psychiatric care models from the perspective of users. This paper demonstrates that it is possible to use a participatory-collaborative approach within the methodologically rigorous confines of a prospective, controlled research design.

A philosophical perspective on the development and application of patient-reported outcomes measures (PROMs)

Questionnaires are a common method in healthcare and clinical research to collect self-reported data on patients' behaviour and outcomes rather than the clinician's perspective. As a consequence there is a plethora of questionnaires and rating forms developed to measure a range of concepts such as health-related quality of life and health status. Given that these measures have been developed within a nomothetic paradigm to enhance our understanding of peoples self-perceived health status by translating complex personal feelings and experiences into a simple numeric score, the patient's illness narrative is lost along the way. This commentary discusses the limitations of the nomothetic approach as completion of a questionnaire is a social and contextually orientated activity and that their development is best viewed within the philosophical tradition of pragmatism, based on sound qualitative methods and rigorous psychometric testing. The commentary discusses the philosophical orientation underpinning PROM development and argues the case for a pragmatic epistemology based on a mixed methods research paradigm which goes beyond the current practice of informing the content validity of a PROM in the early phase of its development but to work towards developing a more composite and holistic picture through mixed methods in the interpretation of a patient's PROM score. Therefore, it is argued that the quality of data obtained will be enhanced but, also importantly and rightly places the participant at the centre of the research.

The Recovering Quality of Life 10-item (ReQoL-10) scale in a first-episode psychosis population: Validation and implications for patient-reported outcome measures (PROMs)

AIM

This study aimed to examine the psychometric performance of the Recovering Quality of Life scale 10-item version (ReQoL-10) using a sample from a Singapore first-episode psychosis intervention program, to explore its clinical and sociodemographic correlates, and to discuss its utility as a patient-reported outcome measure (PROM).

METHODS

Sociodemographic data, duration of untreated psychosis (DUP), and diagnosis were collected from 300 participants. Clinical data, which included baseline and current scores on the Patient Health Questionnaire 9-item version, EuroQoL-5 Dimension 3-level version, Positive and Negative Syndrome Scale, and Global Assessment of Functioning scale, were extracted. The ReQoL-10 was tested for structural validity, internal consistency, and construct validity, and a multiple linear regression determined if any of the baseline factors were statistically significant predictors of the total ReQoL-10 scores.

RESULTS

The mean (SD) total ReQoL-10 score of the sample was 27.8 (7.8). Confirmatory factor analysis confirmed the bifactor model structure of the ReQoL-10. The instrument demonstrated good internal consistency and adequate construct validity. Being older was associated with higher total ReQoL-10 scores, while being married, having a highest educational level of vocational/diploma, longer DUP, and a diagnosis of affective psychosis were associated with lower total ReQoL-10 scores.

CONCLUSIONS

This study has validated the ReQoL-10 as suitable for routine use to measure recovery-specific quality of life in a psychiatric setting among patients with first-episode psychosis, and is a potential tool to initiate recovery conversations. As a PROM, it can facilitate shared decision making, in line with efforts to evaluate and improve quality of care.

Development, reliability, and validity of the case management quality of care scale (CMQS) among clients with first episode psychosis

This paper describes the development and validation of the Case Management Quality Scale (CMQS) to measure the quality of care provided by case managers in an early intervention service for clients with first episode psychosis. A total of 300 client participants from the Singapore Early Psychosis Programme completed a self-administered survey comprising 52 items measuring case management, the Client Satisfaction Questionnaire 8-item version (CSQ-8), Recovery Self-Assessment - person in recovery version (RSA-R), and Working Alliance Inventory - long form client version (WAI-C), and Recovering Quality of Life 10-item version (ReQoL-10). Exploratory factor analysis was conducted on the 52-item scale and the items were reduced using criterion validity. Psychometric properties of the 25-item CMQS were tested using internal consistency and concurrent validity. A 2-factor structure of the CMQS was established comprising two subscales - Therapeutic alliance and Care and empowerment. The test of normality for CMQS and subscales was largely found to be within the acceptable range. The Cronbach's alpha for the CMQS scale was high (0.97) and in terms of convergent validity, the CMQS and the two subscales had a moderate correlation with the CSQ-8, RSA-R, and WAI-C (r ranging from 0.596-0.711) and a mild correlation with the ReQoL-10. In conclusion, the CMQS is a user-friendly, and psychometrically sound instrument that is potentially useful for measuring case manager-client therapeutic alliance, and the level of care and empowerment in a case management service.

Assessment instruments for lower urinary tract dysfunction in children: Symptoms, characteristics and psychometric properties

INTRODUCTION

The use of instruments to assess symptoms of the lower urinary tract has increased mainly when it comes to pediatric urology. The search for a validated and reliable one for use in clinical practice should be a concern of the professionals involved in the care of these children.

OBJECTIVE

The aim of this study was to analyze the assessment instruments for lower urinary tract dysfunction in children regarding: symptoms, characteristics and psychometric properties.

STUDY DESIGN

PubMed and Latin American and Caribbean Literature in Health Sciences databases were searched. The PICO strategy was used to construct the research question and bibliographic search. Keywords included voiding disorders, LUTS, Child, Surveys and Questionnaires.

RESULTS

Nine articles were identified. The construct observed by all instruments was bladder dysfunction and seven instruments also assessed bowel dysfunction. Thirty different events were addressed. In eight instruments the respondents are the parents. The age of the target population ranged from three to eighteen years and regarding the number of items, an average of 21.8 items was observed. Six instruments showed good results of reliability and four presented good results of accuracy.

CONCLUSIONS

The instruments included a wide range of symptoms related to dysfunction. Only two reported all validation and reliability tests, however six instruments showed good reliability results, making them eligible for the cross-cultural adaptation and validation process for use in countries other than those of the instrument's origin.

Individualized and Standardized Outcome Measures: Further Arguments in Favor of Cohabitation

Ashworth et al. (Admin Policy Ment Health Ment Health Serv Res 46:425-428, 2019) recently published a paper in this journal making a cogent argument for the cohabitation of standardized and individualized outcome measures. In the present Point of View article, we consider additional arguments in favor of this cohabitation.

Using a person-generated mental health outcome measure in large clinical trials in Kenya and Pakistan: Self-perceived problem responses in diverse communities

Health care should be informed by the physical, socioeconomic, mental, and emotional well-being of the person, and account for social circumstances and culture. Patient-generated outcome measures can contribute positively to mental health research in culturally diverse populations. In this study, we analysed qualitative responses to the Psychological Outcome Profiles (PSYCHLOPS) Questionnaire-a patient-generated outcome measure based on open-ended questions, and compared the qualitative responses gathered to conventional, nomothetic measures used alongside the PSYCHLOPS in two studies. Data were collected as part of outcome research on a psychological intervention in Pakistan (N = 346) and Kenya (N = 521). Two researchers coded the qualitative responses to the PSYCHLOPS and identified overarching themes. We compared the overarching themes identified to the items in the conventional, nomothetic outcome measures to investigate conceptual equivalence. Using the PSYCHLOPS, the most frequently reported problems in Kenya were financial constraints, poor health, and unemployment. In Pakistan, the most frequent problems were poor health and emotional problems. Most of the person-generated problem concepts were covered also in nomothetic measures that were part of the same study. However, there was no item equivalence in the nomothetic measures for the most frequent PSYCHLOPS problem cited in both countries. Response bias and measurement bias may not be excluded. More research on the use of PSYCHLOPS alongside conventional outcome measures is needed to further explore the extent to which it may bring added value. Use of a PSYCHLOPS semistructured interview schedule and efforts to minimise response biases should be considered.

An emerging framework for fully incorporating public involvement (PI) into patient-reported outcome measures (PROMs)

Patient-reported outcome measures (PROMs) are widely used in the United Kingdom (UK) and internationally to report and monitor patients’ subjective assessments of their symptoms and functional status and also their quality of life. Whilst the importance of involving the public in PROM development to increase the quality of the developed PROM has been highlighted this practice is not widespread. There is a lack of guidance on how public involvement (PI) could be embedded in the development of PROMs, where the roles can be more complex than in other types of research. This paper provides a timely review and sets out an emerging framework for fully incorporating PI into PROM development.

Understanding and using patient experiences as evidence in healthcare priority setting

BACKGROUND

In many countries, committees make priority-setting decisions in order to control healthcare costs. These decisions take into account relevant criteria, including clinical effectiveness, cost-effectiveness, and need, and are supported by evidence usually drawn from clinical and economic studies. These sources of evidence do not include the specific perspective and information that patients can provide about the condition and treatment.

METHODS

Drawing on arguments from political philosophy and ethics that are the ethical basis for many priority-setting bodies, the authors argue that criteria like need and its effects on patients and caregivers are best supported by evidence generated from patients' experiences. Social sciences and mixed-methods research support the generation and collection of robust evidence.

RESULTS

Patient experience is required for a decision-making process that considers all relevant evidence. For fair priority-setting, decision-makers should consider relevant evidence and reasons, so patient experience evidence should not be ignored. Patient experience must be gathered in a way that generates high quality and methodologically rigorous evidence. Established quantitative and qualitative methods can assure that evidence is systematic, adherent to quality standards, and valid. Patient, like clinical, evidence should be subject to a transparent review process.

DISCUSSION

Considering all relevant evidence gives each person an equal opportunity at having their treatment funded. Patient experience gives context to the clinical evidence and also directly informs our understanding of the nature of the condition and its effects, including patients' needs, how to meet them, and the burden of illness. Such evidence also serves to contextualise reported effects of the treatment. The requirement to include patient experience as evidence has important policy implications for bodies that make priority-setting decisions since it proposes that new types of evidence reviews are commissioned and considered.

The experience of mental health service users in health system strengthening: lessons from Uganda

BACKGROUND

Mental, neurological and substance use disorders are a public health burden in Uganda. Mental health service user involvement could be an important strategy for advocacy and improving service delivery, particularly as Uganda redoubles its efforts to integrate mental health into primary health care (PHC). However, little is known on the most effective way to involve service users in mental health system strengthening.

METHODS

This was a qualitative key informant interview study. At national level, 4 interviews were conducted with national level health workers and 3 service user organization representatives. At the district level, 2 interviews were conducted with district level health workers and 5 service user organization representatives. Data were analyzed using content thematic analysis.

FINDINGS

Overall, there was low mental service user participation in health system strengthening at both national and district levels. Health system strengthening activities included policy development, implementation of programs and research. Informants mentioned several barriers to service user involvement in mental health system strengthening. These were grouped into three categories: institutional, community and individual level factors. Institutional level barriers included: limited funding to form, train and develop mental health service user groups, institutional stigma and patronage by founder members of user organizations. Community level barriers included: abject poverty and community stigma. Individual level barriers included: low levels of awareness and presence of self-stigma. Informants also recommended some strategies to enhance service user involvement.

CONCLUSION

The Uganda Ministry of Health should develop a strategy to improve service user participation in mental health system strengthening. This requires an appreciation of the importance of service users in improving service delivery. To address the barriers to service user involvement identified in this study requires concerted efforts by the Uganda Ministry of Health and the district health services, specifically with regard to attitudes of health workers, dealing with stigma at all levels, raising awareness about the rights of service users to participate in health systems strengthening activities, building capacity and financial empowerment of service user organizations.

Acceptability of Patient-Reported Outcome and Experience Measures for Hepatitis C Treatment Among People Who Use Drugs

BACKGROUND

Growing international interest in mechanisms to capture and measure experiences and outcomes of health interventions from the patient perspective has led to the development of patient-reported measures (PRMs) across many areas of medicine. Although PRMs are now well utilised in some settings, the rapidly expanding area of direct-acting antiviral (DAA) treatments for hepatitis C has received remarkably little attention. In addition, questions are also being raised about the extent to which patients have been involved in the development of PRMs, which are primarily designed to reflect the patient perspective. In this context, the aim of this paper was to explore the possibility of developing a new PRM for use in hepatitis C DAA therapy that would also be acceptable to the patient group, in this case people who inject drugs (PWID).

METHOD

The study was based on a participatory design that included a peer researcher and foundational qualitative research including semi-structured interviews with 24 PWID with hepatitis C to inform the development of the PRMs. Stage 2 included four focus groups of six PWID with hepatitis C, who were asked to complete the draft measures and provide feedback.

RESULTS

Participants responded positively to the draft PRMs. The results indicate that participants' concerns during DAA treatment are often not sufficiently attended to in clinical settings. In the light of this finding, participants reported that PRMs have a positive role to play in the negotiation of patients' care during DAA treatment.

CONCLUSIONS

The findings show that utilising a participatory approach to the development of PRMs for DAA HCV treatment with PWID not only provides a way to measure experiences and outcomes of treatment from the patient perspective, but also provides a means for highly marginalised patient groups to have a say in and negotiate their care in ways that might not otherwise be possible.

Public involvement in health outcomes research: lessons learnt from the development of the recovering quality of life (ReQoL) measures

Background

To provide a model for Public involvement (PI) in instrument development and other research based on lessons learnt in the co-production of a recently developed mental health patient reported outcome measure called Recovering Quality of Life (ReQoL). While service users contributed to the project as research participants, this paper focuses on the role of expert service users as research partners, hence referred to as expert service users or PI.

Methods

At every stage of the development, service users influenced the design, content and face validity of the measure, collaborating with other researchers, clinicians and stakeholders who were central to this research. Expert service users were integral to the Scientific Group which was the main decision-making body, and also provided advice through the Expert Service User Group.

Results

During the theme and item generation phase (stage 1) expert service users affirmed the appropriateness of the seven domains of the Patient Reported Outcome Measure (activity, hope, belonging and relationships, self-perception, wellbeing, autonomy, and physical health). Expert service users added an extra 58 items to the pool of 180 items and commented on the results from the face and content validity testing (stage 2) of a refined pool of 88. In the item reduction and scale generation phase (stage 3), expert service users contributed to discussions concerning the ordering and clustering of the themes and items and finalised the measures. Expert service users were also involved in the implementation and dissemination of ReQoL (stage 4). Expert service users contributed to the interpretation of findings, provided inputs at every stage of the project and were key decision-makers. The challenges include additional work to make the technical materials accessible, extra time to the project timescales, including time to achieve consensus from different opinions, sometimes strongly held, and extra costs.

Conclusion

This study demonstrates a successful example of how PI can be embedded in research, namely in instrument development. The rewards of doing so cannot be emphasised enough but there are challenges, albeit surmountable ones. Researchers should anticipate and address those challenges during the planning stage of the project.

Power, Privilege and Knowledge: the Untenable Promise of Co-production in Mental "Health"

This paper examines the concept and practice of coproduction in mental health. By analyzing personal experience as well as the historical antecedents of coproduction, we argue that the site of coproduction is defined by the legacy of the Enlightenment and its notions of "reason" and "the cognitive subject." We show the enduring impact of these notions in producing and perpetuating the power dynamics between professionals, researchers, policy makers and service users within privileged sites of knowledge production, whereby those deemed to lack reason-the mad and the racialized mad in particular-and their knowledge are radically inferiorised. Articulating problems in what is considered knowledge and methods of knowing, we argue that modern "psy" sciences instantiates the privilege of reason as well as of whiteness. We then examine how the survivor movement, and the emergent survivor/mad knowledge base, duplicates white privilege even as it interrogates privileges of reason and cognition. Describing how we grapple with these issues in an ongoing project-EURIKHA-which aims to map the knowledge produced by service users, survivors and persons with psychosocial disabilities globally, we offer some suggestions. Coproduction between researchers, policy makers and those of us positioned as mad, particularly as mad people of color, we argue, cannot happen in knowledge production environments continuing to operate within assumptions and philosophies that privilege reason as well as white, Eurocentric thinking. We seek not to coproduce but to challenge and change thinking and support for psychosocial suffering in contexts local to people's lives.

Patient involvement in improving the evidence base on mental health inpatient care: the PERCEIVE programme

Background

Despite the movement towards care in the community, 40% of the NHS budget on mental health care is still attributed to inpatient services. However, long before the Francis Report highlighted grave shortcomings in inpatient care, there were reports by service user groups on the poor quality of these services in mental health. The programme provides a particular focus on the inclusion of the patient’s perspective in the development and evaluation of evidence.

Objectives

To understand how changes to inpatient care affect the perceptions of the ward by service users and staff by using stakeholder participatory methods.

Design

The programme consisted of four work packages (WPs). (1) Lasting Improvements for Acute Inpatient SEttings (LIAISE): using participatory methods we developed two new scales [Views On Therapeutic Environment (VOTE) for staff and Views On Inpatient CarE (VOICE) for service users]. (2) Client Services Receipt Inventory – Inpatient (CITRINE): working with nurses and service users we developed a health economic measure of the amount of contact service users have with staff. The self-report measure records interactions with staff as well as the number of therapeutic activities attended. (3) Delivering Opportunities for Recovery (DOORWAYS): a stepped-wedge randomised controlled trial to test if training ward nurses to deliver therapeutic group activities would improve the perception of the ward by service users and staff. A total of 16 wards were progressively randomised and we compared the VOICE, VOTE and CITRINE measures before and after the intervention. A total of 1108 service users and 539 staff participated in this trial. (4) Bringing Emergency TreatmenT to Early Resolution (BETTER PATHWAYS) was an observational study comparing two service systems. The first was a ‘triage’ system in which service users were admitted to the triage ward and then either transferred to their locality wards or discharged back into the community within 7 days. The second system was routine care. We collected data from 454 service users and 284 nurses on their perceptions of the wards.

Main outcome measures

The main outcomes for the DOORWAYS and BETTER project were service user and staff perceptions of the ward (VOICE and VOTE, respectively) and the health economic measure was CITRINE. All were developed in WPs 1 and 2.

Results

We developed reliable and valid measures of (1) the perceptions of inpatient care from the perspectives of service users and nurses (VOICE and VOTE) and (2) costs of interactions that were valued by service users (CITRINE). In the DOORWAYS project, after adjusting for legal status, we found weak evidence for benefit (standardised effect of –0.18, 95% CI 0.38 improvement to 0.01 deterioration;p = 0.062). There was only a significant benefit for involuntary patients following the staff training (N582, standardised effect of –0.35, 95% CI –0.57 to –0.12;p = 0.002; interactionp-value 0.006). VOTE scores did not change over time (standardised effect size of 0.04, 95% CI –0.09 to 0.18;p = 0.54). We found no evidence of an improvement in cost-effectiveness (estimated effect of £33, 95% CI –£91 to £146;p = 0.602), but resource allocation did change towards patient-perceived meaningful contacts by an average of £12 (95% CI –£76 to £98;p = 0·774). There were no significant differences between the triage and routine models of admission in terms of better perceptions by service users (estimated effect 0.77-point improvement in VOICE score on the triage ward;p = 0.68) or nurses (estimated effect of 1.68-point deterioration in VOTE on the triage ward;p = 0.38) or in terms of the cost of the length of care provided (£391 higher on triage;p = 0.77).

Strengths and limitations

We have developed measures using methods involving both service users and staff from mental health services. The measures were developed specifically for acute inpatient services and, therefore, cannot be assumed to be useful for other services. For instance, extensions of the measures are under construction for use in mother and baby units. The strength of the BETTER PATHWAYS and DOORWAYS projects is the large-scale data collection. However, we were testing specific services based in inner city areas and stretching to inner urban areas. It may be that different effects would be found in more rural communities or in different types of inpatient care.

Future work

Our database will be used to develop an understanding of the mediating and moderating factors for improving care quality.

Trial registration

Current Controlled Trials ISRCTN06545047.

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 7. See the NIHR Journals Library website for further project information.

Coproduction for feasibility and pilot randomised controlled trials: learning outcomes for community partners, service users and the research team

PLAIN ENGLISH SUMMARY

Co-producing research with members of the public is increasingly recognised as a valuable process. Yet, despite these good intentions, the literature on coproduction has struggled to keep pace with the coproduction 'movement'. There is a lack of clarity regarding acceptable levels of involvement and attempts at standardising approaches appear generic and lack detail. Moreover, relatively little research has captured the views of all the parties involved (academics, service providers and service users).We conducted interviews with all those involved in developing a new online service for depression in Northern Ireland. Our main questions related to how these three very different groups of people worked together over a two-year period to design, develop and deliver the service (e.g. what were the benefits? What would they do differently?)We found that early involvement was a key factor as this promoted equal ownership. There was also a need to be flexible and recognise other workload pressures. Interestingly, service providers and service users were keen to become more involved in data analysis - this is one of the most under-researched and reported areas within the coproduction literature. Finally, we considered how user involvement worked within complex research designs and how this could be improved. Based on this learning, the paper concludes with a simple 3-step framework that others may wish to follow in order to improve coproduction outcomes within interventions.

ABSTRACT

Background Co-production, involving members of the public in research, is increasingly encouraged by research funders. However, reports detailing involvement of the public in the entire research process from design, delivery, analysis and dissemination of findings are lacking. Furthermore, little is known about the lessons learnt from the perspective of the public and researchers; or more specifically lessons learnt when coproducing specific types of research projects, such as feasibility/pilot studies incorporating a randomised controlled trial (RCT) design. This paper aims to provide a more rounded picture of co-production based on the learning outcomes of researchers, their community partners and service users involved in a feasibility/pilot RCT study developing and evaluating an E-health Service for adults with depression. Methods Qualitative research incorporating 11 semi-structured interviews with academic team members (n = 4), community partners (n = 3) and service users with depression (n = 4) Data were analysed using thematic analysis. Results Key factors for successful coproduction include - (1) early involvement at the pre-development stage, including contributing to the scientific grant application; (2) early identification of team strengths and expertise from the outset; (3) regular team meetings and contact (formal or informal) among coproduction partners; (4) a flexible and pragmatic approach to research design (particularly within RCTs); (5) shared decision making and responsibility and (6) recognition of 'other' pressures and providing support to each other. Findings also suggested further scope for involving community partners in data analysis and dissemination through co-authored papers. Those seeking to coproduce interventions or utilise RCT designs should consider tensions between data quality and intervention implementation and ethical issues regarding control groups. Conclusion This paper confirms previous research confirming the benefits of coproduction. However, it also highlights a number of barriers, particularly when using complex research design, such as RCTs. Learning points are summarised in an implementation model for coproducing research. This model may provide a useful guide for considering activities associated with meaningful coproduction. We urge others to test this proposed model more widely in different areas of coproduced research.

Reliability and validity of the self-report version of the apathy evaluation scale in first-episode Psychosis: Concordance with the clinical version at baseline and 12 months follow-up

Negative symptoms have traditionally been assessed based on clinicians' observations. The subjective experience of negative symptoms in people with psychosis may bring new insight. The Apathy Evaluation Scale (AES) is commonly used to study apathy in psychosis and has corresponding self-rated (AES-S) and clinician-rated (AES-C) versions. The aim of the present study was to determine the validity and reliability of the AES-S by investigating its concordance with the AES-C. Eighty-four first-episode (FEP) patients completed the shortened 12-item AES-S and AES-C at baseline (T1) and 12 months (T2). Concordance was studied by degree of correlation, comparison of mean scores, and change and difference between diagnostic groups. The Positive and Negative Symptom Scale (PANSS) was used to study convergent and discriminative properties. High concordance was found between AES-S and AES-C at both T1 and T2 regarding mean values, change from T1 to T2, and the proportion with high levels of apathy. Both versions indicated high levels of apathy in FEP, while associations with PANSS negative symptoms were weaker for AES-S than AES-C. Controlling for depression did not significantly alter results. We concluded that self-rated apathy in FEP patients is in concordance with clinician ratings, but in need of further study.

Participatory research: real or imagined

BACKGROUND

Participatory research has as a central tenet that power relations between researcher and researched be reduced. In the last 20 years, a substantial literature has demonstrated the difficulties inherent in this as well as the troublesome nature of certain central concepts.

AIMS

(1) To describe and illustrate a new form of participatory research where the researchers share at least something with the participants in the research. That is, all are users of mental health services. (2) To reflect on the novel form of participatory research in terms of whether it shares, mitigates or avoids some of the difficulties of more traditional forms and to pose the question: what is a mental health community?

RESULTS

The model described is new in that the researchers have a different status than in conventional participatory research. But it is illuminated by and itself illuminates issues of power relations in research and difficulties in reducing that; gatekeepers and the exclusion of crucial groups of service users; the confusion of demographic representativeness with the silencing of marginalized perspectives; coming out of the academic space and the shifting issue of what counts as 'communities' in mental health.

CONCLUSION

The examples given are moderate in scale and relevant to social psychiatry. Yet they may change methods and the definition of participatory research and at the same time be vitiated by but also illuminate dilemmas already identified in the literature albeit in different formations.

Developing a patient-reported experience questionnaire with and for people who use drugs: A community engagement process in Vancouver's Downtown Eastside

People who use drugs (PWUD) frequently have complex health care needs, yet face multiple barriers to accessing services. Involving PWUD in health service design and evaluation can enhance the quality of data collected and ensure policy and practice improvements reflect the expressed needs of the population. However, PWUD remain largely excluded from the evaluation of health services that directly affect their lives, including development of patient-reported experience measures (PREMS) that have gained prominence in health services research and clinical practice. Detailed descriptions of PWUD participation in survey design are notably absent in the literature. In this commentary, we present a case that demonstrates how PWUD can contribute meaningfully to the development of questionnaires that assess patient-reported health care experiences. We describe the development, implementation and outcomes of a process to engage local drug user organizations in the evaluation of a redesign and reorientation of health service delivery in the Downtown Eastside (DTES) neighborhood of Vancouver, Canada. Through this process, participants contributed critical elements to the design of a patient-reported experience measure, including: (1) identifying unmet service needs in the neighborhood; (2) identifying local barriers and facilitators to care; (3) formulating questions on cultural safety; and (4) improving structure, language and clarity of the questionnaire. We highlight lessons learned from the process, reflecting on the strengths, challenges and ethical considerations associated with community-based approaches to questionnaire development. The workshop model presented here illustrates one flexible and promising approach to enabling meaningful participation of PWUD in questionnaire development.

Showing the Unsayable: Participatory Visual Approaches and the Constitution of 'Patient Experience' in Healthcare Quality Improvement

This article considers the strengths and potential contributions of participatory visual methods for healthcare quality improvement research. It argues that such approaches may enable us to expand our understanding of 'patient experience' and of its potential for generating new knowledge for health systems. In particular, they may open up dimensions of people's engagement with services and treatments which exceed both the declarative nature of responses to questionnaires and the narrative sequencing of self reports gathered through qualitative interviewing. I will suggest that working with such methods may necessitate a more reflexive approach to the constitution of evidence in quality improvement work. To this end, the article will first consider the emerging rationale for the use of visual participatory methods in improvement before outlining the implications of two related approaches-photo-elicitation and PhotoVoice-for the constitution of 'experience'. It will then move to a participatory model for healthcare improvement work, Experience Based Co-Design (EBCD). It will argue that EBCD exemplifies both the strengths and the limitations of adequating visual participatory approaches to quality improvement ends. The article will conclude with a critical reflection on a small photographic study, in which the author participated, and which sought to harness service user perspectives for the design of psychiatric facilities, as a way of considering the potential contribution of visual participatory methods for quality improvement.

The importance of content and face validity in instrument development: lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL)

PURPOSE

Service user involvement in instrument development is increasingly recognised as important, but is often not done and seldom reported. This has adverse implications for the content validity of a measure. The aim of this paper is to identify the types of items that service users felt were important to be included or excluded from a new Recovering Quality of Life measure for people with mental health difficulties.

METHODS

Potential items were presented to service users in face-to-face structured individual interviews and focus groups. The items were primarily taken or adapted from current measures and covered themes identified from earlier qualitative work as being important to quality of life. Content and thematic analysis was undertaken to identify the types of items which were either important or unacceptable to service users.

RESULTS

We identified five key themes of the types of items that service users found acceptable or unacceptable; the items should be relevant and meaningful, unambiguous, easy to answer particularly when distressed, do not cause further upset, and be non-judgemental. Importantly, this was from the perspective of the service user.

CONCLUSIONS

This research has underlined the importance of service users' views on the acceptability and validity of items for use in developing a new measure. Whether or not service users favoured an item was associated with their ability or intention to respond accurately and honestly to the item which will impact on the validity and sensitivity of the measure.

Roadmap for optimizing the clinical utility of emotional stress paradigms in human neuroimaging research

The emotional stress response is relevant to a number of psychiatric disorders, including posttraumatic stress disorder (PTSD) in particular. Research using neuroimaging methods such as functional magnetic resonance imaging (fMRI) to probe stress-related neural processing have provided some insights into psychiatric disorders. Treatment providers and individual patients would benefit from clinically useful fMRI paradigms that provide information about patients' current brain state and responses to stress in order to inform the treatment selection process. However, neuroimaging has not yet made a meaningful impact on real-world clinical practice. This lack of clinical utility may be related to a number of basic psychometric properties that are often overlooked during fMRI task development. The goals of the current review are to discuss important methodological considerations for current human fMRI stress-related paradigms and to provide a roadmap for developing methodologically sound and clinically useful paradigms. This would include establishing various aspects of reliability, including internal consistency, test-retest and multi-site, as well as validity, including face, content, construct, and criterion. In addition, the establishment of standardized normative data from a large sample of participants would support our understanding of how any one individual compares to the general population. Addressing these methodological gaps will likely have a powerful effect on improving the replicability of findings and optimize our chances for improving real-world clinical outcomes.

Routine outcome measurement: recovery, quality of life and co-production

Patient-reported outcome measures (PROMs) are self-rated, but may not take in other aspects of the patient's perspective, such as the inclusion of domains that reflect service-user priorities. The clinician's view still has priority, although this situation has shifted in recent years. The Recovering Quality of Life (ReQoL) offers an advance in this area. Declaration of interest None.

Recovering Quality of Life (ReQoL): a new generic self-reported outcome measure for use with people experiencing mental health difficulties

BACKGROUND

Outcome measures for mental health services need to adopt a service-user recovery focus. Aims To develop and validate a 10- and 20-item self-report recovery-focused quality of life outcome measure named Recovering Quality of Life (ReQoL).

METHOD

Qualitative methods for item development and initial testing, and quantitative methods for item reduction and scale construction were used. Data from >6500 service users were factor analysed and item response theory models employed to inform item selection. The measures were tested for reliability, validity and responsiveness.

RESULTS

ReQoL-10 and ReQoL-20 contain positively and negatively worded items covering seven themes: activity, hope, belonging and relationships, self-perception, well-being, autonomy, and physical health. Both versions achieved acceptable internal consistency, test-retest reliability (>0.85), known-group differences, convergence with related measures, and were responsive over time (standardised response mean (SRM) > 0.4). They performed marginally better than the Short Warwick-Edinburgh Mental Well-being Scale and markedly better than the EQ-5D.

CONCLUSIONS

Both versions are appropriate for measuring service-user recovery-focused quality of life outcomes. Declaration of interest M.B. and J.Co. were members of the research group that developed the Clinical Outcomes in Routine Evaluation (CORE) outcome measures.

What side effects are problematic for patients prescribed antipsychotic medication? The Maudsley Side Effects (MSE) measure for antipsychotic medication

BACKGROUND

Capturing service users' perspectives can highlight additional and different concerns to those of clinicians, but there are no up to date, self-report psychometrically sound measures of side effects of antipsychotic medications. Aim To develop a psychometrically sound measure to identify antipsychotic side effects important to service users, the Maudsley Side Effects (MSE) measure.

METHOD

An initial item bank was subjected to a Delphi exercise (n = 9) with psychiatrists and pharmacists, followed by service user focus groups and expert panels (n = 15) to determine item relevance and language. Feasibility and comprehensive psychometric properties were established in two samples (N43 and N50). We investigated whether we could predict the three most important side effects for individuals from their frequency, severity and life impact.

RESULTS

MSE is a 53-item measure with good reliability and validity. Poorer mental and physical health, but not psychotic symptoms, was related to side-effect burden. Seventy-nine percent of items were chosen as one of the three most important effects. Severity, impact and distress only predicted 'putting on weight' which was more distressing, more severe and had more life impact in those for whom it was most important.

CONCLUSIONS

MSE is a self-report questionnaire that identifies reliably the side-effect burden as experienced by patients. Identifying key side effects important to patients can act as a starting point for joint decision making on the type and the dose of medication.

Measuring patient outcomes in chronic heart failure: psychometric properties of the Care-Related Quality of Life survey for Chronic Heart Failure (CaReQoL CHF)

Background

The Care-Related Quality of Life survey for Chronic Heart Failure (CaReQoL CHF) is a newly developed patient-reported outcome measure (PROM) that measures care-related quality of life in patients diagnosed with chronic heart failure. This study describes the psychometric properties of the questionnaire and its relationship with disease severity and global rating of quality of care.

Method

Insurance companies selected patients with a recorded diagnosis of chronic heart failure and for whom the hospital submitted a billing statement in the last year. Exploratory factor analysis, Cronbach’s alpha and item-rest correlation were used to construct the CaReQoL CHF. Construct validity was assessed by examining the mean values of the CaReQoL CHF scales for the categories of the widely-used New York Heart Association (NYHA) functional classification and by correlating the global rating of quality of care with the CaReQoL CHF scales.

Results

One thousand eighteen patients with chronic heart failure filled out the CaReQoL CHF (RR: 35.7%). The CaReQoL CHF consists of 20 items and three scales: social and emotional problems, physical limitations, and being in safe hands. The mean scores of the three scales differed significantly for the NYHA categories, particularly for the social-emotional problems and physical limitation scales. The ‘being in safe hands’ scale showed a moderate positive correlation with the global rating of quality of care.

Conclusions

The CaReQoL CHF is a concise and valid PROM that matches patients’ priorities in healthcare. It adds a new element to existing quality of life questionnaires for patients with chronic heart failure, that is ‘being in safe hands’ scale. This scale is relevant for patients because they experience anxiety and tension about their condition. Future research should determine whether the CaReQoL CHF can help healthcare providers in daily practice to focus treatment on outcomes of care that are relevant to individual patients.

Developing a measure of mental health service satisfaction for use in low income countries: a mixed methods study

Background

Service satisfaction is integral to quality of care and measures are therefore considered important indicators of quality. Patient’s responses to their experiences of using services are under-researched in the context of mental healthcare in low income countries. Our aim was to use mixed methods to develop a new measure of satisfaction for use among consumers of the new models of mental healthcare which are currently being scaled-up.

Methods

We used qualitative methods to explore the concept of service satisfaction. On the basis of these findings, we developed a new ‘Mental health service satisfaction scale’ (MHSSS v0.0) by adapting existing measures of service satisfaction. We evaluated psychometric properties of the new measure, among a sample of service users with severe mental disorder (SMD) (n = 200) and caregivers (n = 200). Following expert review, a modified version of the measure was developed (MHSSS v1.0) and psychometric properties were examined with data from a second independent sample (n = 150 service users with SMD and n = 150 caregivers).

Results

Factors identified in analysis of the first quantitative sample coincide with core concepts of service satisfaction as reported in the literature and were reflected in the key themes which emerged from our qualitative study: interpersonal factors, efficacy, communication, technical competency and adequacy of facilities. There was generally consensus among caregivers and service users regarding dimensions of satisfaction. However there was evidence of some differences in prioritization. Revisions made to version 0.0 of the Mental Health Service Satisfaction Scale (MHSSS) led to an improved instrument, with excellent internal consistency, convergent validity and factor loadings indicative of a uni-dimensional construct.

Conclusions

Our findings suggest that conceptions of service satisfaction among people accessing a service for SMD are broadly similar with those established in the literature. Our findings indicate that the MHSSS might be a useful candidate for inclusion in the new toolkit of measures needed to facilitate monitoring of service satisfaction which will be crucial to quality improvement.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2126-2) contains supplementary material, which is available to authorized users.

The impact of involvement on researchers: a learning experience

PLAIN ENGLISH SUMMARY

The impacts of involvement in research are often described in terms of the difference made to the research, the people involved and less frequently the researchers. This paper focuses on the researchers' experiences of involvement, based on an evaluation of a pilot project supporting patient/carer involvement in research at Parkinson's UK. Telephone interviews were conducted with researchers from eight different research projects with involvement. The researchers reported gaining new knowledge from patients and carers. They used this knowledge to change their project designs, interventions and new devices. They also gained new skills in communicating with the public. Meeting patients for the first time had a profound impact on some researchers, causing a change in their professional values. Face-to-face contact seemed particularly important to gain a sense of the 'people behind the data', which suggests such meetings may result in impacts beyond those typically achieved through an exchange of documents. Involvement also influenced one researcher's choices and preferences, in terms of who to ask to take part in their study In summary, researchers often learn something new from talking to patients and carers. Facilitating this conversation seems important to maximise the impact of this learning. In future, it might be helpful for evaluations of involvement to ask researchers in more detail about what they learnt from patients/carers and how they applied their new skills and knowledge. This may help to understand how involvement can influence researchers' thinking to have an impact on research.

ABSTRACT

Background The impacts of patient/public involvement are often described in terms of the difference made to the research, the researchers and the people involved. Involvement often impacts on research by influencing the design, delivery and dissemination. Patients/the public report gaining new skills and knowledge, increased self-confidence, and satisfaction from making a difference. There are fewer reports of the impacts on researchers. This paper discusses the findings from an evaluation of a pilot project supporting patient/carer involvement in research at Parkinson's UK, focusing on the researchers' experiences. Methods Semi-structured telephone interviews were conducted with one researcher from each of the eight research projects which involved patients/carers in the pilot. The findings were analysed using theoretical thematic analysis. Results and discussion Learning can be described as acquiring new knowledge, behaviours, skills, values, or preferences. The researchers' reports reflected these different types of learning. They reported gaining new knowledge from patients and carers, which they recognised as distinct from their textbook knowledge of the condition. They used this learning to change their project designs and their new interventions and devices. They also gained new skills in communicating with patients and carers about the aims and significance of their research. Meeting patients for the first time had a profound impact on some researchers causing them to change their professional values. Face-to-face contact seemed particularly important to gain a sense of the 'people behind the data', which suggests such meetings may result in impacts beyond those typically achieved through an exchange of documents. The involvement also influenced one researchers' priorities and preferences, in terms of what questions to ask and of whom, in their project. Conclusions Researchers learn from an exchange of knowledge with patients/ carers, which influences their plans and actions. This seems to be one way that involvement subsequently has an impact on research. Facilitating this exchange seems important to support mutual learning and to enhance the impact on researchers. Future evaluations of involvement might benefit from exploring what researchers learnt from patients/carers and how they applied their new skills and knowledge.

"It is not just about the alcohol": service users' views about individualised and standardised clinical assessment in a therapeutic community for alcohol dependence

Background

The involvement of service users in health care provision in general, and specifically in substance use disorder treatment, is of growing importance. This paper explores the views of patients in a therapeutic community for alcohol dependence about clinical assessment, including general aspects about the evaluation process, and the specific characteristics of four measures: two individualised and two standardised.

Methods

A focus group was conducted and data were analysed using a framework synthesis approach.

Results

Service users welcomed the experience of clinical assessment, particularly when conducted by therapists. The duration of the evaluation process was seen as satisfactory and most of its contents were regarded as relevant for their population. Regarding the evaluation measures, patients diverged in their preferences for delivery formats (self-report vs. interview). Service users enjoyed the freedom given by individualised measures to discuss topics of their own choosing. However, they felt that part of the standardised questions were difficult to answer, inadequate (e.g. quantification of health status in 0–20 points) and sensitive (e.g. suicide-related issues), particularly for pre-treatment assessments.

Conclusions

Patients perceived clinical assessment as helpful for their therapeutic journey, including the opportunity to reflect about their problems, either related or unrelated to alcohol use. Our study suggests that patients prefer to have evaluation protocols administered by therapists, and that measures should ideally be flexible in their formats to accommodate for patient preferences and needs during the evaluation.

Service user and caregiver involvement in mental health system strengthening in low- and middle-income countries: systematic review

Background

The involvement of mental health service users and their caregivers in health system policy and planning, service monitoring and research can contribute to mental health system strengthening, but as yet there have been very few efforts to do so in low- and middle-income countries (LMICs).

Methods

This systematic review examined the evidence and experience of service user and caregiver involvement in mental health system strengthening, as well as models of best practice for evaluation of capacity-building activities that facilitate their greater participation. Both the peer-reviewed and the grey literature were included in the review, which were identified through database searches (MEDLINE, Embase, PsycINFO, Web of Knowledge, Web of Science, Scopus, CINAHL, LILACS, SciELO, Google Scholar and Cochrane), as well as hand-searching of reference lists and the internet, and a snowballing process of contacting experts active in the area. This review included any kind of study design that described or evaluated service user, family or caregiver (though not community) involvement in LMICs (including service users with intellectual disabilities, dementia, or child and adolescent mental health problems) and that were relevant to mental health system strengthening across five categories. Data were extracted and summarised as a narrative review.

Results

Twenty papers matched the inclusion criteria. Overall, the review found that although there were examples of service user and caregiver involvement in mental health system strengthening in numerous countries, there was a lack of high-quality research and a weak evidence base for the work that was being conducted across countries. However, there was some emerging research on the development of policies and strategies, including advocacy work, and to a lesser extent the development of services, service monitoring and evaluation, with most service user involvement having taken place within advocacy and service delivery. Research was scarce within the other health system strengthening areas.

Conclusions

Further research on service user and caregiver involvement in mental health system strengthening in LMICs is recommended, in particular research that includes more rigorous evaluation. A series of specific recommendations are provided based on the review.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1323-8) contains supplementary material, which is available to authorized users.

Patient involvement in the development of patient-reported outcome measures: a scoping review

Background

Patient‐reported outcome measures (PROMs) measure patients’ perspectives on health outcomes and are increasingly used in health care. To capture the patient's perspective, it is essential that patients are involved in PROM development

Objective

This article reviews in what ways and to what extent patients are involved in PROM development and whether patient involvement has increased over time.

Search strategy

Literature was searched in PubMed, EMBASE, MEDLINE and the Cochrane Methodology Register.

Inclusion criteria

Studies were included if they described a new PROM development.

Data extraction

Basic information and information regarding patient involvement in development phases was recorded.

Main results

A total of 189 studies, describing the development of 193 PROMs, were included. Most PROMs were meant for chronic disease patients (n = 59) and measured quality of life (n = 28). In 25.9% of the PROM development studies, no patients were involved. Patients were mostly involved during item development (58.5%), closely followed by testing for comprehensibility (50.8%), while patient involvement in determining which outcome to measure was minimal (10.9%). Some patient involvement took place in the development of most PROMs, but in only 6.7% patients were involved in all aspects of the development. Patient involvement did not increase with time.

Conclusions

Although patient involvement in PROM development is essential to develop valid patient‐centred PROMs, patients are not always involved. When patients are involved, their level of involvement varies considerably. These variations suggest that further attention to building and/or disseminating consensus on requirements for patient involvement in PROM development is necessary.

Design in mind: eliciting service user and frontline staff perspectives on psychiatric ward design through participatory methods

BACKGROUND

Psychiatric ward design may make an important contribution to patient outcomes and well-being. However, research is hampered by an inability to assess its effects robustly. This paper reports on a study which deployed innovative methods to capture service user and staff perceptions of ward design.

METHOD

User generated measures of the impact of ward design were developed and tested on four acute adult wards using participatory methodology. Additionally, inpatients took photographs to illustrate their experience of the space in two wards. Data were compared across wards.

RESULTS

Satisfactory reliability indices emerged based on both service user and staff responses. Black and minority ethnic (BME) service users and those with a psychosis spectrum diagnosis have more positive views of the ward layout and fixtures. Staff members have more positive views than service users, while priorities of staff and service users differ. Inpatient photographs prioritise hygiene, privacy and control and address symbolic aspects of the ward environment.

CONCLUSIONS

Participatory and visual methodologies can provide robust tools for an evaluation of the impact of psychiatric ward design on users.

It's not evidence, it's insight: bringing patients' perspectives into health technology appraisal at NICE

PLAIN ENGLISH SUMMARY

Health technology appraisal involves reviewing the findings from clinical trials and economic data to produce guidance on how health technology should be used. This task is carried out by appraisal committees in NICE. One of the several ways in which patients can feed their views into these committees is via a written patient statement. We asked nine committee members about what difference the information from patients makes to their decision-making. The Committee members reported that written patient statements offer a different perspective when reviewing the clinical and economic data. This can have a profound impact when a committee draws conclusions based solely on the data, which may not reflect the reality of patients' lives. The patients' and carers' input provides meaning to the data, 'bringing the numbers to life'. It identifies if the technology has any wider impacts than what's been reported in the clinical trial, and also if the trial has measured what's important to patients. We conclude that the written patient statement adds value to the decision-making process by helping Committee members to make sense of the clinical and economic data-it makes them look at the evidence 'in a different light'. Patients' stories are very effective in this context, because they have the power to communicate and to challenge Committee members' assumptions. Understanding this difference between analysing research evidence and drawing on patients' insights is important in thinking about what's needed in a written patient statement and the best way to obtain it.

ABSTRACT

Background Health technology appraisal involves reviewing clinical and economic data to inform guidance on the use of technology. In England this task is carried out by appraisal committees within the National Institute for Health and Care Excellence (NICE). Patients are not committee members as they have a vested interest in the outcome, but one of the several ways they are involved is through submitting a written patient statement, which is considered by the committee during its deliberations. We aimed to find out how the written patient statement adds value to the decision-making process by exploring how it is used in practice. Methods Semi-structured interviews were conducted with nine members of NICE appraisal committees. The interviews were transcribed and analysed thematically. We drew on published evidence of the impact of patient involvement on clinical research and our experience of supporting organisations to produce written patient statements to analyse the findings. Results Committee members reported that written patient statements offer a different perspective when evaluating clinical and economic data. This can have a profound impact when a committee draws conclusions based on data that may not reflect the reality of the patient experience. Information from patients and carers also provides context and meaning to the data, by explaining its real-life implications. It identifies wider impacts of a technology that may not have been assessed in a clinical trial, as well as commenting on whether what has been measured in a trial is relevant to patients. The main barrier to using the written patient statement is the misperception that it is a form of research 'evidence', when in fact it takes the form of experiential knowledge - or insight. Conclusions The written patient statement adds value by aiding Committee members in their interpretation of existing evidence - it enables them to consider this evidence 'in a different light'. In this context, patients' experiential knowledge is effective because it is subjective, emotional and anecdotal. It then has the power to communicate and to challenge assumptions based on the data alone. Understanding this difference between using evidence and insights has implications for the content of a written patient statement and the approaches used to obtain it.