Service user- and carer-reported measures of involvement in mental health care planning: methodological quality and acceptability to users

Patient-centered health care planning in acute inpatient mental health settings: a best practice implementation project

OBJECTIVES

The current project aimed to improve the quality of health care by promoting evidence-based practice (EBP) regarding mental health care planning (MHCP) for adult inpatients.

INTRODUCTION

The implementation of the best evidence in the process of developing and documenting nursing care plans is currently an important legal requirement that contributes to increasing the quality of care.

METHODS

This implementation project was based on the JBI evidence implementation approach and included a baseline audit of seven criteria, implementation of strategies, and a follow-up audit. The project was conducted in an acute psychiatric setting at a university hospital in Bucharest, Romania. The sample included 17 nurses and 30 ward patients.

RESULTS

The baseline audit revealed low compliance (33%-37%) for criterion 3 (a comprehensive care plan) and criterion 4 (patient involvement); moderate compliance (55%) for criterion 1 (care plan for all patients); and increased compliance (97%-100%) for criterion 2 (assessment of patient's needs), criterion 5 (education of patients/caregivers), criterion 6 (providing a copy at discharge), and criterion 7 (education of professionals). As a result of implementing the most appropriate strategies, the maximum improvement (100%) was observed across all five audit criteria that were found to be deficient in the baseline audit.

CONCLUSION

The development and implementation of strategies adapted to specific care need to play a key role in the implementation of EBP. In this case, educating nurses, facilitating nurses' access to EBP for care planning, and improving procedures proved effective in achieving maximum compliance with all the audit criteria.

Training to enhance user and carer involvement in mental health-care planning: the EQUIP research programme including a cluster RCT

Background

Service users and carers using mental health services want more involvement in their care and the aim of this research programme was to enhance service user and carer involvement in care planning in mental health services.

Objectives

Co-develop and co-deliver a training intervention for health professionals in community mental health teams, which aimed to enhance service user and carer involvement in care planning. Develop a patient-reported outcome measure of service user involvement in care planning, design an audit tool and assess individual preferences for key aspects of care planning involvement. Evaluate the clinical effectiveness and the cost-effectiveness of the training. Understand the barriers to and facilitators of implementing service user- and carer-involved care planning. Disseminate resources to stakeholders.

Methods

A systematic review, focus groups and interviews with service users/carers/health professionals informed the training and determined the priorities underpinning involvement in care planning. Data from focus groups and interviews were combined and analysed using framework analysis. The results of the systematic review, focus groups/interviews and a review of the training interventions were synthesised to develop the final training intervention. To develop and validate the patient-reported outcome measure, items were generated from focus groups and interviews, and a psychometric analysis was conducted. Patient-reported outcome measure items and a three-round consensus exercise were used to develop an audit tool, and a stated preference survey was undertaken to assess individual preferences for key aspects of care planning. The clinical effectiveness and cost-effectiveness of the training were evaluated using a pragmatic cluster trial with cohort and cross-sectional samples. A nested longitudinal qualitative process evaluation using multiple methods, including semistructured interviews with key informants involved locally and nationally in mental health policy, practice and research, was undertaken. A mapping exercise was used to determine current practice, and semistructured interviews were undertaken with service users and mental health professionals from both the usual-care and the intervention arms of the trial at three time points (i.e. baseline and 6 months and 12 months post intervention).

Results

The results from focus groups (n = 56) and interviews (n = 74) highlighted a need to deliver training to increase the quality of care planning and a training intervention was developed. We recruited 402 participants to develop the final 14-item patient-reported outcome measure and a six-item audit tool. We recruited 232 participants for the stated preference survey and found that preferences were strongest for the attribute ‘my preferences for care are included in the care plan’. The training was delivered to 304 care co-ordinators working in community mental health teams across 10 NHS trusts. The cluster trial and cross-sectional survey recruited 1286 service users and 90 carers, and the primary outcome was the Health Care Climate Questionnaire. Training was positively evaluated. The results showed no statistically significant difference on the primary outcome (the Health Care Climate Questionnaire) (adjusted mean difference –0.064, 95% confidence interval –0.343 to 0.215; p = 0.654) or secondary outcomes at the 6-month follow-up. Overall, the training intervention was associated with a net saving of –£54.00 (95% confidence interval –£193.00 to £84.00), with a net quality-adjusted life-year loss of –0.014 (95% confidence interval –0.034 to 0.005). The longitudinal process evaluation recruited 54 service users, professionals and carers, finding a failure of training to become embedded in routine care.

Limitations

Our pragmatic study was designed to improve service user and care involvement in care planning among routine community mental health services. We intervened in 18 sites with > 300 care co-ordinators. However, our volunteer sites may not be fully representative of the wider population, and we lacked data with which to compare our participants with the eligible population.

Conclusions

We co-developed and co-delivered a training intervention and developed a unidimensional measure of service user and carer involvement in care planning and an audit tool. Despite a high level of satisfaction with the training, no significant effect was found; therefore, the intervention was ineffective. There was a failure of training to become embedded and normalised because of a lack of organisational readiness to accept change. Working with NHS trusts in our ‘Willing Adopters’ programme with enhanced organisational buy-in yielded some promising results.

Future work

Research should focus on developing and evaluating new organisational initiatives in addition to training health-care professionals to address contextual barriers to service and carer involvement in care planning, and explore co-designing and delivering new ways of enhancing service users’ and carers’ capabilities to engage in care planning.

Trial registration

Current Controlled Trials ISRCTN16488358.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 7, No. 9. See the NIHR Journals Library website for further project information.

Public involvement in health outcomes research: lessons learnt from the development of the recovering quality of life (ReQoL) measures

Background

To provide a model for Public involvement (PI) in instrument development and other research based on lessons learnt in the co-production of a recently developed mental health patient reported outcome measure called Recovering Quality of Life (ReQoL). While service users contributed to the project as research participants, this paper focuses on the role of expert service users as research partners, hence referred to as expert service users or PI.

Methods

At every stage of the development, service users influenced the design, content and face validity of the measure, collaborating with other researchers, clinicians and stakeholders who were central to this research. Expert service users were integral to the Scientific Group which was the main decision-making body, and also provided advice through the Expert Service User Group.

Results

During the theme and item generation phase (stage 1) expert service users affirmed the appropriateness of the seven domains of the Patient Reported Outcome Measure (activity, hope, belonging and relationships, self-perception, wellbeing, autonomy, and physical health). Expert service users added an extra 58 items to the pool of 180 items and commented on the results from the face and content validity testing (stage 2) of a refined pool of 88. In the item reduction and scale generation phase (stage 3), expert service users contributed to discussions concerning the ordering and clustering of the themes and items and finalised the measures. Expert service users were also involved in the implementation and dissemination of ReQoL (stage 4). Expert service users contributed to the interpretation of findings, provided inputs at every stage of the project and were key decision-makers. The challenges include additional work to make the technical materials accessible, extra time to the project timescales, including time to achieve consensus from different opinions, sometimes strongly held, and extra costs.

Conclusion

This study demonstrates a successful example of how PI can be embedded in research, namely in instrument development. The rewards of doing so cannot be emphasised enough but there are challenges, albeit surmountable ones. Researchers should anticipate and address those challenges during the planning stage of the project.

Assessing mental health service user and carer involvement in physical health care planning: The development and validation of a new patient-reported experience measure

BACKGROUND

People living with serious mental health conditions experience increased morbidity due to physical health issues driven by medication side-effects and lifestyle factors. Coordinated mental and physical healthcare delivered in accordance with a care plan could help to reduce morbidity and mortality in this population. Efforts to develop new models of care are hampered by a lack of validated instruments to accurately assess the extent to which mental health services users and carers are involved in care planning for physical health.

OBJECTIVE

To develop a brief and accurate patient-reported experience measure (PREM) capable of assessing involvement in physical health care planning for mental health service users and their carers.

METHODS

We employed psychometric and statistical techniques to refine a bank of candidate questionnaire items, derived from qualitative interviews, into a valid and reliable measure involvement in physical health care planning. We assessed the psychometric performance of the item bank using modern psychometric analyses. We assessed unidimensionality, scalability, fit to the partial credit Rasch model, category threshold ordering, local dependency, differential item functioning, and test-retest reliability. Once purified of poorly performing and erroneous items, we simulated computerized adaptive testing (CAT) with 15, 10 and 5 items using the calibrated item bank.

RESULTS

Issues with category threshold ordering, local dependency and differential item functioning were evident for a number of items in the nascent item bank and were resolved by removing problematic items. The final 19 item PREM had excellent fit to the Rasch model fit (x2 = 192.94, df = 1515, P = .02, RMSEA = .03 (95% CI = .01-.04). The 19-item bank had excellent reliability (marginal r = 0.87). The correlation between questionnaire scores at baseline and 2-week follow-up was high (r = .70, P < .01) and 94.9% of assessment pairs were within the Bland Altman limits of agreement. Simulated CAT demonstrated that assessments could be made using as few as 10 items (mean SE = .43).

DISCUSSION

We developed a flexible patient reported outcome measure to quantify service user and carer involvement in physical health care planning. We demonstrate the potential to substantially reduce assessment length whilst maintaining reliability by utilizing CAT.

Evaluating and Quantifying User and Carer Involvement in Mental Health Care Planning (EQUIP): Co-Development of a New Patient-Reported Outcome Measure

International and national health policy seeks to increase service user and carer involvement in mental health care planning, but suitable user-centred tools to assess the success of these initiatives are not yet available. The current study describes the development of a new reliable and valid, interval-scaled service-user and carer reported outcome measure for quantifying user/carer involvement in mental health care planning. Psychometric development reduced a 70-item item bank to a short form questionnaire using a combination of Classical Test, Mokken and Rasch Analyses. Test-retest reliability was calculated using t-tests of interval level scores between baseline and 2-4 week follow-up. Items were worded to be relevant to both service users and carers. Nine items were removed following cognitive debriefing with a service user and carer advisory group. An iterative process of item removal reduced the remaining 61 items to a final 14-item scale. The final scale has acceptable scalability (Ho = .69), reliability (alpha = .92), fit to the Rasch model (χ2(70) = 97.25, p = .02), and no differential item functioning or locally dependent items. Scores remained stable over the 4 week follow-up period, indicating good test-retest reliability. The 'Evaluating the Quality of User and Carer Involvement in Care Planning (EQUIP)' scale displays excellent psychometric properties and is capable of unidimensional linear measurement. The scale is short, user and carer-centred and will be of direct benefit to clinicians, services, auditors and researchers wishing to quantify levels of user and carer involvement in care planning.