Philosophical ruminations on measurement: methodological orientations of patient reported outcome measures (PROMS)

Phenomenology: A Method for the Interpretation of Patient-Reported Outcomes

Patient-reported outcome measures (PROMs) play a central role in clinical research and patient care resulting in a plethora of standardized PROMs to measure a range of constructs, including disease symptoms, health-related quality of life, and health status (Meadows/Reaney) used in a range of settings, including the nursing environment. However, the use of PROMs in drug development and their use in healthcare evaluation do not easily marry together. In drug development, standardization of measurement is key to the interpretation of the formation at a population level with minimal biases. However, in health care, the individual patient perspective, priority, and needs should be taken into account whereas, in the clinical encounter, one has to also deal with what is particular and unique. The purpose of this paper is to describe the characteristics of the phenomenological method as a means within a mixed-method framework, to supplement participants' patient-reported outcome numeric scores with a more in-depth commentary on the essence of the lived health experiences.

The impact of relaxing restrictions on take-home doses during the COVID-19 pandemic on program effectiveness and client experiences in opioid agonist treatment: a mixed methods systematic review

BACKGROUND

The COVID-19 pandemic led to an unprecedented relaxation of restrictions on take-home doses in opioid agonist treatment (OAT). We conducted a mixed methods systematic review to explore the impact of these changes on program effectiveness and client experiences in OAT.

METHODS

The protocol for this review was registered in PROSPERO (CRD42022352310). From Aug.-Nov. 2022, we searched Medline, Embase, CINAHL, PsycInfo, Web of Science, Cochrane Register of Controlled Trials, and the grey literature. We included studies reporting quantitative measures of retention in treatment, illicit substance use, overdose, client health, quality of life, or treatment satisfaction or using qualitative methods to examine client experiences with take-home doses during the pandemic. We critically appraised studies using the Mixed Methods Appraisal Tool. We synthesized quantitative data using vote-counting by direction of effect and presented the results in harvest plots. Qualitative data were analyzed using thematic synthesis. We used a convergent segregated approach to integrate quantitative and qualitative findings.

RESULTS

Forty studies were included. Most were from North America (23/40) or the United Kingdom (9/40). The quantitative synthesis was limited by potential for confounding, but suggested an association between take-home doses and increased retention in treatment. There was no evidence of an association between take-home doses and illicit substance use or overdose. Qualitative findings indicated that take-home doses reduced clients' exposure to unregulated substances and stigma and minimized work/treatment conflicts. Though some clients reported challenges with managing their medication, the dominant narrative was one of appreciation, reduced anxiety, and a renewed sense of agency and identity. The integrated analysis suggested reduced treatment burden as an explanation for improved retention and revealed variation in individual relationships between take-home doses and illicit substance use. We identified a critical gap in quantitative measures of patient-important outcomes.

CONCLUSION

The relaxation of restrictions on take-home doses was associated with improved client experience and retention in OAT. We found no evidence of an association with illicit substance use or overdose, despite the expansion of take-home doses to previously ineligible groups. Including patient-important outcome measures in policy, program development, and treatment planning is essential to ensuring that decisions around take-home doses accurately reflect their value to clients.

The use of patient-reported outcome measures by healthcare professionals in specialized asthma management centers in French-speaking Belgium: A mixed-methods study

INTRODUCTION

Recently, the literature has promoted the use of patient-reported outcome measures (PROMs) in clinical practice as a means to foster patient engagement. However, conditions necessary to support the use of PROMs to encourage asthma patient engagement are not clearly defined. Therefore, we sought (1) to explore the current and ideal use of PROMs by healthcare professionals (HP) in specialized asthma management centers in French-speaking Belgium and (2) to understand under which conditions the use of PROMs contributes to patient engagement.

METHODS

We undertook a mixed-methods study with both anonymous online survey and in-person qualitative semi-structured interviews conducted with HPs to understand their perspectives on the routine use of PROMs. HPs were recruited from 16 asthma centers (French-speaking Belgium) identified via the Belgian Respiratory Society.

RESULTS

Of the 170 HPs identified from the 16 participating centers, 51 (30%) responded to the survey (n = 51) and 11 completed semi-structured interviews. 53% (27/51) of the surveyed HPs reported using PROMs primarily for asthma monitoring and clinical research while all reported that PROMs should primarily be used in practice to facilitate communication with the patient and to address neglected aspects of the care relationship such as the psychosocial aspects of the disease. The qualitative interviews revealed avenues for moving from a medical-centered and utilitarian use of PROMs to a use serving patient engagement. This would require HPs to go beyond their current representation of PROMs, to use instruments offering a more holistic image of the patient, to incorporate PROMs into a digital tool and to integrate PROMs in a patient education process.

CONCLUSION

The main findings of this study suggest relevant avenues for using PROMs in ways that support patient engagement.

A codevelopment process to advance methods for the use of patient-reported outcome measures and patient-reported experience measures with people who are homeless and experience chronic illness

Introduction

People who experience social disadvantage including homelessness suffer from numerous ill health effects when compared to the general public. Use of patient‐reported outcome measures (PROMs) and patient‐reported experience measures (PREMs) enables collection of information from the point of view of the person receiving care. Involvement in research and health care decision‐making, a process that can be facilitated by the use of PROMs and PREMs, is one way to promote equity in care.

Methods

This article reports on a codevelopment and consultation study investigating the use of PROMs and PREMs with people who experience homelessness and chronic illness. Data were analysed according to interpretative phenomenological analysis.

Results

Committee members with lived experience identified three themes for the role of PROMs and PREMs in health care measurement: trust and relationship‐building; health and quality of life; and equity, alongside specific recommendations for the design and administration of PROMs and PREMs. The codevelopment process is reported to demonstrate the meaningful investment in time, infrastructure and relationship‐building required for successful partnership between researchers and people with lived experience of homelessness.

Conclusion

PROMs and PREMs can be meaningful measurement tools for people who experience social disadvantage, but can be alienating or reproduce inequity if they fail to capture complexity or rely on hidden assumptions of key concepts.

Patient or Public Contribution

This study was conducted in active partnership between researchers and people with experience of homelessness and chronic illness, including priority setting for study design, data construction, analysis and coauthorship on this article.

A philosophical perspective on the development and application of patient-reported outcomes measures (PROMs)

Questionnaires are a common method in healthcare and clinical research to collect self-reported data on patients' behaviour and outcomes rather than the clinician's perspective. As a consequence there is a plethora of questionnaires and rating forms developed to measure a range of concepts such as health-related quality of life and health status. Given that these measures have been developed within a nomothetic paradigm to enhance our understanding of peoples self-perceived health status by translating complex personal feelings and experiences into a simple numeric score, the patient's illness narrative is lost along the way. This commentary discusses the limitations of the nomothetic approach as completion of a questionnaire is a social and contextually orientated activity and that their development is best viewed within the philosophical tradition of pragmatism, based on sound qualitative methods and rigorous psychometric testing. The commentary discusses the philosophical orientation underpinning PROM development and argues the case for a pragmatic epistemology based on a mixed methods research paradigm which goes beyond the current practice of informing the content validity of a PROM in the early phase of its development but to work towards developing a more composite and holistic picture through mixed methods in the interpretation of a patient's PROM score. Therefore, it is argued that the quality of data obtained will be enhanced but, also importantly and rightly places the participant at the centre of the research.

Patient-reported Outcome Measures - A Call for More Narrative Evidence

This article puts forward the need to reconsider the current underlying quantitative approach underpinning the application of patient reported outcomes, to a mixed methods approach through the tandem use of patients’ narrative that enables informants in addition to their scores to express the reality of the ways in which their lives are physically and mentally impacted by their health status.

A multi-methods and longitudinal study of patients' perceptions in injectable opioid agonist treatment: Implications for advancing patient-centered methodologies in substance use research

BACKGROUND

Patients' perceptions are vital to the delivery and evaluation of substance use treatment. They are most frequently collected at one time-point and measured using patient satisfaction questionnaires or qualitative methodologies. Interestingly, the findings of these studies often diverge, as satisfaction scores tend to be highly positive, while qualitative findings suggest dissatisfaction and areas for improvement. This divergence limits current understandings of patients' perceptions and their potential change over time in treatment.

OBJECTIVE

This study explores the relationship between open-ended positive and negative perceptions of treatment and patient satisfaction scores over time.

METHODS

The RUTH (Research on the Utilization of Therapeutic Hydromorphone) prospective cohort study included 131 participants receiving injectable diacetylmorphine or hydromorphone in Canada's first injectable opioid agonist treatment (iOAT) program. The study collected the Client Satisfaction Questionnaire (CSQ-8) at eight time-points over an 18-month period. Following a multi-methods approach, the study complemented the CSQ-8 with open-ended positive and negative comments of iOAT. The research team analyzed these comments thematically at each time-point to develop positive and negative perception themes. We then used growth curve modeling to explore the relationship between positive and negative perception themes and patient satisfaction over time.

FINDINGS

Over the eight time-points, six positive and eight negative perception themes emerged, broadly reflecting structural (e.g., expansion of iOAT), process (e.g., schedules), relational (e.g., interactions with providers), and outcome-related (e.g., met/unmet needs) perceptions of iOAT. On average, participants reported high satisfaction (grand mean = 29.2 out of 32), and scores did not significantly change over time. However, we did find significant unexplained variation within participants in their satisfaction trajectories and between participants in their initial satisfaction scores. In conditional growth curve models, the theme "unfavorable interactions with providers" had the strongest independent effect on overall satisfaction trajectories.

CONCLUSIONS

This study provides an example of how open-ended comments can be integrated with patient satisfaction questionnaire data to gather a comprehensive and patient-centered evaluation of substance use treatment. Considering the iOAT context specifically, relational dynamics and daily treatment access were significant predictors of patient satisfaction over time and may be attributes of iOAT that require further investigation.

The development of the PlexiQoL: A patient-reported outcome measure for adults with neurofibromatosis type 1-associated plexiform neurofibromas

Background

To develop and validate a patient‐reported outcome (PRO) measure of quality of life (QoL), specific to patients with Neurofibromatosis Type 1 (NF1)‐associated plexiform neurofibromas (pNFs), suitable for use in clinical efficacy trials. The study was conducted in parallel in the UK and US.

Methods

Qualitative interviews were conducted with patients to generate questionnaire content. Face and content validity of the draft scale was assessed by cognitive debriefing interviews (CDIs). A postal validation survey was conducted to identify the final version of the questionnaire (the PlexiQoL), establish its unidimensionality, and assess its psychometric properties.

Results

Thematic analysis was performed on 42 interview transcripts. Thirty‐one CDIs revealed that patients found the draft scale to be comprehensible, relevant, and easy to complete. The postal validation survey was completed by 273 patients. Rasch analysis identified an 18‐item unidimensional scale that showed excellent internal consistency, reproducibility, and sensitivity to differences in patient‐perceived pNF severity, general health, and the use of pain medication.

Conclusions

The PlexiQoL is the first disease‐specific PRO assessing the ability of adults with NF‐1 associated pNFs to meet their basic human needs. Clinical trials are planned to assess the sensitivity to change of the PlexiQoL in people undergoing treatment for pNFs.

Individualized and Standardized Outcome Measures: Further Arguments in Favor of Cohabitation

Ashworth et al. (Admin Policy Ment Health Ment Health Serv Res 46:425-428, 2019) recently published a paper in this journal making a cogent argument for the cohabitation of standardized and individualized outcome measures. In the present Point of View article, we consider additional arguments in favor of this cohabitation.

Acceptability of Patient-Reported Outcome and Experience Measures for Hepatitis C Treatment Among People Who Use Drugs

BACKGROUND

Growing international interest in mechanisms to capture and measure experiences and outcomes of health interventions from the patient perspective has led to the development of patient-reported measures (PRMs) across many areas of medicine. Although PRMs are now well utilised in some settings, the rapidly expanding area of direct-acting antiviral (DAA) treatments for hepatitis C has received remarkably little attention. In addition, questions are also being raised about the extent to which patients have been involved in the development of PRMs, which are primarily designed to reflect the patient perspective. In this context, the aim of this paper was to explore the possibility of developing a new PRM for use in hepatitis C DAA therapy that would also be acceptable to the patient group, in this case people who inject drugs (PWID).

METHOD

The study was based on a participatory design that included a peer researcher and foundational qualitative research including semi-structured interviews with 24 PWID with hepatitis C to inform the development of the PRMs. Stage 2 included four focus groups of six PWID with hepatitis C, who were asked to complete the draft measures and provide feedback.

RESULTS

Participants responded positively to the draft PRMs. The results indicate that participants' concerns during DAA treatment are often not sufficiently attended to in clinical settings. In the light of this finding, participants reported that PRMs have a positive role to play in the negotiation of patients' care during DAA treatment.

CONCLUSIONS

The findings show that utilising a participatory approach to the development of PRMs for DAA HCV treatment with PWID not only provides a way to measure experiences and outcomes of treatment from the patient perspective, but also provides a means for highly marginalised patient groups to have a say in and negotiate their care in ways that might not otherwise be possible.

Development and early validation of a patient-reported outcome measure to assess sleep amongst people experiencing problems with alcohol or other drugs

Study Objectives

To develop a patient-reported outcome measure to assess sleep amongst people experiencing problems with alcohol or other drugs.

Methods

Item development included secondary analyses of qualitative interviews with drug or alcohol users in residential treatment, a review of validated sleep measures, focus groups with drug or alcohol users in residential treatment, and feedback from drug or alcohol users recruited from community and residential settings. An initial version of the measure was completed by 549 current and former drug or alcohol users (442 in person and 107 online). Analyses comprised classical test theory methods, exploratory and confirmatory factor analysis, measurement invariance assessment, and item response theory (IRT).

Results

The initial measure (30 items) had good content and face validity and was named the Substance Use Sleep Scale (SUSS) by addiction service users. After seven items were removed due to low item-factor loadings, two factors were retained and labeled: "Mind and Body Sleep Problems" (14 items) and "Substance-Related Sleep Problems" (nine items). Measurement invariance was confirmed with respect to gender, age, and administration format. IRT (information) and classical test theory (internal consistency and stability) indicated measure reliability. Standard parametric and nonparametric techniques supported convergent and discriminant validity.

Conclusions

SUSS is an easy-to-complete patient-reported outcome measure of sleep for people with drug or alcohol problems. It can be used by those concerned about their own sleep, and by treatment providers and researchers seeking to better understand, assess, and potentially treat sleep difficulties amongst this population. Further validity testing with larger and more diverse samples is now required.

How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis

BACKGROUND

In this paper, we report the findings of a realist synthesis that aimed to understand how and in what circumstances patient reported outcome measures (PROMs) support patient-clinician communication and subsequent care processes and outcomes in clinical care. We tested two overarching programme theories: (1) PROMs completion prompts a process of self-reflection and supports patients to raise issues with clinicians and (2) PROMs scores raise clinicians' awareness of patients' problems and prompts discussion and action. We examined how the structure of the PROM and care context shaped the ways in which PROMs support clinician-patient communication and subsequent care processes.

RESULTS

PROMs completion prompts patients to reflect on their health and gives them permission to raise issues with clinicians. However, clinicians found standardised PROMs completion during patient assessments sometimes constrained rather than supported communication. In response, clinicians adapted their use of PROMs to render them compatible with the ongoing management of patient relationships. Individualised PROMs supported dialogue by enabling the patient to tell their story. In oncology, PROMs completion outside of the consultation enabled clinicians to identify problematic symptoms when the PROM acted as a substitute rather than addition to the clinical encounter and when the PROM focused on symptoms and side effects, rather than health related quality of life (HRQoL). Patients did not always feel it was appropriate to discuss emotional, functional or HRQoL issues with doctors and doctors did not perceive this was within their remit.

CONCLUSIONS

This paper makes two important contributions to the literature. First, our findings show that PROMs completion is not a neutral act of information retrieval but can change how patients think about their condition. Second, our findings reveal that the ways in which clinicians use PROMs is shaped by their relationships with patients and professional roles and boundaries. Future research should examine how PROMs completion and feedback shapes and is influenced by the process of building relationships with patients, rather than just their impact on information exchange and decision making.

Substance use, sleep and intervention design: insights from qualitative data

Background: Alcohol and other drug use is associated with poor sleep quality and quantity, but there is limited qualitative research exploring substance users' experiences of sleep and few psychosocial sleep interventions for them. Aim: To inform the development of psychosocial interventions to improve sleep amongst people reporting drug/alcohol problems. Method: Qualitative data were collected during a sleep survey. Of the 549 drug/alcohol users completing the survey, 188 (34%) provided additional information about their sleep using a free text box. Responses were analysed via Iterative Categorisation. Findings were reviewed with reference to the Behaviour Change Wheel (BCW). Results: All data were categorised inductively under five headings: (i) sleep quality; (ii) nature of sleep problems; (iii) sleep and substances; (iv) factors improving sleep quality; (v) factors undermining sleep quality. Substance use undermined sleep, but poor sleep often persisted after substance use had ceased. Sleep problems were diverse; as were the causes of, and strategies for dealing with, those problems. Causes and strategies had biological, psychological, social and environmental roots. Conclusions: The BCW facilitated the identification of intervention components that might improve the sleep of people who use substances. These components relate to education, training, enablement, modelling, service provision, guidelines and environment.

A note on contemporary psychometrics

Psychometrics provide the mathematical underpinnings for psychological assessment. From the late 19th century, a plethora of methodological research achievements equipped researchers and clinicians with efficient tools whose practical value becomes more evident in the era of the internet and big data. Nowadays, powerful probabilistic models exist for most types of data and research questions. As the usability of the psychometric scales is better comprehended, there is an increased interest in applied research outcomes. Paradoxically, while the interest in applications for psychometric scales increases, publishing research on the development and/or evaluation of those scales per se, is not welcomed by many relevant journals. This special issue in psychometrics is therefore a great opportunity to briefly review the main ideas and methods used in psychometrics, and to discuss the challenges in contemporary applied psychometrics.

Disturbing sleep and sleepfulness during recovery from substance dependence in residential rehabilitation settings

There is evidence that poor sleep mitigates recovery from substance dependence and increases risk of relapse. However, to date research literature is located within biomedical, clinical and psychological paradigms. To complement the extant work, this article offers a sociological exploration of sleep in the context of recovery from dependence on alcohol and/or other drugs. Drawing on qualitative data generated through interviews with 28 men and women living in residential rehabilitation settings in England, we provide a detailed exploration of sleep practices focusing on how these are enacted throughout the night. We offer the concept of 'sleepfulness' to suggest that sleep should not be understood simply as being other than awake; rather it involves a myriad of associations between diverse actants - human and non-human - that come to 'fill up', enable and assemble sleep. Together these empirical insights and conceptualisations disturb the ontology of sleep and point to the fulsome dimensions of the category.

DEVELOPMENT AND VALIDATION OF 'SURE': A PATIENT REPORTED OUTCOME MEASURE (PROM) FOR RECOVERY FROM DRUG AND ALCOHOL DEPENDENCE

BACKGROUND

Patient Reported Outcome Measures (PROMs) assess health status and health-related quality of life from the patient/service user perspective. Our study aimed to: i. develop a PROM for recovery from drug and alcohol dependence that has good face and content validity, acceptability and usability for people in recovery; ii. evaluate the psychometric properties and factorial structure of the new PROM ('SURE').

METHODS

Item development included Delphi groups, focus groups, and service user feedback on draft versions of the new measure. A 30-item beta version was completed by 575 service users (461 in person [IP] and 114 online [OL]). Analyses comprised rating scale evaluation, assessment of psychometric properties, factorial structure, and differential item functioning.

RESULTS

The beta measure had good face and content validity. Nine items were removed due to low stability, low factor loading, low construct validity or high complexity. The remaining 21 items were re-scaled (Rasch model analyses). Exploratory and confirmatory factor analyses revealed 5 factors: substance use, material resources, outlook on life, self-care, and relationships. The MIMIC model indicated 95% metric invariance across the IP and OL samples, and 100% metric invariance for gender. Internal consistency and test-retest reliability were granted. The 5 factors correlated positively with the corresponding WHOQOL-BREF and ARC subscales and score differences between participant sub-groups confirmed discriminative validity.

CONCLUSION

'SURE' is a psychometrically valid, quick and easy-to-complete outcome measure, developed with unprecedented input from people in recovery. It can be used alongside, or instead of, existing outcome tools.