A theoretical framework for patient-reported outcome measures

Story completion (SC) - an epistemological approach to the interpretation of patient-reported outcomes?

INTRODUCTION

Patient-reported outcome measures (PROMs) quantitative data are increasingly accompanied by qualitative narrative data, which does not always provide the insights required to inform healthcare.

AREAS COVERED

This perspective considers how story completion (SC) may be used to gain a deeper insight on people's experiences and guide interpretation of the meaning of PROM responses.

EXPERT OPINION

Story completion (SC) is an epistemological approach involving participants telling a story in response to a pre-determined 'stem' they have been presented with.

Relating person-centredness to quality-of-life assessments and patient-reported outcomes in healthcare: A critical theoretical discussion

Engagement with the historical and theoretical underpinnings of measuring quality of life (QoL) and patient‐reported outcomes (PROs) in healthcare is important. Ideas and values that shape such practices—and in the endgame, people's lives—might otherwise remain unexamined, be taken for granted or even essentialized. Our aim is to explicate and theoretically discuss the philosophical tenets underlying the practices of QoL assessment and PRO measurement in relation to the notion of person‐centredness. First, we engage with the late‐modern history of the concept of QoL and the act of assessing and measuring it. Working with the historical method of genealogy, we describe the development of both QoL assessments and PRO measures (PROMs) within healthcare by accounting for the contextual conditions for their possibility. In this way, the historical and philosophical underpinnings of these measurement practices are highlighted. We move on to analyse theoretical and philosophical underpinnings regarding the use of PROMs and QoL assessments in clinical practice, as demonstrated in review studies thereof. Finally, we offer a critical analysis regarding the state of theory in the literature and conclude that, although improved person‐centredness is an implied driver of QoL assessments and PROMs in clinical practice, enhanced theoretical underpinning of the development of QoL assessments is called for.

A philosophical perspective on the development and application of patient-reported outcomes measures (PROMs)

Questionnaires are a common method in healthcare and clinical research to collect self-reported data on patients' behaviour and outcomes rather than the clinician's perspective. As a consequence there is a plethora of questionnaires and rating forms developed to measure a range of concepts such as health-related quality of life and health status. Given that these measures have been developed within a nomothetic paradigm to enhance our understanding of peoples self-perceived health status by translating complex personal feelings and experiences into a simple numeric score, the patient's illness narrative is lost along the way. This commentary discusses the limitations of the nomothetic approach as completion of a questionnaire is a social and contextually orientated activity and that their development is best viewed within the philosophical tradition of pragmatism, based on sound qualitative methods and rigorous psychometric testing. The commentary discusses the philosophical orientation underpinning PROM development and argues the case for a pragmatic epistemology based on a mixed methods research paradigm which goes beyond the current practice of informing the content validity of a PROM in the early phase of its development but to work towards developing a more composite and holistic picture through mixed methods in the interpretation of a patient's PROM score. Therefore, it is argued that the quality of data obtained will be enhanced but, also importantly and rightly places the participant at the centre of the research.

Towards a pragmatist dealing with algorithmic bias in medical machine learning

Machine Learning (ML) is on the rise in medicine, promising improved diagnostic, therapeutic and prognostic clinical tools. While these technological innovations are bound to transform health care, they also bring new ethical concerns to the forefront. One particularly elusive challenge regards discriminatory algorithmic judgements based on biases inherent in the training data. A common line of reasoning distinguishes between justified differential treatments that mirror true disparities between socially salient groups, and unjustified biases which do not, leading to misdiagnosis and erroneous treatment. In the curation of training data this strategy runs into severe problems though, since distinguishing between the two can be next to impossible. We thus plead for a pragmatist dealing with algorithmic bias in healthcare environments. By recurring to a recent reformulation of William James's pragmatist understanding of truth, we recommend that, instead of aiming at a supposedly objective truth, outcome-based therapeutic usefulness should serve as the guiding principle for assessing ML applications in medicine.

Validity arguments for patient-reported outcomes: justifying the intended interpretation and use of data

BACKGROUND

Contrary to common usage in the health sciences, the term "valid" refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane's argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health.

METHODS

The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane's five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study.

RESULTS

The general HLQ interpretive argument was built to demonstrate Kane's five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument.

CONCLUSIONS

Kane's five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.

Constructing arguments for the interpretation and use of patient-reported outcome measures in research: an application of modern validity theory

The past 100 years have witnessed an evolution of the meaning of validity and validation within the fields of education and psychology. Validity was once viewed as a property of tests and scales, but is now viewed as the extent to which theory and evidence support proposed interpretations and uses of test scores. Uncertainty about what types of validity evidence were needed motivated the current "argument-based" approach, as reflected in the 2014 Standards for Educational and Psychological Testing. According to this approach, investigators should delineate the assumptions required in order for a proposed interpretation or use to be plausible and then seek evidence that supports or refutes those assumptions. Though validation practices within the field of patient-reported outcome measurement have implicitly included many elements of the argument-based approach, the approach has yet to be explicitly adopted. To facilitate adoption, this article proposes an initial set of assumptions that might be included in most arguments for research-related interpretations and uses of scores from patient-reported outcome measures. The article also includes brief descriptions of the types of evidence that would be best suited for evaluating each assumption. It is hoped that these generic assumptions will stimulate further discussion and debate among quality of life researchers regarding how best to adopt modern validity theory to patient-reported outcome measures.

Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments

OBJECTIVE

Validity refers to the extent to which evidence and theory support the adequacy and appropriateness of inferences based on score interpretations. The health sector is lacking a theoretically-driven framework for the development, testing and use of health assessments. This study used the Standards for Educational and Psychological Testing framework of five sources of validity evidence to assess the types of evidence reported for health literacy assessments, and to identify studies that referred to a theoretical validity testing framework.

METHODS

A systematic descriptive literature review investigated methods and results in health literacy assessment development, application and validity testing studies. Electronic searches were conducted in EBSCOhost, Embase, Open Access Theses and Dissertations and ProQuest Dissertations. Data were coded to the Standards' five sources of validity evidence, and for reference to a validity testing framework.

RESULTS

Coding on 46 studies resulted in 195 instances of validity evidence across the five sources. Only nine studies directly or indirectly referenced a validity testing framework. Evidence based on relations to other variables is most frequently reported.

CONCLUSIONS

The health and health equity of individuals and populations are increasingly dependent on decisions based on data collected through health assessments. An evidence-based theoretical framework provides structure and coherence to existing evidence and stipulates where further evidence is required to evaluate the extent to which data are valid for an intended purpose. This review demonstrates the use of the Standards' theoretical validity testing framework to evaluate sources of evidence reported for health literacy assessments. Findings indicate that theoretical validity testing frameworks are rarely used to collate and evaluate evidence in validation practice for health literacy assessments. Use of the Standards' theoretical validity testing framework would improve evaluation of the evidence for inferences derived from health assessment data on which public health and health equity decisions are based.

Viewing assessments of patient-reported heath status as conversations: Implications for developing and evaluating patient-reported outcome measures

Patient-reported outcome measures (PROMs) are frequently used in research to reflect the patient's perspective. In this commentary, I argue that further improvements can be made in how we develop and evaluate PROMs by viewing assessment as a type of conversation. Philosophically speaking, a PROM assessment can be conceptualized as a formal conversation that serves as a model of an informal, longer, and more nuanced conversation with a research participant about their health experience. Psychologically speaking, evidence from research in survey methodology and discursive psychology shows that respondents to self-report measures behave in ways consistent with the idea that they are doing their best to participate in a conversation, albeit an unusual one. Several suggestions are offered for creating a better conversational context through study materials and PROM instructions, and by improving the yield of cognitive interviews. It is hoped that this commentary can stimulate further discussions in our field regarding how to integrate insights about the conversational nature of assessment from survey research and discursive psychology to better reflect the patient's voice in research.

Validity Beyond Measurement: Why Psychometric Validity Is Insufficient for Valid Psychotherapy Research

In psychotherapy research, "validity" is canonically understood as the capacity of a test to measure what is purported to measure. However, we argue that this psychometric understanding of validity prohibits working researchers from considering the validity of their research. Psychotherapy researchers often use measures with a different epistemic goal than test developers intended, for example when a depression symptom measure is used to indicate "treatment success" (cf. outcome measurement for evidence-based treatment). However, the validity of a measure does not cover the validity of its use as operationalization of another target concept within a research procedure, nor the validity of its function toward an epistemic goal. In this paper, we discuss the importance of considering validity of the epistemic process beyond the validity of measures per se, based on an empirical case example from our psychotherapy study ("SCS", Cornelis et al., 2017). We discuss why the psychometric understanding of validity is insufficient in covering epistemic validity, and we evaluate to what extent the available terminology regarding validity of research is sufficient for working researchers to accurately consider the validity of their overall epistemic process. As psychotherapy research is meant to offer a sound evidence-base for clinical practice, we argue that it is vital that psychotherapy researchers are able to discuss the validity of the epistemic choices made to serve the clinical goal.

How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis

BACKGROUND

In this paper, we report the findings of a realist synthesis that aimed to understand how and in what circumstances patient reported outcome measures (PROMs) support patient-clinician communication and subsequent care processes and outcomes in clinical care. We tested two overarching programme theories: (1) PROMs completion prompts a process of self-reflection and supports patients to raise issues with clinicians and (2) PROMs scores raise clinicians' awareness of patients' problems and prompts discussion and action. We examined how the structure of the PROM and care context shaped the ways in which PROMs support clinician-patient communication and subsequent care processes.

RESULTS

PROMs completion prompts patients to reflect on their health and gives them permission to raise issues with clinicians. However, clinicians found standardised PROMs completion during patient assessments sometimes constrained rather than supported communication. In response, clinicians adapted their use of PROMs to render them compatible with the ongoing management of patient relationships. Individualised PROMs supported dialogue by enabling the patient to tell their story. In oncology, PROMs completion outside of the consultation enabled clinicians to identify problematic symptoms when the PROM acted as a substitute rather than addition to the clinical encounter and when the PROM focused on symptoms and side effects, rather than health related quality of life (HRQoL). Patients did not always feel it was appropriate to discuss emotional, functional or HRQoL issues with doctors and doctors did not perceive this was within their remit.

CONCLUSIONS

This paper makes two important contributions to the literature. First, our findings show that PROMs completion is not a neutral act of information retrieval but can change how patients think about their condition. Second, our findings reveal that the ways in which clinicians use PROMs is shaped by their relationships with patients and professional roles and boundaries. Future research should examine how PROMs completion and feedback shapes and is influenced by the process of building relationships with patients, rather than just their impact on information exchange and decision making.

Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity

BACKGROUND

Data from subjective patient-reported outcome measures (PROMs) are now being used in the health sector to make or support decisions about individuals, groups and populations. Contemporary validity theorists define validity not as a statistical property of the test but as the extent to which empirical evidence supports the interpretation of test scores for an intended use. However, validity testing theory and methodology are rarely evident in the PROM validation literature. Application of this theory and methodology would provide structure for comprehensive validation planning to support improved PROM development and sound arguments for the validity of PROM score interpretation and use in each new context.

OBJECTIVE

This paper proposes the application of contemporary validity theory and methodology to PROM validity testing.

ILLUSTRATIVE EXAMPLE

The validity testing principles will be applied to a hypothetical case study with a focus on the interpretation and use of scores from a translated PROM that measures health literacy (the Health Literacy Questionnaire or HLQ).

DISCUSSION

Although robust psychometric properties of a PROM are a pre-condition to its use, a PROM's validity lies in the sound argument that a network of empirical evidence supports the intended interpretation and use of PROM scores for decision making in a particular context. The health sector is yet to apply contemporary theory and methodology to PROM development and validation. The theoretical and methodological processes in this paper are offered as an advancement of the theory and practice of PROM validity testing in the health sector.

Semantic primes theory may be helpful in designing questionnaires such as to prevent response shift

OBJECTIVES

The purpose of randomized control trials (RCTs) can be the assessment of the direct effect of treatment on health-related quality of life (HRQL). Response shift (RS) theory considers that a change in HRQL scores observed over time cannot be explained solely by a direct effect of a medical condition, it may also result from a change in the way people appraise their HRQL. The RS effect is a potential bias that is liable to compromise efficient assessment of the effect of treatment on HRQL.

STUDY DESIGN AND SETTING

We hypothesize a link between the RS effect on HRQL scores and the level of complexity of HRQL conceptualization.

RESULTS

We discuss how the impact of reconceptualization on scores depends on the complexity of the linguistic definition of a subjective construct and how for reprioritization the impact depends on the dimensionality. The linguistic theory of semantic primes is used to help identify how subjective constructs can be classified according to the complexity of their definitions.

CONCLUSION

Finally, we suggest that the impact of the RS effect on HRQL scores could be avoided (or lessened) if questionnaires were designed with a rule of "the least semantic and psychometric complexity" in mind.

A qualitative inquiry of patient-reported outcomes: the case of lower urinary tract symptoms

BACKGROUND

Patient-reported outcomes are a valuable tool for assessing healthcare, particularly for symptom-based conditions that lack definitive physiological measures of treatment efficacy.

OBJECTIVE

To explore the value of qualitative methods for understanding and developing patient-reported outcomes of medical care for symptom-based conditions by examining the case of lower urinary tract symptoms.

METHODS

Semistructured interviews were conducted with a diverse community sample of 90 respondents who had spoken with a provider about their urinary symptoms. Content and thematic analyses were conducted for the areas of symptom relief, patient adherence, and satisfaction with care according to gender, race or ethnicity, and socioeconomic status.

RESULTS

Across social groups, most patients experienced either no symptom relief or partial relief, reported that they adhered to recommendations, and were satisfied with the care received. The primary reason for no symptom relief was not receiving a treatment recommendation. For patients, even partial relief made symptoms more manageable both physically and emotionally. Satisfaction with care was mediated by the quality of the patient-provider relationship as well as expectations other than symptom relief, particularly for patients of low socioeconomic status.

DISCUSSION

Patients' assessments of the outcomes of seeking medical care for this symptom-based condition broadened the criteria for quality of care beyond providing a cure. For healthcare providers, this can widen the path for meeting patient needs, even without complete symptom relief. For providers and researchers, as the evidence base expands to include patient reports, the context provided by a qualitative approach can enhance understanding of patients' perspectives and the ability to construct meaningful quantitative measures.

Quality of life is a process not an outcome

Quality improvement mechanisms increasingly use outcome measures to evaluate health care providers. This move toward outcome measures is a radical departure from the traditional focus on process measures. More radical still is the proposal to shift from relatively simple and proximal measures of outcome, such as mortality, to complex outcomes, such as quality of life. While the practical, scientific, and ethical issues associated with the use of outcomes such as mortality and morbidity to compare health care providers have been well rehearsed, the specific concerns associated with the use of quality of life measures in quality of care research have received little attention. As with much research on quality of life there is a tendency to assume that the disadvantages are outweighed by the general virtue of "listening" to patients. In this paper we disagree with this assumption and argue that quality of life is a process, not an outcome.

Felicitometric hermeneutics: interpreting quality of life measurements

The use of quality of life (QOL) outcomes in clinical trials is increasing as a number of practical, ethical, methodological, and regulatory reasons for their use have become apparent. It is important, then, that QOL measurements and differences between QOL scores be readily interpretable. We study interpretation in two contexts: when determining QOL and when basing decisions on QOL differences. We consider both clinical situations involving individual patients and research contexts, e.g., randomized clinical trials, involving groups of patients. We note the ethical importance of such understanding: proper interpretation and communication facilitate health care decision making. Communication that facilitates interpretation is of moral significance since better communication can attenuate ethical problems and inform choices. Much of what is communication worthy about QOL assessments is determined by the particular QOL instrument used in the assessment and how it is administered. In practice, these choices will be driven by the purpose of the assessment, but, it is argued, to maximize understanding, we should combine the information garnered from traditional standardized QOL instruments, from individualized QOL assessments, and from a recently proposed dialogic paradigm, where QOL is determined by shared conversation regarding the interpretation of texts. And, while some studies can surely succeed using abbreviated methods of administration (e.g., postal surveys may suffice for certain purposes), we will focus on methods of administration involving interviewer-respondent interaction. We suggest that during the QOL elicitation process, interviewer and respondent should engage in a two-way conversation in order to achieve a shared understanding of the "answers" to QOL "questions" and, finally, to reach a shared interpretation of the individual's QOL.

Interpretability, validity, and the minimum important difference

Patient-reported outcomes are increasingly used as dependent variables in studies regarding the effectiveness of clinical interventions. But patient-reported outcome measures (PROMs) do not provide intuitively meaningful data. For instance, it is not clear what a five point increase or decrease on a particular scale signifies. Establishing 'interpretability' involves making changes in outcomes meaningful. Attempts to interpret PROMs have led to the development of methods for identifying a minimum important difference (MID). In this paper, however, I draw on Charles Taylor's distinction between weak and strong evaluations to suggest that identifying a MID, specifically, a MID that uses a patient-reported reference group, may not provide an adequate interpretation of these measures. Moreover, I argue that the difficulty with interpreting these measures is tied to a larger problem concerning their validity. If researchers wish to interpret PROMs, they may first need to know more about the constructs they attempt to measure, namely, quality of life.

Health policy, patient-centred care and clinical ethics

RATIONALE, AIMS AND OBJECTIVES

Patient-centred care has been a central part of US and UK health policy for over a decade, but, despite its importance, the policy literature often fails to provide an adequate theoretical justification for why and how we should value it. This omission is problematic because it renders the status, content and appropriate evaluation of patient-centredness unclear. In this paper we aim to examine two different accounts of patient-centred care.

METHOD

We draw upon methods of conceptual and ethical analysis.

RESULTS

We argue that neither of the two accounts of patient-centred care identified appropriately grounds patient-centredness because neither of them takes into account the inherently moral nature of terms such as 'respect' and 'dignity', terms that are central to discussions of patient-centred care.

CONCLUSIONS

We suggest that clinical ethics can help to provide a theoretical justification for patient-centred care, and that clinical ethical practices can further patient-centred initiatives through ethics consultation, education and policy development and review.

Choosing a patient-reported outcome measure

There has been much philosophical interest regarding the 'hierarchy of evidence' used to determine which study designs are of most value for reporting on questions of effectiveness, prognosis, and so on. There has been much less philosophical interest in the choice of outcome measures with which the results of, say, an RCT or a cohort study are presented. In this paper, we examine the FDA's recently published guidelines for assessing the psychometric adequacy of patient-reported outcome measures. We focus on their recommendations for demonstrating content validity and also for how researchers should weigh up the sum of psychometric evidence when choosing these measures. We argue that questions regarding judgment and understanding meaning of these measures should play a more central role in determining their adequacy.