Developing a self-report outcome measure for complementary and alternative medicine

Identification, description and appraisal of generic PROMs for primary care: a systematic review

Background

Patients attend primary care with many types of problems and to achieve a range of possible outcomes. There is currently a lack of patient-reported outcome measures (PROMs) designed to capture these diverse outcomes. The objective of this systematic review was to identify, describe and appraise generic PROMs suitable for measuring outcomes from primary care.

Methods

We carried out a systematic Medline search, supplemented by other online and hand-searches. All potentially relevant PROMs were itemised in a long-list. Each PROM in the long-list which met inclusion criteria was included in a short-list. Short-listed PROMs were then described in terms of their measurement properties and construct, based on a previously published description of primary care outcome as three constructs: health status, health empowerment and health perceptions. PROMs were appraised in terms of extent of psychometric testing (extensive, moderate, low) and level of responsiveness (high, medium, low, unknown).

Results

More than 5000 abstracts were identified and screened to identify PROMs potentially suitable for measuring outcomes from primary care. 321 PROMs were long-listed, and twenty PROMs were catalogued in detail. There were five PROMs which measured change directly, without need for a baseline. Although these had less strong psychometric properties, they may be more responsive to change than PROMs which capture status at a point in time. No instruments provided coverage of all three constructs. Of the health status questionnaires, the most extensively tested was the SF-36. Of the health empowerment instruments, the PEI, PAM and heiQ provided the best combination of responsiveness and psychometric testing. The health perceptions instruments were all less responsive to change, and may measure a form of health perception which is difficult to shift in primary care.

Conclusions

This systematic review is the first of its kind to identify papers describing the development and validation of generic PROMs suitable for measuring outcomes from primary care. It identified that: 1) to date, there is no instrument which comprehensively covers the outcomes commonly sought in primary care, and 2) there are different benefits both to PROMs which measure status at a point in time, and PROMs which measure change directly.

Electronic supplementary material

The online version of this article (10.1186/s12875-018-0722-9) contains supplementary material, which is available to authorized users.

Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: refining content validity through cognitive interviews

Background

Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.

Methods

We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.

Results

The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.

Conclusions

We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format

BACKGROUND

Patients receiving complementary and alternative medicine (CAM) therapies often report shifts in well-being that go beyond resolution of the original presenting symptoms. We undertook a research program to develop and evaluate a patient-centered outcome measure to assess the multidimensional impacts of CAM therapies, utilizing a novel mixed methods approach that relied upon techniques from the fields of anthropology and psychometrics. This tool would have broad applicability, both for CAM practitioners to measure shifts in patients' states following treatments, and conventional clinical trial researchers needing validated outcome measures. The US Food and Drug Administration has highlighted the importance of valid and reliable measurement of patient-reported outcomes in the evaluation of conventional medical products. Here we describe Phase I of our research program, the iterative process of content identification, item development and refinement, and response format selection. Cognitive interviews and psychometric evaluation are reported separately.

METHODS

From a database of patient interviews (n = 177) from six diverse CAM studies, 150 interviews were identified for secondary analysis in which individuals spontaneously discussed unexpected changes associated with CAM. Using ATLAS.ti, we identified common themes and language to inform questionnaire item content and wording. Respondents' language was often richly textured, but item development required a stripping down of language to extract essential meaning and minimize potential comprehension barriers across populations. Through an evocative card sort interview process, we identified those items most widely applicable and covering standard psychometric domains. We developed, pilot-tested, and refined the format, yielding a questionnaire for cognitive interviews and psychometric evaluation.

RESULTS

The resulting questionnaire contained 18 items, in visual analog scale format, in which each line was anchored by the positive and negative extremes relevant to the experiential domain. Because of frequent informant allusions to response set shifts from before to after CAM therapies, we chose a retrospective pretest format. Items cover physical, emotional, cognitive, social, spiritual, and whole person domains.

CONCLUSIONS

This paper reports the success of a novel approach to the development of outcome instruments, in which items are extracted from patients' words instead of being distilled from pre-existing theory. The resulting instrument, focused on measuring shifts in patients' perceptions of health and well-being along pre-specified axes, is undergoing continued testing, and is available for use by cooperating investigators.

Acupuncture and chiropractic care for chronic pain in an integrated health plan: a mixed methods study

BACKGROUND

Substantial recent research examines the efficacy of many types of complementary and alternative (CAM) therapies. However, outcomes associated with the "real-world" use of CAM has been largely overlooked, despite calls for CAM therapies to be studied in the manner in which they are practiced. Americans seek CAM treatments far more often for chronic musculoskeletal pain (CMP) than for any other condition. Among CAM treatments for CMP, acupuncture and chiropractic (A/C) care are among those with the highest acceptance by physician groups and the best evidence to support their use. Further, recent alarming increases in delivery of opioid treatment and surgical interventions for chronic pain--despite their high costs, potential adverse effects, and modest efficacy--suggests the need to evaluate real world outcomes associated with promising non-pharmacological/non-surgical CAM treatments for CMP, which are often well accepted by patients and increasingly used in the community.

METHODS/DESIGN

This multi-phase, mixed methods study will: (1) conduct a retrospective study using information from electronic medical records (EMRs) of a large HMO to identify unique clusters of patients with CMP (e.g., those with differing demographics, histories of pain condition, use of allopathic and CAM health services, and comorbidity profiles) that may be associated with different propensities for A/C utilization and/or differential outcomes associated with such care; (2) use qualitative interviews to explore allopathic providers' recommendations for A/C and patients' decisions to pursue and retain CAM care; and (3) prospectively evaluate health services/costs and broader clinical and functional outcomes associated with the receipt of A/C relative to carefully matched comparison participants receiving traditional CMP services. Sensitivity analyses will compare methods relying solely on EMR-derived data versus analyses supplementing EMR data with conventionally collected patient and clinician data.

DISCUSSION

Successful completion of these aggregate aims will provide an evaluation of outcomes associated with the real-world use of A/C services. The trio of retrospective, qualitative, and prospective study will also provide a clearer understanding of the decision-making processes behind the use of A/C for CMP and a transportable methodology that can be applied to other health care settings, CAM treatments, and clinical populations.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT01345409.

Pilot validation of a self-report outcome measure of complementary and alternative medicine

BACKGROUND

We previously developed a multidomain, self-report outcome measure relevant to a wide range of complementary and alternative medical (CAM) therapies. We report the results of a pilot study to validate the measure.

METHODS

Fifty-two patients (60% female, mean age, 53 years) seeking services from CAM providers completed a battery of measures prior to a scheduled visit. The battery included the Positive and Negative Affect Schedule, items from the Brief Pain and Fatigue Inventories, ratings of global quality of life (QOL) and change in physical health, and our pilot measure consisting of six hypothesized subscales (pain, fatigue, physical/functional ability, personal control, existential issues, and general QOL). Internal reliability, item convergence and discrimination, construct, and concurrent validity were assessed.

RESULTS

Pilot measure domains appeared to be internally reliable with five of six alpha coefficients exceeding 0.70. Multitrait scaling analyses demonstrated that most items converged on the domains that they were hypothesized to represent, although item discrimination was demonstrated for the pain domain only. Correlations with standardized assessments of pain, fatigue, and global QOL showed evidence of construct validity. All domain scores differentiated patients classified high versus low in global QOL (P < .01) and four (pain, fatigue, physical/functional ability, and general QOL) differentiated patients who perceived recent changes in their physical health (improvement vs decline; P < .05).

CONCLUSION

These results provide preliminary psychometric data for a pilot measure of CAM outcomes, though further testing is warranted.

The process of whole person healing: "unstuckness" and beyond

OBJECTIVE

The purpose of the present study was to fully explore the descriptions of patients' experiences of change after receipt of whole systems of complementary and alternative medicine (CAM) treatment. The aim was to develop an understanding of "unstuckness," including characterization of states, processes, and modifying factors.

DESIGN

This was a secondary descriptive qualitative analysis, using techniques borrowed from phenomenology and grounded theory.

SETTING/LOCATION

Three existent datasets collected at two different universities in the United States and Canada were used in the secondary analysis.

PARTICIPANTS

Patients with chronic illnesses (including cancer and multiple nonmalignant conditions) who were treated with different packages of care were interviewed for the primary three studies (n = 76 with over 150 interview sessions). Complete data sets from these participants were used in this secondary analysis. OUTCOME MEASURES/DATA COLLECTION TECHNIQUES: Original transcripts were coded asking specific research questions about the experience of change subsequent to whole systems treatments.

RESULTS

Data clearly indicated experiential differences between stuckness, unsticking, and unstuckness. Descriptors and characteristics of each state were identified, as was an initial grounded theory of change or transformation that occurs as an outcome of whole medical systems of CAM.

CONCLUSIONS

The results provide preliminary conceptualizations and descriptions of the impact that CAM whole systems interventions may have on the individual' s life courses. This constitutes a first step in the identification, measurement, and evaluation of whole systems outcomes in a clinical setting. The emerging conceptualization of the process from stuckness to transformation may also provide a link between clinical research and systems science theory.