Pilot validation of a self-report outcome measure of complementary and alternative medicine

Identification, description and appraisal of generic PROMs for primary care: a systematic review

Background

Patients attend primary care with many types of problems and to achieve a range of possible outcomes. There is currently a lack of patient-reported outcome measures (PROMs) designed to capture these diverse outcomes. The objective of this systematic review was to identify, describe and appraise generic PROMs suitable for measuring outcomes from primary care.

Methods

We carried out a systematic Medline search, supplemented by other online and hand-searches. All potentially relevant PROMs were itemised in a long-list. Each PROM in the long-list which met inclusion criteria was included in a short-list. Short-listed PROMs were then described in terms of their measurement properties and construct, based on a previously published description of primary care outcome as three constructs: health status, health empowerment and health perceptions. PROMs were appraised in terms of extent of psychometric testing (extensive, moderate, low) and level of responsiveness (high, medium, low, unknown).

Results

More than 5000 abstracts were identified and screened to identify PROMs potentially suitable for measuring outcomes from primary care. 321 PROMs were long-listed, and twenty PROMs were catalogued in detail. There were five PROMs which measured change directly, without need for a baseline. Although these had less strong psychometric properties, they may be more responsive to change than PROMs which capture status at a point in time. No instruments provided coverage of all three constructs. Of the health status questionnaires, the most extensively tested was the SF-36. Of the health empowerment instruments, the PEI, PAM and heiQ provided the best combination of responsiveness and psychometric testing. The health perceptions instruments were all less responsive to change, and may measure a form of health perception which is difficult to shift in primary care.

Conclusions

This systematic review is the first of its kind to identify papers describing the development and validation of generic PROMs suitable for measuring outcomes from primary care. It identified that: 1) to date, there is no instrument which comprehensively covers the outcomes commonly sought in primary care, and 2) there are different benefits both to PROMs which measure status at a point in time, and PROMs which measure change directly.

Electronic supplementary material

The online version of this article (10.1186/s12875-018-0722-9) contains supplementary material, which is available to authorized users.

The healing experience-how can we capture it?

BACKGROUND

The Integrative Medicine (I-MED) Index, designed to be a universal instrument to assess the 5 levels of healing (physical, energetic, emotional, subconscious, spiritual) among individuals who use complementary therapies, is based on the hypothesis of healing (HH), which considers healing as a dynamic process towards wholeness generally following the same steps. When we pilot-tested the instrument, it became clear that the way participants experience healing in their lives was not as the HH suggested. This led us to ask how patients experience healing and what outcomes they identify as key to their journey.

METHODS

In-depth, semistructured interviews were conducted with 35 individuals with previous healing experiences at 2 wellness centers in Vancouver, BC. Qualitative content analysis and thematic coding were used to analyze the data.

ANALYSIS AND RESULTS

Four themes emerged from participants' stories; they suggested that healing is (1) a personal and subjective experience, (2) a return to wholeness, (3) self-directed and requiring positive intention and (4) experienced in varying degrees. Symptom resolution, goal attainment, changes in social support, mental outlook, and the subtle but significant changes like engaging in life differently were important healing outcomes. Contrary to the assumptions of the HH, healing is an individualized process that does not follow a uniform pattern.

CONCLUSIONS

A lack of consensus on the key concepts of healing has hindered efforts to evaluate the effects of complementary therapies and outcome. The results from this study provide the basis for an instrument to assess individuals' healing experiences in a more dynamic manner.

Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format

BACKGROUND

Patients receiving complementary and alternative medicine (CAM) therapies often report shifts in well-being that go beyond resolution of the original presenting symptoms. We undertook a research program to develop and evaluate a patient-centered outcome measure to assess the multidimensional impacts of CAM therapies, utilizing a novel mixed methods approach that relied upon techniques from the fields of anthropology and psychometrics. This tool would have broad applicability, both for CAM practitioners to measure shifts in patients' states following treatments, and conventional clinical trial researchers needing validated outcome measures. The US Food and Drug Administration has highlighted the importance of valid and reliable measurement of patient-reported outcomes in the evaluation of conventional medical products. Here we describe Phase I of our research program, the iterative process of content identification, item development and refinement, and response format selection. Cognitive interviews and psychometric evaluation are reported separately.

METHODS

From a database of patient interviews (n = 177) from six diverse CAM studies, 150 interviews were identified for secondary analysis in which individuals spontaneously discussed unexpected changes associated with CAM. Using ATLAS.ti, we identified common themes and language to inform questionnaire item content and wording. Respondents' language was often richly textured, but item development required a stripping down of language to extract essential meaning and minimize potential comprehension barriers across populations. Through an evocative card sort interview process, we identified those items most widely applicable and covering standard psychometric domains. We developed, pilot-tested, and refined the format, yielding a questionnaire for cognitive interviews and psychometric evaluation.

RESULTS

The resulting questionnaire contained 18 items, in visual analog scale format, in which each line was anchored by the positive and negative extremes relevant to the experiential domain. Because of frequent informant allusions to response set shifts from before to after CAM therapies, we chose a retrospective pretest format. Items cover physical, emotional, cognitive, social, spiritual, and whole person domains.

CONCLUSIONS

This paper reports the success of a novel approach to the development of outcome instruments, in which items are extracted from patients' words instead of being distilled from pre-existing theory. The resulting instrument, focused on measuring shifts in patients' perceptions of health and well-being along pre-specified axes, is undergoing continued testing, and is available for use by cooperating investigators.

Acupuncture and chiropractic care for chronic pain in an integrated health plan: a mixed methods study

BACKGROUND

Substantial recent research examines the efficacy of many types of complementary and alternative (CAM) therapies. However, outcomes associated with the "real-world" use of CAM has been largely overlooked, despite calls for CAM therapies to be studied in the manner in which they are practiced. Americans seek CAM treatments far more often for chronic musculoskeletal pain (CMP) than for any other condition. Among CAM treatments for CMP, acupuncture and chiropractic (A/C) care are among those with the highest acceptance by physician groups and the best evidence to support their use. Further, recent alarming increases in delivery of opioid treatment and surgical interventions for chronic pain--despite their high costs, potential adverse effects, and modest efficacy--suggests the need to evaluate real world outcomes associated with promising non-pharmacological/non-surgical CAM treatments for CMP, which are often well accepted by patients and increasingly used in the community.

METHODS/DESIGN

This multi-phase, mixed methods study will: (1) conduct a retrospective study using information from electronic medical records (EMRs) of a large HMO to identify unique clusters of patients with CMP (e.g., those with differing demographics, histories of pain condition, use of allopathic and CAM health services, and comorbidity profiles) that may be associated with different propensities for A/C utilization and/or differential outcomes associated with such care; (2) use qualitative interviews to explore allopathic providers' recommendations for A/C and patients' decisions to pursue and retain CAM care; and (3) prospectively evaluate health services/costs and broader clinical and functional outcomes associated with the receipt of A/C relative to carefully matched comparison participants receiving traditional CMP services. Sensitivity analyses will compare methods relying solely on EMR-derived data versus analyses supplementing EMR data with conventionally collected patient and clinician data.

DISCUSSION

Successful completion of these aggregate aims will provide an evaluation of outcomes associated with the real-world use of A/C services. The trio of retrospective, qualitative, and prospective study will also provide a clearer understanding of the decision-making processes behind the use of A/C for CMP and a transportable methodology that can be applied to other health care settings, CAM treatments, and clinical populations.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT01345409.