Evaluating the Statistical Significance of Health-Related Quality-Of-Life Change in Individual Patients

Psychometric evaluation of the Dysphagia Symptom Questionnaire for adults and adolescents with eosinophilic esophagitis

Background

Eosinophilic esophagitis (EoE) is a chronic, inflammatory disease of the esophagus leading to symptoms of esophageal dysfunction; dysphagia is the most common symptom experienced by adults and adolescents.

Objective

We sought to perform a psychometric evaluation of the Dysphagia Symptom Questionnaire (DSQ), a patient-reported outcome measure for patients with EoE.

Methods

Using baseline and week 24 data from the randomized, interventional, multinational phase 3 R668-EE-1774 trial (NCT03633617), the measurement properties of the DSQ-including reliability, construct and known-groups validity, responsiveness, and interpretation of change-were evaluated.

Results

The analysis population comprised 239 patients with EoE (age [mean ± SD], 28.1 ± 13.14 years; 63.6% male; 90.4% White). Intraclass correlation coefficients of 0.92 and 0.97 exceeded the acceptable reliability threshold (≥0.70). Construct validity correlations with EoE symptom and impact measures were moderate at baseline (|r| = 0.44-0.55) and week 24 (|r| = 0.55-0.69), and the DSQ biweekly total score discriminated among groups defined by disease severity. Analyses exploring interpretation of change from baseline on the DSQ biweekly total score indicated thresholds for within-patient improvement ranging from 9 to 23 points; a within-patient improvement from baseline of 13 points or greater could be considered clinically meaningful.

Conclusions

This analysis confirmed that the DSQ has acceptable distributional properties, test-retest reliability, construct validity, and ability to detect change. Therefore, the DSQ is a valid and reliable measure to assess the patient-reported symptom of dysphagia among adult and adolescent patients with EoE in the context of a clinical trial setting.

Care partner-informed meaningful change thresholds for the Clinical Dementia Rating-Sum of Boxes for trials of early Alzheimer's disease

INTRODUCTION

Consensus definitions of meaningful within-patient change (MWPC) on the Clinical Dementia Rating-Sum of Boxes (CDR-SB) are needed. Existing estimates use clinician-rated anchors in clinically diagnosed Alzheimer's disease (AD) populations. Incorporating the care partner perspective offers important insights, and evaluating biomarker-confirmed cohorts aligns estimates with ongoing trials.

METHODS

Anchor-based analyses were conducted to evaluate MWPC on the CDR-SB in early AD (Tauriel; NCT03289143) using Caregiver Global Impression of Change in memory or daily activities.

RESULTS

Across time points and anchors, mean CDR-SB changes associated with the "somewhat worse" category ranged from 1.50 to 2.12 in early AD, 1.07 to 2.06 in mild cognitive impairment-AD, and 1.79 to 2.25 in mild AD.

DISCUSSION

The proposed ranges are appropriate to define meaningful progression on the CDR-SB in similar cohorts and support the interpretation of treatment benefit through MWPC analyses. Thresholds should be calibrated to the context of use; lower/higher thresholds may be applicable in studies of earlier/later disease over shorter/longer durations.

HIGHLIGHTS

Within-patient CDR-SB change thresholds are provided using caregiver-rated anchors. 1.5 to 2.5 points may be an appropriate range in early AD trials of similar durations. Cumulative distribution function plots illustrate the benefit of a given treatment. When selecting thresholds, the target population and study design should be considered.

Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis

PURPOSE

The minimal important change (MIC) is defined as the smallest within-individual change in a patient-reported outcome measure (PROM) that patients on average perceive as important. We describe a method to estimate this value based on longitudinal confirmatory factor analysis (LCFA). The method is evaluated and compared with a recently published method based on longitudinal item response theory (LIRT) in simulated and real data. We also examined the effect of sample size on bias and precision of the estimate.

METHODS

We simulated 108 samples with various characteristics in which the true MIC was simulated as the mean of individual MICs, and estimated MICs based on LCFA and LIRT. Additionally, both MICs were estimated in existing PROMIS Pain Behavior data from 909 patients. In another set of 3888 simulated samples with sample sizes of 125, 250, 500, and 1000, we estimated LCFA-based MICs.

RESULTS

The MIC was equally well recovered with the LCFA-method as using the LIRT-method, but the LCFA analyses were more than 50 times faster. In the Pain Behavior data (with higher scores indicating more pain behavior), an LCFA-based MIC for improvement was estimated to be 2.85 points (on a simple sum scale ranging 14-42), whereas the LIRT-based MIC was estimated to be 2.60. The sample size simulations showed that smaller sample sizes decreased the precision of the LCFA-based MIC and increased the risk of model non-convergence.

CONCLUSION

The MIC can accurately be estimated using LCFA, but sample sizes need to be preferably greater than 125.

Validation of the novel Eosinophilic Esophagitis Impact Questionnaire

BACKGROUND

Eosinophilic esophagitis (EoE) has a detrimental effect on health-related quality of life (HRQOL). The Eosinophilic Esophagitis Impact Questionnaire (EoE-IQ) is a novel patient-reported outcome (PRO) measure assessing the impact of EoE on HRQOL. To assess suitability of the EoE-IQ, its measurement properties were evaluated.

METHODS

Using baseline and week 24 data from the pivotal, randomized, placebo-controlled, multinational phase 3 R668-EE-1774 trial (NCT03633617) of dupilumab, we evaluated EoE-IQ's measurement properties (including reliability, construct and known-groups validity, and ability to detect change) and established the threshold for change in scores that can be considered clinically meaningful.

RESULTS

The analysis population comprised 239 adults and adolescents with EoE. Mean age was 28.1 (standard deviation, 13.14) years; 63.6% were male, and 90.4% were White. Reliability estimates for the EoE-IQ average score exceeded acceptable thresholds for patients who were stable as indicated by ratings of Patient Global Impression of Severity (PGIS) and Change (PGIC) (intraclass correlation coefficients, 0.75 and 0.81). Construct validity correlations with other EoE-specific PRO scores were moderate at baseline (|r|= 0.44-0.60) and moderate to strong at week 24 (|r|= 0.61-0.72). In known-groups analysis, EoE-IQ average score discriminated among groups of patients at varying EoE severity levels defined by PGIS scores. A ≥ 0.6-point reduction in EoE-IQ average score (where scores range from 1 to 5, with higher scores indicating worse HRQOL) from baseline to week 24 can be considered clinically meaningful.

CONCLUSIONS

The EoE-IQ's measurement properties are acceptable, making it a valid, reliable measure of the HRQOL impacts of EoE among adults and adolescents.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03633617. Registered August 14, 2018, https://clinicaltrials.gov/study/NCT03633617 .

Comparison of anchor-based methods for estimating thresholds of meaningful within-patient change using simulated PROMIS PF 20a data under various joint distribution characteristic conditions

PURPOSE

To compare the performance of anchor-based methods for estimating thresholds of meaningful within-patient change (i.e., individual change) of clinical outcome assessments in conditions reflecting data characteristics of small- to medium-sized clinical trials.

METHODS

Datasets were generated from the joint distributions of the PROMIS PF 20a T-score changes and a seven-point global change anchor measure. The 108 simulation conditions (1000 replications per condition) included combinations of three marginal distributions of T-score changes, three improvement percentages in the anchor measure, four levels of responsiveness correlations, and three sample sizes. Threshold estimation methods included mean change, median change, ROC curve, predictive modeling, half SD, and SEM. Relative bias, precision, accuracy, and measurement significance of the estimates were evaluated based on comparison with true thresholds and IRT-based individual reliable changes of PROMIS scores. Quantile regression models were applied to select and interpret effects of simulation conditions on estimation bias.

RESULTS

When PROMIS T-score changes were distributed normally, the predictive modeling method performed best with 50% or more responders identified by the anchor; the mean and median methods were preferred with 30% responders. For skewed distributions, the median method and ROC method gained more advantages. Among the evaluated study conditions, the improvement percentage condition had the most obvious effects on estimation bias.

CONCLUSION

To establish accurate and precise thresholds, clinical researchers are recommended to prioritize study designs with at least 50% anchor-defined responders and strongly responsive target endpoints with highly reliable scoring calibration and to select optimal anchor-based methods given the data characteristics.

Interpreting Patient-Reported Outcome Scores: Pediatric Inflammatory Bowel Disease as a Use Case

OBJECTIVE

To demonstrate how to interpret Patient-Reported Outcomes Measurement Information System (PROMIS) pediatric patient-reported outcome measure (PROM) scores for patients with pediatric inflammatory bowel disease (IBD).

METHODS

Using data from a prospective cohort study of patients ages 8 to 23 years with IBD (n = 1049), we established disease-specific percentiles and computed the minimal clinically important difference (MCID) change score for 6 pediatric PROMs. We applied these results, general population percentiles, and the reliable change index to interpret PROM scores in a clinical trial sample of patients ages 8 to 20 years with IBD (n = 294) in which PROMIS PROMs were obtained at baseline and 3 months later.

RESULTS

Application of general population percentiles showed that the clinical trial sample at baseline had moderately worse self-reported health than the general population (22% of patients at or above the 95th percentile on Fatigue; 21% on Pain Interference). IBD-specific percentiles showed that the sample was somewhat worse than the reference IBD sample (8% of patients at or above the 95th percentile on Fatigue; 11% on Pain Interference). Application of the MCID threshold indicated that among the subgroup of patients that improved by 15 or more on the short Pediatric Crohn's Disease Activity Index (n = 38), 45% also improved on IBD Symptoms, 47% for Fatigue, and 65% for Pain Interference.

CONCLUSION

This study established IBD-specific percentiles for 6 pediatric PROMIS measures and demonstrated the application of percentiles and other methods for interpreting PROM scores.

Likely change indexes improve estimates of individual change on patient-reported outcomes

PURPOSE

Individual change on a patient-reported outcome (PRO) measure can be assessed by statistical significance and meaningfulness to patients. We explored the relationship between these two criteria by varying the confidence levels of the coefficient of repeatability (CR) on the Patient-Reported Outcomes Measurement Information System (R) Physical Function (PF) 10a (PF10a) measure.

METHODS

In a sample of 1129 adult cancer patients, we estimated individual-change thresholds on the PF10a from baseline to 6 weeks later with the CR at 50%, 68%, and 95% confidence. We also assessed agreement with group- and individual-level thresholds from anchor-based methods [mean change and receiver operating characteristic (ROC) curve] using a PF-specific patient global impression of change (PGIC).

RESULTS

CRs at 50%, 68%, and 95% confidence were 3, 4, and 7 raw score points, respectively. The ROC- and mean-change-based thresholds for deterioration were -4 and -6; for improvement they were both 2. Kappas for agreement between anchor-based thresholds and CRs for deterioration ranged between κ = 0.65 and 1.00, while for improvement, they ranged between 0.35 and 0.83. Agreement between the PGIC and all CRs always fell below "good" (κ < 0.40) for deterioration (0.30-0.33) and were lower for improvement (0.16-0.28).

CONCLUSIONS

In comparison to the CR at 95% confidence, CRs at 50% and 68% confidence (considered likely change indexes) have the advantage of maximizing the proportion of patients appropriately classified as changed according to statistical significance and meaningfulness.

Assessing the Significance of Individual Change in 2 Samples of Patients in Treatment for Low Back Pain Using 5 Different Statistical Indicators

OBJECTIVE

The purpose of this study was to estimate the significance of individual change using 5 statistical indicators in 2 samples of patients treated for low back pain.

METHODS

This secondary analysis used observational and clinical trial data from 2 samples of patients with low back pain to compare 5 ways of estimating significant individual change on the Impact Stratification Score (ISS) administered at the following 2 time points: 3 months apart in an observational study of 1680 patients undergoing chiropractic care, and 6 weeks apart in a randomized trial of 750 active-duty military personnel with low back pain. The following 5 methods were compared: (1) standard deviation index; (2) standard error of measurement (SEM); (3) standard error of estimate (SEE); (4) standard error of prediction (SEP); and (5) the reliable change index (RCI). The ISS is the sum of the Patient-Reported Outcomes Measurement Information System (PROMIS)-29 v2.1 physical function, pain interference, and pain intensity scores and is scored to have a possible range of 8 (least impact) to 50 (greatest impact).

RESULTS

The amount of change on the ISS needed for significant individual change in both samples was 5 for the SEM and for the SEE and 7 for the SEP and RCI.

CONCLUSIONS

The results of the current study provide some preliminary support for use of the SEP or the RCI to identify significant individual change and provide estimated thresholds of individual change that can be used for the ISS. The SEP and RCI estimates of significant change were consistent with retrospective ratings of change of at least moderately better in prior research. These 2 were less likely than other methods to classify people with low back pain as responders who have not actually gotten better (false positive). In contrast, the SEM and SEE were less likely to miss real change (false negative).

Development of the "Kalmer" relaxation intervention: co-design with stroke survivors with aphasia

PURPOSE

Anxiety is common after stroke and more prevalent in survivors with aphasia. Relaxation is an effective first-line therapy. The current study aimed to obtain the perspectives of stroke survivors with aphasia to inform the development of an accessible, technology-based, relaxation intervention.

MATERIALS AND METHODS

Qualitative co-design methods were used with twelve people with aphasia after stroke. The "Kalmer" Relaxation treatment package materials were iteratively based on participants' experiences and preferences; barriers and facilitators to treatment compliance were explored. Participants were also asked to consider how the intervention might be evaluated in a research trial.

RESULTS

A thematic analysis highlighted the importance and need for the development of an appropriate and inclusive relaxation product, to be implemented by health professionals early post-stroke. Several behavioural strategies to improve treatment adherence were recommended. Participants had varying perspectives on clinically meaningful treatment outcomes.

CONCLUSIONS

Overall, a co-designed accessible relaxation product was viewed as a necessary component of usual stroke care. Acceptability and feasibility and preliminary efficacy of the "Kalmer" intervention should be trialled in future studies. IMPLICATIONS FOR REHABILITATIONCo-designing psychological interventions for people with aphasia after stroke is needed to meet the needs of this at-risk population.Technology-based relaxation interventions to manage anxiety after stroke are viewed positively by people with aphasia and deemed acceptable and feasible.Clinical trials of these co-designed relaxation interventions are required before recommending integration into routine practice.

Using Item Response Theory to Identify Responders to Treatment: Examples with the Patient-Reported Outcomes Measurement Information System (PROMIS®) Physical Function Scale and Emotional Distress Composite

The reliable change index has been used to evaluate the significance of individual change in health-related quality of life. We estimate reliable change for two measures (physical function and emotional distress) in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item health-related quality of life measure (PROMIS-29 v2.1). Using two waves of data collected 3 months apart in a longitudinal observational study of chronic low back pain and chronic neck pain patients receiving chiropractic care, and simulations, we compare estimates of reliable change from classical test theory fixed standard errors with item response theory standard errors from the graded response model. We find that unless true change in the PROMIS physical function and emotional distress scales is substantial, classical test theory estimates of significant individual change are much more optimistic than estimates of change based on item response theory.

Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS

The psychometric process used to establish a relationship between the scores of two (or more) instruments is generically referred to as linking. When two instruments with the same content and statistical test specifications are linked, these instruments are said to be equated. Linking and equating procedures have long been used for practical benefit in educational testing. In recent years, health outcome researchers have increasingly applied linking techniques to patient-reported outcome (PRO) data. However, these applications have some noteworthy purposes and associated methodological questions. Purposes for linking health outcomes include the harmonization of data across studies or settings (enabling increased power in hypothesis testing), the aggregation of summed score data by means of score crosswalk tables, and score conversion in clinical settings where new instruments are introduced, but an interpretable connection to historical data is needed. When two PRO instruments are linked, assumptions for equating are typically not met and the extent to which those assumptions are violated becomes a decision point around how (and whether) to proceed with linking. We demonstrate multiple linking procedures-equipercentile, unidimensional IRT calibration, and calibrated projection-with the Patient-Reported Outcomes Measurement Information System Depression bank and the Patient Health Questionnaire-9. We validate this link across two samples and simulate different instrument correlation levels to provide guidance around which linking method is preferred. Finally, we discuss some remaining issues and directions for psychometric research in linking PRO instruments.

Responsiveness of Nepali version of Oswestry Disability Index (ODI) on individuals with non-specific low back pain

BACKGROUND

Low back pain (LBP) is a common musculoskeletal problem, associated with disability and high societal costs. The Oswestry Disability Index (ODI) is among the most commonly used patient reported outcome measures to measure disability due to LBP. Evidence supporting the reliability and validity of the Nepali Version of Oswestry Disability Index (NODI) exists, but its responsiveness is yet to be assessed.

OBJECTIVE

We aimed to assess the responsiveness of NODI in participants with non-specific low back pain.

METHODS

The study included 102 (Male 41, Female 61) participants with non-specific low back pain, attending the physiotherapy outpatient department of a tertiary care hospital and nearby community. The NODI was administered to the patients at baseline and again 2 weeks later along with a 7-item Nepali Version of Global Rating of Change (GROC-NP). Responsiveness of NODI was assessed by plotting Receivers Operating Characteristics (ROC) curve.

RESULTS

The area under curve (AUC) of NODI was 0.88. The best cut-off point on the NODI for improvement on the GROC-NP or the minimal clinical important change (MIC) was 4.22 and ranged from 3.11 to 6.34. The sensitivity and specificity was 77.4% and 84.2% respectively.

CONCLUSION

NODI is a responsive scale which can discriminate between participants whose level of disability due to LBP is stable or improving. The result for minimal clinically important change, sensitivity and specificity are consistent with other cross culturally adopted versions.

Clinical Use of PROMIS, Neuro-QoL, TBI-QoL, and Other Patient-Reported Outcome Measures for Individual Adult Clients with Cognitive and Language Disorders

Patient-reported outcome measures (PROMs) assess health outcomes from the patient's perspective. The National Institutes of Health has invested in the creation of numerous PROMs that comprise the PROMIS, Neuro-QoL, and TBI-QoL measurement systems. Some of these PROMs are potentially useful as primary or secondary outcome measures, or as contextual variables for the treatment of adults with cognitive/language disorders. These PROMs were primarily created for clinical research and interpretation of group means. They also have potential for use with individual clients; however, at present there is only sparse evidence and direction for this application of PROMs. Previous research by Cohen and Hula (2020) described how PROMs could support evidence-based practices in speech-language pathology. This companion article extends upon that work to present clinicians with implementation information about obtaining, administering, scoring, and interpreting PROMs for individual clients with cognitive/language disorders. This includes considerations of the type and extent of communication support that is appropriate, implications of the relatively large measurement error that accompanies individual scores and pairs of scores, and recommendations for applying minimal detectable change values depending on the clinician's desired level of measurement precision. However, more research is needed to guide the interpretation of PROM scores for an individual client.

Between-group minimally important change versus individual treatment responders

PURPOSE

Estimates of the minimally important change (MIC) can be used to evaluate whether group-level differences are large enough to be important. But responders to treatment have been based upon group-level MIC thresholds, resulting in inaccurate classification of change over time. This article reviews options and provides suggestions about individual-level statistics to assess whether individuals have improved, stayed the same, or declined.

METHODS

Review of MIC estimation and an example of misapplication of MIC group-level estimates to assess individual change. Secondary data analysis to show how perceptions about meaningful change can be used along with significance of individual change.

RESULTS

MIC thresholds yield over-optimistic conclusions about responders to treatment because they classify those who have not changed as responders.

CONCLUSIONS

Future studies need to evaluate the significance of individual change using appropriate individual-level statistics such as the reliable change index or the equivalent coefficient of repeatability. Supplementing individual statistical significance with retrospective assessments of change is desirable.

Assessing baseline dependency of anchor-based minimal important change (MIC): don't stratify on the baseline score!

PURPOSE

The minimal important change (MIC) of a patient-reported outcome measure (PROM) is often suspected to be baseline dependent, typically in the sense that patients who are in a poorer baseline health condition need greater improvement to qualify as minimally important. Testing MIC baseline dependency is commonly performed by creating two or more subgroups, stratified on the baseline PROM score. This study's purpose was to show that this practice produces biased subgroup MIC estimates resulting in spurious MIC baseline dependency, and to develop alternative methods to evaluate MIC baseline dependency.

METHODS

Datasets with PROM baseline and follow-up scores and transition ratings were simulated with and without MIC baseline dependency. Mean change MICs, ROC-based MICs, predictive MICs, and adjusted MICs were estimated before and after stratification on the baseline score. Three alternative methods were developed and evaluated. The methods were applied in a real data example for illustration.

RESULTS

Baseline stratification resulted in biased subgroup MIC estimates and the false impression of MIC baseline dependency, due to redistribution of measurement error. Two of the alternative methods require a second baseline measurement with the same PROM or another correlated PROM. The third method involves the construction of two parallel tests based on splitting the PROM's item set. Two methods could be applied to the real data.

CONCLUSION

MIC baseline dependency should not be tested in subgroups based on stratification on the baseline PROM score. Instead, one or more of the suggested alternative methods should be used.

Critical consideration of assessment methods for clinically significant changes of mental distress after psycho-oncological interventions

OBJECTIVES

Considering the heterogeneity of cancer entities and the associated disease progression, personalized care of patients is increasingly emphasized in psycho-oncology. This individualization makes the use of measurements of individual clinically significant change important when studying the efficacy and effectiveness of psycho-oncological care. Two conceptualizations for the measurement of clinical significance are critically contrasted in this study: the Reliable Change Index (RCI) and the Minimal Important Difference (MID) method.

METHODS

In total, 2,121 cancer patients participated in the study and a subsample of 708 patients was reassessed about 4 months later. Psychological distress was measured using the Hospital Anxiety and Depression Scale. We evaluated two measures of clinical significance (RCI, MID) by comparing the respective numbers of improved, unimproved, and deteriorated patients.

RESULTS

Individually significant changes were observed with both methods; however, determined rates of improvement differed substantially: MID (66.67%) and RCI (48.23%). Most importantly, according to MID, 17.93% of patients were identified as being improved, although their respective improvements were not statistically significant and thus unreliable.

CONCLUSIONS

The benefits of RCI outweigh MID, and therefore, the RCI is recommended as a measure to assess change.

Low-Value Proton Pump Inhibitor Prescriptions Among Older Adults at a Large Academic Health System

BACKGROUND

Older adults are particularly vulnerable to complications from proton pump inhibitor (PPI) drugs. We sought to characterize the prevalence of potentially low-value PPI prescriptions among older adults to inform a quality improvement (QI) intervention.

METHODS

We created a cohort of patients, aged 65 years or older, receiving primary care at a large academic health system in 2018. We identified patients currently prescribed any PPI using the electronic health record (EHR) medication list (current defined as September 1, 2018). A geriatrician, a gastroenterologist, a QI expert, and two primary care physicians (PCPs) created multidisciplinary PPI appropriateness criteria based on evidenced-based guidelines. Supervised by a gastroenterologist and PCP, two internal medicine residents conducted manual chart reviews in a random sample of 399 patients prescribed PPIs. We considered prescriptions potentially low value if they lacked a guideline-based (1) short-term indication (gastroesophageal reflux disease [GERD]/peptic ulcer disease/Helicobacter pylori gastritis/dyspepsia) or (2) long-term (>8 weeks) indication (severe/refractory GERD/erosive esophagitis/Barrett esophagus/esophageal adenocarcinoma/esophageal stricture/high gastrointestinal bleeding risk/Zollinger-Ellison syndrome). We used the Wilson score method to calculate 95% confidence intervals (CIs) on low-value PPI prescription prevalence.

RESULTS

Among 69 352 older adults, 8729 (12.6%) were prescribed a PPI. In the sample of 399 patients prescribed PPIs, 63.9% were female; their mean age was 76.2 years, and they were seen by 169 PCPs. Of the 399 prescriptions, 143 (35.8%; 95% CI = 31.3%-40.7%) were potentially low value-of which 82% began appropriately (eg, GERD) but then continued long term without a guideline-based indication. Among 169 PCPs, 32 (18.9%) contributed to 59.2% of potentially low-value prescriptions.

CONCLUSION

One in eight older adults were prescribed a PPI, and over one-third of prescriptions were potentially low-value. Most often, appropriate short-term prescriptions became potentially low value because they lacked long-term indications. With most potentially low-value prescribing concentrated among a small subset of PCPs, interventions targeting them and/or applying EHR-based automatic stopping rules may protect older adults from harm. J Am Geriatr Soc 67:2600-2604, 2019.

Can Methods Developed for Interpreting Group-level Patient-reported Outcome Data be Applied to Individual Patient Management?

BACKGROUND

Patient-reported outcome (PRO) data may be used at 2 levels: to evaluate impacts of disease and treatment aggregated across individuals (group-level) and to screen/monitor individual patients to inform their management (individual-level). For PRO data to be useful at either level, we need to understand their clinical relevance.

PURPOSE

To provide clarity on whether and how methods historically developed to interpret group-based PRO research results might be applied in clinical settings to enable PRO data from individual patients to inform their clinical management and decision-making.

METHODS

We first differentiate PRO-based decision-making required at group versus individual levels. We then summarize established group-based approaches to interpretation (anchor-based and distribution based), and more recent methods that draw on item calibrations and qualitative research methods. We then assess the applicability of these methods to individual patient data and individual-level decision-making.

FINDINGS

Group-based methods provide a range of thresholds that are useful in clinical care: some provide screening thresholds for patients who need additional clinical assessment and/or intervention, some provide thresholds for classifying an individual's level of severity of symptoms or problems with function, and others provide thresholds for meaningful change when monitoring symptoms and functioning over time during or after interventions. Availability of established cut-points for screening and symptom severity, and normative/reference values, may play into choice of PRO measures for use in clinical care. Translatability of thresholds for meaningful change is more problematic because of the greater reliability needed at the individual-level versus group-level, but group-based methods may provide lower bound estimates. Caution is needed to set thresholds above bounds of measurement error to avoid "false-positive changes" triggering unwarranted alerts and action in clinic.

CONCLUSIONS

While there are some challenges in applying available methods for interpreting group-based PRO results to individual patient data and clinical care-including myriad contextual factors that may influence an individual patient's management and decision-making-they provide a useful starting point, and should be used pragmatically.

Group and Individual-level Change on Health-related Quality of Life in Chiropractic Patients With Chronic Low Back or Neck Pain

STUDY DESIGN

A prospective observational study.

OBJECTIVE

The aim of this study was to evaluate group-level and individual-level change in health-related quality of life among persons with chronic low back pain or neck pain receiving chiropractic care in the United States.

SUMMARY OF BACKGROUND DATA

Chiropractors treat chronic low back and neck pain, but there is limited evidence of the effectiveness of their treatment METHODS.: A 3-month longitudinal study of 2024 patients with chronic low back pain or neck pain receiving care from 125 chiropractic clinics at six locations throughout the United States was conducted. Ninety-one percent of the sample completed the baseline and 3-month follow-up survey (n = 1835). Average age was 49, 74% females, and most of the sample had a college degree, were non-Hispanic White, worked full-time, and had an annual income of $60,000 or more. Group-level (within-group t tests) and individual-level (coefficient of repeatability) changes on the Patient-Reported Outcomes Measurement Information System (PROMIS-29) v2.0 profile measure was evaluated: six multi-item scales (physical functioning, pain, fatigue, sleep disturbance, social health, emotional distress) and physical and mental health summary scores.

RESULTS

Within-group t tests indicated significant group-level change (P < 0.05) for all scores except for emotional distress, and these changes represented small improvements in health (absolute value of effect sizes ranged from 0.08 for physical functioning to 0.20 for pain). From 13% (physical functioning) to 30% (PROMIS-29 v2.0 Mental Health Summary Score) got better from baseline to 3 months later according to the coefficient of repeatability.

CONCLUSION

Chiropractic care was associated with significant group-level improvement in health-related quality of life over time, especially in pain. But only a minority of the individuals in the sample got significantly better ("responders"). This study suggests some benefits of chiropractic on functioning and well-being of patients with low back pain or neck pain.

LEVEL OF EVIDENCE

3.

Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma

Background

The Asthma Symptom Diary was developed to assess severity of symptoms in patients with moderate to severe asthma, and has evidence supporting reliability and validity. Only limited information is available on sensitivity to change and responder definitions for the Asthma Symptom Diary.

Objectives

Main study objectives were to evaluate sensitivity to change and provide responder definitions for clinically meaningful effects for the Asthma Symptom Diary.

Methods

This is a secondary analysis of Phase II clinical trial data in patients with moderate to severe asthma, Asthma Symptom Diary (ASD) was collected daily during the 24-week study. The Asthma Control Questionnaire and the Patient Global Assessment were collected at baseline, and week 12 and 24. Analysis of covariance (ANCOVA) models were used to evaluate sensitivity to change in Asthma Symptom Diary scores after 12 and 24 weeks of treatment. Anchor-based methods, using Asthma Control Questionnaire and Patient Global Assessment defined anchors, were used to identify minimal important differences and various responder criteria for changes in mean 7-day ASD score, symptomatic days, and minimal symptom days.

Results

Sample was 59% female, 81% White, with a mean age of 47.3 (SD = 13.6) years. ANCOVAs demonstrated significant differences in baseline to week 12 and week 24 changes in mean 7-day Asthma Symptom Diary scores and symptomatic days by Asthma Control Questionnaire (all p < 0.001) and Patient Global Assessment anchors (all p < 0.001). Meaningful responders, from the patient’s perspective, were defined as improvements of 0.5–0.6 points (SD = 0.6; scale range 0 to 4) in mean 7-day Asthma Symptom Diary scores, and as a reduction of 2 to 3 Asthma Symptom Diary-based symptomatic days.

Conclusion

The Asthma Symptom Diary was responsive to changes in clinical status in patients with moderate to severe asthma. Responder definitions were identified, including symptomatic days, for evaluating individual level treatment effects in clinical trials.

Effect of Mobile Device-Supported Single-Patient Multi-crossover Trials on Treatment of Chronic Musculoskeletal Pain: A Randomized Clinical Trial

IMPORTANCE

Individually designed single-patient multi-crossover (n-of-1) trials can facilitate tailoring of treatments directed at various conditions, including chronic musculoskeletal pain (CMSP) but are potentially burdensome, which may limit uptake in research and practice.

OBJECTIVES

To determine whether patients randomized to participate in an n-of-1 trial supported by a mobile health (mHealth) app would experience less pain and improved global health, adherence, satisfaction, and shared decision making compared with patients assigned to usual care.

DESIGN, SETTING, AND PARTICIPANTS

This randomized clinical trial compared participation in an individualized, mHealth-supported n-of-1 trial vs usual care. The participating 215 patients had CMSP for at least 6 weeks, had a smartphone or tablet with a data plan, were enrolled in northern California from July 2014 through July 2016, and were followed for up to 1 year by 48 clinicians in academic, community, Veterans Affairs, and military settings.

INTERVENTIONS

Intervention patients met with their clinicians and used a desktop interface to select treatments and trial parameters for an n-of-1 trial comparing 2 pain-management regimens. The mHealth app provided reminders to take designated treatments on assigned days and to upload responses to daily questions on pain and treatment-associated adverse effects. Control patients received care as usual.

MAIN OUTCOMES AND MEASURES

The primary outcome was change in the PROMIS (Patient-Reported Outcomes Measurement Information System) pain-related interference 8-item short-form scale (full scale range, 41-78) from baseline to 6 months. Secondary outcomes included patient-reported pain intensity, overall health, analgesic adherence, trust in clinician, satisfaction with care, medication-related shared decision making, and, for the n-of-1 group only, participant engagement and experience.

RESULTS

Among 215 patients (108 randomized to the n-of-1 intervention and 107 to control), 102 (47%) were women, and the mean (SD) age was 55.5 (11.1) years. At the 6-month follow-up, pain interference was reduced in both groups, though there was no difference between the intervention and control groups (-1.36 points; 95% CI, -2.91 to 0.19 points; P = .09). There were no advantages in secondary outcomes for intervention patients vs control patients except for higher medication-related shared decision making at 6 months (between-group difference, 11.9 points; 95% CI, 2.6-21.2 points; P = .01). Among patients assigned to the n-of-1 group, 88% (n = 86) affirmed that the mHealth app could help people like them manage their pain.

CONCLUSIONS AND RELEVANCE

In this population of patients with CMSP, mHealth-supported n-of-1 trials were feasible and associated with a satisfactory user experience, but n-of-1 trial participation did not significantly improve pain interference at 6 months vs usual care.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02116621.

How Do Psychiatrists Apply the Minimum Clinically Important Difference to Assess Patient Responses to Treatment?

Symptom report scales are used in clinical practice to monitor patient outcomes. Using them permits the definition of a minimum clinically important difference (MCID) beyond which a patient may be judged as having responded to treatment. Despite recommendations that clinicians routinely use MCIDs in clinical practice, statisticians disagree about how MCIDs should be used to evaluate individual patient outcomes and responses to treatment. To address this issue, we asked how clinicians actually use MCIDs to evaluate patient outcomes in response to treatment. Sixty-eight psychiatrists made judgments about whether hypothetical patients had responded to treatment based on their pre- and posttreatment change scores on the widely used Positive and Negative Syndrome Scale. Psychiatrists were provided with the scale's MCID on which to base their judgments. Our secondary objective was to assess whether knowledge of the patient's genotype influenced psychiatrists' responder judgments. Thus, psychiatrists were also informed of whether patients possessed a genotype indicating hyperresponsiveness to treatment. While many psychiatrists appropriately used the MCID, others accepted a far lower posttreatment change as indicative of a response to treatment. When psychiatrists accepted a lower posttreatment change than the MCID, they were less confident in such judgments compared to when a patient's posttreatment change exceeded the scale's MCID. Psychiatrists were also less likely to identify patients as responders to treatment if they possessed a hyperresponsiveness genotype. Clinicians should recognize that when judging patient responses to treatment, they often tolerate lower response thresholds than warranted. At least some conflate their judgments with information, such as the patient's genotype, that is irrelevant to a post hoc response-to-treatment assessment. Consequently, clinicians may be at risk of persisting with treatments that have failed to demonstrate patient benefits.

The From Survivor to Thriver program: RCT of an online therapist-facilitated program for rape-related PTSD

This study evaluated the efficacy of the From Survivor to Thriver program, an interactive, online therapist-facilitated cognitive-behavioral program for rape-related PTSD. Eighty-seven college women with rape-related PTSD were randomized to complete the interactive program (n=46) or a psycho-educational self-help website (n=41). Both programs led to large reductions in interview-assessed PTSD at post-treatment (interactive d=2.22, psycho-educational d=1.10), which were maintained at three month follow-up. Both also led to medium- to large-sized reductions in self-reported depressive and general anxiety symptoms. Follow-up analyses supported that the therapist-facilitated interactive program led to superior outcomes among those with higher pre-treatment PTSD whereas the psycho-educational self-help website led to superior outcomes for individuals with lower pre-treatment PTSD. Future research should examine the efficacy and effectiveness of online interventions for rape-related PTSD including whether treatment intensity matching could be utilized to maximize outcomes and therapist resource efficiency.

Change in Health-Related Quality-of-Life at Group and Individual Levels Over Time in Patients Treated for Chronic Myofascial Neck Pain

BACKGROUND

This study evaluated change in health-related quality of life at the group and individual levels in a consecutive series of patients with chronic myofascial neck pain.

METHODS

Fifty patients with chronic neck pain self-administered the Short Form-36 Version 2 (SF-36 v2) before treatment and 6 weeks later. Internal consistency reliability was estimated for the 8 scale scores and Mosier's formula was used to estimate reliability of the physical and mental health composite scores. Significance of group-level change was estimated using within-group t statistics. Significance of individual change was evaluated by reliable change index.

RESULTS

Statistically significant ( P < .05) group mean improvement over time was found for all SF-36 scores. At the individual level, 20% of the possible changes were statistically significant (17% improvement, 3% decline).

CONCLUSIONS

Estimating the significance of individual change in health-related quality of life adds important information in comparing different treatment modalities for chronic myofascial neck pain.

Cognitive-Behavioral Treatment of Posttraumatic Stress in Patients With Implantable Cardioverter Defibrillators: Results From a Randomized Controlled Trial

Approximately 20% of patients with implantable cardioverter defibrillators (ICDs) suffer from posttraumatic stress disorder (PTSD) due to a history of cardiac arrest, device implantation, and ICD shock. There has been very little examination of treatment of PTSD symptoms in these patients. This study evaluated the effect of a specific cognitive-behavioral therapy (CBT) intervention for ICD patients with high levels of PTSD symptoms: a manualized program consisting of 8 telephone sessions with a trained counselor, a patient education book, and a stress management procedure on compact disc. Participants were 193 ICD patients, who were randomized to CBT or usual cardiac care (UCC) who completed self-report surveys at the time of recruitment and 6 and 12 months after initial measurement. Previous publication on the primary research evaluation questions reported that the CBT condition resulted in greater improvement on PTSD and depression symptoms than the UCC for the general population of ICD patients, but did not evaluate the effect on those with elevated symptoms of PTSD. The authors conducted secondary analyses of the effect of treatment on high and low PTSD symptom groups based on a cutoff for the Impact of Event Scale-Revised (Weiss & Marmar, 1997). Participants in the CBT group who had high symptoms experienced significantly greater symptom reduction from baseline to 12 months (d = 2.44, p = .021) than the UCC group (d = 1.12). Participants with low symptoms had small reductions regardless of group assignment (d = 0.16, p = .031). ICD-focused CBT was sufficient to produce a large, statistically significant reduction in PTSD symptoms in ICD patients with indications for treatment.

The challenge of measuring intra-individual change in fatigue during cancer treatment

PURPOSE

To evaluate how well three different patient-reported outcomes (PROs) measure individual change.

METHODS

Two hundred and fourteen patients (from two sites) initiating first or new chemotherapy for any stage of breast or gastrointestinal cancer participated. The 13-item FACIT Fatigue scale, a 7-item PROMIS^® Fatigue Short Form (PROMIS 7a), and the PROMIS^® Fatigue computer adaptive test (CAT) were administered monthly online for 6 months. Reliability of measured change was defined, under a population mixed effects model, as the ratio of estimated systematic variance in rate of change to the estimated total variance of measured individual differences in rate of change. Precision of individual measured change, the standard error of measurement of change, was given by the square root of the rate-of-change sampling variance. Linear and quadratic models were examined up to 3 and up to 6 months.

RESULTS

A linear model for measured change showed the following by 6 and 3 months, respectively: PROMIS CAT (0.363 and 0.342); PROMIS SF (0.408 and 0.533); FACIT (0.459 and 0.473). Quadratic models offered no noteworthy improvement over linear models. Both reliability and precision results demonstrate the need to improve the measurement of intra-individual change.

CONCLUSIONS

These results illustrate the challenge of reliably measuring individual change in fatigue with a level of confidence required for intervention. Optimizing clinically useful measurement of intra-individual differences over time continues to pose a challenge for PROs.

Effects of Prior Intensive Insulin Therapy and Risk Factors on Patient-Reported Visual Function Outcomes in the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) Cohort

IMPORTANCE

Preservation of vision in patients with diabetes mellitus is critical. Interventions to improve glycemic control through early intensive treatment of diabetes reduce rates of severe retinopathy and preserve visual acuity.

OBJECTIVE

To assess the effects of prior intensive insulin treatment and risk factors on patient-reported visual function in the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) cohort.

DESIGN, SETTING, AND PARTICIPANTS

Cohort study of 1184 participants with type 1 diabetes from the DCCT/EDIC study (randomized clinical trial followed by an observational follow-up study) who completed the 25-item National Eye Institute Visual Function Questionnaire (NEI-VFQ-25) during EDIC years 17 through 20 (September 1, 2009, through April 30, 2014) in 28 institutions across the United States and Canada.

MAIN OUTCOMES AND MEASURES

The primary outcome was the composite NEI-VFQ-25 score. Secondary outcomes were visual acuity (measured by the Early Treatment Diabetic Retinopathy Study protocol), retinopathy level (determined by masked grading of stereoscopic color fundus photographs), and NEI-VFQ-25 subscale scores. The composite NEI-VFQ-25 scale and its subscales were scored 0 to 100, corresponding to poor to excellent function, respectively.

RESULTS

The overall average NEI-VFQ-25 score for 1184 DCCT/EDIC participants (mean [SD] age, 52.3 [6.9] years; 48% female) with a 30-year duration of diabetes was high (all participants: median, 91.7; interquartile range [IQR], 89.7-96.9; intensive treatment [n = 605]: median, 94.7; IQR, 91.0-97.2; conventional treatment [n = 579]: median, 94.0; IQR, 88.4-96.1; P = .006 for intensive vs conventional). After adjustment for sex, age, hemoglobin A1c level, and retinopathy level at DCCT baseline, the former intensive treatment group had a significant, albeit modest, improvement in overall NEI-VFQ-25 score compared with the former conventional diabetes treatment group (median difference, -1.0; 95% CI, -1.7 to -0.3; P = .006). This beneficial treatment effect was fully attributed to the prior glycemic control in DCCT (explained treatment effect: 100%). Those with visual acuity worse than 20/100 reported the largest decline in visual function (median difference, -21.0; 95% CI, -40.5 to -1.6; P = .03).

CONCLUSIONS AND RELEVANCE

In the DCCT/EDIC cohort, patient-reported visual function remains high in both treatment groups, comparable to previous reports of overall health-related quality of life. Intensive diabetes therapy modestly improved NEI-VFQ-25 score 30 years after the start of the DCCT, the benefit underestimated owing to more nonparticipants from the conventional treatment group. Visual acuity had the greatest effect on patient-reported visual function from among all risk factors.

TRIAL REGISTRATION

clinicaltrials.gov Identifiers: NCT00360815 and NCT00360893.

Best (but oft-forgotten) practices: expressing and interpreting associations and effect sizes in clinical outcome assessments

This article reviews methods used to facilitate the interpretation and evaluation of group-level differences in clinical outcome assessments. These methods complement and supplement tests of statistical significance. Examples, including studies in nutrition, are used to illustrate the application of the interpretation methods for group-level comparisons from experimental or observational studies. In addition, specific pitfalls of evaluating change in meta-analysis studies are described. A set of recommendations is provided. This review is intended as an introduction for the novice and as a refresher for the experienced researcher.

Best (but oft-forgotten) practices: expressing and interpreting associations and effect sizes in clinical outcome assessments

This article reviews methods used to facilitate the interpretation and evaluation of group-level differences in clinical outcome assessments. These methods complement and supplement tests of statistical significance. Examples, including studies in nutrition, are used to illustrate the application of the interpretation methods for group-level comparisons from experimental or observational studies. In addition, specific pitfalls of evaluating change in meta-analysis studies are described. A set of recommendations is provided. This review is intended as an introduction for the novice and as a refresher for the experienced researcher.

Evaluating clinically significant changes in health-related quality of life: A sample of relatives of patients with severe traumatic brain injury

The objective of the study was to investigate change and predictors of change in health-related quality of life (HRQoL) in relatives of patients with severe traumatic brain injury (TBI) during rehabilitation, and to analyse associations between changes in HRQoL and symptoms of anxiety and depression. The Vitality (VT), Mental Health (MH), Social Function (SF) and the Role Emotional (RE) scales from the Short Form 36, and the anxiety and depression scales from the Symptom Checklist-90 - Revised were used. Of the 62 relatives, 24.6% experienced a reliable improvement on the VT scale, 53.2% on the MH scale, 27.4% on the SF scale and 16.1% on the RE scale. Of the relatives, 24.0% experienced clinically significant change (CSC) on the VT scale, 19.6% on the MH scale, 21.6% on the SF scale, and 19.2% on the RE scale. Relatives' age and higher patient Glasgow Coma Scale score predicted the experience of CSC in RE, and change on the Functional Independence Measure in patients predicted CSC on the MH scale. Improvements in VT as well as MH were associated with improvement in symptoms of anxiety and depression, and improvement in SF was associated with improvement in symptoms of depression. About one-fifth of the sample experienced a CSC on one of the four HRQoL measures. Relatives experiencing CSC tended to be related to patients who showed more improvement during rehabilitation. Improvements in HRQoL were associated with improvements in symptoms of anxiety and depression.

Practice Guidelines for the Assessment of Clinically Significant Treatment Outcomes in the Children's Mental Health System

The use of program evaluation to monitor client change and improve intervention effectiveness is gaining increasing importance in the mental health field. However, there is a lack of literature available in community-based clinics for those who desire to evaluate the effectiveness of services. Through this article the authors review the literature on the best methods to assess clinically significant treatment outcomes in community-based children's mental health services. The strengths and weaknesses of commonly recommended methods of evaluating change are discussed (i.e., reliable change index, percentage of improvement, normative comparisons, and effect size) using a dataset from a community-based parenting program (N = 308).

Positive Outcomes of Varicose Vein Surgery: The Patient Perspective

BACKGROUND

Patient-reported outcomes are increasingly recognized as important to understanding outcomes of medical interventions such as varicose vein surgery (VVS). Our aim was to compare positive outcomes of VVS as defined by several patient-reported measures, and to identify baseline characteristics associated with positive outcomes of VVS.

METHODS

A secondary analysis of the UK Patient-Reported Outcome Measures database was conducted on patients undergoing VVS, in the period 2009-2011 who completed the generic EQ-5D (index and visual analog scale [VAS] summary scores) and disease-specific Aberdeen varicose vein questionnaire (AVVQ). Surgical outcome was defined as positive if pre/post change scores exceeded half a standard deviation of mean baseline scores. Logistic regression models were used to identify significant predictors of positive outcomes, including age, gender, and baseline health.

RESULTS

Of 9,113 patients analyzed (71% females, 57% aged >50 years), positive outcomes were identified in 62% using the AVVQ, 43% based on EQ-5D index scores, and 24% according to EQ-VAS; 10% improved on all three measures. Patients with poorer baseline functioning (AVVQ scores ≥ 11) were more likely to have a positive outcome based on the EQ-5D index (odds ratio [OR] 1.23, 95% confidence interval [CI] 1.11-1.36) and EQ-VAS (OR 1.30, 95% CI 1.14-1.47).

CONCLUSIONS

Defining surgery as successful will clearly depend on how health-related quality of life (HRQL) is operationalized and the criteria used to identify meaningful change. Across a range of criteria, a consistently greater proportion of patients had positive outcomes in terms of VV-related functioning (via AVVQ) compared with those who improved in terms of generic health (via EQ-index), or self-rated health (EQ-VAS).

Modifying measures based on differential item functioning (DIF) impact analyses

OBJECTIVE

Measure modification can impact comparability of scores across groups and settings. Changes in items can affect the percent admitting to a symptom.

METHODS

Using item response theory (IRT) methods, well-calibrated items can be used interchangeably, and the exact same item does not have to be administered to each respondent, theoretically permitting wider latitude in terms of modification.

RESULTS

Recommendations regarding modifications vary, depending on the use of the measure. In the context of research, adjustments can be made at the analytic level by freeing and fixing parameters based on findings of differential item functioning (DIF). The consequences of DIF for clinical decision making depend on whether or not the patient's performance level approaches the scale decision cutpoint. High-stakes testing may require item removal or separate calibrations to ensure accurate assessment.

DISCUSSION

Guidelines for modification based on DIF analyses and illustrations of the impact of adjustments are presented.

Lumbar computerized adaptive test and Modified Oswestry Low Back Pain Disability Questionnaire: relative validity and important change

STUDY DESIGN

Retrospective analysis of longitudinal, observational cohort data.

OBJECTIVES

To compare discriminating ability and minimal clinically important improvement (MCII) calculated using functional status (FS) measures estimated from the lumbar computerized adaptive test (LCAT) and Modified Oswestry Low Back Pain Disability Questionnaire (ODQ).

BACKGROUND

The LCAT and ODQ are commonly used to estimate FS in patients seeking outpatient therapy but have not been compared directly.

METHODS

Data from 8198 adult patients who completed the LCAT and ODQ at intake were analyzed, 3379 (41%) of whom completed both surveys at discharge. Global ratings of change data were available for 980 patients. Discriminating ability of FS estimates from the LCAT and ODQ was estimated using relative validity, calculated by dividing F values from LCAT and ODQ analyses of covariance for important risk-adjustment variables. MCII was estimated using receiver-operating-characteristic analyses by quartiles of intake FS values, and areas under the curves were compared.

RESULTS

Relative validity ratios favored the LCAT for age (3.7; 95% confidence interval [CI]: 2.0, 8.9), acuity (1.3; 95% CI: 1.1, 1.6), comorbidities (1.8; 95% CI: 1.3, 2.6), and surgical history (1.8; 95% CI: 1.2, 2.9). MCII cut scores per quartile favored the LCAT. Receiver-operating-characteristic areas under the curves were not different.

CONCLUSION

FS measures estimated by both questionnaires had similar psychometric characteristics. The LCAT FS estimates tended to be more discriminating than ODQ FS estimates. MCII cut scores by quartile of intake FS favored the LCAT. Given the need to be efficient and precise in estimating measures of FS, particularly in older patients, results favor the LCAT in busy, automated outpatient therapy clinics, which are increasingly serving an aging population.

From Survivor to Thriver: A Pilot Study of an Online Program for Rape Victims

Approximately 15% to 20% of women have been victims of rape and close to a third report current rape-related PTSD or clinically significant depression or anxiety. Unfortunately, very few distressed rape victims seek formal help. This suggests a need to develop alternative ways to assist the many distressed victims of sexual violence. Online treatment programs represent a potentially important alternative strategy for reaching such individuals. The current paper describes a pilot evaluation of an online, therapist-facilitated, self-paced cognitive behavioral program for rape victims. Five college women with current rape-related PTSD were recruited to complete the From Survivor to Thriver (S to T) program in a lab setting over the course of 7 weeks. After completing the program, 4 participants reported clinically significant reductions in PTSD symptoms and no longer met criteria for PTSD. All participants reported clinically significant reductions in vulnerability fears and 4 reported significant reductions in negative trauma-related cognitions. Implications of the results for further development of the S to T program and how clinicians could utilize this program in treating rape-related PTSD are discussed.

Agreement about identifying patients who change over time: cautionary results in cataract and heart failure patients

BACKGROUND

Preference-based measures of health-related quality of life all use the same dead = 0.00 to perfect health = 1.00 scale, but there are substantial differences among measures.

OBJECTIVE

The objective was to examine agreement in classifying patients as better, stable, or worse.

METHODS

The EQ-5D, Health Utilities Index Mark 2 and Mark 3, Quality of Well-Being-Self-Administered scale, Short-Form 36 (Short-Form 6D), and disease-targeted measures were administered prospectively in 2 clinical cohorts. The study was conducted at academic medical centers: University of California, Los Angeles; University of California, San Diego; University of Wisconsin-Madison; and University of Southern California. Patients undergoing cataract extraction surgery with lens replacement completed the 25-item National Eye Institute Visual Function Questionnaire (NEI-VFQ-25). Patients newly referred to congestive heart failure specialty clinics completed the Minnesota Living with Heart Failure Questionnaire (MLHF). In both cohorts, subjects completed surveys at baseline and at 1 and 6 months. The NEI-VFQ-25 and MLHF were used as gold standards to assign patients to categories of change. Agreement was assessed using κ.

RESULTS

There were 376 cataract patients recruited. Complete data for baseline and the 1-month follow-up were available on all measures for 210 cases. Using criteria specified by Altman, agreement was poor for 6 of 9 pairs of comparisons and fair for 3 pairs. There were 160 heart failure patients recruited. Complete data for baseline and the 6-month follow-up were available for 86 cases. Agreement was negligible for 5 pairs and fair for 1. The study was conducted on selected patients at a few academic medical centers.

CONCLUSIONS

The results underscore the lack of interchangeability among different preference-based measures.

Interpreting patient-reported outcome results: US FDA guidance and emerging methods

In recent years, the US FDA has become more critical of instruments used to measure patient-reported outcomes (PROs) in clinical trials. To facilitate decisions related to the approval of drugs, labels and promotional claims based on PROs, the FDA created the Study Endpoints and Label Development (SEALD) group. SEALD has developed a PRO guidance related to the use of PRO measures used to support drug approvals and label claims, including recommendations for establishing thresholds for meaningful change at the individual level (i.e., defining a responder). This article examines in detail the FDA-recommended methodology for defining a responder and analyzing responder-based PRO measure results. We also present other responder analysis approaches for consideration in furthering the precision and interpretation of this methodology.

Minimally important differences of the UCLA Scleroderma Clinical Trial Consortium Gastrointestinal Tract Instrument

OBJECTIVE

To provide minimally important difference (MID) estimates for the UCLA Scleroderma Clinical Trial Consortium Gastrointestinal Tract 2.0 (UCLA SCTC GIT 2.0) in a longitudinal observational cohort.

METHODS

We administered the UCLA SCTC GIT 2.0 to 115 patients with systemic sclerosis (SSc) at 2 timepoints 6 months apart. The UCLA SCTC GIT 2.0 has 7 multi-item scales: Reflux, Distension/Bloating, Diarrhea, Fecal Soilage, Constipation, Emotional Well-being, and Social Functioning and a total GIT score. All scales are scored from 0 [better health-related quality of life (HRQOL)] to 3 (worse HRQOL) except the diarrhea and constipation scales (ranges 0-2 and 0-2.5, respectively). Patients also rated their overall and upper and lower GIT involvement during the second visit using a response scale with options "much better; somewhat better; almost the same; somewhat worse; or much worse." The minimally changed group was defined by those reporting they were somewhat better or somewhat worse compared to first visit.

RESULTS

Study participants were 84% female and 81% white with a mean disease duration of 6.9 years. The MID estimates for improvement ranged from 0.07 for the Social Functioning scale to 0.36 for the Emotional Well-being scale. For worsening, the MID estimates ranged from 0.06 for the Fecal Soilage scale to 0.21 for the Social Functioning scale.

CONCLUSION

We provide MID estimates for the UCLA SCTC GIT 2.0 scales. This information can aid in interpreting scale scores in future randomized controlled trials and observational studies.

Baseline dependency of minimal clinically important improvement

BACKGROUND

Minimal clinically important improvement (MCII) is the smallest outcome measure change important to patients. Research suggests that MCII is dependent on patients' baseline functional status measures.

OBJECTIVE

The purposes of this study were: (1) to confirm whether MCII is dependent on patients' admission scores and (2) to test whether MCII is dependent on selected demographic characteristics.

STUDY DESIGN AND SETTING

This was a prospective, longitudinal, observational cohort study of 6,651 patients with orthopedic knee impairments treated in 332 outpatient rehabilitation clinics in 27 states in the United States.

OUTCOME MEASURES

Patient self-reports of functional status (FS) from the Lower Extremity Functional Scale were assessed using a computerized adaptive testing application (0-100 scale).

METHODS

An anchored-based longitudinal method, with a 15-point Likert-type scale (-7 to +7), was used to provide a global rating of change (GROC). The MCII threshold for the GROC was defined at a cut-score of +3 or greater and was determined using nonparametric receiver operating characteristic curve analysis for each of the following variables: sex, symptom acuity, age group, and quartile of baseline FS scores.

RESULTS

The results showed that MCII was dependent on patient baseline and demographic characteristics. Patients who were male, were younger, had more-acute symptoms, or had lower FS scores at admission required more FS change to report meaningful change.

LIMITATIONS

As this study was a secondary analysis, how the length of treatment mediated the relationship between the independent and dependent variables was unclear.

CONCLUSIONS

Although a single MCII index may provide a standard cut-score defining the smallest FS change that is meaningful to patients, researchers and clinicians should be aware that MCII is context specific and not a fixed attribute. Current results may help researchers, clinicians, and policy makers to interpret FS change related to the importance of the change to the patient.

Interpreting patient-reported outcome results: US FDA guidance and emerging methods

In recent years, the US FDA has become more critical of instruments used to measure patient-reported outcomes (PROs) in clinical trials. To facilitate decisions related to the approval of drugs, labels and promotional claims based on PROs, the FDA created the Study Endpoints and Label Development (SEALD) group. SEALD has developed a PRO guidance related to the use of PRO measures used to support drug approvals and label claims, including recommendations for establishing thresholds for meaningful change at the individual level (i.e., defining a responder). This article examines in detail the FDA-recommended methodology for defining a responder and analyzing responder-based PRO measure results. We also present other responder analysis approaches for consideration in furthering the precision and interpretation of this methodology.

Minimally important differences of the gout impact scale in a randomized controlled trial

OBJECTIVE

The Gout Impact Scale (GIS) is a gout-specific quality of life instrument that assesses impact of gout during an attack and impact of overall gout. The GIS has five scales and each is scored from 0 to 100 (worse health). Our objective was to assess minimally important differences (MIDs) for the GIS administered in a randomized controlled trial (RCT) assessing rilonacept vs placebo for prevention of gout flares during initiation of allopurinol therapy.

METHODS

Trial subjects (n = 83) included those with two or more gout flares (self-reported) in the past year. Of these, 73 had data for Weeks 8 vs 4 and formed the MID analysis group and were analysed irrespective of the treatment assignment. Subjects completed the GIS and seven patient-reported anchors. Subjects with a one-step change (e.g. from very poor to poor) were considered as the MID group for each anchor. The mean change in GIS scores and effect size (ES) was calculated for each anchor's MID group. The average of these created the overall summary MID statistics for each GIS. An ES of 0.2-0.5 was considered to represent MID estimates. Results. Trial subjects (n = 73) were males (96.0%), White (90.4%), with mean age of 50.5 years and serum uric acid of 9.0 mg/dl. The mean change score for the MID improvement group for scales ranged from -5.24 to -7.61 (0-100 scale). The ES for the MID improvement group for the four scales ranged from 0.22 to 0.38.

CONCLUSION

The MID estimates for GIS scales are between 5 and 8 points (0-100 scale). This information can aid in interpreting the GIS results in future gout RCTs. Trial Registration. Clinicaltrials.gov, www.clinicaltrials.gov, NCT00610363.

Ability to detect change in patient function: responsiveness designs and methods of calculation

In hand clinics, the goal of enabling patient improvement is fostered by the use of assessments with the ability to detect change (responsiveness). Thus, for commonly used assessments, investigations are needed to determine a standardized change index, the amount of change exceeding error estimates (minimal detectable change or MDC), and the amount of change shown to make a clinically relevant difference (clinically important difference or CID). The purpose of study was to serve as an introduction for hand therapists to responsiveness designs and indices and to highlight their application within the clinical setting. The study design used was a narrative review. Method papers and research studies addressing responsiveness were selected and summarized. Currently, several good studies of responsiveness have been conducted. However, there is a need to move beyond the calculation of standardized change indices to include the calculation of clinically meaningful values. For many of the assessments used in hand clinics, there is still a call for investigation of the amount of change, which exceeds error estimates (MDC) and the amount of change shown to make a clinically relevant difference (CID).

LEVEL OF EVIDENCE

N/A.