Interpreting patient-reported outcome results: US FDA guidance and emerging methods

Comparable performance of the Kansas City Cardiomyopathy Questionnaire in patients with heart failure with preserved and reduced ejection fraction

BACKGROUND

Despite the growing epidemic of heart failure with preserved ejection fraction (HFpEF), no valid measure of patients' health status (symptoms, function, and quality of life) exists. We evaluated the Kansas City Cardiomyopathy Questionnaire (KCCQ), a validated measure of HF with reduced EF, in patients with HFpEF.

METHODS AND RESULTS

Using a prospective HF registry, we dichotomized patients into HF with reduced EF (EF≤ 40) and HFpEF (EF≥50). The associations between New York Heart Association class, a commonly used criterion standard, and KCCQ Overall Summary and Total Symptom domains were evaluated using Spearman correlations and 2-way ANOVA with differences between patients with HF with reduced EF and HFpEF tested with interaction terms. Predictive validity of the KCCQ Overall Summary scores was assessed with Kaplan-Meier curves for death and all-cause hospitalization. Covariate adjustment was made using Cox proportional hazards models. Internal reliability was assessed with Cronbach's α. Among 849 patients, 200 (24%) had HFpEF. KCCQ summary scores were strongly associated with New York Heart Association class in both patients with HFpEF (r=-0.62; P<0.001) and HF with reduced EF (r=-0.55; P=0.27 for interaction). One-year event-free rates by KCCQ category among patients with HFpEF were 0 to 25=13.8%, 26 to 50=59.1%, 51 to 75=73.8%, and 76 to 100=77.8% (log rank P<0.001), with no significant interaction by EF (P=0.37). The KCCQ domains demonstrated high internal consistency among patients with HFpEF (Cronbach's α=0.96 for overall summary and ≥0.69 in all subdomains).

CONCLUSIONS

Among patients with HFpEF, the KCCQ seems to be a valid and reliable measure of health status and offers excellent prognostic ability. Future studies should extend and replicate our findings, including the establishment of its responsiveness to clinical change.

Decay of impact after self-management education for people with chronic illnesses: changes in anxiety and depression over one year

BACKGROUND

In people with chronic illnesses, self-management education can reduce anxiety and depression. Those benefits, however, decay over time. Efforts have been made to prevent or minimize that "decay of impact", but they have not been based on information about the decay's characteristics, and they have failed. Here we show how the decay's basic characteristics (prevalence, timing, and magnitude) can be quantified. Regarding anxiety and depression, we also report the prevalence, timing, and magnitude of the decay.

METHODS

Adults with various chronic conditions participated in a self-management educational program (n = 369). Data were collected with the Hospital Anxiety and Depression Scale four times over one year. Using within-person effect sizes, we defined decay of impact as a decline of ≥0.5 standard deviations after improvement by at least the same amount. We also interpret the results using previously-set criteria for non-cases, possible cases, and probable cases.

RESULTS

Prevalence: On anxiety, decay occurred in 19% of the participants (70/369), and on depression it occurred in 24% (90/369). Timing: In about one third of those with decay, it began 3 months after the baseline measurement (6 weeks after the educational program ended). Magnitude: The median magnitudes of decay on anxiety and on depression were both 4 points, which was about 1 standard deviation. Early in the follow-up year, many participants with decay moved into less severe clinical categories (e.g., becoming non-cases). Later, many of them moved into more severe categories (e.g., becoming probable cases).

CONCLUSIONS

Decay of impact can be identified and quantified from within-person effect sizes. This decay occurs in about one fifth or more of this program's participants. It can start soon after the program ends, and it is large enough to be clinically important. These findings can be used to plan interventions aimed at preventing or minimizing the decay of impact.

Interpreting effect sizes and clinical relevance of pharmacological interventions for fibromyalgia

Duloxetine, milnacipran, and pregabalin are approved by the United States Food and Drug Administration for the management of fibromyalgia. A number of meta-analyses, pooled analyses, and systematic reviews have been published in recent years involving the efficacy of these three medications for pain in fibromyalgia. Despite being based on the same clinical data, some analyses found these treatments to have a clinically relevant effect on pain, while others concluded that the advantages were small or of questionable clinical relevance. This commentary discussed possible reasons behind these differing conclusions and explored ways of evaluating the clinical relevance of pharmacological treatments for fibromyalgia. In particular, we considered: (1) the importance of judicious and careful interpretation of average treatment effect size and the recognition that average treatment effect sizes do not always tell the whole story; (2) the utility of individual patient response data to assess clinical relevance; and (3) the importance of considering pain reduction within the context of other benefits due to the presence of associated symptoms in patients with fibromyalgia.

An evaluation of the FDA responder endpoint for IBS-C clinical trials: analysis of data from linaclotide Phase 3 clinical trials

BACKGROUND

Our objective was to evaluate the performance of the Food and Drug Administration (FDA) Responder Endpoint for clinical trials in IBS-C, using data from two large Phase 3 clinical trials of linaclotide. The FDA interim endpoint requires that, for 50% of trial weeks, patients report ≥30% decrease in Abdominal Pain at its worst and (in the same week) an increase in Complete Spontaneous Bowel Movements (CSBMs) of ≥1 from baseline.

METHODS

Anchor-based methodology was used to estimate thresholds of clinically meaningful change using symptom-specific patient rating of change questions (PRCQs) and symptom severity questions. The diagnostic accuracy of the FDA Responder Endpoint was assessed using sensitivity/specificity-based methods.

KEY RESULTS

Using anchor-based methods, the estimates of the clinically meaningful improvement thresholds for Abdominal Pain ranged from 25.9% to 32.4% and thresholds for increase in weekly CSBM rate ranged from 1.4 to 1.6 CSBMs per week. Compared with the symptom-specific PRCQs for patient rating of relief, the FDA Responder Endpoint has a sensitivity of 60.7%, a specificity of 93.5%, and an accuracy of 82.0%. Changing the number of weeks required to be a responder or the percentage improvement in the Abdominal Pain criteria did not result in notable improvement in the accuracy of the FDA Responder Endpoint.

CONCLUSIONS & INFERENCES

The FDA Responder Endpoint for IBS-C clinical trials represents clinically meaningful improvements in IBS-C symptoms for patients with excellent specificity and reasonable sensitivity.

Don't middle your MIDs: regression to the mean shrinks estimates of minimally important differences

Minimal important differences (MIDs) for patient-reported outcomes (PROs) are often estimated by selecting a clinical variable to serve as an anchor. Then, differences in the clinical anchor regarded as clinically meaningful or important can be used to estimate the corresponding value of the PRO. Although these MID values are sometimes estimated by regression techniques, we show that this is a biased procedure and should not be used; alternative methods are proposed.

Interpretation of patient-reported outcomes

A patient-reported outcome is any report on the status of a patient’s health condition that comes directly from the patient. Clear and meaningful interpretation of patient-reported outcome scores are fundamental to their use as they can be valuable in designing studies, evaluating interventions, educating consumers, and informing health policy makers involved with regulatory, reimbursement, and advisory agencies. Interpretation of patient-reported outcome scores, however, is often not well understood because of insufficient data or lack of experience or clinical understanding to draw from. This article provides an update review on two broad approaches – anchor-based and distributed-based – aimed at enhancing the understanding and meaning of patient-reported outcome scores. Anchor-based approaches include percentages based on thresholds, criterion-group interpretation, content-based interpretation, and clinical important difference. Distributed-based approaches include effect size, probability of relative benefit, and responder analysis and cumulative proportions. A third strategy called mediation analysis, which can elucidate a health condition measured by a patient-reported outcome in the context of an intervention’s mechanism of action, is also highlighted and illustrated. Mediation analysis in the context of interpretation of patient-reported outcome scores is a relatively new development. The logic and rationale of the three methods are expressed generally. While the three approaches themselves are not new, some applications of them taken from their examples published in the past few years are original and coalesced in this article to add real-life implications of the different methodologies in one integrated report.

Development and validation of a symptom index for advanced hepatobiliary and pancreatic cancers: the National Comprehensive Cancer Network Functional Assessment of Cancer Therapy (NCCN-FACT) Hepatobiliary-Pancreatic Symptom Index (NFHSI)

BACKGROUND

The 45-item Functional Assessment of Cancer Therapy-Hepatobiliary (FACT-Hep) questionnaire assesses health-related quality of life in patients with liver, bile duct, and pancreatic cancers. Although the FACT-Hep was initially derived from patient input, this study's researchers sought to verify adequate coverage of items by soliciting open-ended input from patients with advanced disease.

METHODS

As part of a larger study in collaboration with the National Comprehensive Cancer Network (NCCN), 50 people (60% male, 80% caucasian, average age 60.4 years) with stage 3 or 4 hepatobiliary or pancreatic cancer were recruited. Participants generated and ranked up to 10 important symptoms and concerns that physicians should monitor when assessing the value of chemotherapy. Patients were also able to provide open-ended, qualitative information that was evaluated systematically. Ten expert physicians also provided input on priority symptoms.

RESULTS

The resulting 18-item NCCN-FACT Hepatobiliary-Pancreatic Symptom Index (NFHSI-18) demonstrated high internal consistency (α = .89) and moderate to strong correlations with measures of physical well-being (ρ = .76), emotional well-being (ρ = 0.52), and functional well-being (ρ = 0.57). Scores on the NFHSI-18 were also highly correlated with the original hepatobiliary scale of the FACT-Hep (ρ = .82; all P < .001). Compared with patients with better performance status, patients with poor performance status had worse NFHSI-18 symptom scores, F(3,47) = 9.74; P = .0003.

CONCLUSIONS

The NFHSI-18 assesses symptoms of importance to patients with hepatobiliary and pancreatic cancers and demonstrates promising measurement properties. The scale is a good candidate for brief symptom assessment in clinical trials.

Quality of life and technology: impact on children and families with diabetes

Ensuring quality of life (QOL) while maintaining glycemic control within targets is an important challenge in type 1 and type 2 diabetes treatment. For children with diabetes, QOL includes enjoying meals, feeling safe in school, and perceiving positive, supportive relationships with parents, siblings, and friends. Yet many treatment-related and psychosocial barriers can interfere with a child's QOL and their ability to manage diabetes effectively. Diabetes management also imposes considerable lifestyle demands that are difficult and often frustrating for children to negotiate at a young age. Recent advances in diabetes medications and technologies have improved glycemic control in children with diabetes. Two widely used technologies are the insulin pump and continuous glucose monitoring (CGM) system. These technologies provide patients with more flexibility in their daily life and information about glucose fluctuations. Several studies report improvements in glycemic control in children with type 1 diabetes using the insulin pump or sensor-augmented pump therapy. Importantly, these technologies may impact QOL for children and families with diabetes, although they are rarely used or studied in the treatment of children with type 2 diabetes. Further, emerging closed loop and web- and phone-based technologies have great potential for supporting diabetes self-management and perhaps QOL. A deeper understanding and appreciation of the impact of diabetes technology on children's and parents' QOL is critical for both the medical and psychological care of diabetes. Thus, the purpose of this review is to discuss the impact of new diabetes technologies on QOL in children, adolescents and families with type 1 diabetes.

Validation of the Knee Injury and Osteoarthritis Outcome Score subscales for patients with articular cartilage lesions of the knee

BACKGROUND

The Knee Injury and Osteoarthritis Outcome Score (KOOS) assesses acute and chronic knee injuries or early-onset osteoarthritis in young, active patients. The United States Food and Drug Administration guidelines recommend that patient-reported outcome instruments used to support clinical trial label claims should demonstrate content validity using patient input and have acceptable psychometric properties in the target population. To use the KOOS subscales in safety and efficacy trials assessing new treatments for patients with articular cartilage lesions, additional validation work, using input from patients with articular cartilage lesions, was necessary.

PURPOSE

Qualitative and quantitative evaluations of the KOOS subscales' validity among patients with articular cartilage lesions were conducted to support their use as clinically meaningful end points in clinical trials.

STUDY DESIGN

Cohort study (diagnosis); Level of evidence, 2.

METHODS

For qualitative analysis, cognitive interviews involving concept elicitation and cognitive debriefing with the KOOS items were conducted with 15 participants aged 25 to 52 years. Participants either were candidates for cartilage repair or had undergone cartilage repair 6 months or more before the study. For the quantitative analysis, a psychometric evaluation of the KOOS was conducted with clinical trial data from 54 patients, aged 18 to 55 years, evaluating the Cartilage Autograft Implantation System in the United States (n = 29) and the European Union (n = 25). Data were collected before surgery and at 7 postsurgical visits up to 12 months. Internal consistency and test-retest reliability, construct validity, responsiveness, and estimates of the minimal detectable change (MDC) were assessed. Test-retest reliability was assessed using data from months 2 and 3 on a subset of stable patients.

RESULTS

Qualitative research confirmed that concepts measured on the KOOS are important to patients with articular cartilage lesions. Most participants reported the KOOS was comprehensive and appropriate. In the quantitative research, KOOS subscales showed excellent internal consistency reliability (range, .74-.97 at baseline) and test-retest reliability (range, .78-.82). Construct validity results supported hypothesized relationships, with significant correlations (r ≥ .50) in the expected directions. Responsiveness analyses demonstrated excellent sensitivity to change; standardized response means ranged from 0.8 to 1.2, and MDC estimates ranged from 7.4 to 12.1.

CONCLUSION

The study results support the use of the KOOS subscales among patients with articular cartilage lesions.

Cumulative response curves to enhance interpretation of treatment differences on the Self-Esteem And Relationship questionnaire for men with erectile dysfunction

What's known on the subject? and What does the study add? Studies on erectile dysfunction (ED) therapies rely heavily on patient-reported outcomes (PROs) to measure efficacy on treatment response. A challenge when using PROs is interpretation of the clinical meaning of changes in scores. A responder analysis provides a threshold score to indicate whether a change in score qualifies a patient as a responder. However, a major consideration with responder analysis is the sometimes arbitrary nature of defining the threshold for a response. By contrast, cumulative response curves (CRCs) display patient response rates over a continuum of possible thresholds, thus eliminating problems with a rigid threshold definition, allowing for a variety of response thresholds to be examined simultaneously, and encompassing all data. With respect to the psychosocial factors addressed in the Self-Esteem And Relationship questionnaire in ED, CRCs clearly, distinctly, and meaningfully highlighted the favourable profiles of responses to sildenafil compared with placebo. CRCs for PROs in urology can provide a clear, transparent and meaningful visual depiction of efficacy data that can supplement and complement other analyses.

OBJECTIVE

To use cumulative response curves (CRCs) to enrich meaning and enhance interpretation of scores on the Self-Esteem And Relationship (SEAR) questionnaire with respect to treatment differences for men with erectile dysfunction (ED).

PATIENTS AND METHODS

This post hoc analysis used data from all patients who took at least one dose of study drug and had at least one post-baseline efficacy evaluation in a previously published 12-week, multicentre, randomized, double-blind, placebo-controlled trial of flexible-dose (25, 50, or 100 mg) sildenafil citrate (Viagra) in adult men with ED who had scored ≤ 75 out of 100 on the Self-Esteem subscale of the SEAR questionnaire. CRCs were used on the numeric change in transformed SEAR scores from baseline to end-of-study for each SEAR component. The horizontal axis of the CRC represented change from baseline on the SEAR score, and the vertical axis represented the percentage of patients experiencing that change or greater. The differences between CRCs for the sildenafil group vs the placebo group were assessed using the Kolmogorov-Smirov test.

RESULTS

For each of the SEAR components, there was essentially no overlap in the CRCs between the sildenafil group (n = 113) and placebo group (n = 115 or 116, depending on the component), showing that a greater percentage of sildenafil recipients compared with placebo recipients had a more favourable change across the spectrum of response thresholds (P ≤ 0.01). Previous research showed that a 10-point score increase is the minimal clinically meaningful improvement for most SEAR components. In the sildenafil vs placebo groups, a ≥10-point score increase occurred in 72 vs 37% of patients, respectively, on the Sexual Relationship Satisfaction domain, 71 vs 41% on the Confidence domain, 76 vs 49% on the Self-Esteem subscale, 60 vs 44% on the Overall Relationship Satisfaction subscale, and 75 vs 38% on the Overall score.

CONCLUSIONS

With respect to the psychosocial factors addressed in the SEAR questionnaire, CRCs clearly, distinctly, and meaningfully highlighted the favourable profiles of responses to sildenafil compared with placebo. CRCs for patient-reported outcomes in urology can provide a clear, transparent, and meaningful visual depiction of efficacy data that can supplement and complement other analyses.

Conceptual and Analytical Considerations toward the Use of Patient-Reported Outcomes in Personalized Medicine

BACKGROUND

Patient-reported outcomes (PROs) can play an important role in personalized medicine. PROs can be viewed as an important fundamental tool to measure the extent of disease and the effect of treatment at the individual level, because they reflect the self-reported health state of the patient directly. However, their effective integration in personalized medicine requires addressing certain conceptual and methodological challenges, including instrument development and analytical issues.

OBJECTIVES

To evaluate methodological issues, such as multiple comparisons, missing data, and modeling approaches, associated with the analysis of data related to PRO and personalized medicine to further our understanding on the role of PRO data in personalized medicine.

DISCUSSION

There is a growing recognition of the role of PROs in medical research, but their potential use in customizing healthcare is not widely appreciated. Emerging insights into the genetic basis of PROs could potentially lead to new pathways that may improve patient care. Knowledge of the biologic pathways through which the various genetic predispositions propel people toward negative or away from positive health experiences may ultimately transform healthcare. Understanding and addressing the conceptual and methodological issues in PROs and personalized medicine are expected to enhance the emerging area of personalized medicine and to improve patient care. This article addresses relevant concerns that need to be considered for effective integration of PROs in personalized medicine, with particular reference to conceptual and analytical issues that routinely arise with personalized medicine and PRO data. Some of these issues, including multiplicity problems, handling of missing values-and modeling approaches, are common to both areas. It is hoped that this article will help to stimulate further research to advance our understanding of the role of PRO data in personalized medicine.

CONCLUSION

A robust conceptual framework to incorporate PROs into personalized medicine can provide fertile opportunity to bring these two areas even closer and to enhance the way a specific treatment is attuned and delivered to address patient care and patient needs.

Asthma outcome measures

Asthma is a common chronic disease with underlying inflammation of the airway. Advances in science have led to increased understanding of the heterogeneous nature of asthma and its complex mechanisms. Traditionally, asthma-practice guidelines have focused on optimizing lung function and the US FDA has required increases in lung function and reduction of exacerbation as primary outcomes in clinical trials of new asthma therapeutics. Improved lung function is a critical indicator of bronchodilator therapy, but the importance of long-term asthma control while maintained on controller medication is increasingly emphasized. The NIH asthma guidelines suggest the use of patient-reported outcomes, including health-related quality-of-life measures, to assess asthma control. Clinical practices and research studies concerning asthma can benefit from harmonizing the major outcome measures so that comparisons across studies can be made. In this article, we review common asthma outcome measures with a focus on recent efforts to harmonize outcomes for therapeutic clinical trials in asthma.

Agreement about identifying patients who change over time: cautionary results in cataract and heart failure patients

BACKGROUND

Preference-based measures of health-related quality of life all use the same dead = 0.00 to perfect health = 1.00 scale, but there are substantial differences among measures.

OBJECTIVE

The objective was to examine agreement in classifying patients as better, stable, or worse.

METHODS

The EQ-5D, Health Utilities Index Mark 2 and Mark 3, Quality of Well-Being-Self-Administered scale, Short-Form 36 (Short-Form 6D), and disease-targeted measures were administered prospectively in 2 clinical cohorts. The study was conducted at academic medical centers: University of California, Los Angeles; University of California, San Diego; University of Wisconsin-Madison; and University of Southern California. Patients undergoing cataract extraction surgery with lens replacement completed the 25-item National Eye Institute Visual Function Questionnaire (NEI-VFQ-25). Patients newly referred to congestive heart failure specialty clinics completed the Minnesota Living with Heart Failure Questionnaire (MLHF). In both cohorts, subjects completed surveys at baseline and at 1 and 6 months. The NEI-VFQ-25 and MLHF were used as gold standards to assign patients to categories of change. Agreement was assessed using κ.

RESULTS

There were 376 cataract patients recruited. Complete data for baseline and the 1-month follow-up were available on all measures for 210 cases. Using criteria specified by Altman, agreement was poor for 6 of 9 pairs of comparisons and fair for 3 pairs. There were 160 heart failure patients recruited. Complete data for baseline and the 6-month follow-up were available for 86 cases. Agreement was negligible for 5 pairs and fair for 1. The study was conducted on selected patients at a few academic medical centers.

CONCLUSIONS

The results underscore the lack of interchangeability among different preference-based measures.