Likely change indexes improve estimates of individual change on patient-reported outcomes

Health-Related Quality of Life During Carfilzomib-Lenalidomide-Dexamethasone Consolidation: Findings From the Multiple Myeloma CONPET Study

BACKGROUND

In the CONPET study, multiple myeloma patients with abnormal 18FDG positron emission/computed tomography scan after upfront autologous stem cell transplantation were treated with four cycles of carfilzomib-lenalidomide-dexamethasone (KRd). Side effect registrations show that carfilzomib might cause dyspnea, cough, respiratory tract infections, and heart failure. The aims were to investigate patient-reported shortness of breath and dyspnea during KRd consolidation.

METHODS

To assess shortness of breath, patients completed the Functional Assessment of Cancer Therapy-Pulmonary Symptom Index (FACT-PSI) and the EORTC QLQ-C30 to assess dyspnea. Shortness of breath was defined as decrease in FACT-PSI score or starting/increasing diuretic drugs. Mixed effect logistic regression was used for the effect analysis. Linear mixed model and clinical relevance were used to investigate dyspnea.

RESULTS

A total of 50 patients were included, median age 62 years (interquartile range 54-67). 17% reported shortness of breath at Day 15 Cycles 1-4 versus 11% at Day 1 Cycles 2-4, Cycle 4 Day 29, and 1 month posttreatment (p-value 0.048). Compared with baseline, patients reported significant, and clinically relevant worsening in dyspnea during consolidation.

CONCLUSION

Our study confirmed earlier findings of carfilzomib causing shortness of breath during KRd administration and revealed dyspnea during consolidation compared to baseline.

TRIAL REGISTRATION

Clinicaltrials.gov: NCT03314636, EudraCT: 2017-000586-72.

Clinically meaningful improvements after gene therapy for aromatic L-amino acid decarboxylase deficiency (AADCd) in the Peabody Developmental Motor Scale, Second Edition (PDMS-2) and correlation with Bayley-III scores and motor milestones

BACKGROUND

Aromatic L-amino acid decarboxylase deficiency (AADCd) is a rare genetic disorder characterized by movement disorders, motor and autonomic dysfunction, and developmental delays. The gene therapy eladocagene exuparvovec has become available in some regions; pooled clinical trial results demonstrate continuous long-term improvement in motor development and cognitive function. We sought to characterize clinically meaningful change in motor function, as measured by Total Peabody Developmental Motor Scales-Second Edition (PDMS-2) score, and assess correlations with cognition and language domains of the Bayley-III and motor milestone (MM) achievement.

METHODS

Data from N = 30 patients from three single-arm clinical studies of eladocagene exuparvovec were analyzed. Anchor-based estimation of the meaningful score difference (MSD) of Total PDMS-2 score was conducted using mean-difference and receiver operating characteristic curve (ROC) approaches. MM achievement served as the anchor defining meaningful change.

RESULTS

An MSD of 40 points for Total PDMS-2 score was selected for analysis as it yielded specificity > 0.95 using the ROC approach, and generally aligned with the mean-difference approach. Cumulative incidence analysis reflected that 50% of patients treated with eladocagene exuparvovec may achieve the MSD of 40-point change in Total PDMS-2 score at 6 months, and 86% at 18 months. Correlations between measures were of large magnitude and improved over time (Month 6: r = 0.599 [p = 0.0032]; Month 18: r = 0.796 [p = 0.0002]; Month 60: r = 0.861 [p = 0.0007]).

CONCLUSIONS

The MSD of 40 points for Total PDMS-2 score enables the interpretation of changes observed in patients with AADCd, and suggests that treatment with eladocagene exuparvovec leads to significant improvements in motor and cognitive function.

A perspective on the use of patient-reported experience and patient-reported outcome measures in ambulatory healthcare

INTRODUCTION

Patient-reported experience measures (PREMs) are patient reports about their healthcare, whereas patient-reported outcome measures (PROMs) are reports about their functioning and well-being regarding physical, mental, and social health. We provide a perspective on using PREMs and PROMs in ambulatory healthcare.

AREAS COVERED

We conducted a narrative review of the literature about using PREMs and PROMs in research and clinical practice, identified challenges and possibilities for addressing them, and provided suggestions for future research and clinical practice.

EXPERT OPINION

Substantial progress in using PREMs and PROMs has occurred during the last half-century. Collecting and reporting PREMs to clinicians in ambulatory care settings has improved communication with patients, diagnosis, and treatment, which may improve patients' health. Optimal use requires appropriate data analysis, minimizing implementation barriers, and facilitating interpretation of PREMs and PROMs in clinical practice. Also, formal structures and processes that include patient and family input into care improvement are needed (e.g. patient and family advisory councils as partners in co-design and coproduction of quality improvement). PREMs and PROMs have been used primarily in more affluent countries (e.g. the United States, Australia, United Kingdom, Netherlands, Japan, and Portugal), but this is expected to increase in many countries.

Significant individual change should be used as a lower bound for anchor based estimates of meaningful change on patient-reported outcome scores

Interpretation of patient-reported outcome (PRO) scores has been supported by identifying score thresholds or ranges that indicate clinical importance. There has been a recent focus on the estimation of meaningful within patient change (MWPC). While much attention has been focused on anchor-based methods, some researchers prefer that a lower bound to these estimates should exceed a change score that could be observed due to measurement error alone as a safeguard against misclassifying individual patients as changed when they have not. The standard error of measurement (SEM) is often used as the lower bound of anchor estimates. Here, we argue that the SEM is not an the best lower bound for MWPCs. Instead, statistically significant individual change as calculated by the reliable change index (RCI) should be used as the lower bound. Our argument is based on two points. First, conceptually, the SEM does not provide specific enough information to serve as a lower bound for MWPCs, which should be based on the level of observed score change that is unlikely to be due to chance alone. Second, the SEM is not appropriate for direct application to observed scores, and requires a multiplier when examining observed change instead of true change. We conclude with recommendations for using the RCI with a thoughtful range of p-values in combination with anchor estimates.

Interstitial Lung Disease Patients' Global Impressions of Symptoms, Severity Ratings, and Meaningfulness of Changes Over Time

Rationale: In interstitial lung disease (ILD), symptoms drive impairments in health-related quality of life. Patient-reported outcome measures (PROMs) can assess whether interventions change symptom severity. The meaningfulness of change in a PROM score is estimated by anchoring it to a related variable for which meaningful change has been previously established. Patient global impressions of severity (PGISs) are single-item PROMs that may make trustworthy anchors, but, for ILD, the meaningfulness of change in PGIS items for shortness of breath (SOB), cough, and fatigue/low energy are unknown. Objectives: To improve understanding of how patients with ILD rate and categorize symptoms, how differing levels of symptom severity affect lived experiences, and how patients derive and apply meaningfulness to change in symptoms. Methods: We used one-on-one interviews and an electronic survey to collect data from patients with various forms of ILD. Interviews were conducted to provide richness and context to survey responses. We conducted certain analyses with respondents stratified by supplemental oxygen use. Results: Interviewees (N = 18) confirmed SOB, cough, and fatigue/low energy as the most bothersome symptoms of ILD. Among 298 survey respondents, on a PGIS for SOB with a 0-4 numeric rating scale, on average, those who used supplemental O2 had more severe SOB than nonusers, and most respondents considered a 2-point change meaningful for worsening (45.5%) or improvement (47.2%). On a PGIS with a five-option ordinal response scale, for SOB, most considered a 1-category change meaningful for worsening (49.8%) and a 2-category change meaningful for improvement (42.3); for cough frequency, most respondents considered a 1-category change on the five-option ordinal response scale meaningful for worsening (48.2%) or improvement (45.0%). Survey responses for SOB at the present time versus 3 months earlier (patient global impressions of change) were biased toward the present state. Conclusions: PGISs can be used as anchors for meaningful change analyses of PROMs that assess SOB or cough in patients with ILD. Patient global impressions of change demonstrate present-state bias and should not be used. Patients' descriptions paint a vivid picture of lived experience with varying levels of symptom severity and can help contextualize change scores.

The minimal important change is not a universal fixed value across diagnoses when using the FAOS and FAAM in patients undergoing elective foot and ankle surgery

PURPOSE

This study aimed to calculate region and diagnosis-specific minimal important changes (MICs) of the Foot and Ankle Outcome Score (FAOS) and the Foot and Ankle Ability Measure (FAAM) in patients requiring foot and ankle surgery and to assess their variability across different foot and ankle diagnoses.

METHODS

The study used routinely collected data from patients undergoing elective foot and ankle surgery. Patients had been invited to complete the FAOS and FAAM preoperatively and at 3-6 months after surgery, along with two anchor questions encompassing change in pain and daily function. Patients were categorised according to region of pathology and subsequent diagnoses. MICs were calculated using predictive modelling (MICPRED) and receiver operating characteristic curve (MICROC) method and evaluated according to strict credibility criteria.

RESULTS

Substantial variability of the MICs between forefoot and ankle/hindfoot region was observed, as well as among specific foot and ankle diagnoses, with MICPRED and MICROC values ranging from 7.8 to 25.5 points and 9.4 to 27.8, respectively. Despite differences between MICROC and MICPRED estimates, both calculation methods exhibited largely consistent patterns of variation across subgroups, with forefoot conditions systematically showing smaller MICs than ankle/hindfoot conditions. Most MICs demonstrated high credibility; however, the majority of the MICs for the FAOS symptoms subscale and forefoot conditions exhibited insufficient or low credibility.

CONCLUSION

The MICs of the FAOS and FAAM vary across foot and ankle diagnoses in patients undergoing elective foot and ankle surgery and should not be used as a universal fixed value, but recognised as contextual parameters. This can help clinicians and researchers in more accurate interpretation of the FAOS and FAAM change scores.

LEVEL OF EVIDENCE

Level IV.

Validity, reliability, responsiveness, and clinically meaningful change threshold estimates of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16)

BACKGROUND

Breast cancer is one of the most common cancers in women. Patient-reported outcome measures are used to evaluate patients' health-related quality of life in clinical breast cancer studies. This study evaluated the structure, validity, reliability, and responsiveness of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16) subscales in a clinical trial featuring patients with advanced/metastatic breast cancer (aBC), and estimated NFBSI-16 meaningful change thresholds.

METHODS

Data from 101 patients with aBC enrolled in a phase II trial (Xenera-1) were included for psychometric evaluation of the NFBSI-16. Subscale structure was evaluated by assessing inter-item correlations, item-total correlations, and internal consistency (cycles 2 and 5). Validity was assessed using scale-level convergent validity (cycles 2 and 5) and known-groups (Baseline). Reliability was analysed via test-retest at cycles 3-4, and responsiveness to improvement and worsening was evaluated at cycles 5, 7, and 9. Meaningful change thresholds were estimated using anchor-based methods (supported by distribution-based methods) at cycles 5, 7, and 9.

RESULTS

NFBSI-16 internal consistency was acceptable, but item-total correlations suggested that its subscales and the GP5 item (side-effect of treatment) scores may be preferred over a total score. Convergent and known-groups evidence supported NFBSI-16 validity. Test-retest reliability was good to excellent for Total and DRS-P (disease-related symptoms: physical) scales, and moderate for the GP5 item. Responsiveness to worsening was generally demonstrated, but responsiveness to improvement could not be demonstrated due to limited observed improvement. Anchor-based meaningful change thresholds were estimated for DRS-P and Total scores.

CONCLUSION

This study provides evidence that the NFBSI-16 has desirable psychometric properties for use in clinical studies in aBC. It also provides estimates of group- and individual-level meaningful change thresholds to facilitate score interpretation in future aBC research.

Linking QuickDASH and PROMIS Upper-Extremity Computer-Adaptive Test Scores in Hand Surgery: A Crosswalk Study

PURPOSE

Assessment of patient-reported outcome measures (PROMs) for hand and upper-extremity surgery patients using measures such as the Quick Disabilities of the Arm, Shoulder, and Hand (qDASH), as well as general measures including the Patient-Reported Outcomes Measurement Information System Upper Extremity Physical Function domain via a Computer-Adaptive Test (PROMIS UE CAT), has become commonplace. The aim of this study was to link, for crosswalking, the qDASH measure to both versions of the PROMIS UE CAT (v1.2 and v2.0).

METHODS

We included 18,944 hand and upper-extremity patients who completed both versions of the PROMIS UE CAT and the qDASH at the same clinical encounter. Shoulder pathology was excluded. Score linkage was performed using the R package equate, and multiple equating models (linear regression, identity, mean, linear, equipercentile, and circle-arc models) were used to establish crosswalk tables.

RESULTS

Mean qDASH and PROMIS UE CAT v1.2 scores were 38.2 (SD = 23.1) and 36.6 (SD = 9.8), respectively. Mean qDASH and PROMIS UE CAT v2.0 scores were 37.3 (SD = 21.8) and 38.3 (SD = 10.4), respectively. Pearson correlations had very strong linear relationships between the qDASH and the PROMIS UE CAT v1.2 and PROMIS UE CAT v2.0 (r = -0.83 [-0.84, -0.92] and r = -0.80 [-0.81, -0.80], respectively). For the equipercentile equating models, the intraclass correlation coefficient (ICC) had very strong positive relationships to linking measures with ICC = 0.85 (0.84, 0.86) for the qDASH-UE CAT v1.2 crosswalk and ICC = 0.83 (0.82, 0.84) for the qDASH-UE CAT v2.0 crosswalk.

CONCLUSIONS

The linkages establish crosswalk tables using equipercentile equating models to convert the PROMIS UE CAT v1.2 and v2.0 scores to the qDASH and vice versa.

CLINICAL RELEVANCE

This study provides crosswalk tables for commonly collected PROMs in hand surgery, increasing the comparability of results between centers using different PROMs to study the same conditions or treatments.

The challenge of using patient reported outcome measures in clinical practice: how do we get there?

BACKGROUND

As patient-reported outcome measures (PROMs) become available to clinicians for routine clinical decision-making, many wonder how to define a meaningful change in a patient's PROM score. Some PROMs have a specific threshold that indicates meaningful change, but since those numbers are based on population averages, they do not necessarily apply to the varying experiences of each individual patient. Rather than viewing this as a weakness of PROMs, it is worth considering how clinicians use other existing measures in clinical decision-making-and whether PROMs can be used similarly. BODY: An informal survey of 43 clinicians reported using measures such as weight, blood pressure, and blood chemistry to inform clinical decision-making. Although clinicians were very consistent with what constituted a meaningful change for some measures (e.g., ECOG performance status), other measures had considerable variability (e.g., weight), often informed by their specialization (for example, differing thresholds for meaningful weight change for adult primary care, pediatrics, and oncology). For interpreting change in measures, they relied on clinical experience (44%), published literature (38%), and established guidelines (35%). In open-response comments, many clarified that the results of any measure had to be taken in the context of each individual patient before making treatment decisions. In short, clinicians already apply individualized clinical judgment when interpreting score changes in existing clinical measures. As clinicians gain familiarity with PROMs, PROMs will likely be utilized in the same way.

CONCLUSION

Like other clinical measures from weight to blood chemistry, change in a PROM score is but one piece of a patient's clinical story. Rather than relying on a hard-and-fast number for defining clinically meaningful change in a PROM score, providers should-and many already do-consider the full scope of a patient's experience as they make treatment decisions.

Estimating individual health-related quality of life changes in low back pain patients

BACKGROUND

There is a need to evaluate different options for estimating individual change in health-related quality of life for patients with low back pain.

METHODS

Secondary analysis of data collected at baseline and 6 weeks later in a randomized trial of 749 adults with low back pain receiving usual medical care (UMC) or UMC plus chiropractic care at a small hospital at a military training site or two large military medical centers. The mean age was 31; 76% were male and 67% were White. The study participants completed the Patient-Reported Outcomes Measurement Information System (PROMIS®)-29 v 1.0 physical function, pain interference, pain intensity, fatigue, sleep disturbance, depression, anxiety, satisfaction with participation in social roles, physical summary, and mental health summary scores (T-scored with mean = 50 and standard deviation (SD) = 10 in the U.S. general population).

RESULTS

Reliability estimates at the baseline ranged from 0.700 to 0.969. Six-week test-retest intraclass correlation estimates were substantially lower than these estimates: the median test-retest intraclass correlation for the two-way mixed-effects model was 0. 532. Restricting the test-retest reliability estimates to the subset who reported they were about the same as at baseline on a retrospective rating of change item increased the median test-retest reliability to 0.686. The amount of individual change that was statistically significant varied by how reliability was estimated, and which SD was used. The smallest change needed was found when internal consistency reliability and the SD at baseline were used. When these values were used, the amount of change needed to be statistically significant (p < .05) at the individual level ranged from 3.33 (mental health summary scale) to 12.30 (pain intensity item) T-score points.

CONCLUSIONS

We recommend that in research studies estimates of the magnitude of individual change needed for statistical significance be provided for multiple reliability and standard deviation estimates. Whenever possible, patients should be classified based on whether they 1) improved significantly and perceived they got better, 2) improved significantly but did not perceive they were better, 3) did not improve significantly but felt they got better, or 4) did not improve significantly or report getting better.

Does Scoring Method Impact Estimation of Significant Individual Changes Assessed by Patient-Reported Outcome Measures? Comparing Classical Test Theory Versus Item Response Theory

OBJECTIVES

This study aimed to examine the ability of classical test theory (CTT) and item response theory (IRT) scores assessed by Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures to identify significant individual changes in the setting of clinical studies, using both simulated and empirical data.

METHODS

We used simulated data to compare the estimation of significant individual changes between CTT and IRT scores across different conditions and a clinical trial data set to verify the simulation results. We calculated reliable change indexes to estimate significant individual changes.

RESULTS

For small true change, IRT scores showed a slightly higher rate of classifying change groups than CTT scores and were comparable with CTT scores for a shorter test length. Additionally, IRT scores were found to have a prominent advantage in the classification rates of change groups for medium to high true change over CTT scores. Such an advantage became prominent in a longer test length. The empirical data analysis results using an anchor-based approach further supported the above findings that IRT scores can more accurately classify participants into change groups than CTT scores.

CONCLUSIONS

Given that IRT scores perform better, or at least comparably, in most conditions, we recommend using IRT scores to estimate significant individual changes and identify responders to treatment. This study provides evidence-based guidance in detecting individual changes based on CTT and IRT scores under various measurement conditions and leads to recommendations for identifying responders to treatment for participants in clinical trials.

Patient-reported outcomes of maintenance rucaparib in patients with recurrent ovarian carcinoma in ARIEL3, a phase III, randomized, placebo-controlled trial

PURPOSE

To compare NFOSI-18 Disease Related Symptoms - Physical (DRSP), Total score, and side effect bother between maintenance rucaparib (600 mg twice daily) vs. placebo in the phase III ARIEL3 trial.

METHODS

ARIEL3 (NCT01968213) included patients with ovarian carcinoma who responded to second-line or later platinum-based chemotherapy. The NFOSI-18 DRS-P and Total scales were secondary endpoints. The NFOSI-18 contains a side effect impact item (GP5): "I am bothered by side effects of treatment." We compared treatment arms on change from baseline of DRS-P and Total scores using mixed models with repeated measures (MRMM). Time to first and confirmed deterioration of NFOSI-18 DRS-P and Total scales were analyzed using Cox regression. We also calculated the proportion of patients reporting moderate to high side effect bother on GP5.

RESULTS

In the intention-to-treat (ITT) cohort, mean change from baseline favored the placebo. Compared to placebo, rucaparib was associated with higher risk of deterioration [e.g., 4-point deteriorator definition hazard ratio (HR): 1.85; 95% CI: 1.46, 2.36; median time to first deterioration on DRSP: 1.9 vs. 7.0 months]. Confirmed deterioration results resembled those for first deterioration. Proportions of patients reporting moderate/high side effect bother on GP5 fluctuated around 20% across treatment cycles. Results in BRCA mutant and homologous recombination deficient cohorts were generally similar to those from the ITT cohort.

CONCLUSION

This placebo-controlled study in the maintenance therapy setting provides a unique view of the impact of PARP inhibition on the patient-reported outcomes that are commonly used in ovarian cancer clinical trials. Information regarding the adverse side effect impact of PARP inhibitors should be weighed against their clinical benefit.

Psychometric performance of the Primary Mitochondrial Myopathy Symptom Assessment (PMMSA) in a randomized, double-blind, placebo-controlled crossover study in subjects with mitochondrial disease

BACKGROUND

The Primary Mitochondrial Myopathy Symptom Assessment (PMMSA) is a 10-item patient-reported outcome (PRO) measure designed to assess the severity of mitochondrial disease symptoms. Analyses of data from a clinical trial with PMM patients were conducted to evaluate the psychometric properties of the PMMSA and to provide score interpretation guidelines for the measure.

METHODS

The PMMSA was completed as a daily diary for approximately 14 weeks by individuals in a Phase 2 randomized, placebo-controlled crossover trial evaluating the safety, tolerability, and efficacy of subcutaneous injections of elamipretide in patents with mitochondrial disease. In addition to the PMMSA, performance-based assessments, clinician ratings, and other PRO measures were also completed. Descriptive statistics, psychometric analyses, and score interpretation guidelines were evaluated for the PMMSA.

RESULTS

Participants (N = 30) had a mean age of 45.3 years, with the majority of the sample being female (n = 25, 83.3%) and non-Hispanic white (n = 29, 96.6%). The 10 PMMSA items assessing a diverse symptomology were not found to form a single underlying construct. However, four items assessing tiredness and muscle weakness were grouped into a "general fatigue" domain score. The PMMSA Fatigue 4 summary score (4FS) demonstrated stable test-retest scores, internal consistency, correlations with the scores produced by reference measures, and the ability to differentiate between different global health levels. Changes on the PMMSA 4FS were also related to change scores produced by the reference measures. PMMSA severity scores were higher for the symptom rated as "most bothersome" by each subject relative to the remaining nine PMMSA items (most bothersome symptom mean = 2.88 vs. 2.18 for other items). Distribution- and anchor-based evaluations suggested that reduction in weekly scores between 0.79 and 2.14 (scale range: 4-16) may represent a meaningful change on the PMMSA 4FS and reduction in weekly scores between 0.03 and 0.61 may represent a responder for each of the remaining six non-fatigue items, scored independently.

CONCLUSIONS

Upon evaluation of its psychometric properties, the PMMSA, specifically the 4FS domain, demonstrated strong reliability and construct-related validity. The PMMSA can be used to evaluate treatment benefit in clinical trials with individuals with PMM. Trial registration ClinicalTrials.gov identifier, NCT02805790; registered June 20, 2016; https://clinicaltrials.gov/ct2/show/NCT02805790 .