Development and preliminary validation of a diagnostic prediction model to optimise outpatient management of patients with urolithiasis using urinary stones and intervention quality of life (USIQoL) measure

PURPOSE

Patients with urinary calculi undergo resource-intensive follow-up. Application of a PROM, Urinary Stones and Intervention Quality of Life (USIQoL), can potentially optimise current practices if it matches the outcomes of traditional follow-up. Our objective was to develop, and conduct, a preliminary validation of the USIQoL based prediction model to aid triage.

METHODS

We performed a two phase prospective cohort study. The 1st phase included development of the USIQoL-based decision model using multicentre data. The 2nd phase involved prospective single-blind external validation for the outpatient application. The aim was to evaluate correlations between the USIQoL scores and key predictors; clinical outcomes and global health ratings (EuroQoL EQ-5D). We used statistical analysis to validate USIQoL cut-off scores to aid triage and the decision to intervene.

RESULTS

Of 503 patients invited, 91% (n = 455, Development [305] and Validation [150]; M = 308, F = 147) participated. The relationship between USIQoL domain scores and clinical outcomes was consistently significant (estimated odds: PPH 1.24, p < 0.001, 95% CI 1.13-1.36; PSH 1.22, p < 0.001, 95% CI 1.12-1.33). The ROC values for the model were ≥ 0.75. The optimum domain cut-off scores were derived with rising scores implying increased need to intervene. The model demonstrated satisfactory sensitivity (0.81-0.89) and specificity (0.36-0.47).

CONCLUSIONS

The study demonstrates satisfactory correlation between the USIQoL and clinical outcomes making this model a valid aid for triage and optimising outpatient management with the cut-off scores able to identify high risk patients who need active treatment.

Patient-Reported Outcome Measure Development and Validation: A Primer for Clinicians

A comprehensive definition of health includes the assessment of patient experiences of a disease and its treatment. These patient experiences are best captured by standardized patient-reported outcome (PRO) instruments. A PRO is reported directly by the patient (or caregiver) and provides the patient's perspective into how a disease and its treatment impact their lives. PRO instruments are typically standardized, validated questionnaires with items that are scaled and can be combined to represent an underlying health-related construct such as physical, social, and role functioning, psychological well-being, symptoms, pain, and quality of life. Over the past few decades, PROs have become increasingly used in clinical trials as endpoints to better understand treatment benefits from the patient's perspective and in clinical practice to identify unmet needs of patients, health risk surveillance, and monitor outcomes of care. In this paper, we describe the process for developing standardized PRO instruments, from conceptual model development through instrument validation.

Effect size varies based on calculation method and may affect interpretation of treatment effect: an illustration using randomised clinical trials in osteoarthritis

BACKGROUND

To illustrate how (standardised) effect sizes (ES) vary based on calculation method and to provide considerations for improved reporting.

METHODS

Data from three trials of tanezumab in subjects with osteoarthritis were analyzed. ES of tanezumab versus comparator for WOMAC Pain (outcome) was defined as least squares difference between means (mixed model for repeated measures analysis) divided by a pooled standard deviation (SD) of outcome scores. Three approaches to computing the SD were evaluated: Baseline (the pooled SD of WOMAC Pain values at baseline [pooled across treatments]); Endpoint (the pooled SD of these values at the time primary endpoints were assessed); and Median (the median pooled SD of these values based on the pooled SDs across available timepoints). Bootstrap analyses were used to compute 95% confidence intervals (CI).

RESULTS

ES (95% CI) of tanezumab 2.5 mg based on Baseline, Endpoint, and Median SDs in one study were - 0.416 (- 0.796, - 0.060), - 0.195 (- 0.371, - 0.028), and - 0.196 (- 0.373, - 0.028), respectively; negative values indicate pain improvement. This pattern of ES differences (largest with Baseline SD, smallest with Endpoint SD, Median SD similar to Endpoint SD) was consistent across all studies and doses of tanezumab.

CONCLUSION

Differences in ES affect interpretation of treatment effect. Therefore, we advocate clearly reporting individual elements of ES in addition to its overall calculation. This is particularly important when ES estimates are used to determine sample sizes for clinical trials, as larger ES will lead to smaller sample sizes and potentially underpowered studies.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02697773, NCT02709486, and NCT02528188.

Reference Values for the German Version of the Quality of Life after Brain Injury in Children and Adolescents (QOLIBRI-KID/ADO) from a General Population Sample

Assessment of health-related quality of life (HRQoL) after pediatric traumatic brain injury (TBI) has been limited in children and adolescents due to a lack of disease-specific instruments. To fill this gap, the Quality of Life after Traumatic Brain Injury for Children and Adolescents (QOLIBRI-KID/ADO) Questionnaire was developed for the German-speaking population. Reference values from a comparable general population are essential for comprehending the impact of TBI on health and well-being. This study examines the validity of the German QOLIBRI-KID/ADO in a general pediatric population in Germany and provides reference values for use in clinical practice. Overall, 1997 children and adolescents aged 8-17 years from the general population and 300 from the TBI population participated in this study. The questionnaire was tested for reliability and validity. A measurement invariance (MI) approach was used to assess the comparability of the HRQoL construct between both samples. Reference values were determined by percentile-based stratification according to factors that significantly influenced HRQoL in regression analyses. The QOLIBRI-KID/ADO demonstrated strong psychometric properties. The HRQoL construct was measured largely equivalently in both samples, and reference values could be provided. The QOLIBRI-KID/ADO was considered reliable and valid for assessing HRQoL in a general German-speaking pediatric population, allowing for clinically meaningful comparisons between general and TBI populations.

Unsupervised item response theory models for assessing sample heterogeneity in patient-reported outcomes measures

PURPOSE

Unsupervised item-response theory (IRT) models such as polytomous IRT based on recursive partitioning (IRTrees) and mixture IRT (MixIRT) models can be used to assess differential item functioning (DIF) in patient-reported outcome measures (PROMs) when the covariates associated with DIF are unknown a priori. This study examines the consistency of results for IRTrees and MixIRT models.

METHODS

Data were from 4478 individuals in the Alberta Provincial Project on Outcome Assessment in Coronary Heart Disease registry who received cardiac angiography in Alberta, Canada, and completed the Hospital Anxiety and Depression Scale (HADS) depression subscale items. The partial credit model (PCM) based on recursive partitioning (PCTree) and mixture PCM (MixPCM) were used to identify covariates associated with differential response patterns to HADS depression subscale items. Model covariates included demographic and clinical characteristics.

RESULTS

The median (interquartile range) age was 64.5(15.7) years, and 3522(78.5%) patients were male. The PCTree identified 4 terminal nodes (subgroups) defined by smoking status, age, and body mass index. A 3-class PCM fits the data well. The MixPCM latent classes were defined by age, disease indication, smoking status, comorbid diabetes, congestive heart failure, and chronic obstructive pulmonary disease.

CONCLUSION

PCTree and MixPCM were not consistent in detecting covariates associated with differential interpretations of PROM items. Future research will use computer simulations to assess these models' Type I error and statistical power for identifying covariates associated with DIF.

Translation, cultural adaptation, and validation of the Chinese standardized outcomes in nephrology-hemodialysis fatigue (C-SONG-HD fatigue) scale: a study of Chinese patients undergoing hemodialysis

OBJECTIVE

This study aimed to translate and culturally adapt the standardized outcomes in nephrology-hemodialysis fatigue (SONG-HD fatigue) scale and to assess the psychometric properties of the Chinese version of the SONG-HD fatigue (C-SONG-HD fatigue) scale.

METHODS

Forward and back translations were used to translate the SONG-HD fatigue scale into Chinese. We used the C-SONG-HD fatigue scale to survey Chinese patients undergoing hemodialysis (HD) in China. We examined the distribution of responses and floor and ceiling effects. Cronbach's alpha and McDonald's omega coefficient, intraclass coefficients, and Spearman correlations were used to assess internal consistency reliability, test-retest reliability, and convergent validity, respectively. Responsiveness was also evaluated.

RESULTS

In total, 489 participants across southeast China, northwest China, and central China completed the study. The C-SONG-HD fatigue scale had good internal consistency (Cronbach's alpha coefficient 0.861, omega coefficient 0.916), test-retest reliability (intraclass correlation coefficient 0.695), and convergent validity (Spearman correlation 0.691). The analysis of all first-time HD patients did not show notable responsiveness, and only patients with temporary vascular access had good responsiveness with an effect size (ES) of 0.54, a standardized response mean (SRM) of 0.85, and a standard error of measurement (SEM) of 0.77.

CONCLUSION

The Chinese version of the SONG-HD fatigue scale showed satisfactory reliability and validity in patients undergoing hemodialysis (HD) in China. It could be used as a tool to measure the fatigue of Chinese HD patients.

Health-related quality of life in pre-pubertal children with pediatric growth hormone deficiency: 12-month results from a phase 3 clinical trial of once-weekly somatrogon versus once-daily somatropin

OBJECTIVE

Treatment of pediatric growth hormone deficiency (pGHD) with daily injection of recombinant human growth hormone (somatropin) aims to increase height velocity and improve health-related quality of life (HRQoL). The Quality of Life in Short Stature Youth (QoLISSY) questionnaire was administered in a phase 3 clinical trial that evaluated efficacy and safety of once-weekly somatrogon versus once-daily somatropin in children with pGHD (ClinicalTrials.gov no NCT02968004).

METHODS

Treatment-naïve prepubertal children with pGHD received once-weekly somatrogon or once-daily somatropin for 12 months. The QoLISSY core module (physical/social/emotional subscales) was administered at baseline and 12 months after treatment initiation. QoLISSY-Parent was completed by parents/caregivers of children <7 years old and some parents/caregivers of children ≥7 years old; children ≥7 years old self-completed QoLISSY-Child.

RESULTS

Baseline characteristics were similar between treatment groups (N = 117). Among children <7 years old, QoLISSY-Parent total and subscale scores showed similarly improved HRQoL at 12 months relative to baseline in both treatment groups. Self-reported QoLISSY-Child total and subscale scores in children ≥7 years old indicated HRQoL improvements at 12 months that were numerically better with somatrogon than somatropin (similar results with QoLISSY-Parent in this age group). At both time points, children reported better HRQoL than perceived by their parents/caregivers.

CONCLUSION

Treatment for 12 months with once-weekly somatrogon or once-daily somatropin resulted in comparable improvements in HRQoL among children with pGHD. Lower HRQoL perceived by parents/caregivers possibly reflect children's tendency to emphasize adaptation. These results suggest that evaluation of HRQoL could help support treatment decisions in children with pGHD treated with growth hormone.

Quantitative and Qualitative Exploration of Meaningful Change on the Vineland Adaptive Behavior Scales (Vineland™-II) in Children and Adolescents with Autism Without Intellectual Disability Following Participation in a Clinical Trial

Purpose

The VinelandTM Adaptive Behavior Scale is often used in autism spectrum disorder (ASD) trials. The Adaptive Behavior Composite Score (VABS-ABC) is the standardized overall score (the average of the Socialization, Communication and Daily Living skills domains), and the standardized 2-Domain Composite Score (VABS-2DC) is a novel outcome measure (average of the Socialization and Communication domains). A within-person meaningful change threshold (MCT) has not been established for the VABS-2DC. This paper presents a quantitative and qualitative interpretation of what constitutes a meaningful change in these scores to individuals with ASD without Intellectual Disability (ID; IQ≥70) and their families, as reported by their study partners (SPs).

Participants and Methods

Data were obtained from the aV1ation clinical trial in children and adolescents with ASD and associated exit interviews. The intent-to-treat (ITT) clinical trial population included 308 individuals with autism (85.4% male; average age: 12.4 years [standard deviation (SD)=2.97]); 124 in the child cohort (aged 5 to 12 years; average age: 9.4 years [SD=1.86]), and 184 in the adolescent cohort (aged 13 to 17 years; average age: 14.5 years [SD=1.39]). Study partners of 86 trial participants were included in the Exit Interview Population (EIP): participants represented were 83.7% male, average age: 12.3 years [SD=2.98]). Anchor and distribution-based methods were used to estimate within-person change to support a responder definition, to aid interpretation of the clinical trial data; qualitative data were used to contextualize the meaning of changes observed.

Results

A within-person MCT range of 4 to 8 points was proposed for both VABS-ABC and VABS-2DC, which was associated with at least a 1-point improvement on 4 different anchors. Evidence for this within-person MCT was further supported by qualitative data, which suggested any change was considered meaningful to the individual with ASD, as reported by their SP, no matter what the magnitude.

Conclusion

A change in standardized score of 4 to 8 points constitutes a within-person MCT on both VABS-ABC and novel VABS-2DC in those with ASD and no ID. A change of this, or more, was reported by the SPs in this trial to be meaningful and highly impactful upon the individuals with ASD and their family.

Measurement of treatment burden in patients with multimorbidity in the Netherlands: translation and validation of the Multimorbidity Treatment Burden Questionnaire (NL-MTBQ)

BACKGROUND

Multimorbidity is a growing problem. The number and complexity of (non-)pharmaceutical treatments create a great burden for patients. Treatment burden refers to the perception of the weight of these treatments, and is associated with multimorbidity. Measurement of treatment burden is of great value for optimizing treatment and health-related outcomes.

OBJECTIVE

We aim to translate and validate the Multimorbidity Treatment Burden Questionnaire (MTBQ) for use in the Dutch population with multimorbidity and explore the level of treatment burden.

METHODS

Translating the MTBQ into Dutch included forward-backward translation, piloting, and cognitive interviewing (n = 8). Psychometric properties of the questionnaire were assessed in a cross-sectional study of patients with multimorbidity recruited from a panel in the Netherlands (n = 959). We examined item properties, dimensionality, internal consistency reliability, and construct validity. The level of treatment burden in the population was assessed.

RESULTS

The mean age among 959 participants with multimorbidity was 69.9 (17-96) years. Median global NL-MTBQ score was 3.85 (interquartile range 0-9.62), representing low treatment burden. Significant floor effects were found for all 13 items of the instrument. Factor analysis supported a single-factor structure. The NL-MTBQ had high internal consistency (α = 0.845), and provided good evidence on the construct validity of the scale.

CONCLUSION

The Dutch version of the 13-item MTBQ is a single-structured, valid, and compact patient-reported outcome measure to assess treatment burden in primary care patients with multimorbidity. It could identify patients experiencing high treatment burden, with great potential to enhance shared decision-making and offer additional support.

From Meaningful Outcomes to Meaningful Change Thresholds: A Path to Progress for Establishing Digital Endpoints

This paper examines the use of digital endpoints (DEs) derived from digital health technologies (DHTs), focusing primarily on the specific considerations regarding the determination of meaningful change thresholds (MCT). Using DHTs in drug development is becoming more commonplace. There is general acceptance of the value of DHTs supporting patient-centric trial design, capturing data outside the traditional clinical trial setting, and generating DEs with the potential to be more sensitive to change than conventional assessments. However, the transition from exploratory endpoints to primary and secondary endpoints capable of supporting labeling claims requires these endpoints to be substantive with reproducible population-specific values. Meaningful change represents the amount of change in an endpoint measure perceived as important to patients and should be determined for each digital endpoint and given population under consideration. This paper examines existing approaches to determine meaningful change thresholds and explores examples of these methodologies and their use as part of DE development: emphasizing the importance of determining what aspects of health are important to patients and ensuring the DE captures these concepts of interest and aligns with the overarching endpoint strategy. Examples are drawn from published DE qualification documentation and responses to qualification submissions under review by the various regulatory authorities. It is the hope that these insights will inform and strengthen the development and validation of DEs as drug development tools, particularly for those new to the approaches to determine MCTs.

Development of the Bleeding and Pelvic Discomfort Scale for Use in Women With Heavy Menstrual Bleeding Associated With Uterine Fibroids

OBJECTIVES

This study aimed to define a cardinal symptom burden measure based on items from the Uterine Fibroid Symptom and Quality of Life questionnaire for use as a clinical trial endpoint.

METHODS

Exploratory factor analysis was computed to assess the Uterine Fibroid Symptom and Quality of Life symptom severity scale factor structure, using phase 2 data. Pooled blinded data from phase 3 studies were used for the confirmatory factor analysis and the psychometric evaluation of the new measure. Exit interviews in 30 patients from phase 3 studies provided additional qualitative evidence. A meaningful change threshold was determined using anchor-based analyses supported by patient feedback in the exit interviews.

RESULTS

Three factors emerged from the exploratory factor analysis. Factor 1, called the bleeding and pelvic discomfort (BPD) scale, consists of cardinal symptoms, measuring menstrual distress owing to heavy bleeding, passing blood clots, and feeling tightness or pressure in pelvic area. Patients generally understood the items in the scale and the recall period as intended. The BPD scale had good item performance and internal consistency reliability, strong item-to-total correlations, good item discrimination, known-groups validity, and ability to detect change. A 20-point change on the BPD scale was determined as the clinically meaningful change threshold.

CONCLUSIONS

The BPD scale assesses symptom burden owing to bleeding, passing blood clots, and pelvic pressure. The subscale is based on a subset of items selected to measure the cardinal symptoms of uterine fibroids in a clinical trial setting. The responder threshold evaluates whether patients experience a meaningful treatment benefit over the on-treatment period.

Comparison of anchor-based methods for estimating thresholds of meaningful within-patient change using simulated PROMIS PF 20a data under various joint distribution characteristic conditions

PURPOSE

To compare the performance of anchor-based methods for estimating thresholds of meaningful within-patient change (i.e., individual change) of clinical outcome assessments in conditions reflecting data characteristics of small- to medium-sized clinical trials.

METHODS

Datasets were generated from the joint distributions of the PROMIS PF 20a T-score changes and a seven-point global change anchor measure. The 108 simulation conditions (1000 replications per condition) included combinations of three marginal distributions of T-score changes, three improvement percentages in the anchor measure, four levels of responsiveness correlations, and three sample sizes. Threshold estimation methods included mean change, median change, ROC curve, predictive modeling, half SD, and SEM. Relative bias, precision, accuracy, and measurement significance of the estimates were evaluated based on comparison with true thresholds and IRT-based individual reliable changes of PROMIS scores. Quantile regression models were applied to select and interpret effects of simulation conditions on estimation bias.

RESULTS

When PROMIS T-score changes were distributed normally, the predictive modeling method performed best with 50% or more responders identified by the anchor; the mean and median methods were preferred with 30% responders. For skewed distributions, the median method and ROC method gained more advantages. Among the evaluated study conditions, the improvement percentage condition had the most obvious effects on estimation bias.

CONCLUSION

To establish accurate and precise thresholds, clinical researchers are recommended to prioritize study designs with at least 50% anchor-defined responders and strongly responsive target endpoints with highly reliable scoring calibration and to select optimal anchor-based methods given the data characteristics.

Functional Assessment of Chronic Illness Therapy-Fatigue is a reliable and valid measure in patients with active ankylosing spondylitis

Background

The Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-F) scale has demonstrated good internal consistency and responsiveness to changes in clinical status among patients with ankylosing spondylitis (AS). We aimed to further evaluate the psychometric properties of the FACIT-F scale in adult patients with AS.

Methods

Measurement properties of the FACIT-F scale were evaluated using data from tofacitinib phase 2/3 (NCT01786668/NCT03502616) studies in adult patients with active AS.

Results

Second-order confirmatory factor modeling supported the measurement structure of the FACIT-F scale (Bentler’s comparative fit index ≥ 0.91), and FACIT-F demonstrated excellent internal consistency (Cronbach’s coefficient α ≥ 0.88) and test–retest reliability (Intraclass Correlation Coefficient ≥ 0.75). Correlation coefficients between FACIT-F and other patient-reported outcomes generally exceeded 0.40, supporting convergent validity. Meaningful within-patient change was estimated as 3.1–6.3 for FACIT-F total score, and 1.4–2.8 and 1.7–3.6 for FACIT-F Experience and Impact domain scores, respectively. Large (effect size ≥ 1.17 standard deviation units), statistically significant differences in FACIT-F domain/total scores between ‘no disease activity’ (Patient Global Assessment of Disease Activity [PtGA] = 0) and ‘very active disease’ (PtGA = 10) patient groups supported known-groups validity. Ability to detect change was evidenced by an approximately linear relationship between changes in FACIT-F and PtGA scores.

Conclusions

FACIT-F is a reliable and valid measure for evaluating fatigue in adult patients with active AS.

Trial registration: ClinicalTrials.gov; NCT01786668 (registered 6 February 2013, https://clinicaltrials.gov/ct2/show/NCT01786668) and NCT03502616 (registered 11 April 2018, https://clinicaltrials.gov/ct2/show/NCT03502616).

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-022-00508-0.

Ankylosing spondylitis (AS) is a disease that causes pain and stiffness in the spine. AS is a chronic disease. Most people with AS experience fatigue (a feeling of being very tired or exhausted). People with more severe AS tend to have more fatigue and a lower level of well-being. Because of this, fatigue is an important symptom to measure in studies of AS. The Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-F) is a questionnaire that measures fatigue. It has 13 questions that assess level of fatigue over a week’s time. The FACIT-F has not been studied in depth in people with AS. Therefore, we analyzed results from two clinical trials. We found that FACIT-F was a reliable and valid way to measure fatigue in adults with AS. This makes it a suitable tool to use in AS clinical trials.

Establishing thresholds for meaningful within-individual change using longitudinal item response theory

Purpose

Thresholds for meaningful within-individual change (MWIC) are useful for interpreting patient-reported outcome measures (PROM). Transition ratings (TR) have been recommended as anchors to establish MWIC. Traditional statistical methods for analyzing MWIC such as mean change analysis, receiver operating characteristic (ROC) analysis, and predictive modeling ignore problems of floor/ceiling effects and measurement error in the PROM scores and the TR item. We present a novel approach to MWIC estimation for multi-item scales using longitudinal item response theory (LIRT).

Methods

A Graded Response LIRT model for baseline and follow-up PROM data was expanded to include a TR item measuring latent change. The LIRT threshold parameter for the TR established the MWIC threshold on the latent metric, from which the observed PROM score MWIC threshold was estimated. We compared the LIRT approach and traditional methods using an example data set with baseline and three follow-up assessments differing by magnitude of score improvement, variance of score improvement, and baseline-follow-up score correlation.

Results

The LIRT model provided good fit to the data. LIRT estimates of observed PROM MWIC varied between 3 and 4 points score improvement. In contrast, results from traditional methods varied from 2 to 10 points—strongly associated with proportion of self-rated improvement. Best agreement between methods was seen when approximately 50% rated their health as improved.

Conclusion

Results from traditional analyses of anchor-based MWIC are impacted by study conditions. LIRT constitutes a promising and more robust analytic approach to identifying thresholds for MWIC.

WOMAC Meaningful Within-patient Change: Results From 3 Studies of Tanezumab in Patients With Moderate-to-severe Osteoarthritis of the Hip or Knee

OBJECTIVE

To define meaningful within-patient change (MWPC) in the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC).

METHODS

Data were analyzed separately from 3 phase III clinical trials (ClinicalTrials.gov: NCT02697773, NCT02709486, NCT02528188) of tanezumab, a novel treatment intended for the relief of signs and symptoms of moderate-to-severe osteoarthritis (OA), administered subcutaneously every 8 weeks. Patients with moderate-to-severe OA of the hip or knee completed the WOMAC and patient global assessment of OA (PGA-OA) at regular timepoints. A repeated measures longitudinal model with change in WOMAC Pain, Physical Function, or Stiffness domain score as the outcome and change in PGA-OA as the anchor was used to establish MWPC for WOMAC domains.

RESULTS

In the 3 studies, there were 688, 844, and 2948 subjects available for analyses, respectively. Analysis showed that a linear relationship between changes in WOMAC domains and changes in PGA-OA was supported and justified. Moreover, the relationships between these changes were very similar for 2 trials and close for the third. The estimated MWPC for the 3 WOMAC domains were from 0.84-1.16 (0-10 numerical rating scale) and from 12.50-16.23%, depending on study and domain, that corresponded to a 1-category change on PGA-OA. For a 2-category change those values were from 1.68-2.31 and from 25.01-32.46%, respectively.

CONCLUSION

These results establish MWPCs for WOMAC domains, at the individual patient level, for patients with moderate-to-severe OA of the hip or knee. [ClinicalTrials.gov: NCT02697773, NCT02709486, and NCT02528188].

Responsiveness of the Brazilian IdFAI questionnaire in patients with chronic ankle instability

PURPOSE

This study aimed to determine the responsiveness of the Brazilian version of the Identification of Functional Ankle Instability (IdFAI) questionnaire in students who received an eight-week treatment for chronic ankle instability (CAI).

METHODS

Twenty-five college students (aged 23.12 ± 2.80 years) with CAI, as identified by the IdFAI questionnaire, were recruited. We used distribution and anchor-based methods to assess the responsiveness of the questionnaire, and its ability to determine clinical changes in participants. Eleven anchors were used: Visual Analog Scale for instability (VAS-i); Cumberland Ankle Instability Tool (CAIT); Isometric dorsiflexion, plantar flexion, inversion, and eversion muscle strength measured using a manual dynamometer; Dynamic balance as assessed through the Star Excursion Balance Test (SEBT-Y); Active ankle dorsiflexion range of motion as measured using the weight-bearing lunge test; and Functional performance assessment using three hop tests: single hop, triple crossover hop, and side hop. The distribution-based method used effect size (ES) and standardized response mean (SRM), whereas the anchor-based method used paired t-tests. Both methods allowed the calculation of the minimal important difference (MID).

RESULTS

The Brazilian IdFAI showed high responsiveness, with a large magnitude of change (ES = 1.34) and a high responsiveness index (SRM = 1.28) when assessed after a treatment for CAI. The IdFAI total score (p < .001) and all the 11 anchors [VAS-i (p < .001); CAIT (p < .001); Isometric dorsiflexion (p < .001), plantar flexion (p < .001), inversion (p < .001), and eversion (p < .001) muscle strength; SEBT-Y (p < .001); Lunge test (p = .002); Single hop (p < .001); triple crossover hop (p < .001); and side hop tests (p < .001)] showed significant differences. The anchor and distribution-based methods demonstrated MID values of 3.72 and 1.49-2.27, respectively.

CONCLUSION

The Brazilian IdFAI questionnaire is a patient-reported outcome measure sensitive to clinical changes in individuals with CAI. It can be used as an identification of patients with CAI, and as a parameter to verify clinical changes of clinical trials or therapeutic interventions in the population with CAI.

Measurement invariance of the Seattle Angina Questionnaire in coronary artery disease

PURPOSE

The Seattle Angina Questionnaire (SAQ) is a widely used patient-reported measure of health status in patients with coronary artery disease. Comparisons of SAQ scores amongst population groups and over time rely on the assumption that its factorial structure is invariant. This study evaluates the measurement invariance of the SAQ across different demographic and clinical groups and over time.

METHODS

Data were obtained from the Alberta Provincial Project on Outcome Assessment in Coronary Heart Disease registry, a registry of patients who received coronary angiogram in Alberta, Canada. The study cohort consists of adult patients who completed the paper-based version of the 16-item Canadian version of the SAQ (SAQ-CAN) 2 weeks and 1-year post-coronary angiogram between 2009 and 2016. Multi-group confirmatory factor analysis was used to assess configural, weak, strong, and strict measurement invariance across age groups, sex, angina type, treatment, and over time. Model fit was assessed using the comparative fit index and root mean square error of approximation.

RESULTS

Of the 8101 patients included in these analysis, 1300 (16.1%) were at least 75 years old, while 1755 (21.7%) were female, 5154 (63.6%) were diagnosed with acute coronary syndrome, 1177 (14.5%) received coronary artery bypass graft treatment, and 3279 had complete data on the SAQ-CAN at both occasions. There was evidence of strict invariance across age, sex, and angina type, and treatment groups, but partial strict invariance was established over time.

CONCLUSION

SAQ-CAN can be used to compare the health status of coronary artery disease patients across population groups and over time.

Addressing Bias in Responder Analysis of Patient-Reported Outcomes

INTRODUCTION

Quantitative patient-reported outcome (PRO) measures ideally are analyzed on their original scales and responder analyses are used to aid the interpretation of those primary analyses. As stated in the FDA PRO Guidance for Medical Product Development (2009), one way to lend meaning and interpretation to such a PRO measure is to dichotomize between values where within-patient changes are considered clinically important and those that are not. But even a PRO scale with a cutoff score that discriminates well between responder and non-responders is fraught with some misclassification.

METHODS

Using estimates of sensitivity and specificity on classification of responder status from a PRO instrument, formulas are provided to correct for such responder misclassification under the assumption of no treatment misclassification. Two case studies from sexual medicine illustrate the methodology.

RESULTS

Adjustment formulas on cell counts for responder misclassification are a direct extension of correction formulas for misclassification on disease from a two-way cross-classification table of disease (yes, no) and exposure (yes, no). Unadjusted and adjusted estimates of treatment effect are compared in terms of odds ratio, response ratio, and response difference. In the two case studies, there was considerable underestimation of treatment effect.

DISCUSSION AND CONCLUSIONS

The methodology can be applied to different therapeutic areas. Limitations of the methodology, such as when adjusted cell estimates become negative, are highlighted. The role of anchor-based methodology is discussed for obtaining estimates of sensitivity and specificity on responder classification. Correction for treatment effect bias from misclassification of responder status on PRO measures can lead to more trustworthy interpretation and effective decision-making. Clinicaltrials.gov: NCT00343200.

A New Procedure to Assess When Estimates from the Cumulative Link Model Can Be Interpreted as Differences for Ordinal Scales in Quality of Life Studies

Purpose

Assessing the clinical importance of an exposure effect on a quality of life (QoL) score often requires quantifying the effect in terms of a difference in scores. Using the linear regression model (LRM) for this purpose assumes the ordinal score is a proxy for an underlying continuous variable, but the analysis offers no assessment for the validity of the assumption. We propose an approach that assesses the proxy assumption and estimates the exposure effect by using the cumulative link model (CLM).

Patients and methods

CLM is a well-established regression model that assumes an ordinal score is an ordered category generated from applying thresholds to a latent continuous variable. Our approach assesses the proxy assumption by testing whether these thresholds are equidistant. We compared the performance of CLM and LRM using simulated ordinal data and illustrated their application to the effect of time since diagnosis on five subscales of fatigue among breast cancer survivors measured using the Multidimensional Fatigue Inventory.

Results

CLM had good performance in estimating the difference in means with simulated ordinal data satisfying the proxy assumption, even when the outcome had only a few categories. When the proxy assumption was inadequate, both the CLM and LRM had biased estimates with poor coverage. The proxy assumption was appropriate for four of the five subscales in our real data application to fatigue scores, which highlighted the importance of assessing the proxy assumption to avoid reporting invalid estimates in terms of the difference in scores.

Conclusion

The proxy assumption is critical to the interpretation of the exposure effect on the difference in mean QoL scores. CLM offers a valid test for the presence of an association, a method for assessing the proxy assumption, and when the assumption is adequate, an assessment for clinical significance using the difference in means.

Development and Content Validation of Pruritus and Symptoms Assessment for Atopic Dermatitis (PSAAD) in Adolescents and Adults with Moderate-to-Severe AD

Introduction

Most patient-reported outcome (PRO) instruments that measure atopic dermatitis (AD) symptoms do not have sufficient documented evidence of content validity to satisfy regulatory agency guidance for inclusion in product-labelling claims in the USA or Europe. The objective of this study was to develop a PRO instrument in accordance with regulatory agency guidance to assess daily AD symptoms during the course of therapy and to establish its content validity and psychometric properties.

Methods

The Pruritus and Symptoms Assessment for Atopic Dermatitis (PSAAD) daily diary was developed based on qualitative interviews with US adolescents and adults with mild-to-severe AD. Content validity, test–retest reliability, internal consistency reliability, clinically important difference, clinically important responder, convergent validity, and known-group validity were evaluated using correlational and regression methods from phase 2b data from US adults with moderate-to-severe AD who were treated with abrocitinib.

Results

Patient interviews conducted with US adolescents and adults with mild-to-severe AD identified 11 relevant symptoms (itch, dryness, redness, flaking, discolouration, pain, bleeding, cracking, bumps, swelling, and weeping/oozing) for inclusion in the PSAAD instrument. All PSAAD psychometric parameters were acceptable based on phase 2b data from US adults with moderate-to-severe AD. Convergent validity and known-group validity were confirmed by significant correlations between PSAAD and six other PRO measures (r = 0.24–0.91, all p ≤ 0.01) and Dermatology Life Quality Index category (p ≤ 0.0001), respectively.

Conclusions

Evidence supports the PSAAD instrument validity, reliability, responsiveness and definitions of clinically important changes/differences for adults with moderate-to-severe AD.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13555-020-00474-9.

Content validity of a sleep numerical rating scale and a sleep diary in adults and adolescents with moderate-to-severe atopic dermatitis

Background

The intense itching associated with atopic dermatitis (AD) often causes patients to experience severe sleep disturbance. Here, we describe the results of a two-phase concept elicitation and cognitive interview study to establish the content validity of a sleep disturbance numerical rating scale (SD NRS) and a Consensus Sleep Diary adapted for adults and adolescents with moderate-to-severe AD (CSD-AD©).

Results

In phase I, a concept elicitation conducted in 20 adults and 10 adolescents with moderate-to-severe AD revealed that the following sleep-related issues were important and relevant: nighttime awakening (87%), trouble falling asleep (73%), feeling unrested (53%), daytime fatigue or sleepiness (53%), and feeling as if they did not get enough sleep (33%). The frequency and extent of sleep disturbance varied substantially from day to day due to varying degrees of itching and flares, medication use, and changes in the weather. All participants understood the SD NRS question, with most finding it easy or very easy to understand (100% of adults and 90% of adolescents) and most understanding the anchors as intended (95% of adults, and 100% of adolescents). Most participants (94% of adults, and 90% of adolescents) indicated that they would consider a one- or two-point change meaningful on the SD NRS. The CSD-AD© was revised based on participant feedback, and tested during phase II in a convenience sample of six adults and four adolescents from phase I. The changes made to the CSD-AD© were confirmed to be relevant and understandable. All patients were able to provide an answer to each item in the CSD-AD©, and most were able to estimate the duration of nighttime awakenings, daytime naps, and dozing.

Conclusions

The study supported the content validity of the SD NRS and CSD-AD© in adults and adolescents with moderate-to-severe AD. It also emphasized the importance of using these instruments daily when assessing the benefit of a new treatment on sleep quality in this population.

Supplementary Information

Supplementary information accompanies this paper at 10.1186/s41687-020-00265-y.

Tofacitinib in Patients with Ulcerative Colitis: Inflammatory Bowel Disease Questionnaire Items in Phase 3 Randomized Controlled Induction Studies

BACKGROUND

Tofacitinib is an oral, small molecule Janus kinase inhibitor for the treatment of ulcerative colitis (UC). We examined the effect of tofacitinib induction treatment on Inflammatory Bowel Disease Questionnaire (IBDQ) items in adults with moderate to severe UC.

METHODS

Data were pooled from the randomized, 8‑week, double-blind, phase 3 OCTAVE Induction 1 and 2 studies. The IBDQ was self-administered by patients at baseline, week 4, and week 8, with higher scores indicating better health-related quality of life (HRQoL). Change from baseline in IBDQ items was analyzed for 10 mg of tofacitinib twice daily (BID) vs placebo using a linear mixed-effects model, with no multiplicity adjustment performed. Effect sizes were calculated. Subgroup analyses by tumor necrosis factor inhibitor (TNFi) experience were performed.

RESULTS

Significant improvements (nominal P < 0.05) were observed in all IBDQ items with 10 mg of tofacitinib BID vs placebo at weeks 4 and 8. For the overall population, the largest treatment differences across all items were reported for "bowel movements been loose" at weeks 4 and 8, and "problem with rectal bleeding" at week 8 (mean treatment differences all 1.1; both in bowel symptoms domain). These items also showed the largest effect sizes. Treatment benefits were generally slightly numerically higher in TNFi-experienced vs TNFi-naïve patients.

CONCLUSIONS

Tofacitinib induction therapy improved all IBDQ items vs placebo in patients with UC, reflecting improvements in HRQoL, with greatest benefits reported in bowel symptoms domain items (Funded by Pfizer Inc; OCTAVE Induction 1 and OCTAVE Induction 2; ClinicalTrials.gov, NCT01465763 and NCT01458951, respectively).

Translation, cultural adaptation and validation of the Chinese Multimorbidity Treatment Burden Questionnaire(C-MTBQ): a study of older hospital patients

Background

Due to an ageing population, multimorbidity is becoming more common. Treatment burden (the effort required of patients to look after their health and the impact this has on their wellbeing) is prevalent in patients with multimorbidity. The Multimorbidity Treatment Burden Questionnaire (MTBQ) is a patient-reported outcome measure of treatment burden that has been validated amongst patients with multimorbidity in the UK. The aim of this study was to translate and culturally adapt the MTBQ into Chinese and to assess its reliability and validity in elderly patients with multimorbidity in hospital.

Methods

The original English version of the MTBQ was translated into Chinese using Brislin’s model of cross-culture translation. The C-MTBQ was piloted on a sample of 30 elderly patients with multimorbidity prior to being completed by 156 Chinese elderly patients with multimorbidity recruited from a hospital in Zhengzhou, China. We examined the proportion of missing data, the distribution of responses and floor and ceiling effects for each question. Factor analysis, Cronbach’s alpha, intraclass coefficient and Spearman’s rank correlations assessed dimensional structure, internal consistency reliability, test-retest reliability and criterion validity, respectively.

Results

The average age of the respondents was 73.5 years (range 60–99 years). The median C-MTBQ global score was 20.8 (interquartile range 12.5–29.2). Significant floor effects were seen for all items. Factor analysis supported a three-factor structure. The C-MTBQ had high internal consistency (Cronbach’s alpha coefficient, 0.76) and test-retest reliability (the intraclass correlation coefficient, 0.944), the correlations between every item and global scores scored > 0.4. The scale content validity index(S-CVI) was 0.89, and the item level content validity index(I-CVI)was 0.83 ~ 1.00. The criterion validity was 0.875.

Conclusion

The Chinese version of MTBQ showed satisfactory reliability and validity in elderly patients with multimorbidity, and could be used as a tool to measure treatment burden of elderly patients with multimorbidity in hospital.

Considerations for development of an evidence dossier to support the use of mobile sensor technology for clinical outcome assessments in clinical trials

BACKGROUND

Mobile sensors offer enormous potential for the collection of informative clinical endpoints in clinical trials to support regulatory decision making and product labelling. There are currently no specific guidelines on the information needed to enable regulators to review and accept proposed endpoints derived from mobile sensors for use in drug development trials.

OBJECTIVE

The purpose of this working group report is to recommend the structure and content of an evidence dossier intended to support whether a clinical endpoint derived from mobile sensor data is fit-for-purpose for use in regulatory submissions for drug approvals.

EVIDENCE DOSSIER

The structure and content of a dossier to provide evidence supporting the use of a sensor-derived clinical endpoint is described. Sections include clinical endpoint definition and positioning, the concept of interest, the context of use, clinical validation and interpretation, study implementation, and analytical validity with sensor performance verification in support of the selected sensor.

CONCLUSIONS

In the absence of definitive regulatory guidance, this report provides a considered approach to compiling a comprehensive body of evidence to justify acceptance of mobile sensors for support of new drug applications.

Considerations in testing treatment effects on transient event driven health status changes measured by patient reported outcomes

Many treatments and drugs are intended to reduce the occurrence of negative events of interest, control the severity of the events, accelerate recovery from the events, or a combination of these effects. While assessing the clinical effect is typically the primary objective of a trial, testing the treatment effect on the health status of patients based on patient reported outcome (PRO) can be a useful component in determining the value of a treatment. Analysis of PROs in this setting, however, face the following challenges: the PRO value immediately after the event occurrence is often not captured, and the effect of the event on health status measured by the PRO is transient as subjects recover over time. Therefore, traditional statistical methods used to assess treatment effects suffer from low power for PROs. In this manuscript, we apply a kernel smoothing technique to estimate before- and after-event PRO values. We also propose new test outcomes based on observed and estimated PRO values and evaluate tests that focus on the tail distributions. We demonstrate that the tail distribution tests using the new outcomes can achieve high power under certain conditions.

Validity and Clinically Meaningful Changes in the Psychosexual Daily Questionnaire and Derogatis Interview for Sexual Function Assessment: Results From the Testosterone Trials

BACKGROUND

Limited information is available on the performance characteristics of 2 questionnaires commonly used in clinical research, the Psychosexual Daily Questionnaire (PDQ) and the Derogatis Interview for Sexual Function (DISF)-II Assessment, especially in older men with low testosterone (T) and impaired sexual function.

AIM

To determine reliability of PDQ and DISF-II by assessing the correlation within and between domains in the questionnaires and to define clinically meaningful changes in sexual activity (PDQ question 4 [Q4]) and desire (DISF-II sexual desire domain [SDD]) domains.

METHODS

Data from 470 men participating in the T Trials were used to calculate Spearman correlation coefficients of individual items and total score among questionnaires to determine convergent and construct validity. Clinically meaningful changes for sexual desire and activity were determined by randomly dividing the sample into training and validation sets. Anchor- and distribution-based clinically meaningful change criteria were defined in the training set, and selected changes were evaluated in the validation set.

OUTCOMES

Validity of the PDQ and DISF-II and clinically meaningful changes in sexual desire and activity were determined in older men in T Trials.

RESULTS

Moderate to strong correlations were shown within and between domains from different questionnaires. Using Patient Global Impression of Change as an anchor, clinically meaningful change in PDQ sexual activity was ≥0.6, and in DISF-SDD was ≥5.0. Applying these change cut-points to the validation set, a greater proportion of T-treated men achieved clinically meaningful improvement in their sexual desire and activity compared to placebo-treated men.

CLINICAL IMPLICATIONS

The PDQ-Q4 and DISF-II-SDD can be used to reliably assess clinically meaningful changes in sexual activity and sexual desire in hypogonadal men treated with T.

STRENGTHS & LIMITATIONS

Strengths of this study include a large sample size, long trial duration, and inclusion of men with low libido and unequivocally low T levels. Limitations include using data from a single study that enrolled only older hypogonadal men, and only 1 anchor for both sexual desire and activity.

CONCLUSION

Moderate to strong correlations were demonstrated within and between different sexual domains of the PDQ and DISF-II confirming construct and convergent validity. Clinically meaningful improvement in elderly hypogonadal men was change of ≥0.6 score in the PDQ-Q4 and ≥5.0 in the DISF-SDD. Improvements in sexual activity and desire in the T Trials were modest but clinically meaningful. Wang C, Stephens-Shields AJ, DeRogatis LR, et al. Validity and Clinically Meaningful Changes in the Psychosexual Daily Questionnaire and Derogatis Interview for Sexual Function Assessment: Results From the Testosterone Trials. J Sex Med 2018;15:997-1009.

Stability Enhanced Variable Selection for a Semiparametric Model with Flexible Missingness Mechanism and Its Application to the ChAMP Study

This paper is motivated by the analytical challenges we encounter when analyzing the ChAMP (Chondral Lesions And Meniscus Procedures) study, a randomized controlled trial to compare debridement to observation of chondral lesions in arthroscopic knee surgery. The main outcome, WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index) pain score, is derived from the patient's responses to the questionnaire collected in the study. The major goal is to identify potentially important variables that contribute to this outcome. In this paper, the model of interest is a semiparametric model for the pain score. To address the missing data issue, we adopt a flexible missingness mechanism which is much more versatile in practice than a single parametric model. Then we propose a pairwise conditional likelihood approach to estimate the unknown parameter in the semiparametric model without the need of modeling its nonparametric counterpart nor the missingness mechanism. For variable selection we apply a regularization approach with a variety of stability enhanced tuning parameter selection methods. We conduct comprehensive simulation studies to evaluate the performance of the proposed method. We also apply the proposed method to the ChAMP study to demonstrate its usefulness.

Establishing clinically-relevant terms and severity thresholds for Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting

PURPOSE

Patient-Reported Outcomes Measurement Information System^® (PROMIS^®) physical function, cognitive function, and sleep disturbance measures are increasingly used in cancer care. However, there is limited guidance for interpreting the clinical meaning of scores. This study aimed to apply bookmarking, a standard setting methodology, to identify PROMIS score thresholds in the context of cancer care.

METHODS

Using item parameters, we constructed vignettes of five items covering the range of possible scores. Focus groups were held with cancer care providers and people with cancer. Terminology for categorizing levels of severity was explored. Participants rank ordered vignettes by severity and then placed bookmarks between vignettes representing different levels of severity. Group discussion was held until consensus on bookmark placement was reached.

RESULTS

Clinicians selected "within normal limits," "mild," "moderate," and "severe" to describe levels of severity. Both patients and clinicians were able to apply these labels, but there was not unanimous support for any set of descriptors. Clinicians and patients agreed on all severity thresholds for sleep disturbance. For cognitive and physical function, clinicians and patients agreed on the threshold between "within normal limits" and "mild." However, patients required greater dysfunction than clinicians before applying "moderate" and "severe" labels.

CONCLUSIONS

Bookmarking can be applied to develop provisional score interpretation for PROMIS measures. Patients and clinicians were frequently consistent in their bookmark placement. When there was variance, patients required more dysfunction before assigning more severity. Additional research with other cancer samples is needed to evaluate the replicability and generalizability of our findings.

Clinically Meaningful Change in Sexual Desire in the Psychosexual Daily Questionnaire in Older Men from the TTrials

BACKGROUND

A recent study of older men participating in the Testosterone Trials (TTrials) defined a clinically meaningful change in the Psychosexual Daily Questionnaire (PDQ) question 4 in hypogonadal men age ≥65 years. This study defines clinically meaningful change in the same population for sexual desire assessed by PDQ question 1.

AIM

To determine a clinically meaningful change in the answers to question 1 of the PDQ in hypogonadal older men.

METHODS

Participants in the Sexual Function Trial of the TTrials were randomly divided into a training and test set. Anchor-based methods, including regression analysis, receiver operating characteristic curves, and empirical cumulative distribution functions, were used to determine a clinically meaningful change on question 1 in the training set, and the selected threshold was evaluated in the test set for an effect of testosterone treatment.

RESULTS

A clinically meaningful increase in question 1 of the PDQ was determined to be ≥0.7 points.

CLINICAL IMPLICATIONS

Question 1 of the PDQ can be used to assess sexual desire in response to testosterone treatment.

STRENGTHS & LIMITATIONS

Data were obtained from a single large study of older hypogonadal men.

CONCLUSION

Clinically meaningful improvement of sexual desire is a change of ≥0.7 in the score of question 1 of the PDQ. Stephens-Shields AJ, Wang C, Preston P, et al. Clinically Meaningful Change in Sexual Desire in the Psychosexual Daily Questionnaire in Older Men from the TTrials. J Sex Med 2019;16:951-953.

Use of Both Qualitative and Quantitative Methods to Estimate Meaningful Change Thresholds for Key Endpoints in Pediatric Asthma Trials

INTRODUCTION

Diary-derived symptom score and rescue medication use endpoints, such as symptom-free days (SFDs) and rescue medication-free days (RFD), are frequently used as clinical trial endpoints. Estimates of meaningful change for SFDs and RFDs have not been generated in pediatric populations. This research aimed to generate evidence supporting estimates of the individual within-patient changes that constitute an important or meaningful change in SFDs, RFDs, and updated estimates on the Childhood Asthma Control Test (C-ACT) in pediatric asthma populations aged 5-11 years.

METHODS

Semistructured, qualitative interviews were conducted with children (ages 8-11 years) who had asthma and parents/caregivers of children (4-11 years) with asthma. Before the interview (4-9 days) participants were asked to complete a morning and evening diary.

RESULTS

On average, parent/caregiver estimates of the difference in SFDs between a "very bad" and a "little bad" week for their children's asthma were largely concordant with the values reported by their children (differences of 1.8 and 1.4 SFDs, respectively). Both parents/caregivers and children were able to articulate what a meaningful level of change would be on the C-ACT at the item level. This qualitative study generated C-ACT item-level meaningful change estimates in the region of 1-3 category change, which potentially suggests that, if scaled up to represent C-ACT total score, this would lead to change estimates of 7-15 points.

CONCLUSIONS

Our findings suggest that both children with asthma and parents/caregivers can quantitatively estimate and to some extent qualitatively articulate meaningful change in SFDs and RFDs.

Important Group Differences on the Functional Assessment of Cancer Therapy-Kidney Symptom Index Disease-Related Symptoms in Patients with Metastatic Renal Cell Carcinoma

BACKGROUND

The Functional Assessment of Cancer Therapy-Kidney Symptom Index Disease-Related Symptoms (FKSI-DRS) is important to gauge clinical benefit in metastatic renal cell carcinoma (mRCC).

OBJECTIVES

To estimate important difference (ID) in FKSI-DRS scores that is considered to be meaningful when comparing treatment effect between groups, using mRCC trial data.

METHODS

Data were derived from two pivotal phase III mRCC trials comparing sunitinib versus interferon alfa (N = 750) in first-line mRCC, and axitinib versus sorafenib (N = 723) in second-line mRCC. The change from baseline in FKSI-DRS score was examined as a function of a set of anchors using the repeated-measures model. Several anchors were evaluated: FKSI item "I am bothered by side effects of treatment," EuroQol five-dimensional questionnaire utility score, and adverse events.

RESULTS

When the "I am bothered by side effects of treatment" score was used as an anchor, the ID ranged between 1.2 and 1.3 points. When change in the EuroQol five-dimensional questionnaire utility score was used as an anchor, the FKSI-DRS ID ranged between 0.62 and 0.63 points. Selecting the adverse events that corresponded to a maximum worsening in the FKSI-DRS score in either trial, the ID ranged between 0.62 and 0.74 points.

CONCLUSIONS

Among patients undergoing treatment for mRCC, between-group differences in FKSI-DRS scores as low as 1 point might be meaningful.

A quantitative method for measuring the relationship between an objective endpoint and patient reported outcome measures

Patient reported outcome measures (PROMs) become increasingly important for assessing the effectiveness of a drug or medical device. In order for a PROM to be claimed in labeling, the PROM has to be valid, reliable and able to detect a change if the targeted disease status changes. One approach to assess the quality of a patient reported outcome measure (PROM) is to investigate the association between the PROM and an objective clinical endpoint measuring the status of a disease/condition. However, methods assessing the association between continuous and discrete variables are limited, especially for correlated measurements. In this paper, we propose a method to assess such association with any type of samples with or without correlation. The method involves estimating the probability revealing the status of a subject’s disease/condition (called truth thereafter) through the subject’s reported outcomes. The probability is a conditional probability revealing truth given the relative location of the subject’s objective outcome compared to the subject-specific latent threshold in the objective endpoint. A consistent estimator for the probability is derived. The operating characteristics of the consistent estimator are illustrated using simulation. Our method is applied to hypothetical clinical trial data generated for an ophthalmic device as an illustration.

Important differences and meaningful changes for the Functional Assessment of Cancer Therapy-Cognitive Function (FACT-Cog)

Background

We estimated clinically important, group-level differences in self-reported cognitive function for the Functional Assessment of Cancer Therapy-Cognitive Function (FACT-Cog) instrument. We also investigated individual level change that could be considered meaningful for cancer survivors affected by cognitive impairment following chemotherapy, and that could be used for responder analyses. We used data from a multi-site randomized controlled trial in 242 participants that evaluated a web-based intervention for improving self-reported cognitive functioning in adult cancer survivors who reported cognitive impairment and who had adjuvant chemotherapy in the previous 6–60 months. We used anchor and distribution methods to estimate a range of clinically important differences (CIDs) and investigated meaningful change thresholds (MCTs) for the FACT-Cog and the Perceived Cognitive Impairments (PCI) subscale, post-intervention and at six-month follow-up with empirical cumulative distribution functions. Our primary anchor was the patient reported cognitive function subscale of the European Organization for Research and Treatment of Cancer Quality of Life-Cognitive Functioning Scale (EORTC-CF).

Results

Most participants were female (95%) breast cancer survivors (89%). Correlation of changes in the FACT-Cog and the EORTC-CF were 0.55 post-intervention and 0.61 at follow-up. Anchor-based CID estimates for the FACT-Cog using our primary anchor were 11.3 points (post) and 8.8 (follow-up), which corresponds to a standardized effect size of 0.49 and 0.38; 8.6% and 6.6% of the total scale’s range. Anchor-based CID estimates for the FACT-Cog PCI subscale were 7.4 (post) and 4.6 points (follow-up), which corresponds to a standardized effect size of 0.50 and 0.31; 10.3% and 6.4% of the PCI range). Empirical cumulative distribution functions of change in FACT-Cog demonstrating possible MCTs showed that anchor change of none, minimally better and much better were well separated.

Conclusions

The CID and MCT estimates from this manuscript can help in the design, analysis and interpretation of self-reported cognitive function in cancer patients and survivors.

A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs)

BACKGROUND

Patient-reported outcome measures (PROMs) could play an important role in identifying patients' needs and goals in clinical encounters, improving communication and decision-making with clinicians, while making care more patient-centred. Comprehensive evidence that PROMS are an effective intervention is lacking in single randomised controlled trials (RCTs).

METHODS

A systematic search was performed using controlled vocabulary related to the terms: clinical care setting and patient-reported outcome. English language studies were included if they were a RCT with a PROM as an intervention in a patient population. Included studies were analysed and their methodologic quality was appraised using the Cochrane Risk of Bias tool. The protocol was registered with PROSPERO (CRD42016034182).

RESULTS

Of 4302 articles initially identified, 115 underwent full-text review resulting in 22 studies reporting on 25 comparisons. The majority of included studies were conducted in USA (11), among cancer patients (11), with adult participants only (20). Statistically significant and robust improvements were reported in the pre-specified outcomes of the process of care (2) and health care (3). Additionally, five, eight and three statistically significant but possibly non-robust findings were reported in the process of care, health and patient satisfaction outcomes, respectively.

CONCLUSIONS

Overall, studies that compared PROM to standard care either reported a positive effect or were not powered to find pre-specified differences. There is justification for the use of a PROM as part of standard care, but further adequately powered studies on their use in different contexts are necessary for a more comprehensive evidence base.

Does Having Open Access to Care Improve Patient Experience? A Case-Control Study

This study investigated whether primary and specialist care practices utilizing open access to care (OA) receive better patient experience scores than propensity-matched control practices without OA. From March 2010 to December 2014, 711 physicians classified as having OA in their practice, indicated by scoring 15 or higher on the OA checklist, were propensity matched to practices without OA. Patient experience was measured with 5 composites: timely care, communication, staff quality, care coordination, and overall physician rating. Minimally important differences in patient experience ratings were calculated between OA and control practices to determine optimal OA checklist scores. OA positively affected most composite domains for specialist practices, except physician rating, but minimally affected primary care practices. Practices scoring 19 or higher on the OA checklist had significantly higher patient-experience scores than matched controls. The authors recommend practices strive for 20 or higher on the OA checklist to see significant improvements in patient experience ratings.

Clinically important difference on the Erectile Dysfunction Inventory of Treatment Satisfaction questionnaire in patients with erectile dysfunction

AIM

To determine what constitutes a clinically important difference (CID) on the Erectile Dysfunction Inventory of Treatment Satisfaction (EDITS), an 11-item validated questionnaire assessing treatment satisfaction used in clinical trials for patients with erectile dysfunction (ED).

METHODS

Erectile Dysfunction Inventory of Treatment Satisfaction data were evaluated from a double-blind, fixed-dose trial of 279 men aged 18-65 years with ED who were treated with sildenafil 50 or 100 mg or placebo. The primary anchor measure was the erectile function (EF) domain of the International Index of Erectile Function (IIEF), which has a 4-point minimal CID. The CID on the EDITS index score was determined using a regression analysis comparing EDITS and IIEF EF scores at the end of the 8-week treatment. A similar analysis was performed for EDITS and the Erection Hardness Score (EHS) instrument, a single-item questionnaire measuring hardness, which was used as a secondary anchor measure.

RESULTS

Erectile Dysfunction Inventory of Treatment Satisfaction and IIEF EF domain scores were highly correlated (Pearson correlation coefficient = 0.75). EDITS total scores across treatments at week 8 averaged (mean ± standard deviation [SD]) 67.5 ± 21.6 (range, 0-100; higher scores indicate greater treatment satisfaction); IIEF EF domain scores averaged 22.2 ± 6.9 (range, 1-30; higher scores indicate higher erectile functioning). The calculated CID for EDITS scores was 9.5 (95% CI, 8.5-10.4; 0.44 SD units), corresponding to a medium effect size. EDITS and EHS instrument scores also correlated highly (Pearson correlation coefficient = 0.64). Placebo-adjusted EDITS mean scores were more than twice the CID, at 23 (95% CI, 17-28) and 28 (95% CI, 23-33) for the 50- and 100-mg doses, respectively.

CONCLUSION

Approximately 10 points on the EDITS index score is considered a CID. Serving as a benchmark, this finding aids interpretation of the clinical relevance of a difference in mean EDITS index scores between treatments for patients with ED.

Criteria for gauging response to sodium oxybate for narcolepsy

Our objective was to define responder criteria using an anchor-based approach for frequency of cataplexy attacks and excessive daytime sleepiness in patients with narcolepsy undergoing sodium oxybate treatment. We used pooled data from two randomized, placebo-controlled, double-blind, multicentre 4- and 8-week trials of sodium oxybate for narcolepsy with cataplexy and analysed using receiver operator characteristics analysis. The percentage change in frequency of weekly cataplexy attacks and the Epworth Sleepiness Scale outcomes were compared with Clinical Global Impression of Change ratings, used as the anchor to define true response. Participants (n = 336) were 39% male, 89% white, with a mean age of 41.5 (15.3) years, reporting a median of 20.5 cataplexy attacks per week and a mean Epworth Sleepiness score of 17.5 at baseline. A majority (51%) were Much Improved or Very Much Improved based on Clinical Global Impression of Change ratings, considered a true response to treatment. Area under the curve values for % reduction in cataplexy attacks (77%) and % change in sleepiness score (78%) supported response definition thresholds of 46% and 12%, respectively. Classification using either response definition agreed with the anchor for approximately 71% of participants. Cataplexy response definition was more sensitive (cataplexy = 0.77, Epworth Sleepiness Scale = 0.69), while sleepiness was more specific (cataplexy = 0.66, Epworth Sleepiness Scale = 0.75). Both responder definitions showed a dose-response relationship with sodium oxybate, demonstrating their validity using an external criterion. Weekly cataplexy attacks and Epworth Sleepiness Scale can be used to help document clinical response to narcolepsy treatment using criteria of 46% and 12% reductions, respectively.

Statistical controversies in cancer research: using standardized effect size graphs to enhance interpretability of cancer-related clinical trials with patient-reported outcomes

Patient reported outcomes (PROs) are becoming increasingly important in cancer studies, particularly with the emphasis on patient centered outcome research. However, multiple PROs, using different scales, with different directions of favorability are often used within a trial, making interpretation difficult. To enhance interpretability, we propose the use of a standardized effect size graph, which shows all PROs from a study on the same figure, on the same scale. Plotting standardized effects with their 95% confidence intervals (CIs) on a single graph clearly showing the null value conveys a comprehensive picture of trial results. We demonstrate how to create such a graph using data from a randomized controlled trial that measured 12 PROs at two time points. The 24 effect sizes and CIs are shown on one graph and clearly indicate that the intervention is effective and sustained.

Determining the longitudinal validity and meaningful differences in HRQL of the PedsQL™ Sickle Cell Disease Module

Background

Detecting change in health status over time and ascertaining meaningful changes are critical elements when using health-related quality of life (HRQL) instruments to measure patient-centered outcomes. The PedsQL™ Sickle Cell Disease module, a disease specific HRQL instrument, has previously been shown to be valid and reliable. Our objectives were to determine the longitudinal validity of the PedsQL™ Sickle Cell Disease module and the change in HRQL that is meaningful to patients.

Methods

An ancillary study was conducted utilizing a multi-center prospective trial design. Children ages 4–21 years with sickle cell disease admitted to the hospital for an acute painful vaso-oclusive crisis were eligible. Children completed HRQL assessments at three time points (in the Emergency Department, one week post-discharge, and at return to baseline (One to three months post-discharge). The primary outcome was change in HRQL score. Both distribution (effect size, standard error of measurement (SEM)) and anchor (global change assessment) based methods were used to determine the longitudinal validity and meaningful change in HRQL. Changes in HRQL meaningful to patients were identified by anchoring the change scores to the patient’s perception of global improvement in pain.

Results

Moderate effect sizes (0.20–0.80) were determined for all domains except the Communication I and Cognitive Fatigue domains. The value of 1 SEM varied from 3.8–14.6 across all domains. Over 50% of patients improved by at least 1 SEM in Total HRQL score. A HRQL change score of 7–10 in the pain domains represented minimal perceived improvement in HRQL and a HRQL change score of 18 or greater represented moderate to large improvement.

Conclusions

The PedsQL™ Sickle Cell Disease Module is responsive to changes in HRQL in patients experiencing acute painful vaso-occlusive crises. The study data establish longitudinal validity and meaningful change parameters for the PedsQL™ Sickle Cell Disease Module.

Trial Registration

ClinicalTrials.gov (study identifier: NCT01197417). Date of registration: 08/30/2010

Electronic supplementary material

The online version of this article (doi:10.1186/s12955-017-0700-2) contains supplementary material, which is available to authorized users.

Structural equation modeling-based effect-size indices were used to evaluate and interpret the impact of response shift effects

OBJECTIVES

The investigation of response shift in patient-reported outcomes (PROs) is important in both clinical practice and research. Insight into the presence and strength of response shift effects is necessary for a valid interpretation of change.

STUDY DESIGN AND SETTING

When response shift is investigated through structural equation modeling (SEM), observed change can be decomposed into change because of recalibration response shift, change because of reprioritization and/or reconceptualization response shift, and change because of change in the construct of interest. Subsequently, calculating effect-size indices of change enables evaluation and interpretation of the clinical significance of these different types of change.

RESULTS

Change was investigated in health-related quality of life data from 170 cancer patients, assessed before surgery and 3 months after surgery. Results indicated that patients deteriorated on general physical health and general fitness and improved on general mental health. The decomposition of change showed that the impact of response shift on the assessment of change was small.

CONCLUSION

SEM can be used to enable the evaluation and interpretation of the impact of response shift effects on the assessment of change, particularly through calculation of effect-size indices. Insight into the occurrence and clinical significance of possible response shift effects will help to better understand changes in PROs.

What is quality of life and how do we measure it? Relevance to Parkinson's disease and movement disorders

Health-related quality of life is a patient-reported outcome that complements clinical evaluation and provides information about disease activity and effects of the treatment. The objective of this review is to present the conceptual framework, the measures, and some of their most relevant applications in the field of Parkinson's disease and movement disorders. Health-related quality of life is a subjective, individual, and multidimensional construct, and its main dimensions are physical, mental, and social, besides global perceptions of health and personal domains. Health-related quality of life measurement is carried out by means of questionnaires or scales, ideally self-applied by patients, and has a diversity of important applications for clinical practice, research, and health policy. Movement disorders and Parkinson's disease are complex conditions impacting all components of patients' health-related quality of life. The use of health-related quality of life tools provides important information on a variety of aspects that are important to patients while complementing clinical evaluations. In particular, studies using this kind of assessment can identify and monitor the most important health-related quality of life determinant factors, allowing tailored assistance and prioritized interventions. In addition, maintaining or improving the patients' health-related quality of life is an objective of care for chronic diseases and, therefore, it has to be monitored over time and as an outcome of clinical trials. Several methods are available for the interpretation of the change in scores of health-related quality of life measures, although a definitive agreement on the most appropriate method is yet to be determined. Presently, health-related quality of life assessment is an important outcome for research and management of chronic conditions such as Parkinson's disease and other movement disorders. © 2016 International Parkinson and Movement Disorder Society.

Diagnosing Hidradenitis Suppurativa

Three diagnostic criteria must be met for hidradenitis suppurativa: typical lesions, occurrence in one or more of the predilection areas, and that it is chronic and/or recurrent. Several outcome measures are used, including patient-reported pain and itch scales, Dermatology Life Quality Index, and Skindex. Hidradenitis suppurativa is associated with significant comorbidities that must be addressed in the evaluation of the patients.

Characterizing neuropathic pain profiles: enriching interpretation of painDETECT

Purpose

To psychometrically evaluate painDETECT, a patient-reported screening questionnaire for neuropathic pain (NeP), for discriminating among sensory pain symptoms (burning, tingling/prickling, light touching, sudden pain attacks/electric shock-type pain, cold/heat, numbness, and slight pressure).

Methods

The seven-item version of painDETECT provides an overall score that targets only sensory symptoms, while the nine-item version adds responses on two items to the overall score, covering pain course pattern and pain radiation. Both versions have relevance in terms of characterizing broad NeP. The nine- and seven-item versions of painDETECT were administered to subjects with confirmed NeP across six conditions identified during office visits to US community-based physicians. Responses on the sensory symptom items were dichotomized into “at least moderate” (ie, moderate, strongly, very strongly) relative to the combined other responses (never, hardly noticed, slightly). Logistic regression of dichotomized variables on the total painDETECT score provided probabilities of experiencing each symptom across the range of painDETECT scores.

Results

Both painDETECT versions discriminated among the symptoms with similar probabilities across the score ranges. Using these data, the probability of moderately experiencing each pain sensory item was estimated for a particular score, providing a pain profile. Additionally, the likelihood of experiencing each sensation was determined for a discrete increase in score, ie, the odds of at least a moderate sensation of burning (versus less than a moderate sensation) was 1.29 for a 1-point increase, 3.52 for a 5-point increase, and 12.42 for every 10-point increase in the nine-item painDETECT score.

Conclusion

painDETECT differentiates pain profiles across the range of scores such that, for a particular score, the probability of experiencing at least a moderate sensation of each symptom was determined and compared. These results can help characterize NeP symptomatology, enrich interpretation of painDETECT scores, and provide a basis for individualizing NeP management.

Systematic collection of patient reported outcome research data: A checklist for clinical research professionals

Understanding the human experience is no longer an outcome explored strictly by social and behavioral researchers. Increasingly, biomedical researchers are also including patient reported outcomes (PROs) in their clinical research studies not only due to calls for increased patient engagement in research but also healthcare. Collecting PROs in clinical research studies offers a lens into the patient's unique perspective providing important information to industry sponsors and the FDA. Approximately 30% of trials include PROs as primary or secondary endpoints and a quarter of FDA new drug, device and biologic applications include PRO data to support labeling claims. In this paper PRO, represents any information obtained directly from the patient or their proxy, without interpretation by another individual to ascertain their health, evaluate symptoms or conditions and extends the reference of PRO, as defined by the FDA, to include other sources such as patient diaries. Consumers and clinicians consistently report that PRO data are valued, and can aide when deciding between treatment options; therefore an integral part of clinical research. However, little guidance exists for clinical research professionals (CRPs) responsible for collecting PRO data on the best practices to ensure quality data collection so that an accurate assessment of the patient's view is collected. Therefore the purpose of this work was to develop and validate a checklist to guide quality collection of PRO data. The checklist synthesizes best practices from published literature and expert opinions addressing practical and methodological challenges CRPs often encounter when collecting PRO data in research settings.

Best (but oft-forgotten) practices: expressing and interpreting associations and effect sizes in clinical outcome assessments

This article reviews methods used to facilitate the interpretation and evaluation of group-level differences in clinical outcome assessments. These methods complement and supplement tests of statistical significance. Examples, including studies in nutrition, are used to illustrate the application of the interpretation methods for group-level comparisons from experimental or observational studies. In addition, specific pitfalls of evaluating change in meta-analysis studies are described. A set of recommendations is provided. This review is intended as an introduction for the novice and as a refresher for the experienced researcher.

Best (but oft-forgotten) practices: expressing and interpreting associations and effect sizes in clinical outcome assessments

This article reviews methods used to facilitate the interpretation and evaluation of group-level differences in clinical outcome assessments. These methods complement and supplement tests of statistical significance. Examples, including studies in nutrition, are used to illustrate the application of the interpretation methods for group-level comparisons from experimental or observational studies. In addition, specific pitfalls of evaluating change in meta-analysis studies are described. A set of recommendations is provided. This review is intended as an introduction for the novice and as a refresher for the experienced researcher.

Simplified Interpretation of the Erectile Function Domain of the International Index of Erectile Function

INTRODUCTION

This report describes a post hoc analysis of data from a randomized, double-blinded, placebo-controlled, flexible-dose, sildenafil trial in men with erectile dysfunction.

AIMS

To simplify interpretation of erectile function (EF) domain scores of the International Index of Erectile Function (IIEF).

METHODS

Men at least 18 years old with erectile dysfunction were randomized to receive sildenafil or placebo for 12 weeks. Men taking nitrates or nitric oxide donors were excluded. Responses for each IIEF EF domain question (questions 1-5 and 15) were combined into two broad categories ("success" for responses of the two most favorable categories of a question and "no success" for other responses). Each question was expressed in a logistic regression model (sildenafil and placebo groups combined) as a function of overall EF domain score.

MAIN OUTCOME MEASURES

IIEF EF domain score and items.

RESULTS

A four-point increase in the IIEF EF domain score was associated with an odds ratio of success of 6.1 for getting an erection, 29.2 for having a firm erection, 10.0 for able to penetrate,12.8 for maintaining erection, 4.0 for maintaining erection to completion, and 3.7 for erection confidence. An EF domain score of 22 was associated with a probability of success of 81% for getting an erection, 86% for having a firm erection, 89% for able to penetrate, 67% for maintaining an erection, 70% for maintaining an erection to completion, and 32% for erection confidence. For an EF domain score of 16, the corresponding probabilities of success were 22%, 4%, 20%, 4%, 22%, and 6%, respectively.

CONCLUSION

These results provide stakeholders with a simplified and meaningful interpretation of IIEF EF domain scores based on six key aspects of EF.