Patient versus proxy response on global health scales: no meaningful DIFference

Does Differential Item Functioning Jeopardize the Comparability of Health-Related Quality of Life Assessment Between Patients and Proxies in Patients with Moderate-to-Severe Traumatic Brain Injury?

BACKGROUND

Health-related quality of life (HRQoL) is clearly recognized as a patient-important outcome in patients with traumatic brain injury (TBI). Patient-reported outcomes are therefore often used and supposed to be directly reported by the patients without interpretation of their responses by a physician or anyone else. However, patients with TBI are often unable to self-report because of physical and/or cognitive impairments. Thus, proxy-reported measures, e.g., family members, are often used on the patient's behalf. Yet, many studies have reported that proxy and patient ratings differ and are noncomparable. However, most studies usually do not account for other potential confounding factors that may be associated with HRQoL. In addition, patients and proxies can interpret some items of the patient-reported outcomes differently. As a result, item responses may not only reflect patients' HRQoL but also the respondent's (patient or proxy) own perception of the items. This phenomenon, called differential item functioning (DIF), can lead to substantial differences between patient-reported and proxy-reported measures and compromise their comparability, leading to highly biased HRQoL estimates. Using data from the prospective multicenter continuous hyperosmolar therapy in traumatic brain-injured patients study (240 patients with HRQoL measured with the Short Form-36 (SF-36)), we assessed the comparability of patients' and proxies' reports by evaluating the extent to which items perception differs (i.e., DIF) between patients and proxies after controlling for potential confounders.

METHODS

Items at risk of DIF adjusting for confounders were examined on the items of the role physical and role emotional domains of the SF-36.

RESULTS

Differential item functioning was evidenced in three out of the four items of the role physical domain measuring role limitations due to physical health problems and in one out of the three items of the role emotional domain measuring role limitations due to personal or emotional problems. Overall, despite an expected similar level of role limitations between patients who were able to respond and those for whom proxies responded, proxies tend to give more pessimistic responses than patients in the case of major role limitations and more optimistic responses than patients in the case of minor limitations.

CONCLUSIONS

Patients with moderate-to-severe TBI and proxies seem to have different perceptions of the items measuring role limitations due to physical or emotional problems, questioning the comparability of patient and proxy data. Therefore, aggregating proxy and patient responses may bias HRQoL estimates and alter medical decision-making based on these patient-important outcomes.

The use of PROMIS measures in clinical research in stroke patients, a systematic literature review

Objective

To systematically describe the use and outcomes of Patient-Reported Outcomes Measurement Information System (PROMIS) measures in clinical studies in populations with stroke.

Data Sources

A systematic search on the use of PROMIS measures in clinical stroke studies in 9 electronic databases.

Study Selection

Studies had to be original, reporting on outcome data using PROMIS measures in populations with stroke (ischemic and/or hemorrhagic), from January 1st, 2007. Initially, 174 unique studies met the inclusion criteria. In 2 steps, titles, abstracts and full-text articles were screened for eligibility (2 authors independently).

Data Extraction

From the selected articles, study characteristics, type of PROMIS measures, and its outcomes were extracted by 2 authors independently. The authors discussed their views to achieve consensus. A third author was consulted if necessary.

Data Synthesis

In total, 27 studies (24,366 patients) were included, predominantly from the United States (22); most study populations were hospital-based (20); the number of patients ranged from 30-3283. In general, patients had no or mild symptoms (median modified Rankin scale 1). Two different generic PROMIS measures were reported (PROMIS Global Health, PROMIS 29) and 9 PROMIS measures focusing on specific domains (sleep, pain, physical functioning, self-efficacy, satisfaction with social roles, depression, anxiety, cognition, fatigue). These match the International Classification of Functioning, Disability, and Health (ICF) domains mentioned in the Core Set for Stroke. The measures were administered 1-55 months after stroke. Outcome data are provided. Pooling of data was not achieved because of a large variety in study characteristics (inclusion criteria, follow-up moments, data processing).

Conclusions

The PROMIS measures in this review could be relevant from a patient's perspective, covering ICF core set domains for patients with stroke. The large variety in study characteristics hampers comparisons across populations. Many different outcome measures are used to report results of stroke rehabilitation studies.

Differential item functioning to validate setting of delivery compatibility in PROMIS-global health

PURPOSE

Patient-reported outcomes measures (PROMs) such as PROMIS are increasingly utilized in healthcare to assess patient perception and functional status, but the effect of delivery setting remains to be fully investigated. To our knowledge, no current study establishes the absence of differential item functioning (DIF) across delivery setting for these PROMIS- Global Health (PROMIS-GH) measures among orthopedic patients. We sought to investigate the correlation of PROMIS-GH scores across in-clinic versus remote delivery by evaluating DIF within the Global Physical Health (GPH) and Global Mental Health (GMH) items. We hypothesize that the setting of delivery of the GPH and GMH domains of PROMIS-GH will not impact the results of the measure, allowing direct comparison between the two delivery settings.

METHODS

Five thousand and seven hundred and eighty-five complete PROMIS-Global Health measures were analyzed retrospectively using the 'Lordif' package on the R platform. DIF was measured for GPH and GMH domains across setting of response (in-clinic vs remote) during the pre-operative period, immediate post-operative period, and 1-year post-operative period using Monte Carlo estimation. McFadden pseudo-R2 thresholds (> 0.02) were used to assess the magnitude of DIF for individual PROMIS items.

RESULTS

No GPH or GMH items contained in the PROMIS-GH instrument yielded DIF across in-clinic vs remote delivery setting during the pre-operative, immediate post-operative, or 1-year post-operative window.

CONCLUSION

The GPH and GMH domains within the PROMIS-GH instrument may be delivered in the clinic or remotely with comparable accuracy. This cross-delivery setting validation analysis may aid to improve the quality of patient care by allowing mixed platform PROMIS-GH data tailored to individual patient circumstance.

Optimal Methods for Reducing Proxy-Introduced Bias on Patient-Reported Outcome Measurements for Group-Level Analyses

BACKGROUND

Caregivers, or proxies, often complete patient-reported outcomes (PROs) on behalf of patients; yet, research has demonstrated proxies rate patient outcomes worse than patients rate their own outcomes. To improve interpretability of PROs in group-level analyses, our study aimed to identify optimal approaches for reducing proxy-introduced bias in the analysis of PROs.

METHODS

Data were simulated based on 200 patients with stroke and their proxies who both completed 9 PROMIS domains as part of a cross-sectional study. The sample size was varied as 50, 100, 200, and 500, and the proportion of patients with proxy-respondents was varied as 10%, 20%, and 50%. Six methods for handling proxy-completions were investigated: (1) complete case analysis; (2) proxy substitution; (3) Method 2 plus proxy adjustment; (4) Method 3 including inverse-probability of treatment weighting; (5) multiple imputation; (6) linear equating. These methods were evaluated by comparing average bias in PROMIS T-scores (estimated versus observed patient-only responses), as well as by comparing estimated regression coefficients to models using patient-only responses.

RESULTS

Overall mean T-score differences ranged from 0 to 1.75. The range of mean differences varied by the 6 methods with methods 1 and 5 providing estimates closest to the observed mean. In regression models, all but inverse-probability of treatment weighting resulted in low bias when proxy-completions were 10% to 20%. With 50% proxy-completions, method 5 resulted in less accurate estimations while methods 1 to 3 provided less proxy-introduced bias. Bias remained low across domain and varying sample sizes but increased with larger percentages of proxy-respondents.

CONCLUSIONS

Our study found modest proxy-introduced bias when estimating PRO scores or regression estimates across multiple domains of health. This bias remained low, even when sample size was 50 and there were large proportions of proxy-completions. While many of these methods can be chosen for including proxies in stroke PRO research with <20% proxy-respondents, proxy substitution with adjustment resulted in low bias with 50% proxy-respondents.

Patient-proxy agreement on change in acute stroke patient-reported outcome measures: a prospective study

Objectives

Research has indicated proxies overestimate symptoms on patients’ behalves, however it is unclear whether patients and proxies agree on meaningful change across domains over time. The objective of this study is to assess patient-proxy agreement over time, as well as agreement on identification of meaningful change, across 10 health domains in patients who underwent acute rehabilitation following stroke.

Methods

Stroke patients were recruited from an ambulatory clinic or inpatient rehabilitation unit, and were included in the study if they were undergoing rehabilitation. At baseline and again after 30 days, patients and their proxies completed PROMIS Global Health and eight domain-specific PROMIS short forms. Reliability of patient-proxy assessments at baseline, follow-up, and the change in T-score was evaluated for each domain using intra-class correlation coefficients (ICC(2,1)). Agreement on meaningful improvement or worsening, defined as 5+ T-score points, was compared using percent exact agreement.

Results

Forty-one patient-proxy dyads were included in the study. Proxies generally reported worse symptoms and functioning compared to patients at both baseline and follow-up, and reported less change than patients. ICCs for baseline and change were primarily poor to moderate (range: 0.06 (for depression change) to 0.67 (for physical function baseline)), and were better at follow-up (range: 0.42 (for anxiety) to 0.84 (for physical function)). Percent exact agreement between indicating meaningful improvement versus no improvement ranged from 58.5–75.6%. Only a small proportion indicated meaningful worsening.

Conclusions

Patient-proxy agreement across 10 domains of health was better following completion of rehabilitation compared to baseline or change. Overall change was minimal but the majority of patient-proxy dyads agreed on meaningful change. Our study provides important insight for clinicians and researchers when interpreting change scores over time for questionnaires completed by both patients and proxies.

Mental Health of Parents and Primary Caregivers by Sex and Associated Child Health Indicators

Poor mental health among parents or primary caregivers is associated with poor mental and physical health in children; however, research often excludes the mental health of male caregivers including fathers. This analysis examines associations between caregiver mental health by caregiver sex and child health indicators (i.e., child's general health; child's history of diagnosed mental, behavioral, or developmental disorders (MBDDs)). Using parent-reported data on 97,728 US children aged 0-17 years from the National Survey of Children's Health (2016-2018), we estimated nationally representative, weighted proportions of children with parents or primary caregivers with poor mental health by caregiver sex, prevalence ratios (PR), and 95% confidence intervals (CI) for child health indicators by caregiver mental health and sex. Nationally, 7.2% of children had at least one caregiver with poor mental health; 2.8% had any male caregiver; and 5.1% had any female caregiver with poor mental health. Compared to children with all male caregivers with good mental health, children with any male caregiver with poor mental health were more likely to have poor general health (PR: 4.9, CI: 3.0-8.0) and have ≥1 diagnosed MBDDs (PR: 1.9, CI: 1.7-2.1); this remained significant when controlling for caregiver and household characteristics. Findings were similar when comparing children with any female caregiver with poor mental health to children with all female caregivers with good mental health. Our findings support previously published recommendations that promoting mental health among all types of caregivers by addressing gaps in research on fathers and male caregivers may further promote child health and wellness.

Psychometric Properties of the Proxy-Reported Life-Space Assessment in Institutionalized Settings (LSA-IS-Proxy) for Older Persons with and without Cognitive Impairment

(1) Background: Life-space mobility assessments for institutionalized settings are scarce and there is a lack of comprehensive validation and focus on persons with cognitive impairment (CI). This study aims to evaluate the psychometric properties of the Life-Space Assessment for Institutionalized Settings by proxy informants (LSA-IS-proxy) for institutionalized, older persons, with and without CI. (2) Methods: Concurrent validity against the self-reported version of the LSA-IS, construct validity with established construct variables, test-retest reliability, sensitivity to change during early multidisciplinary geriatric rehabilitation treatment, and feasibility (completion rate, floor/ceiling effects) of the LSA-IS-proxy, were assessed in 94 hospitalized geriatric patients (83.3 ± 6.1 years), with and without CI. (3) Results: The LSA-IS-proxy total score showed good-to-excellent agreement with the self-reported LSA-IS (Intraclass Correlations Coefficient, ICC_3,1 = 0.77), predominantly expected small-to-high correlations with construct variables (r = 0.21–0.59), good test–retest reliability (ICC_3,1 = 0.74), significant sensitivity to change over the treatment period (18.5 ± 7.9 days; p < 0.001, standardized response mean = 0.44), and excellent completion rates (100%) with no floor/ceiling effects. These results were predominantly confirmed for the sub-scores of the LSA-IS-proxy and were comparable between the sub-groups with different cognitive status. (4) Conclusions: The LSA-IS-proxy has proven to be feasible, valid, reliable, and sensitive to change in hospitalized, geriatric patients with and without CI.

Cross-specialty PROMIS-global health differential item functioning

PURPOSE

To investigate the functioning of the PROMIS-Global Health (PROMIS-GH) across clinical setting, patient age, and medical complexity by evaluating differential item functioning (DIF) within the Global Physical Health (GPH) and Global Mental Health (GMH) domains. To our knowledge, no study demonstrates lack of differential item functioning (DIF) for PROMIS-GH across these populations. We hypothesize that the PROMIS-GH domains of GMH and GPH will perform similarly when compared across these populations.

METHODS

Seven thousand nine hundred and seventy four complete PROMIS Global Health measures were retrospectively analyzed using the 'Lordif' package on the R platform. DIF was investigated for both GMH and GPH across clinical environment (Orthopedic Surgery, Family Medicine, & Internal Medicine), age group (≤ 53, > 53-66, > 66), and Charlson Comorbidity Index (CCI:0, CCI:1, CCI:2 +) using quasi Monte Carlo estimation. To assess the significance of DIF, Wald tests were used with the Benjamini & Hochberg procedure.

RESULTS

No items contained in the GMH or GPH demonstrated DIF across age groups, medical complexity, or clinical environment.

CONCLUSION

Items assessing the domains of GMH and GPH within the PROMIS-GH function comparably across treatment setting, age category, and medical comorbidities. The PROMIS-Global Health holds potential to facilitate interdisciplinary patient care and patient optimization prior to surgical intervention.

Magnitude and Variability of Stroke Patient-Proxy Disagreement Across Multiple Health Domains

OBJECTIVES

To quantify the extent and variability of bias introduced when caregivers, or proxies, complete patient-reported outcome measures (PROM) on behalf of stroke patients.

DESIGN

Cross-sectional survey study conducted between July 2018 and November 2019.

SETTING

Ambulatory clinic of a cerebrovascular center or rehabilitation unit.

PARTICIPANTS

A consecutive sample of stroke patients (N=200) and their proxies who were able and willing to complete PROMs. Proxies completed PROMs as they believed the patient would answer.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

PROMs included Neuro-QoL cognitive function, PROMIS physical function, social role satisfaction, anxiety, fatigue, pain interference, sleep disturbance, Patient Health Questionnaire-9 translated to PROMIS Depression, and PROMIS Global Health.

RESULTS

The study included 200 stroke patients (age, 62.2±13.3; 41.5% women) and their proxies (age 56.5±13.9; 70% women, 72% spouses). Proxies reported worse functioning and more symptoms across all PROM domains compared with patients (average difference, 0.3-3.0 T score points). Reliability between dyad responses was moderate across all domains (intraclass correlation coefficients (2,1), 0.49-0.76) and effect sizes were small (d=0.04-0.35). Cognitive function, anxiety, and depression had the lowest agreement, whereas physical function, pain, and sleep had the highest agreement based on the Bland-Altman method. At the individual level, a large proportion of dyads had meaningfully different scores across domains (range, 40%-57%; dyads differed >5 T score points). Few predictors of disagreement were identified through multinomial regression models.

CONCLUSIONS

At the aggregate level, small differences were detected between stroke patient-proxy pairs, with lower agreement on more subjective domains. At the individual level, a large proportion of dyads reported meaningfully different scores on all domains, affecting the interpretability of proxy responses on PROMs in a clinical setting.

Health-related quality of life 90 days after stroke assessed by the International Consortium for Health Outcome Measurement standard set

BACKGROUND AND PURPOSE

Stroke has detrimental effects in multiple health domains not captured by routine scales. The International Consortium for Health Outcome Measurement has developed a standardized set for self-reported assessment to overcome this limitation. The aim was to assess this set in acute stroke care.

METHODS

Consecutive patients with acute ischaemic stroke, transient ischaemic attack or intracerebral hemorrhage were enrolled. Demographics, living situation and cardiovascular risk factors were collected from medical records and interviews. The Patient-reported Outcomes Measurement Information System 10-Question Short Form (PROMIS-10) and the Patient Health Questionnaire-4 (PHQ-4) were conducted 90 days after admission. Linear and logistic regression analyses were used to identify predictors of outcome. The study is registered at ClinicalTrials.gov, NCT03795948.

RESULTS

In all, 1064 patients were enrolled; mean age was 71.6 years, 51% were female, and median National Institutes of Health Stroke Scale (NIHSS) on admission was 3. Diagnosis was acute ischaemic stroke in 74%, transient ischaemic attack in 20% and intracerebral hemorrhage in 6%. 673 patients were available for outcome evaluation at 90 days; of these 90 (13%) had died. In survivors, t scores of PROMIS-10 physical and mental health were 40.3 ± 6.17 and 44.3 ± 8.63, compared to 50 ± 10 in healthy populations. 16% reported symptoms indicating depression or anxiety on the PHQ-4. Higher NIHSS, prior stroke and requiring help pre-stroke predicted lower values in physical and mental health scores. Higher NIHSS and diabetes were associated with anxiety or depression.

CONCLUSIONS

Integrated in the routine of acute stroke care, systematic assessment of patient-reported outcomes reveals impairments in physical and mental health. Main predictors are severity of stroke symptoms and comorbidities such as hypertension and diabetes.

Proxy reliability of the 12-item world health organization disability assessment schedule II among adult patients with mental disorders

PURPOSE

Despite the wide usage of World Health Organization Disability Assessment Schedule II (WHODAS 2.0) in psychiatry research and clinical practice, there was limited knowledge on its proxy reliability among people with mental disorders. This paper aimed to compare the 12-item WHODAS 2.0 responses of adult patients with mental disorders to their family caregivers.

METHODS

In this study, 205 pairs of patients with mental disorders and primary family caregivers were consecutively recruited from one inpatient mental health department in a large hospital in China. All participants completed the 12-item version WHODAS 2.0 to assess patients' functioning in the 30 days prior to the hospitalization. Measurement invariance, including configural, metric and scalar invariance, was tested across patient and proxy groups, using multi-group confirmatory factor analysis. Agreement between patients and proxies was examined by paired Wilcoxon tests and intraclass correlation coefficients (ICC). Subgroup analyses for proxy reliability were conducted within strata of proxy kinship and patient psychiatric diagnosis.

RESULTS

The 12-item WHODAS 2.0 achieved configural, metric and partial scalar invariance across patient and proxy groups. Unsatisfactory consistency was found for most items (ICC < 0.75, P < 0.05), especially for items on Cognition, Getting along, Life activities, and Participation in society (ICC < 0.4, P < 0.05). Spouses agreed with patients more often than parents (ICC ≥ 0.4, P < 0.05). The paired Wilcoxon tests found that impairment of patients with psychotic disorders tended to be overestimated by proxies while proxies tended to underestimate impairment of patients with mood disorders.

CONCLUSION

Our study reveals inconsistency between self and proxy reports in the 12-item WHODAS 2.0 among adult patients with mental disorders. When proxy reports is needed, spouses are preferred than parents. We should be aware of proxies' impairment overestimation among patients with psychotic disorders and underestimation among patients with mood disorders.

Associations between self-rated health and the assessments of anchoring vignettes in cardiovascular patients

Background/Objective

Assessments of health can be biased by response shift effects. One method for detecting such effects is the use of anchoring vignettes. The aim of this study was to analyze the relationship between participants' self-assessed health state and their assessments of these vignettes.

Method

A total of 342 cardiovascular patients assessed their own state of health on a 0-100 visual analogue scale. The patients additionally assessed two vignettes featuring fictional persons suffering from specific complaints. A sample of the general population (N = 1,236) served as controls.

Results

The participants rated the health state of the vignette character featuring physical problems as being significantly better than the general population did (effect size: d = 0.53). The group difference in the assessment of the vignette featuring primarily mental health problems was lower (d = -0.17). Participants' assessments of the vignettes were positively correlated with their assessments of their own health state (r = .26 and r = .10) and with several quality of life variables.

Conclusions

Anchoring vignettes are a useful tool for detecting response shift effects.