Using Patient Health Questionnaire-9 item parameters of a common metric resulted in similar depression scores compared to independent item response theory model reestimation

Converting PROMIS®-29 v2.0 profile data to SF-36 physical and mental component summary scores in patients with cardiovascular disorders

BACKGROUND

Health-related quality of life (HRQL) has become an important outcome parameter in cardiology. The MOS 36-ltem Short-Form Health Survey (SF-36) and the PROMIS-29 are two widely used generic measures providing composite HRQL scores. The domains of the SF-36, a well-established instrument utilized for several decades, can be aggregated to physical (PCS) and mental (MCS) component summary scores. Alternative scoring algorithms for correlated component scores (PCSc and MCSc) have also been suggested. The PROMIS-29 is a newer but increasingly used HRQL measure. Analogous to the SF-36, physical and mental health summary scores can be derived from PROMIS-29 domain scores, based on a correlated factor solution. So far, scores from the PROMIS-29 are not directly comparable to SF-36 results, complicating the aggregation of research findings. Thus, our aim was to provide algorithms to convert PROMIS-29 data to well-established SF-36 component summary scores.

METHODS

Data from n = 662 participants of the Berlin Long-term Observation of Vascular Events (BeLOVE) study were used to estimate linear regression models with either PROMIS-29 domain scores or aggregated PROMIS-29 physical/mental health summary scores as predictors and SF-36 physical/mental component summary scores as outcomes. Data from a subsequent assessment point (n = 259) were used to evaluate the agreement between empirical and predicted SF-36 scores.

RESULTS

PROMIS-29 domain scores as well as PROMIS-29 health summary scores showed high predictive value for PCS, PCSc, and MCSc (R2 ≥ 70%), and moderate predictive value for MCS (R2 = 57% and R2 = 40%, respectively). After applying the regression coefficients to new data, empirical and predicted SF-36 component summary scores were highly correlated (r > 0.8) for most models. Mean differences between empirical and predicted scores were negligible (|SMD|<0.1).

CONCLUSIONS

This study provides easy-to-apply algorithms to convert PROMIS-29 data to well-established SF-36 physical and mental component summary scores in a cardiovascular population. Applied to new data, the agreement between empirical and predicted SF-36 scores was high. However, for SF-36 mental component summary scores, considerably better predictions were found under the correlated (MCSc) than under the original factor model (MCS). Additionally, as a pertinent byproduct, our study confirmed construct validity of the relatively new PROMIS-29 health summary scores in cardiology patients.

Establishing the measurement precision of the patient health questionnaire in an adolescent sample

BACKGROUND

Technically sound measures are necessary for accurately identifying youth at risk for depression, but many studies rely on classical test theory metrics or adult samples to evaluate measures. This study examined the use of the PHQ-8, a common and freely available pediatric depression screener, in an adolescent sample using item response theory (IRT).

METHODS

Secondary analyses were conducted on data from a study conducted in Midwestern middle schools in which 1224 youth completed the PHQ-8 as part of a battery of surveys. Polytomous IRT analyses (a Graded Response Model) were used to evaluate the PHQ-8. Items were examined for their ability to distinguish between respondents of different latent depression severity and for differential item functioning (DIF) across demographic categories.

RESULTS

All PHQ-8 items had adequate discriminative abilities. Items measuring anhedonia and psychomotor disturbances performed relatively poorly, and items measuring somatic symptoms (appetite and sleep) were most informative when respondents endorsed extreme response options ("not at all" or "nearly every day"). No DIF was found across grade level or race, but several items were flagged for DIF by gender and student income level.

LIMITATIONS

These results might not be generalizable to a broader youth population due to administration setting and the unique demographic characteristics of this sample (76.0 % African American).

CONCLUSIONS

Tools such as the PHQ-8 are appropriate to quickly screen for depression in adolescents, but further scrutiny of adolescent response patterns is warranted. Future research should examine items measuring anhedonia and psychomotor and somatic disturbances in adolescents.

Evaluations of the sum-score-based and item response theory-based tests of group mean differences under various simulation conditions

The use of patient-reported outcomes measures is gaining popularity in clinical trials for comparing patient groups. Such comparisons typically focus on the differences in group means and are carried out using either a traditional sum-score-based approach or item response theory (IRT)-based approaches. Several simulation studies have evaluated different group mean comparison approaches in the past, but the performance of these approaches remained unknown under certain uninvestigated conditions (e.g. under the impact of differential item functioning (DIF)). By incorporating some of the uninvestigated simulation features, the current study examines Type I error, statistical power, and effect size estimation accuracy associated with group mean comparisons using simple sum scores, IRT model likelihood ratio tests, and IRT expected-a-posteriori scores. Manipulated features include sample size per group, number of items, number of response categories, strength of discrimination parameters, location of thresholds, impact of DIF, and presence of missing data. Results are summarized and visualized using decision trees.

Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS

The psychometric process used to establish a relationship between the scores of two (or more) instruments is generically referred to as linking. When two instruments with the same content and statistical test specifications are linked, these instruments are said to be equated. Linking and equating procedures have long been used for practical benefit in educational testing. In recent years, health outcome researchers have increasingly applied linking techniques to patient-reported outcome (PRO) data. However, these applications have some noteworthy purposes and associated methodological questions. Purposes for linking health outcomes include the harmonization of data across studies or settings (enabling increased power in hypothesis testing), the aggregation of summed score data by means of score crosswalk tables, and score conversion in clinical settings where new instruments are introduced, but an interpretable connection to historical data is needed. When two PRO instruments are linked, assumptions for equating are typically not met and the extent to which those assumptions are violated becomes a decision point around how (and whether) to proceed with linking. We demonstrate multiple linking procedures-equipercentile, unidimensional IRT calibration, and calibrated projection-with the Patient-Reported Outcomes Measurement Information System Depression bank and the Patient Health Questionnaire-9. We validate this link across two samples and simulate different instrument correlation levels to provide guidance around which linking method is preferred. Finally, we discuss some remaining issues and directions for psychometric research in linking PRO instruments.

Psychometric Data Linking Across HIV and Substance Use Cohorts

Psychometric data linking of psychological and behavioral questionnaires can facilitate the harmonization of data across HIV and substance use cohorts. Using data from the Collaborating Consortium of Cohorts Producing NIDA Opportunities (C3PNO), we demonstrate how to capitalize on previous linking work with a common linked depression metric across multiple questionnaires. Cohorts were young men who have sex with men (MSM), substance-using MSM, HIV/HCV cocaine users, and HIV-positive patients. We tested for differential item functioning (DIF) by comparing C3PNO cohort data with general population data. We also fit a mixed-effects model for depression, entering HIV-status and recent opioid/heroin use as fixed effects and cohort as a random intercept. Our results suggest a minimal level of DIF between the C3PNO cohorts and general population samples. After linking, descriptive statistics show a wide range of depression score means across cohorts. Our model confirmed an expected positive relationship between substance use and depression, though contrary to expectations, no significant association with HIV status. The study reveals the likely role of cohort differences, associated patient characteristics, study designs, and administration settings.

Using subdomain-specific item sets affected PROMIS physical function scores differently in cardiology and rheumatology patients

OBJECTIVES

The Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) item bank has been developed to standardize patient-reported PF across medical fields. However, evidence of scoring equivalence across cardiology and rheumatology patients is still missing. Therefore, this study aims to investigate both (1) the extent of disease-related differential item functioning (DIF) and (2) the impact of the disease group on using subdomain-specific item sets for generating PROMIS PF scores in cardiology and rheumatology patients.

STUDY DESIGN AND SETTING

Ordinal regression was used to evaluate DIF between cardiology (n = 201) and rheumatology (n = 200) inpatients. To explore the disease-specific impact of PF subdomains on scoring, we compared scores derived from the full item bank with scores derived from subdomain-specific item sets for each disease group.

RESULTS

DIF was detected in 18 items, predominately from the upper extremity subdomain. When upper extremity items were used, cardiology patients reached systematically higher scores than using the full item bank. Rheumatology patients scored substantially higher when mobility items were used.

CONCLUSION

Applying the PROMIS PF metric to disease-specific item sets including items from differing subdomains may lead to biased comparisons of PF levels across disease groups. Disease-specific item parameters should be provided for items showing DIF, and subdomain-related content balancing is recommended for scoring the generic PROMIS PF construct.

Estimating person parameters via item response model and simple sum score in small samples with few polytomous items: A simulation study

BACKGROUND

The Item Response Theory (IRT) is becoming increasingly popular for item analysis. Theoretical considerations and simulation studies suggest that parameter estimates will become precise only by utilizing many items in large samples.

METHOD

A simulation study focusing on a single scale was performed on data with (a) n = 40, 60, 80, 120, 200, 300, 500, and 900 cases utilizing (b) 4, 8, 16, or 32 items. The items were (c) symmetrically distributed vs. skew (skewness 0, 1, and 2). Item loadings were (d) homogeneous vs. heterogeneous. Item loadings were (e) low vs. high. Half of the items had (f) a correlated error or not. The number of answering categories (g) was four vs. five. A total of 10% of each item had missing values. The ability-estimates from the IRT model and the simple sum score served as criteria for evaluating the results.

RESULTS

The ability-estimate from the IRT model outperformed the sum score when there were many items, skewed distributed items, and the item loadings were heterogeneous and high. The sum score outperformed the ability-estimate when there were few items, nonskewed items, and homogeneous and low item loadings. However, convergence rates were partly low in small samples. Correlated errors affected, both negatively, the ability-estimate and the sum score.

CONCLUSION

With skew item distributions and heterogeneous item loadings, utilizing an IRT model is recommended. However, with few items, many cases are required, conversely, with few cases many items. With few items and few cases, the sum score performs better.

A 67-item stress resilience item bank showing high content validity was developed in a psychosomatic sample

OBJECTIVES

To develop the first item bank to measure stress resilience (SR) in clinical populations.

STUDY DESIGN AND SETTING

Qualitative item development resulted in an initial pool of 131 items covering a broad theoretical SR concept. These items were tested in n = 521 patients at a psychosomatic outpatient clinic. Exploratory and confirmatory factor analysis, as well as other state-of-the-art item analyses and item response theory were used for item evaluation and calibration of the final item bank.

RESULTS

Of the initial item pool of 131 items, we excluded 64 items (54 factor loading <0.5, four residual correlations >0.3, two nondiscriminative item response curves, and four differential item functioning). The final set of 67 items indicated sufficient model fit in confirmatory factor analysis and item response theory analyses. In addition, a 10-item short form with high measurement precision (SE ≤ 0.32 in a theta range between -1.8 and +1.5) was derived. Both the SR item bank and the SR short form were highly correlated with an existing static legacy tool (Connor-Davidson Resilience Scale).

CONCLUSION

The final SR item bank and 10-item short form showed good psychometric properties. When further validated, they will be ready to be used within a framework of computer-adaptive tests for a comprehensive assessment of the stress construct.

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term 'metric'

OBJECTIVE

To examine the use of the term 'metric' in health and social sciences' literature, focusing on the interval scale implication of the term in Modern Test Theory (MTT).

MATERIALS AND METHODS

A systematic search and review on MTT studies including 'metric' or 'interval scale' was performed in the health and social sciences literature. The search was restricted to 2001-2005 and 2011-2015. A Text Mining algorithm was employed to operationalize the eligibility criteria and to explore the uses of 'metric'. The paradigm of each included article (Rasch Measurement Theory (RMT), Item Response Theory (IRT) or both), as well as its type (Theoretical, Methodological, Teaching, Application, Miscellaneous) were determined. An inductive thematic analysis on the first three types was performed.

RESULTS

70.6% of the 1337 included articles were allocated to RMT, and 68.4% were application papers. Among the number of uses of 'metric', it was predominantly a synonym of 'scale'; as adjective, it referred to measurement or quantification. Three incompatible themes 'only RMT/all MTT/no MTT models can provide interval measures' were identified, but 'interval scale' was considerably more mentioned in RMT than in IRT.

CONCLUSION

'Metric' is used in many different ways, and there is no consensus on which MTT metric has interval scale properties. Nevertheless, when using the term 'metric', the authors should specify the level of the metric being used (ordinal, ordered, interval, ratio), and justify why according to them the metric is at that level.

Screening for Behavioral Health Conditions in Primary Care Settings: A Systematic Review of the Literature

BACKGROUND

Mounting evidence indicates that early recognition and treatment of behavioral health disorders can prevent complications, improve quality of life, and help reduce health care costs. The aim of this systematic literature review was to identify and evaluate publicly available, psychometrically tested tools that primary care physicians (PCPs) can use to screen adult patients for common mental and substance use disorders such as depression, anxiety, and alcohol use disorders.

METHODS

We followed the Institute of Medicine (IOM) systematic review guidelines and searched PubMed, PsycINFO, Applied Social Sciences Index and Abstracts, Cumulative Index to Nursing and Allied Health Literature, and Health and Psychosocial Instruments databases to identify literature addressing tools for screening of behavioral health conditions. We gathered information on each tool's psychometrics, applicability in primary care, and characteristics such as number of items and mode of administration. We included tools focused on adults and the most common behavioral health conditions; we excluded tools designed for children, youth, or older adults; holistic health scales; and tools screening for serious but less frequently encountered disorders, such as bipolar disorder.

RESULTS

We identified 24 screening tools that met the inclusion criteria. Fifteen tools were subscales stemming from multiple-disorder assessments or tools that assessed more than one mental disorder or more than one substance use disorder in a single instrument. Nine were ultra-short, single-disorder tools. The tools varied in psychometrics and the extent to which they had been administered and studied in primary care settings.

DISCUSSION

Tools stemming from the Patient Health Questionnaire had the most testing and application in primary care settings. However, numerous other tools could meet the needs of primary care practices. This review provides information that PCPs can use to select appropriate tools to incorporate into a screening protocol.

Measurement invariance and general population reference values of the PROMIS Profile 29 in the UK, France, and Germany

PURPOSE

Comparability of patient-reported outcome measures over different languages is essential to allow cross-national research. We investigate the comparability of the PROMIS Profile 29, a generic health-related quality of life measure, in general population samples in the UK, France, and Germany and present general population reference values.

METHODS

A web-based survey was simultaneously conducted in the UK (n = 1509), France (1501), and Germany (1502). Along with the PROMIS Profile 29, we collected sociodemographic information as well as the EQ-5D. We tested measurement invariance by means of multigroup confirmatory factor analysis (CFA). Differences in the health-related quality of life between countries were modeled by linear regression analysis. We present general population reference data for the included PROMIS domains utilizing plausible value imputation and quantile regression.

RESULTS

Multigroup CFA of the PROMIS Profile 29 showed that factor means are insensitive to potential measurement bias except in one item. We observed significant differences in patient-reported health between countries, which could be partially explained by the differences in overall ratings of health. The physical function and pain interference scales showed considerable floor effects in the normal population in all countries.

CONCLUSIONS

Scores derived from the PROMIS Profile 29 are largely comparable across the UK, France, and Germany. Due to the use of plausible value imputation, the presented general population reference values can be compared to data collected with other PROMIS short forms or computer-adaptive tests.

An initial psychometric evaluation of the German PROMIS v1.2 Physical Function item bank in patients with a wide range of health conditions

OBJECTIVES

To translate the PROMIS Physical Function (PF) item bank version 1.2 into German and to investigate psychometric properties of resulting full bank and seven derived short forms.

DESIGN

Cross-sectional psychometric study.

SETTING

Inpatient and outpatient clinics of the Department of Psychosomatic Medicine at Charité-Universitätsmedizin Berlin, Germany.

SUBJECTS

A total of 10 adult patients with various chronic diseases participated in cognitive debriefing interviews. The final item bank was administered to n = 266 adult patients with a broad range of medical conditions.

INTERVENTIONS

Patient-reported outcome assessment as part of routine care.

MAIN MEASURES

PROMIS v1.2 PF bank; MOS SF-36 PF scale (PF-10).

RESULTS

Cross-cultural adaptation of the item bank followed established guidelines. For the final German translation, the corrected item-total correlations ranged from 0.44 to 0.84. Cronbach's alpha was high for each PROMIS PF short form ( α = 0.88-0.96). The full PROMIS PF bank and most short forms correlated highly with the SF-36 PF-10 ( r = 0.85-0.90), with the exception of PROMIS Upper Extremity ( r = 0.64). PROMIS Upper Extremity showed ceiling effects and lower agreement with the full bank than other short forms. Unidimensionality was supported for all PROMIS PF measures using traditional factor analysis and nonparametric item response theory.

CONCLUSION

The German PROMIS PF bank was found to be conceptually equivalent to the English version and fulfilled the psychometric requirements for use of short forms in clinical practice. Future studies should pay particular attention to samples with upper extremity functional limitations to further investigate the dimensional structure of PF as conceptualized according to PROMIS.

Varying the item format improved the range of measurement in patient-reported outcome measures assessing physical function

Background

Physical function (PF) is a core patient-reported outcome domain in clinical trials in rheumatic diseases. Frequently used PF measures have ceiling effects, leading to large sample size requirements and low sensitivity to change. In most of these instruments, the response category that indicates the highest PF level is the statement that one is able to perform a given physical activity without any limitations or difficulty. This study investigates whether using an item format with an extended response scale, allowing respondents to state that the performance of an activity is easy or very easy, increases the range of precise measurement of self-reported PF.

Methods

Three five-item PF short forms were constructed from the Patient-Reported Outcomes Measurement Information System (PROMIS®) wave 1 data. All forms included the same physical activities but varied in item stem and response scale: format A (“Are you able to …”; “without any difficulty”/“unable to do”); format B (“Does your health now limit you …”; “not at all”/“cannot do”); format C (“How difficult is it for you to …”; “very easy”/“impossible”). Each short-form item was answered by 2217–2835 subjects. We evaluated unidimensionality and estimated a graded response model for the 15 short-form items and remaining 119 items of the PROMIS PF bank to compare item and test information for the short forms along the PF continuum. We then used simulated data for five groups with different PF levels to illustrate differences in scoring precision between the short forms using different item formats.

Results

Sufficient unidimensionality of all short-form items and the original PF item bank was supported. Compared to formats A and B, format C increased the range of reliable measurement by about 0.5 standard deviations on the positive side of the PF continuum of the sample, provided more item information, and was more useful in distinguishing known groups with above-average functioning.

Conclusions

Using an item format with an extended response scale is an efficient option to increase the measurement range of self-reported physical function without changing the content of the measure or affecting the latent construct of the instrument.

Electronic supplementary material

The online version of this article (doi:10.1186/s13075-017-1273-5) contains supplementary material, which is available to authorized users.