Psychometric properties of the patient-reported outcomes measurement information system scale v1.2: global health (PROMIS-GH) in a Dutch general population

Purpose

To assess the psychometric properties of the Dutch-Flemish Patient-Reported Outcome Measurement Information System Scale v1.2 – Global Health (PROMIS-GH).

Methods

The PROMIS-GH (also referred to as PROMIS-10) was administered to 4370 persons from the Dutch general population. Unidimensionality (CFI ≥ 0.95; TLI ≥ 0.95; RMSEA ≤ 0.06; SRMR ≤ 0.08), local independence (residual correlations < 0.20), monotonicity (H > 0.30), model fit with the Graded Response Model (GRM, p < 0.001), internal consistency (alpha > 0.75), precision (total score information across the latent trait), measurement invariance (no Differential Item Functioning [DIF]), and cross-cultural validity (no DIF for language, Dutch vs. United States English) of its subscales, composed of four items each, Global Mental Health (GMH) and Global Physical Health (GPH), were assessed.

Results

Confirmatory factor analyses, on both subscales, revealed slight departures from unidimensionality for GMH (CFI = 0.98; TLI = 0.95, RMSEA = 0.22; SRMR = 0.04) and GPH (CFI = 0.99; TLI = 0.97; RMSEA = 0.12; SRMR = 0.03). Local independence, monotonicity, GRM model fit, internal consistency, precision and cross-cultural validity were supported. However, Global10 (emotional problems) showed misfit on the GMH subscale, while Global08 (fatigue) presented DIF for age.

Conclusion

The psychometric properties of the PROMIS-GH in the Dutch population were considered acceptable. Sufficient local independence, monotonicity, GRM fit, internal consistency, measurement invariance and cross-cultural validity were found. If future studies find similar results, structural validity of the GMH could be enhanced by improving or replacing Global10 (emotional problems).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12955-021-01855-0.

Common patient-reported outcomes across ICHOM Standard Sets: the potential contribution of PROMIS®

BACKGROUND

The International Consortium for Health Outcomes Measurement (ICHOM) develops condition-specific Standard Sets of outcomes to be measured in clinical practice for value-based healthcare evaluation. Standard Sets are developed by different working groups, which is inefficient and may lead to inconsistencies in selected PROs and PROMs. We aimed to identify common PROs across ICHOM Standard Sets and examined to what extend these PROs can be measured with a generic set of PROMs: the Patient-Reported Outcomes Measurement Information System (PROMIS®).

METHODS

We extracted all PROs and recommended PROMs from 39 ICHOM Standard Sets. Similar PROs were categorized into unique PRO concepts. We examined which of these PRO concepts can be measured with PROMIS.

RESULTS

A total of 307 PROs were identified in 39 ICHOM Standard Sets and 114 unique PROMs are recommended for measuring these PROs. The 307 PROs could be categorized into 22 unique PRO concepts. More than half (17/22) of these PRO concepts (covering about 75% of the PROs and 75% of the PROMs) can be measured with a PROMIS measure.

CONCLUSION

Considerable overlap was found in PROs across ICHOM Standard Sets, and large differences in terminology used and PROMs recommended, even for the same PROs. We recommend a more universal and standardized approach to the selection of PROs and PROMs. Such an approach, focusing on a set of core PROs for all patients, measured with a system like PROMIS, may provide more opportunities for patient-centered care and facilitate the uptake of Standard Sets in clinical practice.

Construct validity, responsiveness, and utility of change indicators of the Dutch-Flemish PROMIS item banks for depression and anxiety administered as computerized adaptive test (CAT): A comparison with the Brief Symptom Inventory (BSI)

We evaluated construct validity, responsiveness, and utility of change indicators of the Dutch-Flemish PROMIS adult v1.0 item banks for Depression and Anxiety administered as computerized adaptive test (CAT). Specifically, the CATs were compared to the Brief Symptom Inventory (BSI) using pre- and re-test data of adult patients treated for common mental disorders (N = 400; median pre-to-re-test interval = 215 days). Construct validity was evaluated with Pearson's correlations and Cohen's ds; responsiveness with Pearson's correlations and pre-post effect sizes (ES); utility of change indicators with kappa coefficients and percentages of (dis)agreement. The results showed that the PROMIS CATs measure similar constructs as matching BSI scales. Under the assumption of measuring similar constructs, the CAT and BSI Depression scales were similarly responsive. For the Anxiety scales, we found a higher responsiveness for CAT (ES = 0.64) compared to the BSI (ES = 0.50). Finally, both CATs categorized the change scores of more patients as changed compared to matching BSI scales, indicating that the PROMIS CATs may be more able to detect actual change than the BSI. Based on these findings, the PROMIS CATs may be considered a modest improvement over matching BSI scales as tools for reviewing treatment progress with patients. We discuss several additional differences between the PROMIS CATs and the BSI to help test users choose instruments. These differences include the adopted measurement theory (Item Response Theory vs. Classical Test Theory), the mode of administration (CAT vs. fixed items), and the area of application (universal vs. predominantly clinical). (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Operationalization of the Brief ICF Core Set for Hearing Loss: An ICF-Based e-Intake Tool in Clinical Otology and Audiology Practice

OBJECTIVES

According to the International Classification of Functioning, Disability and Health (ICF), functioning reflects the interplay between an individual's body structures and functions, activities, participation, environmental, and personal factors. To be useful in clinical practice, these concepts need to be operationalized into a practical and integral instrument. The Brief ICF Core Set for Hearing Loss (CSHL) provides a minimum standard for the assessment of functioning in adults with hearing loss. The objective of the present study was to operationalize the Brief CSHL into a digital intake tool that could be used in the otology-audiology practice for adults with ear and hearing problems as part of their intake assessment.

DESIGN

A three-step approach was followed: (1) Selecting and formulating questionnaire items and response formats, using the 27 categories of the Brief CSHL as a basis. Additional categories were selected based on relevant literature and clinical expertise. Items were selected from existing, commonly used disease-specific questionnaires, generic questionnaires, or the WHO's official descriptions of ICF categories. The response format was based on the existing item's response categories or on the ICF qualifiers. (2) Carrying out an expert survey and a pilot study (using the three-step test interview. Relevant stakeholders and patients were asked to comment on the relevance, comprehensiveness, and comprehensibility of the items. Results were discussed in the project group, and items were modified based on consensus. (3) Integration of the intake tool into a computer-based system for use in clinical routine.

RESULTS

The Brief CSHL was operationalized into 62 items, clustered into six domains: (1) general information, including reason for visit, sociodemographic, and medical background; (2) general body functions; (3) ear and hearing structures and functions; (4) activities and participation (A&P); (5) environmental factors (EF); and (6) personal factors (mastery and coping). Based on stakeholders' responses, the instructions of the items on A&P and EF were adapted. The three-step test interview showed that the tool had sufficient content validity but that some items on EF were redundant. Overall, the stakeholders and patients indicated that the intake tool was relevant and had a logical and clear structure. The tool was integrated in an online portal.

CONCLUSIONS

In the current study, an ICF-based e-intake tool was developed that aims to screen self-reported functioning problems in adults with an ear/hearing problem. The relevance, comprehensiveness, and comprehensibility of the originally proposed item list was supported, although the stakeholder and patient feedback resulted into some changes of the tool on item-level. Ultimately, the functioning information obtained with the tool could be used to promote patient-centered ear and hearing care taking a biopsychosocial perspective into account.

From statistics to clinics: the visual feedback of PROMIS® CATs

Background

To reduce the burden of completing Patient-Reported Outcome Measures (PROMs), PROMIS® Computerized Adaptive Tests (CATs) are being implemented in pediatric clinical practice. We aimed to develop recommendations for visual feedback options for PROMIS CATs on individual item and domain score level as an evidence-based feedback recommendation for PROMIS CATs is lacking.

Methods

Focus groups were held with clinicians who use the KLIK PROM portal. Literature-based feedback options were provided to initiate group discussion. Data was analyzed using thematic coding method. Additionally, a questionnaire was sent out to assess patients’ (12-18y) and parents’ (child 0-18y) preference for individual item feedback. Data was analyzed using descriptive statistics.

Results

Six focus groups were held (N = 28 clinicians). Regarding individual item feedback, showing the complete item bank, with only responses to administered items in traffic light colors was preferred. For domain scores, line graphs were preferred, including numerical (T-)scores, reference and cut-off lines, and traffic light colors. Separate graphs per domain, ranked in order of importance and harmonization of directionality (‘higher = better’) were considered important. Questionnaire results (N = 31 patients/N = 131 parents) showed that viewing their own item responses was preferred above receiving no item feedback by 58.1% of the patients and 77.1% of the parents.

Conclusions

Based on the outcomes and after discussion with the Dutch-Flemish PROMIS National Center, recommendations for PROMIS CAT feedback options were developed. PROMIS CATs can now be used in clinical practice to help clinicians monitor patient outcomes, while reducing the burden of completing PROMs for patients significantly.

Psychometric properties of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Pediatric Anger Scale in the Dutch general population

This study aimed to validate the PROMIS® pediatric v2.0 Anger scale in the Dutch general population, provide reference data, and compare reliability and relative efficiency between the full-length scale, its short-form, computerized adaptive test (CAT), and Pediatric Quality of Life Inventory (PedsQLTM) emotional functioning (EF) subscale scores. Children (N = 1,328), representative of the Dutch population, were asked to complete the PROMIS pediatric Anger scale (8-18 years) and PedsQLTM (8-17 years). A graded response model (GRM) was fit to the data. Structural validity was assessed by checking item-fit statistics (S-X2, p < .001 = misfit). For construct validity, a moderate correlation (Pearson's r > 0.50) was expected between the Anger scale and PedsQLTM EF subscale score. Dutch mean T score based on the U.S. model was calculated to provide reference data and cut-offs. Standard error of measurement (SE(θ)) was used to assess reliability (SE(θ) < .32 = .90 reliability). Relative efficiency was calculated (1 - SE(θ)2/N items) to compare how good the measures performed relative to the amount of items administered. In total, 527 children completed the PROMIS pediatric Anger scale, of which 482 completed the PedsQLTM. Structural validity was sufficient as no items displayed misfit (S-X2 = 22.9-40.3, p > .001). The Anger scale score correlated moderately (Pearson's r = .64) with the PedsQLTM EF subscale score. Dutch mean T score was 44.20 (SD = 11.39), with cut-offs of >52.2 for moderate and ≥62.3 for severe symptoms. Reliable measurements were obtained at the population mean and >2SD in the clinically relevant direction. CAT outperformed all other measures in efficiency. The PROMIS pediatric Anger scale displayed sufficient psychometric properties within the Dutch population and reference data are available. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Patient-reported outcome measures ( PROMs): making sense of individual PROM scores and changes in PROM scores over time

Patient-reported outcome measures (PROMs) are increasingly being used in nephrology care. However, in contrast to well-known clinical measures such as blood pressure, health-care professionals are less familiar with PROMs and the interpretation of PROM scores is therefore perceived as challenging. In this paper, we provide insight into the interpretation of PROM scores by introducing the different types and characteristics of PROMs, and the most relevant concepts for the interpretation of PROM scores. Concepts such as minimal detectable change, minimal important change and response shift are explained and illustrated with examples from nephrology care.

A Patient-Reported Outcomes Measurement Information System Short Form for Measuring Physical Function During Geriatric Rehabilitation: Test-Retest Reliability, Construct Validity, Responsiveness, and Interpretability

OBJECTIVES

To study the test-retest reliability and measurement error, construct validity, responsiveness, interpretability, and floor/ceiling effects of a Patient-Reported Outcomes Measurement Information System (PROMIS) short form designed to measure physical function in geriatric rehabilitation patients (PROMIS-PF-GR).

DESIGN

Prospective cohort.

SETTING AND PARTICIPANTS

Inpatient geriatric rehabilitation patients.

METHODS

We evaluated the test-retest reliability by re-administering PROMIS-PF-GR 3 to 5 days after the admission measurement. The intraclass correlation coefficient (ICC) was calculated to determine test-retest reliability; an ICC of ≥0.70 was considered sufficient. Measurement error was established by calculating the standard error of measurement and smallest detectable change. Construct validity and responsiveness were determined by testing a priori formulated hypotheses (criterion: ≥75% hypothesis not rejected). Interpretability was evaluated by calculating the minimal important change using predictive modeling and a global rating as criterion for change. Floor/ceiling effects were established by calculating the percentage patients with the minimum/maximum raw score (criterion: ≤15%) at admission and discharge.

RESULTS

A total of 207 patients participated in the study [mean ± standard deviation age (80 ± 8.3 years), 58% female]. More than one-half of patients (56%) reported to be improved during rehabilitation. The ICC was 0.79 (95% confidence interval 0.70-0.84), the standardized error of measurement was 3.8, and the smallest detectable change 10.6. None of the 4 hypotheses for construct validity were rejected; 2 out of 5 hypotheses for responsiveness were rejected. The minimal important change was 8.0 (95% confidence interval 4.1-12.5). No floor/ceiling effects were found.

CONCLUSIONS AND IMPLICATIONS

The PROMIS-PG-GR showed sufficient test-retest reliability, measurement error, and construct validity. We did not find sufficient evidence for responsiveness, which may be due to the unexplained weak correlation between the PROMIS change score and the Global Rating Scale. We still recommend the use the PROMIS-PG-GR for measuring self-reported physical function in geriatric rehabilitation.

International application of PROMIS computerized adaptive tests: US versus country-specific item parameters can be consequential for individual patient scores

OBJECTIVE

PROMIS offers computerized adaptive tests (CAT) of patient-reported outcomes, using a single set of US-based IRT item parameters across populations and language-versions. The use of country-specific item parameters has local appeal, but also disadvantages. We illustrate the effects of choosing US or country-specific item parameters on PROMIS CAT T-scores.

STUDY DESIGN AND SETTING

Simulations were performed on response data from Dutch chronic pain patients (n = 1110) who completed the PROMIS Pain Behavior item bank. We compared CAT T-scores obtained with (1) US parameters; (2) Dutch item parameters; (3) US item parameters for DIF-free items and Dutch item parameters (rescaled to the US metric) for DIF items; (4) Dutch item parameters for all items (rescaled to the US metric).

RESULTS

Without anchoring to a common metric, CAT T-scores cannot be compared. When scores were rescaled to the US metric, mean differences in CAT T-scores based on US vs. Dutch item parameters were negligible. However, 0.9%-4.3% of the T-score differences were larger than 5 points (0.5 SD).

CONCLUSION

The choice of item parameters can be consequential for individual patient scores. We recommend more studies of translated CATs to examine if strategies that allow for country-specific item parameters should be further investigated.

Uptake of core outcome sets by clinical trialists publishing in major medical journals: Protocol

Background: Outcome heterogeneity, selective reporting, and choosing outcomes that do not reflect needs and priorities of stakeholders, limit the examination of health intervention effects, particularly in late phase trials. Core outcome sets (COS) are a proposed solution to these issues. A COS is an agreed-upon, standardised set of outcomes that should be measured and reported as a minimum in all trials in a specific area of health or healthcare. COS are intended to increase standardisation of outcome measurement and reporting to better enable comparisons between, and synthesis of findings of trials in a particular health area.  Methods: This study will examine late phase trials, published between October 2019 and March 2020 (inclusive), in the following five medical journals: New England Journal of Medicine, Journal of the American Medical Association, Lancet, BMJ, and Annals of Internal Medicine. Trials will be examined to determine if they refer to a COS, and whether they use a COS. Trialists for each identified trial will subsequently be contacted to complete an online survey examining trialists' awareness of, and decisions to search for and use a COS. Discussion: This study will provide important information on uptake of COS by later phase trialists in major medical journals, and the views of these trialists on COS use in trials. These findings will inform approaches to increasing awareness and uptake of COS in future health trials.

EULAR recommendations for the reporting of ultrasound studies in rheumatic and musculoskeletal diseases (RMDs)

OBJECTIVE

To produce European League Against Rheumatism (EULAR) recommendations for the reporting of ultrasound studies in rheumatic and musculoskeletal diseases (RMDs).

METHODS

Based on the literature reviews and expert opinion (through Delphi surveys), a taskforce of 23 members (12 experts in ultrasound in RMDs, 9 in methodology and biostatistics together with a patient research partner and a health professional in rheumatology) developed a checklist of items to be reported in every RMD study using ultrasound. This checklist was further refined by involving a panel of 79 external experts (musculoskeletal imaging experts, methodologists, journal editors), who evaluated its comprehensibility, feasibility and comprehensiveness. Agreement on each proposed item was assessed with an 11-point Likert scale, grading from 0 (total disagreement) to 10 (full agreement).

RESULTS

Two face-to-face meetings, as well as two Delphi rounds of voting, resulted in a final checklist of 23 items, including a glossary of terminology. Twenty-one of these were considered 'mandatory' items to be reported in every study (such as blinding, development of scoring systems, definition of target pathologies) and 2 'optional' to be reported only if applicable, such as possible confounding factors (ie, ambient conditions) or experience of the sonographers.

CONCLUSION

An EULAR taskforce developed a checklist to ensure transparent and comprehensive reporting of aspects concerning research and procedures that need to be presented in studies using ultrasound in RMDs. This checklist, if widely adopted by authors and editors, will greatly improve the interpretability of study development and results, including the assessment of validity, generalisability and applicability.

Harms reported by patients in rheumatology drug trials: a systematic review of randomized trials in the cochrane library from an OMERACT working group

BACKGROUND

Underreporting of harms in randomized controlled trials (RCTs) may lead to incomplete or erroneous assessments of the perceived benefit-to-harm profile of an intervention. To compare benefit with harm in clinical practice and future clinical studies, adverse event (AE) profiles including severity need to be understood. Even though patients report harm symptoms earlier and more frequently than clinicians, rheumatology RCTs currently do not provide a reporting framework from the patient's perspective regarding harms. Our objective for this meta-research project was to identify AEs in order to determine harm clusters and whether these could be self-reported by patients. Our other objective was to examine reported severity grading of the reported harms.

METHODS

We considered primary publications of RCTs eligible if they were published between 2008 and 2018 evaluating pharmacological interventions in patients with a rheumatic or musculoskeletal condition and if they were included in Cochrane reviews. We extracted data on harms such as reported AE terms together with severity (if described), and categorized AE- and severity-terms into overall groups. We deemed all AEs with felt components appropriate for patient self-reporting.

RESULTS

The literature search identified 187 possible Cochrane reviews, of which 94 were eligible for evaluation, comprising 1,297 articles on individual RCTs. Of these RCTs, 93 pharmacological trials met our inclusion criteria (including 31,023 patients; representing 20,844 accumulated patient years), which reported a total of 21,498 AEs, corresponding to 693 unique reported terms for AEs. We further sub-categorized these terms into 280 harm clusters (i.e., themes). AEs appropriate for patient self-reporting accounted for 58% of the AEs reported. Among the reported AEs, we identified medical terms for all of the 117 harm clusters appropriate for patient reporting and lay language terms for 86%. We intended to include severity grades of the reported AEs, but there was no evidence for systematic reporting of clinician- or patient-reported severity in the primary articles of the 93 trials. However, we identified 33 terms suggesting severity, but severity grading was discernible in only 9%, precluding a breakdown by severity in this systematic review.

CONCLUSIONS

Our results support the need for a standardized framework for patients' reporting of harms in rheumatology trials. Reporting of AEs with severity should be included in future reporting of harms, both from the patients' and investigators' perspectives.

REGISTRATION

PROSPERO: CRD42018108393.

Translation and content validity of the Dutch Impact of Vision Impairment questionnaire assessed by Three-Step Test-Interviewing

BACKGROUND

Patients suffering from exsudative retinal diseases may experience severe central vision loss and this might have impact on their daily activities and quality of life. To measure the disabilities these patients may have, the use of the Impact of Vision Impairment Profile questionnaire is recommended. The aim of this study was to translate the original English 28-item Impact of Vision Impairment Profile (IVI) into the Dutch language and evaluate its comprehensibility, comprehensiveness and relevance as evidence of content validity. The translation process was performed using standardized methods. Content validity was assessed by cognitive debriefing using a Three-Step Test-Interview (TSTI) method for participants diagnosed with exudative retinal diseases. Step 1 and 2 focused on assessment of comprehensibility of items, step 3 on comprehensiveness and relevance. Audio-recorded qualitative data was analyzed using Atlas.ti. Data regarding comprehensibility problems was further categorized into item-specific problems and general problems.

RESULTS

Few minor discrepancies in wording were found after translation. After conducting 12 cognitive interviews, data saturation was reached. All participants reported comprehensibility problems resulting from specific items, these were; sentence structure, vocabulary and formulation, influence of conditions or composite items and influence of comorbid disorders. Several general comprehensibility problems resulting from instructions or response categories were detected. The main general comprehensibility problem resulted from the layout of the Dutch-IVI. Most participants considered the included items as relevant and indicated that they covered the problems that occur due to vision impairment.

CONCLUSIONS

Minor problems in the Dutch translation were detected and adjusted. The layout and instructions of the Dutch-IVI resulted in some comprehensibility problems. The Dutch-IVI appeared to be at risk of being interpreted as a generic patient reported outcome measure, instead of a disease-specific instrument, mainly due to the influence of co-morbidities. Adaptations should improve validity and reliability of the Dutch-IVI, however, cross-cultural comparisons may be at stake.

The impact of lockdown during the COVID-19 pandemic on mental and social health of children and adolescents

PURPOSE

During the COVID-19 pandemic in the Netherlands, governmental regulations resulted in a lockdown for adults as well as children/adolescents. Schools were closed and contact with other people was limited. In this cross-sectional, population-based study, we aimed to investigate the mental/social health of children/adolescents during COVID-19 lockdown.

METHODS

Two representative samples of Dutch children/adolescents (8-18 years) before COVID-19 (2018, N = 2401) and during lockdown (April 2020, N = 844) were compared on the Patient-Reported Outcomes Measurement Information System (PROMIS) domains: global health, peer relationships, anxiety, depressive symptoms, anger, sleep-related impairment by linear mixed models and calculating relative risks (RR (95% CI)) for the proportion of severe scores. Variables associated with worse mental/social health during COVID-19 were explored through multivariable regression models. The impact of COVID-19 regulations on the daily life of children was qualitatively analyzed.

RESULTS

Participants reported worse PROMIS T-scores on all domains during COVID-19 lockdown compared to before (absolute mean difference range 2.1-7.1 (95% CI 1.3-7.9). During lockdown, more children reported severe Anxiety (RR = 1.95 (1.55-2.46) and Sleep-Related Impairment (RR = 1.89 (1.29-2.78) and fewer children reported poor Global Health (RR = 0.36 (0.20-0.65)). Associated factors with worse mental/social health were single-parent family, ≥ three children in the family, negative change in work situation of parents due to COVID-19 regulations, and a relative/friend infected with COVID-19. A large majority (> 90%) reported a negative impact of the COVID-19 regulations on daily life.

CONCLUSION

This study showed that governmental regulations regarding lockdown pose a serious mental/social health threat on children/adolescents that should be brought to the forefront of political decision-making and mental healthcare policy, intervention, and prevention.

Assessing baseline dependency of anchor-based minimal important change (MIC): don't stratify on the baseline score!

PURPOSE

The minimal important change (MIC) of a patient-reported outcome measure (PROM) is often suspected to be baseline dependent, typically in the sense that patients who are in a poorer baseline health condition need greater improvement to qualify as minimally important. Testing MIC baseline dependency is commonly performed by creating two or more subgroups, stratified on the baseline PROM score. This study's purpose was to show that this practice produces biased subgroup MIC estimates resulting in spurious MIC baseline dependency, and to develop alternative methods to evaluate MIC baseline dependency.

METHODS

Datasets with PROM baseline and follow-up scores and transition ratings were simulated with and without MIC baseline dependency. Mean change MICs, ROC-based MICs, predictive MICs, and adjusted MICs were estimated before and after stratification on the baseline score. Three alternative methods were developed and evaluated. The methods were applied in a real data example for illustration.

RESULTS

Baseline stratification resulted in biased subgroup MIC estimates and the false impression of MIC baseline dependency, due to redistribution of measurement error. Two of the alternative methods require a second baseline measurement with the same PROM or another correlated PROM. The third method involves the construction of two parallel tests based on splitting the PROM's item set. Two methods could be applied to the real data.

CONCLUSION

MIC baseline dependency should not be tested in subgroups based on stratification on the baseline PROM score. Instead, one or more of the suggested alternative methods should be used.

Psychometric Properties of the Pediatric Patient-Reported Outcomes Measurement Information System Item Banks in a Dutch Clinical Sample of Children With Juvenile Idiopathic Arthritis

Objective

To assess the psychometric properties of 8 pediatric Patient‐Reported Outcomes Measurement Information System (PROMIS) item banks in a clinical sample of children with juvenile idiopathic arthritis (JIA).

Methods

A total of 154 Dutch children (mean ± SD age 14.4 ± 3.0 years; range 8–18 years) with JIA completed 8 pediatric version 1.0 PROMIS item banks (anger, anxiety, depressive symptoms, fatigue, pain interference, peer relationships, physical function mobility, physical function upper extremity) twice and the Pediatric Quality of Life Inventory (PedsQL) and the Childhood Health Assessment Questionnaire (C‐HAQ) once. Structural validity of the item banks was assessed by fitting a graded response model (GRM) and inspecting GRM fit (comparative fit index [CFI], Tucker‐Lewis index [TLI], and root mean square error of approximation [RMSEA]) and item fit (S‐X² statistic). Convergent validity (with PedsQL/C‐HAQ subdomains) and discriminative validity (active/inactive disease) were assessed. Reliability of the item banks, short forms, and computerized adaptive testing (CAT) was expressed as the SE of theta (SE[θ]). Test–retest reliability was assessed using intraclass correlation coefficients (ICCs) and smallest detectable change.

Results

All item banks had sufficient overall GRM fit (CFI >0.95, TLI >0.95, RMSEA <0.08) and no item misfit (all S‐X²P > 0.001). High correlations (>0.70) were found between most PROMIS T scores and hypothesized PedsQL/C‐HAQ (sub)domains. Mobility, pain interference, and upper extremity item banks were able to discriminate between patients with active and inactive disease. Regarding reliability, PROMIS item banks outperformed legacy instruments. Post hoc CAT simulations outperformed short forms. Test–retest reliability was strong (ICC >0.70) for all full‐length item banks and short forms, except for the peer relationships item bank.

Conclusion

The pediatric PROMIS item banks displayed sufficient psychometric properties for Dutch children with JIA. PROMIS item banks are ready for use in clinical research and practice for children with JIA.

First Validation of the Full PROMIS Pain Interference and Pain Behavior Item Banks in Patients With Rheumatoid Arthritis

OBJECTIVE

Pain interference and pain behavior are highly relevant outcomes in patients with rheumatoid arthritis (RA). The Patient-Reported Outcomes Measurement Information System (PROMIS) is a universally applicable set of item banks measuring patient-reported health, and if applied as computerized adaptive tests (CATs), more efficiently and precisely than current instruments. The objective was to study the psychometric properties of the Dutch-Flemish PROMIS pain interference (PROMIS-PI) and the PROMIS pain behavior (PROMIS-PB) item banks in patients with RA.

METHODS

A total of 2,029 patients with RA completed the full PROMIS-PI (version 1.1, 40 items), and 1,554 patients completed the full PROMIS-PB (version 1.1, 39 items). The following psychometric properties were studied: unidimensionality, local dependence, monotonicity and graded response model (GRM) fit, cross-cultural validity (differential item functioning [DIF] for language [Dutch versus Flemish]), other forms of measurement invariance, construct validity, reliability, and floor and ceiling effects.

RESULTS

The PROMIS-PI and PROMIS-PB banks were sufficiently unidimensional (Omega-hierarchical [Omega-H] 0.99, 0.95, and explained common variance 0.95, 0.78, respectively), had negligible local dependence (0.3-1.4% of item pairs), good monotonicity (H 0.75, 0.46), and a good GRM model fit (no misfitting items). Furthermore, both item banks showed good cross-cultural validity (no DIF for language), measurement invariance (no DIF for age, sex, administration mode, and disease activity), good construct validity (all hypotheses met), high reliability (>0.90 in the range of patients with RA), and an absence of floor and ceiling effects (0% minimum or maximum score, respectively).

CONCLUSION

Both PROMIS-PI and PROMIS-PB banks showed good psychometric properties in patients with RA and can be used as CATs in research and clinical practice.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

BACKGROUND

Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values.

METHODS

We conducted a 3-round Delphi study involving 52 panelists.

RESULTS

Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee.

CONCLUSION

We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

Proposal to extend the PROMIS® item bank v2.0 'Ability to Participate in Social Roles and Activities': item generation and content validity

PURPOSE

Previous research indicated that the Patient-Reported Outcomes Measurement Information System (PROMIS®) item bank v2.0 'Ability to Participate in Social Roles and Activities' may miss subdomains of social participation. The purpose of this study was to generate items for these missing subdomains and to evaluate their content validity.

METHODS

A three-step approach was followed: (1) Item generation for 16 International Classification of Functioning Disability and Health subdomains currently not covered by the item bank; (2) Evaluation of content validity of generated items through expert review (n = 20) and think-aloud interviews with a purposeful sample of people with and without (chronic) health conditions (n = 10), to assess item comprehensibility, relevance, and comprehensiveness; and 3) Item revision based on the results of step 2, in a consensus procedure.

RESULTS

First, 48 items were generated. Second, overall, content experts indicated that the generated items were relevant. Furthermore, based on experts' responses, items were simplified and 'participation in social media' was identified as an important additional subdomain of social participation. Additionally, 'participating in various social roles simultaneously' was identified as a missing item. Based on the responses of the interviewed adults items were simplified. Third, in total 17 items, covering 17 subdomains, were proposed to be added to the original item bank.

DISCUSSION

The relevance, comprehensibility and comprehensiveness of the 17 proposed items were supported. Whether the proposed extension of the item bank leads to better psychometric properties of the item bank should be tested in a large-scale field study.

Changes in the ability to participate in and satisfaction with social roles and activities in patients in outpatient rehabilitation

Background

One of the main aims of rehabilitation is to improve participation. Patient-Reported Outcomes Measurement Information System (PROMIS®) item banks ‘Ability to Participate in Social roles and Activities, (PROMIS-APS) and ‘Satisfaction with Social Roles and Activities’ (PROMIS-SPS) are promising options to measure participation, but the literature on PROMIS measures of (satisfaction with) participation across diagnoses in rehabilitation is limited. Therefore, the objective of this study was to describe levels of and changes in participation, as assessed with the PROMIS-APS and the PROMIS-SPS short forms, of patients in outpatient rehabilitation.

Methods

This study had quantitative, observational design with assessments at admission and discharge. Consecutive patients treated between April and August 2018 receiving outpatient multidisciplinary rehabilitation were the population of this study. The following diagnosis categories were included: brain injury (e.g. stroke), spinal cord and nerve injury, neuromuscular disorder (e.g. lateral sclerosis), amputation, musculoskeletal condition (e.g. osteoarthritis) or heart or lung disease (e.g. myocardial infarction, chronic obstructive pulmonary disease). The main patient-reported outcomes (PRO) of this study were the short form of the PROMIS-APS (8 items, Dutch general population reference score 50.6 [SD 9.5]), and PROMIS-SPS (8 items, Dutch general population reference score 47.5 [SD 8.3].

Results

Of the 1279 patients invited, 777 (61%) completed the online forms at admission. Of those, 329 patients were invited at discharge, with 209 (64%) completing the forms. The mean (SD) T-scores of the PROMIS-APS and PROMIS-SPS were lower at admission (42.7 [SD 7.4]; (41.4 [SD 7.7]) and discharge (43.6 [SD 7.2]; (43.7 [SD 7.8]) than the Dutch general population. The change scores of the PROMIS-APS and PROMIS-SPS were 1.2 (95% CI 0.4–1.9; p = 0.004; effect size 0.16), and 2.4 (95% CI 1.6–3.2; p < 0.0001; effect size 0.31), respectively. In all diagnostic subgroups with > 30 paired measurements statistically significant improvements of PROMIS-APS, PROMIS-SPS or both were seen.

Conclusions

Patients undergoing outpatient rehabilitation had, both at admission and discharge, considerably lower PROMIS-APS and PROMIS-SPS T-scores short forms than the general Dutch population, and showed small T-score improvements at discharge.

Uptake of core outcome sets by clinical trialists publishing in major medical journals: Protocol

Background: Outcome heterogeneity, selective reporting, and choosing outcomes that do not reflect needs and priorities of stakeholders, limit the examination of health intervention effects, particularly in late phase trials. Core outcome sets (COS) are a proposed solution to these issues. A COS is an agreed-upon, standardised set of outcomes that should be measured and reported as a minimum in all trials in a specific area of health or healthcare. COS are intended to increase standardisation of outcome measurement and reporting to better enable comparisons between, and synthesis of findings of trials in a particular health area.  Methods: This study will examine late phase trials, published between October 2019 and March 2020 (inclusive), in the following five medical journals: New England Journal of Medicine, Journal of the American Medical Association, Lancet, BMJ, and Annals of Internal Medicine. Trials will be examined to determine if they refer to a COS, and whether they use a COS. Trialists for each identified trial will subsequently be contacted to complete an online survey examining trialists' awareness of, and decisions to search for and use a COS. Discussion: This study will provide important information on uptake of COS by later phase trialists in major medical journals, and the views of these trialists on COS use in trials. These findings will inform approaches to increasing awareness and uptake of COS in future health trials.

Unlike ROC analysis, a new IRT method identified clinical thresholds unbiased by disease prevalence

OBJECTIVE

This study introduces a new method to establish clinical thresholds for multi-item tests, based on item response theory (IRT), as an alternative to receiver operating characteristic (ROC) analysis. The performance of IRT method was examined and compared with the ROC method across multiple simulated data sets and in a real data set.

STUDY DESIGN AND SETTING

Simulated data sets (sample size: 1,000) varied in means and variability of the test scores and the prevalence of disease. The true clinical threshold was defined as a predetermined location on the latent trait underlying the questionnaire, with its corresponding expected test score. The real data set (sample size: 295) comprised Hospital Anxiety Depression Scale (HADS) depression scores and Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition major depressive disorder (MDD) diagnoses.

RESULTS

The IRT method recovered the clinical thresholds without bias, whereas the ROC method identified thresholds that were biased by the prevalence of disease. Mild MDD was clinically diagnosed in 23%, moderate MDD in 12%, and severe MDD in 14% of the participants. The IRT method identified the following HADS depression score thresholds for mild, moderate, and severe MDD: 10.7, 13.2, and 15.1, respectively.

CONCLUSION

The new IRT method identifies clinical thresholds that are unbiased by disease prevalence.

Development of a Patient-Reported Outcomes Measurement Information System (PROMIS®) short form for measuring physical function in geriatric rehabilitation patients

Purpose

To develop and test the validity of a Patient-Reported Outcomes Measurement Information System (PROMIS®) short form for measuring physical function of geriatric rehabilitation patients.

Methods

Experts selected items from the Dutch-Flemish PROMIS v1.2 Physical Function (PROMIS-PF) item bank and proposed new items to develop the PROMIS-PF short form for geriatric rehabilitation (PROMIS-PF-GR). Patients evaluated its content validity. Structural validity was assessed by evaluating unidimensionality (confirmatory, exploratory, and bi-factor analyses [criterion: Omega H > 0.80 and ECV > 0.60]), local independence (criterion: residual correlation < 0.20) ,and monotonicity (criterion: H_i-coefficient ≥ 0.30). Measurement invariance was assessed by evaluating Differential Item Functioning (DIF) between geriatric rehabilitation patients and people from the general population using ordinal logistic regression. Internal consistency was assessed by calculating Cronbach’s alpha (criterion: alpha ≥ 0.70).

Results

Experts selected 24 items from the PROMIS-PF item bank and proposed one new item which was not included in the short form. Patients considered the 24 items relevant and containing essential information. The PROMIS-PF-GR’s psychometric properties were evaluated in 207 patients (mean age ± SD, 80.0 ± 8.3 year; 58% female). The 24 items were found to be sufficiently unidimensional (Omega H = 0.82, ECV = 0.70), locally independent (98.7% item pairs), and monotone (all ≥ 0.32). Five items were flagged for DIF, but their impact on the total score was negligible. Cronbach’s alpha was 0.94.

Conclusion

The PROMIS-PF-GR was developed from the PROMIS-PF and has good content validity, structural validity, measurement invariance, and internal consistency in Dutch geriatric rehabilitation patients. We recommend to confirm the content validity of the PROMIS-PF-GR in other countries.

Psychometric properties of the Patient-Reported Outcomes Measurement Information System (PROMIS) Sleep Disturbance and Sleep-Related Impairment item banks in adolescents

Sleep problems have a high prevalence and negative daytime consequences in adolescents. Current sleep measures for this age group have limitations. The Patient‐Reported Outcomes Measurement Information System (PROMIS^®) developed sleep item banks for adults. In a previous validation study, these item banks were adapted to a shortened version for adolescents. The current study aimed to further explore the psychometric properties of the 11‐item Sleep‐Related Impairment and 23‐item Sleep Disturbance item banks in Dutch adolescents. We investigated structural validity by testing item response theory assumptions and model fit; measurement invariance by performing differential item functioning analyses; performance as a computerized adaptive test; reliability by marginal reliability estimates and test–retest reliability (intraclass correlation coefficients and limits of agreement); and construct validity by hypothesis testing. Additionally, we provide mean values for the item banks. The study sample consisted of 1,046 adolescents (mean age 14.3 ± 1.6), including 1,013 high‐school students and 33 sleep‐clinic patients. The Sleep Disturbance‐23 showed lack of unidimensionality, but had sufficient test–retest reliability, and could distinguish between adolescents with and without sleep or health issues. The Sleep‐Related Impairment‐11 showed sufficient unidimensionality and model fit and was thus tested as a computerized adaptive test, demonstrating an equal amount of reliable measures to the full item bank. Furthermore, the Sleep‐Related Impairment‐11 could distinguish between adolescents with and without sleep or health issues and test–retest reliability was moderate. The use of both item banks in the full form and the use of the Sleep‐related Impairment‐11 as a computer adaptive test is recommended.

Graded response model fit, measurement invariance and (comparative) precision of the Dutch-Flemish PROMIS® Upper Extremity V2.0 item bank in patients with upper extremity disorders

BACKGROUND

The Dutch-Flemish PROMIS® Upper Extremity (DF-PROMIS-UE) V2.0 item bank was recently developed using Item Response Theory (IRT). Unknown for this bank are: (1) if it is legitimate to calculate IRT-based scores for short forms and Computerized Adaptive Tests (CATs), which requires that the items meet the assumptions of and fit the IRT-model (Graded Response Model [GRM]);(2) if it is legitimate to compare (sub) groups of patients using this measure, which requires measurement invariance; and (3) the precision of the estimated patients' scores for patients with different levels of functioning and compared to legacy measures. Aims were to evaluate (1) the assumptions of and fit to the GRM, (2) measurement invariance and (3) (comparative) precision of the DF-PROMIS-UE v2.0.

METHODS

Cross-sectional data were collected in Dutch patients with upper extremity disorders. Assessed were IRT-assumptions (unidimensionality [bi-factor analysis], local independence [residual correlations], monotonicity [coefficient H]), GRM item fit, measurement invariance (absence of Differential Item Functioning [DIF] due to age, gender, center, duration, and location of complaints) and precision (standard error of IRT-based scores across levels of functioning). To study measurement invariance for language [Dutch vs. English], additional US data were used. Legacy instruments were the Disability of the Arm, Shoulder and Hand (DASH), the QuickDASH and the Michigan Hand Questionnaire (MHQ).

RESULTS

In total 521 Dutch (mean age ± SD = 51 ± 17 years, 49% female) and 246 US patients (mean age ± SD = 48 ± 14 years, 69% female) participated. The DF-PROMIS-UE v2.0 item bank was sufficiently unidimensional (Omega-H = 0.80, Explained Common Variance = 0.68), had negligible local dependence (four out of 1035 correlations > 0.20), good monotonicity (H = 0.63), good GRM fit (no misfitting items) and demonstrated sufficient measurement invariance. Precise estimates (Standard Error < 3.2) were obtained for most patients (7-item short form, 88.5%; standard CAT, 91.3%; and, fixed 7-item CAT, 87.6%). The DASH displayed better reliability than the DF-PROMIS-UE short form and standard CAT, the QuickDASH displayed comparable reliability. The MHQ-ADL displayed better reliability than the DF-PROMIS-UE short form and standard CAT for T-scores between 28 and 50. For patients with low function, the DF-PROMIS-UE measures performed better.

CONCLUSIONS

The DF-PROMIS-UE v2.0 item bank showed sufficient psychometric properties in Dutch patients with UE disorders.

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

It is often unrealistic to assume normally distributed latent traits in the measurement of health outcomes. If normality is violated, the item response theory (IRT) models that are used to calibrate questionnaires may yield parameter estimates that are biased. Recently, IRT models were developed for dealing with specific deviations from normality, such as zero-inflation (“excess zeros”) and skewness. However, these models have not yet been evaluated under conditions representative of item bank development for health outcomes, characterized by a large number of polytomous items. A simulation study was performed to compare the bias in parameter estimates of the graded response model (GRM), polytomous extensions of the zero-inflated mixture IRT (ZIM-GRM), and Davidian Curve IRT (DC-GRM). In the case of zero-inflation, the GRM showed high bias overestimating discrimination parameters and yielding estimates of threshold parameters that were too high and too close to one another, while ZIM-GRM showed no bias. In the case of skewness, the GRM and DC-GRM showed little bias with the GRM showing slightly better results. Consequences for the development of health outcome measures are discussed.

Development of a framework with tools to support the selection and implementation of patient-reported outcome measures

BACKGROUND

Patient reported outcomes (PROs) provide information on a patient's health status coming directly from the patient. Measuring PROs with patient reported outcome measures (PROMs) has gained wide interest in clinical practice for individual patient care, as well as in quality improvement, and for providing transparency of outcomes to stakeholders through public reporting. However, current knowledge of selecting and implementing PROMs for these purposes is scattered, and not readily available for clinicians and quality managers in healthcare organizations. The objective of this study is to develop a framework with tools to support the systematic selection, implementation and evaluation of PROs and PROMs in individual patient care, for quality improvement and public reporting.

METHODS

We developed the framework in a national project in the Netherlands following a user-centered design. The development process of the framework contained five iterative components: (a) identification of existing tools, (b) identification of user requirements and designing steps for selection and implementation of PROs and PROMs, (c) discussing a prototype of the framework during a national workshop, (d) developing a web version, (e) pre-testing of the framework. A total of 40 users with different perspectives (clinicians, patient representatives, quality managers, purchasers, researchers) have been consulted.

RESULTS

The final framework is presented as the PROM-cycle that consists of eight steps in four phases: (1) goal setting, (2) selecting PROs and PROMs, (3) developing and testing of quality indicator(s), (4) implementing and evaluating the PROM(s) and indicator(s). Users emphasized that the first step is the key element in which the why, for whom and setting of the PROM has to be defined. This information is decisive for the following steps. For each step the PROM-cycle provides guidance and tools, with instruments, checklists, methods, handbooks, and standards supporting the process.

CONCLUSION

We developed a framework to support the selection and implementation of PROs and PROMs. Each step provides guidance and tools to support the process. The PROM-cycle and its tools are publicly available and can be used by clinicians, quality managers, patient representatives and other experts involved in using PROMS. Through periodic evaluation and updates, tools will be added for national and international use of the PROM-cycle.

Pain, fatigue, depressive symptoms and sleep disturbance in young adults with cerebral palsy

PURPOSE

Investigate pain, fatigue, depressive symptoms and sleep disturbance in young adults with cerebral palsy compared to references.

MATERIALS AND METHODS

Young adults with cerebral palsy (n = 97, aged 21-34 years) and age-matched references from the general population (n = 190) rated pain using a numeric rating scale and fatigue, depressive symptoms, sleep disturbance and global health using Patient-Reported Outcomes Measurement Information System® short forms. Scores were compared between cerebral palsy subgroups and the reference population. Correlation coefficients and linear regression analyses assessed interrelationships of health issues and associations with global health.

RESULTS

Individuals with Gross Motor Function Classification System level I had less pain, fatigue and depressive symptoms, while individuals with levels II and III-V had more pain (53% and 56%, p < 0.001) and those with levels III-V more fatigue (39%, p = 0.035) than references (pain: 26%, fatigue: 14%). Pain and fatigue were more interrelated (correlation coefficients: 0.71 vs. 0.41) and stronger associated with global mental health in individuals with cerebral palsy.

CONCLUSIONS

Young adults with Gross Motor Function Classification System levels II-V report more pain and those with levels III-V report more fatigue than references. Pain and fatigue are highly interrelated and specifically relate to mental health in individuals with cerebral palsy.Implications for rehabilitationExcept for those in the highest level of motor function, young adults with cerebral palsy report higher levels of pain and fatigue compared to the general population of the same age.Pain and fatigue are strongly interrelated and associated with mental health in young adults with cerebral palsy.The present study recommends to monitor pain and fatigue in young adults with cerebral palsy with low levels of gross motor function.We advise rehabilitation professionals to consider combined treatment for both pain and fatigue.

Responsiveness and interpretability of the Animated Activity Questionnaire for assessing activity limitations of patients with hip or knee osteoarthritis

PURPOSE

The aim of the study was to determine the responsiveness and interpretability of the Animated Activity Questionnaire (AAQ), an online questionnaire in which osteoarthritis patients select animations that best match their performance of daily activities.

METHODS

A longitudinal study was carried out, in which 94 patients with hip or knee osteoarthritis were assessed at baseline, and 3 and 6 months after treatment (conservative and surgical). Responsiveness was assessed by means of testing hypotheses about expected correlations between change in AAQ, a Global Rating Scale of change (GRS) and change in the Activities of Daily Living subscale of the Hip disability or Knee injury and Osteoarthritis Outcome Score (H/KOOS), and a combination of performance-based tests (the 30 s chair-stand test, the timed up-and-go test and the nine-step stair climbing test). The minimal important change (MIC) was estimated by means of the receiving operating characteristics (ROC) method.

RESULTS

The correlations of the AAQ with the H/KOOS were as expected, but other correlations were lower than anticipated. The area under the ROC curve was 0.74 at 6 months. At 3 months' follow-up, the correlations were too low to calculate a MIC. A total of 20% of the results at 3 months and 80% of the results at 6 months were in accordance with the hypotheses. The MIC was 9 points at 6 months.

CONCLUSIONS

The AAQ was sufficiently responsive at the six-months follow-up, but not at the three-month follow-up. The MIC at the 6-month follow-up (9 points) was slightly lower than the smallest detectable change of 14 points found in a previous study.

Comparison of a video versus paper questionnaire on functional limitation in lower limb osteoarthritis

INTRODUCTION

The video Animated Activity Questionnaire (AAQ) was developed to assess the impact of lower limb osteoarthritis (OA) on daily functional activities. The objective of the study was to compare the video and the HOOS/KOOS paper questionnaires and to assess the effect of order of administration.

MATERIAL AND METHODS

Patients recruited in the KHOALA cohort were randomized in two groups: AAQ questionnaire first (AAQ-first group) and HOOS (hip)/KOOS (knee) questionnaire first (H/KOOS-first group). Within group differences between AAQ and HOOS/KOOS scores were compared using a Student t-test. The Spearman correlation coefficient between AAQ score and HOOS/KOOS score was calculated in each group then compared, using Fisher z-transformation.

RESULTS

Among 200 randomized patients, 188 (65.8 years, 66.0% women) completed the questionnaires: 99 in the AAQ-first group and 89 in the H/KOOS-first group. The AAQ score was 85.9 (SD: 13.7) in the AAQ-first versus 87.8 (SD: 13.1) in the H/KOOS-first group (p = 0.34). The H/KOOS score was 72.5 (SD: 21.2) in the AAQ-first versus 73.5 (SD: 18.4) in the H/KOOS-first group (p = 0.71). The Spearman correlation coefficient between AAQ and H/KOOS in the AAQ-first was 0.84[0.77-0.89] and 0.73[0.61-0.81] in H/KOOS-first group. These correlations differed between groups significantly (p = 0.02).

CONCLUSION

This study found video AAQ and paper HOOS/KOOS questionnaire highly correlated, with a moderate but significant effect of order administration of video and paper questionnaires evidencing a stronger correlation when the videos were viewed first.

Practical Significance of Longitudinal Measurement Invariance Violations in the Dutch-Flemish PROMIS Item Banks for Depression and Anxiety: An Illustration With Ordered-Categorical Data

We investigated longitudinal measurement invariance in the Dutch-Flemish PROMIS adult v1.0 item banks for Depression and Anxiety using two clinical samples with mood and anxiety disorders (n = 640 and n = 528, respectively). Factor analysis was used to evaluate whether the item banks were sufficiently unidimensional at two test-occasions and whether the measured constructs remained the same over time. The results indicated that the item banks were sufficiently unidimensional, but the thresholds and residual variances of the constructs changed over time. However, using tentative rules of thumb, these invariance violations did not substantially affect the endorsement of a specific response category of a specific item at a specific test-occasion. Furthermore, the impact on the mean latent change scores of the item banks remained below the proposed cutoff value for substantial bias. These findings suggest that the invariance violations lacked practical significance for test-users, meaning that the item banks provide sufficiently invariant latent factor scores for use in clinical practice.

Structural validity and construct validity of the Dutch-Flemish PROMIS® physical function-upper extremity version 2.0 item bank in Dutch patients with upper extremity injuries

INTRODUCTION

Aim of this study was to validate the Dutch-Flemish Patient-Reported Outcomes Measurement Information System Physical Function - Upper Extremity version 2.0 item bank in patients with upper extremity injuries.

MATERIALS AND METHODS

Cross-sectional study. Structural validity was assessed using Confirmatory Factor Analysis examining unidimensionality. In addition, a bi-factor model was fitted. Internal consistency was assessed by Cronbach's alpha. Construct validity was examined by assessing correlations with legacy instruments Disability of Arm Shoulder and Hand, Patient Reported Wrist Evaluation and Michigan Hand Questionnaire subscale Activities in Daily Life.

RESULTS

A total of 303 patients (144 female) with mean age of 50 years (standard deviation 18) were included. Confirmatory Factor Analysis showed Comparative Fit Index of 0.94, a Tucker Lewis Index of 0.93, a Root Mean Square Error of Approximation of 0.12 and a Standardized Root Mean Residual of 0.09. Factor loadings were all above 0.70. Bifactor analysis showed an omega-H of 0.79 and Explained Common Variance of 0.67. The correlations with the legacy instruments were as expected or higher than expected.

CONCLUSION

The Dutch-Flemish Patient-Reported Outcomes Measurement Information System Physical Function - Upper Extremity version 2.0 item bank measures a unidimensional trait and sufficient construct validity was found.IMPLICATIONS FOR REHABILITATIONCompleting Patient Reported Outcomes is time-consuming for patients and interpretability of outcomes is sometimes unclear due to some variation in psychometric properties.Computerized Adaptive Testing reduces the burden for patients by using an algorithm which decreases the amount of questions that need to be answered to 4 to 7 items.The Dutch-Flemish Patient-Reported Outcomes Measurement Information System Physical Function - Upper Extremity version 2.0 item bank measures a unidimensional trait and has sufficient structural validity, internal consistency and construct validity.After calibration of the Patient-Reported Outcomes Measurement Information System Physical Function - Upper Extremity version 2.0, the item bank is operable to use with Computerized Adaptive Testing.

Uptake of the OMERACT-OARSI Hip and Knee Osteoarthritis Core Outcome Set: Review of Randomized Controlled Trials from 1997 to 2017

OBJECTIVE

To assess the uptake of the OMERACT-OARSI (Outcome Measures in Rheumatology- Osteoarthritis Research Society International) core outcome set (COS) domains in hip and/or knee osteoarthritis (OA) trials.

METHODS

There were 382 trials of hip and/or knee OA identified from the ClinicalTrial.gov registry from 1997 to 2017. Frequency of COS adoption was assessed by year and per 5-yearly phases.

RESULTS

COS adoption decreased from 61% between 1997 and 2001 to 38% between 2012 and 2016. Pain (95%) and physical function (86%) were most consistently adopted. Patient's global assessment (48%) was the principal missing domain.

CONCLUSION

Limited adoption of the COS domains indicates that further consideration to improve uptake is required.

The OMERACT-OARSI Core Domain Set for Measurement in Clinical Trials of Hip and/or Knee Osteoarthritis

OBJECTIVE

To update the 1997 OMERACT-OARSI (Outcome Measures in Rheumatology-Osteoarthritis Research Society International) core domain set for clinical trials in hip and/or knee osteoarthritis (OA).

METHODS

An initial review of the COMET database of core outcome sets (COS) was undertaken to identify all domains reported in previous COS including individuals with hip and/or knee OA. These were presented during 5 patient and health professionals/researcher meetings in 3 continents (Europe, Australasia, North America). A 3-round international Delphi survey was then undertaken among patients, healthcare professionals, researchers, and industry representatives to gain consensus on key domains to be included in a core domain set for hip and/or knee OA. Findings were presented and discussed in small groups at OMERACT 2018, where consensus was obtained in the final plenary.

RESULTS

Four previous COS were identified. Using these, and the patient and health professionals/researcher meetings, 50 potential domains formed the Delphi survey. There were 426 individuals from 25 different countries who contributed to the Delphi exercise. OMERACT 2018 delegates (n = 129) voted on candidate domains. Six domains gained agreement as mandatory to be measured and reported in all hip and/or knee OA clinical trials: pain, physical function, quality of life, and patient's global assessment of the target joint, in addition to the mandated core domain of adverse events including mortality. Joint structure was agreed as mandatory in specific circumstances, i.e., depending on the intervention.

CONCLUSION

The updated core domain set for hip and/or knee OA has been agreed upon. Work will commence to determine which outcome measurement instrument should be recommended to cover each core domain.

Psychometric properties of the Dutch-Flemish Patient-Reported Outcomes Measurement Information System Pain Behavior item bank in patients with musculoskeletal complaints

We studied the psychometric properties of the 39-item v1.1 Dutch-Flemish Patient-Reported Outcomes Measurement Information System Pain Behavior item bank in a sample of 1,602 patients with musculoskeletal complaints. We evaluated the assumptions of the underlying item response theory (IRT) model (unidimensionality and local dependency with confirmatory factor analyses), and monotonicity with scalability coefficients). We studied the IRT model fit of all items and estimated the item parameters of the IRT model. Differential item functioning (DIF) was studied for age and gender, and DIF for language was studied as a measure of cross-cultural validity. Confirmatory factor analyses showed suboptimal fit of a unidimensional model, but a bifactor model showed low risk of bias when a unidimensional model was assumed (Omega H = .92, explained common variance of .70). Fifteen item pairs (2%) were locally dependent. Five items showed poor scalability. All items fitted the IRT model; slope parameters ranged from .60 to 2.00, and threshold parameters from -2.05 to 6.80. One item showed DIF for age, 1 item DIF for gender, and 5 items showed DIF for language, but the impact on total scores was low. Our study shows limitations of the Dutch-Flemish Patient-Reported Outcomes Measurement Information System Pain Behavior item bank when used in a primary care population with musculoskeletal complaints. PERSPECTIVE: We studied the psychometric properties of the Dutch-Flemish Patient-Reported Outcomes Measurement Information System Pain Behavior item bank in a large primary care population of patients with musculoskeletal complaints. It showed that the Pain Behavior item bank has limitations when used in this population.

A qualitative study investigating the meaning of participation to improve the measurement of this construct

PURPOSE

The purpose of this study was to improve the measurement of participation. Research questions were as follows: (1) What constitutes participation according to adults? (2) Do they mention participation subdomains that are not covered in the Patient-Reported Outcomes Measurement Information System (PROMIS) item bank "Ability to Participate in Social Roles and Activities"?

METHODS

Semi-structured interviews were conducted with 46 adults from the general population. Interviews were thematically analysed using the International Classification of Functioning, Disability and Health (ICF) as conceptual framework. Thereafter, assigned codes were compared to PROMIS item bank.

RESULTS

Participants mentioned a variety of participation subdomains that were meaningful to them, such as socializing and employment. All subdomains could be classified into the ICF. The following subdomains were not covered by the PROMIS item bank: acquisition of necessities, education life, economic life, community life, and religion and spirituality. Also a distinction between remunerative (i.e. paid) and non-remunerative (i.e. unpaid) employment, and domestic life was missing. Several ICF sub-codes were not mentioned, such as ceremonies.

CONCLUSIONS

Many participation subdomains were mentioned to be meaningful. As several of these subdomains are not covered in the PROMIS item bank, it may benefit from extension with new (patient-)reported subdomains of participation.

Measuring health-related quality of life in colorectal cancer patients: systematic review of measurement properties of the EORTC QLQ-CR29

INTRODUCTION

The EORTC QLQ-CR29 is a patient-reported outcome measure to evaluate health-related quality of life among colorectal cancer patients in research and clinical practice. The aim of this systematic review was to investigate whether the initial positive results regarding the measurement properties of the QLQ-CR29 are confirmed in subsequent studies.

METHODS

A systematic search of Embase, Medline, PsycINFO, and Web of Science was conducted to identify studies investigating the measurement properties of the QLQ-CR29 published up to January 2019. For the 11 included studies, data were extracted, methodological quality was assessed, results were synthesized, and evidence was graded according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on the measurement properties: structural validity, internal consistency, reliability, measurement error, construct validity (hypothesis testing, including known-group comparison, convergent and divergent validity), cross-cultural validity, and responsiveness.

RESULTS

Internal consistency was rated as "sufficient," with low evidence. Reliability was rated as "insufficient," with moderate evidence. Construct validity (hypothesis testing; known-group comparison, convergent and divergent validity) was rated as "inconsistent," with moderate evidence. Structural validity, measurement error, and responsiveness were rated as "indeterminate" and could therefore not be graded.

CONCLUSION

This review indicates that current evidence supporting the measurement properties of the QLQ-CR29 is limited. Additionally, better quality research is needed, taking into account the COSMIN methodology.

Measuring positive health: for now, a bridge too far

OBJECTIVES

Huber et al. introduced a new concept 'positive health', defined as 'the ability to adapt and self-manage in the face of social, physical and emotional challenges' and suggested a conceptual model comprising six domains covering 32 aspects. Our aim was to generate items and pilot test an outcome measurement instrument for measuring 'positive health' in Dutch adult citizens.

STUDY DESIGN

A mixed-method study: a literature search, a qualitative study with interviews and a quantitative ranking study for the development phase, to be followed by a content validity study for the validation phase.

METHODS

We developed items based on the concept elicitation study of Huber et al. A ranking study with end users, Dutch citizens and members of an Expert Group was performed for item reduction. Content validity of the prefinal questionnaire was evaluated.

RESULTS

A prefinal 46-item questionnaire was developed. The results of the content validity study, however, showed major concerns with regard to relevance, comprehensiveness and comprehensibility of the questionnaire.

CONCLUSIONS

Because of major concerns regarding the conceptual model of 'positive health', it was not possible to develop a valid questionnaire to measure 'positive health'. Future research should focus on the refinement of the conceptualization of 'positive health' before an adequate measurement instrument could be developed that can be used for outcome measurement purposes.

The Female Sexual Function Index (FSFI)-A Systematic Review of Measurement Properties

INTRODUCTION

The Female Sexual Function Index (FSFI) is a patient-reported outcome measure measuring female sexual dysfunction. The FSFI-19 was developed with 6 theoretical subscales in 2000. In 2010, a shortened version became available (FSFI-6).

AIM

To investigate the measurement properties of the FSFI-19 and FSFI-6.

METHODS

A systematic search was performed of Embase, Medline, and Web of Science for studies that investigated measurement properties of the FSFI-19 or FSFI-6 up to April 2018. Data were extracted and analyzed according to COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate, and quality of evidence as very high, high, moderate, or low.

MAIN OUTCOME MEASURES

The Main Outcome Measure is the evidence of a measurement property, and the quality of evidence based on the COSMIN guidelines.

RESULTS

83 studies were included. Concerning the FSFI-19, the evidence for internal consistency was sufficient and of moderate quality. The evidence for reliability was sufficient but of low quality. The evidence for criterion validity was sufficient and of high quality. The evidence for structural validity was inconsistent of low quality. The evidence for construct validity was inconsistent of moderate quality. Concerning the FSFI-6, the evidence for criterion validity was sufficient of moderate quality. The evidence for internal consistency was rated as indeterminate. The evidence for reliability was inconsistent of low quality. The evidence for construct validity was inconsistent of very low quality. No information was available on structural validity of the FSFI-6, and measurement error, responsiveness, and cross-cultural validity of both FSFI-6 and FSFI-19.

CLINICAL IMPLICATIONS

Conflicting and lack of evidence for some of the measurement properties of the FSFI-19 and FSFI-6 indicates the importance of further research on the validity of these patient-reported outcome measures. We advise researchers who use the FSFI-19 to perform confirmatory factor analyses and report the factor structure found in their sample. Regardless of these concerns, the FSFI-19 and FSFI-6 have strong criterion validity. Pragmatically, they are good screening tools for the current definition of female sexual dysfunction.

STRENGTH & LIMITATION

A strong point of the review is the use of predefined guidelines. A limitation is the use of a precise rather than a sensitive search filter.

CONCLUSIONS

The FSFI requires more research on structural validity (FSFI-19 and FSFI-6), reliability (FSFI-6), construct validity (FSFI-19), measurement error (FSFI-19 and FSFI-6), and responsiveness (FSFI-19 and FSFI-6). Further corroboration of measurement invariance (both across cultures and across subpopulations) in the factor structure of the FSFI-19 is necessary, as well as tests for the unidimensionality of the FSFI-6. Neijenhuijs KI, Hooghiemstra N, Holtmaat K, et al. The Female Sexual Function Index (FSFI)-A Systematic Review of Measurement Properties. J Sex Med 2019;16:640-660.

Improving outcome reporting in clinical trial reports and protocols: study protocol for the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT)

Background

Inadequate and poor quality outcome reporting in clinical trials is a well-documented problem that impedes the ability of researchers to evaluate, replicate, synthesize, and build upon study findings and impacts evidence-based decision-making by patients, clinicians, and policy-makers. To facilitate harmonized and transparent reporting of outcomes in trial protocols and published reports, the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT) is being developed. The final product will provide unique InsPECT extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines.

Methods

The InsPECT SPIRIT and CONSORT extensions will be developed in accordance with the methodological framework created by the EQUATOR (Enhancing the Quality and Transparency of Health Research Quality) Network for reporting guideline development. Development will consist of (1) the creation of an initial list of candidate outcome reporting items synthesized from expert consultations and a scoping review of existing guidance for reporting outcomes in trial protocols and reports; (2) a three-round international Delphi study to identify additional candidate items and assess candidate item importance on a 9-point Likert scale, completed by stakeholders such as trial report and protocol authors, systematic review authors, biostatisticians and epidemiologists, reporting guideline developers, clinicians, journal editors, and research ethics board representatives; and (3) an in-person expert consensus meeting to finalize the set of essential outcome reporting items for trial protocols and reports, respectively. The consensus meeting discussions will be independently facilitated and informed by the empirical evidence identified in the primary literature and through the opinions (aggregate rankings and comments) collected via the Delphi study. An integrated knowledge translation approach will be used throughout InsPECT development to facilitate implementation and dissemination, in addition to standard post-development activities.

Discussion

InsPECT will provide evidence-informed and consensus-based standards focused on outcome reporting in clinical trials that can be applied across diverse disease areas, study populations, and outcomes. InsPECT will support the standardization of trial outcome reporting, which will maximize trial usability, reduce bias, foster trial replication, improve trial design and execution, and ultimately reduce research waste and help improve patient outcomes.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3248-0) contains supplementary material, which is available to authorized users.

Physical Activity Questionnaires for Pregnancy: A Systematic Review of Measurement Properties

Background

In order to assess physical activity (PA) during pregnancy, it is important to choose the instrument with the best measurement properties.

Objectives

To systematically summarize, appraise, and compare the measurement properties of all self-administered questionnaires assessing PA in pregnancy.

Methods

We searched PubMed, Embase, and SPORTDiscus with the following inclusion criteria: (i) the study reported at least one measurement property (reliability, criterion validity, construct validity, responsiveness) of a self-administered questionnaire; (ii) the questionnaire intended to measure PA; (iii) the questionnaire was evaluated in healthy pregnant women; and (iv) the study was published in English. We evaluated results, quality of individual studies, and quality of evidence using a standardized checklist (Quality Assessment of Physical Activity Questionnaires [QAPAQ]) and the GRADE (Grading of Recommendation, Assessment, Development, and Evaluation) approach.

Results

Seventeen articles, reporting 18 studies of 11 different PA questionnaires (17 versions), were included. Most questionnaire versions showed insufficient measurement properties. Only the French and Turkish versions of the Pregnancy Physical Activity Questionnaire (PPAQ) showed both sufficient reliability and construct validity. However, all versions of the PPAQ pooled together showed insufficient construct validity. The quality of individual studies was usually high for reliability but varied considerably for construct validity. Overall, the quality of evidence was very low to moderate.

Conclusions

We recommend the PPAQ to assess PA in pregnancy, although the pooled results revealed insufficient construct validity. The lack of appropriate standards in data collection and processing criteria for objective devices in measuring PA during pregnancy attenuates the quality of evidence. Therefore, research on the validity of comparison instruments in pregnancy followed by consensus on validation reference criteria and standards of PA measurement is needed.

Electronic supplementary material

The online version of this article (10.1007/s40279-018-0961-x) contains supplementary material, which is available to authorized users.