A computer-adaptive disability instrument for lower extremity osteoarthritis research demonstrated promising breadth, precision, and reliability

Patient-reported outcome measures used in patients undergoing total knee arthroplasty

Patient-reported outcome measures (PROMs) are being used increasingly in total knee arthroplasty (TKA). We conducted a systematic review aimed at identifying psychometrically sound PROMs by appraising their measurement properties. Studies concerning the development and/or evaluation of the measurement properties of PROMs used in a TKA population were systematically retrieved via PubMed, Web of Science, Embase, and Scopus. Ratings for methodological quality and measurement properties were conducted according to updated COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. Of the 155 articles on 34 instruments included, nine PROMs met the minimum requirements for psychometric validation and can be recommended to use as measures of TKA outcome: Oxford Knee Score (OKS); OKS-Activity and Participation Questionnaire (OKS-APQ); 12-item short form Knee Injury and Osteoarthritis Outcome (KOOS-12); KOOS Physical function Short form (KOOS-PS); Western Ontario and McMaster Universities Arthritis Index-Total Knee Replacement function short form (WOMAC-TKR); Lower Extremity Functional Scale (LEFS); Forgotten Joint Score (FJS); Patient's Knee Implant Performance (PKIP); and University of California Los Angeles (UCLA) activity score. The pain and function subscales in WOMAC, as well as the pain, function, and quality of life subscales in KOOS, were validated psychometrically as standalone subscales instead of as whole instruments. However, none of the included PROMs have been validated for all measurement properties. Thus, further studies are still warranted to evaluate those PROMs. Use of the other 25 scales and subscales should be tempered until further studies validate their measurement properties. Cite this article: Bone Joint Res 2021;10(3):203-217.

Development and Evaluation of the Boston University Osteoarthritis Functional Pain Short Form (BU-OA-FPS)

BACKGROUND

Pragmatic studies have gained popularity, thus emphasizing the need for patient-reported outcomes (PRO) to be integrated into electronic health records.

OBJECTIVE

This study describes the development of a customized short form from the Boston University Osteoarthritis Functional Assessment PRO (BU-OA-PRO) for a specific pragmatic clinical trial.

METHODS

A Functional Pain Short Form was created from an existing item bank of deidentified data in the BU-OA-PRO. Item response theory (IRT) methods were used to select items. Reliability was measured with the Cronbach alpha, then with IRT simulation methods. To examine validity, ceiling and floor effects, correlations between the short-form scores and scores from the BU-OA-PRO and the Western Ontario McMasters University Osteoarthritis Index (WOMAC) Pain and Difficulty subscales, and the area under the curve (AUC) were calculated. A minimum detectable change at 90% confidence (MDC90) was calculated based on a calibration sample.

RESULTS

The BU-OA-PRO was reduced from 126 items to 10 items to create the BU-OA Functional Pain Short Form (BU-OA-FPS). The Cronbach alpha indicated high internal consistency (0.91), and reliability distribution estimates were 0.96 (uniform) and 0.92 (normal). Low ceiling effects (4.57%) and floor effects (0%) were found. Moderate-to-high correlations between the BU-OA-PRO and BU-OA-FPS were found with WOMAC Pain (BU-OA-FPS = 0.67; BU-OA-PRO = 0.64) and Difficulty (BU-OA-FPS = 0.73; BU-OA-PRO = 0.69) subscales. The correlation between the BU-OA-PRO and BU-OA-FPS was 0.94. The AUC ranged from 0.80 to 0.88. The MDC90 was approximately 6 standardized points.

CONCLUSIONS

The BU-OA-FPS provides reliable and valid measurement of functional pain. Pragmatic studies may consider the BU-OA-FPS for use in electronic health records to capture outcomes.

Detecting functional change in response to exercise in knee osteoarthritis: a comparison of two computerized adaptive tests

Background

The intent of this study was to examine and compare the ability to detect change of two patient reported outcome (PRO) instruments that use a computerized adaptive test (CAT) approach to measurement. The Patient Reported Outcomes Measurement Information System (PROMIS®) Physical Function scale is a generic PRO, while the Osteoarthritis Computerized Adaptive Test (OA-CAT) is an osteoarthritis-specific PRO.

Methods

This descriptive, longitudinal study was conducted in a community setting, involving individuals from the greater Boston area. Inclusion criteria: age > 50, self-reported doctor-diagnosed knee osteoarthritis (OA) and knee pain. The PROMIS® Physical Function CAT and OA-CAT Functional Difficulty scale were administered at baseline and at the conclusion of a 6-week exercise program. Effect sizes (ES) were calculated for both measures, and bootstrap methods were used to construct confidence intervals and to test for significant ES differences between the measures.

Results

The OA-CAT Functional Difficulty scale achieved an ES of 0.62 (0.43, 0.87) compared to the PROMIS® Physical Function CAT ES of 0.42 (0.24, 0.63). ES estimates for the two CAT measures were not statistically different.

Conclusions

The condition-specific OA-CAT and generic PROMIS® Physical Function CAT both demonstrated the ability to detect change in function. While the OA-CAT scale showed larger effect size, no statistically significant difference was found in the effect size estimates for the generic and condition-specific CATs. Both CATs have potential for use in arthritis research.

Trial registration

This trial is registered with ClinicalTrials.gov on 6/21/11 (Identifier NCT01394874)

Electronic Quality of Life Assessment Using Computer-Adaptive Testing

Background

Quality of life (QoL) questionnaires are desirable for clinical practice but can be time-consuming to administer and interpret, making their widespread adoption difficult.

Objective

Our aim was to assess the performance of the World Health Organization Quality of Life (WHOQOL)-100 questionnaire as four item banks to facilitate adaptive testing using simulated computer adaptive tests (CATs) for physical, psychological, social, and environmental QoL.

Methods

We used data from the UK WHOQOL-100 questionnaire (N=320) to calibrate item banks using item response theory, which included psychometric assessments of differential item functioning, local dependency, unidimensionality, and reliability. We simulated CATs to assess the number of items administered before prespecified levels of reliability was met.

Results

The item banks (40 items) all displayed good model fit (P>.01) and were unidimensional (fewer than 5% of t tests significant), reliable (Person Separation Index>.70), and free from differential item functioning (no significant analysis of variance interaction) or local dependency (residual correlations < +.20). When matched for reliability, the item banks were between 45% and 75% shorter than paper-based WHOQOL measures. Across the four domains, a high standard of reliability (alpha>.90) could be gained with a median of 9 items.

Conclusions

Using CAT, simulated assessments were as reliable as paper-based forms of the WHOQOL with a fraction of the number of items. These properties suggest that these item banks are suitable for computerized adaptive assessment. These item banks have the potential for international development using existing alternative language versions of the WHOQOL items.

Sensitivity to Change of a Computer Adaptive Testing Instrument for Outcome Measurement After Hip and Knee Arthroplasty and Periacetabular Osteotomy

Study Design Clinical measurement study. Background Computer adaptive testing (CAT) methods may allow detection of change across the continuum of osteoarthritis (OA) care. Objective To evaluate the sensitivity to change of a self-report OA CAT instrument (OA-CAT) following surgery. Methods Core measures consisted of the 5-item OA-CAT function, pain, and disability scales; the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC); the University of California at Los Angeles activity rating scale; and the Medical Outcomes Study 12-Item Short-Form Health Survey (SF-12), which were administered in 3 samples. Fifty-three patients with hip dysplasia completed the core measures, the Hip disability and Osteoarthritis Outcome Score physical function short form (HOOS-PS), and the Modified Harris Hip Score (MHHS) before periacetabular osteotomy, and at 6 months, 1 year, and 2 years after periacetabular osteotomy. The hip (n = 62) and knee (n = 66) arthroplasty samples completed core measures and the MHHS or the Knee Society's Knee Scoring System at baseline and at 3-month follow-up. Mean change, floor and ceiling effects (percent), and effect size were calculated. Results For osteotomy, the 6-month physical function effect sizes for the OA-CAT, WOMAC, HOOS-PS, MHHS, and SF-12 physical component summary scores were 0.66 (95% confidence interval [CI]: 0.08, 1.61), 0.78 (95% CI: 0.56, 1.10), 0.91 (95% CI: 0.70, 1.21), 0.64 (95% CI: 0.22, 1.07), and 0.87 (95% CI: 0.53, 1.38), respectively. Effect-size trends were all increased at 1 year, and most were level at 2 years. For hip arthroplasty, the OA-CAT, WOMAC, MHHS, and SF-12 effect sizes were 1.27 (95% CI: 0.88, 1.84), 1.50 (95% CI: 1.20, 1.80), 0.68 (95% CI: 0.35, 1.04), and 0.56 (95% CI: 0.29, 0.88), respectively. For knee arthroplasty, the OA-CAT, WOMAC, Knee Society Knee Scoring System, and SF-12 effect sizes were 0.81 (95% CI: 0.56, 1.14), 0.85 (95% CI: 0.61, 1.10), 0.09 (95% CI: -0.22, 0.40), and -0.01 (95% CI: -0.39, 0.31), respectively. The OA-CAT and SF-12 demonstrated smaller ceiling effects than the HOOS-PS and other instruments, especially at 1 and 2 years. Administration time was less for the OA-CAT than for the WOMAC physical function subscale. Conclusion The OA-CAT shows potential for outcome measurement after hip and knee surgery. Larger studies are needed to better understand relative performance. J Orthop Sports Phys Ther 2016;46(9):756-767. Epub 5 Aug 2016. doi:10.2519/jospt.2016.6442.

Sensitivity to Change of Patient-Preference Measures for Pain in Patients With Knee Osteoarthritis: Data From Two Trials

Objective

In osteoarthritis (OA) clinical trials, a pain measure that is most sensitive to change is considered optimal. We compared sensitivity to change of patient‐reported pain outcomes, including a patient‐preference measure (where the patient nominates an activity that aggravates their pain).

Methods

We used data from 2 trials of patients with confirmed (American College of Rheumatology criteria) knee OA: a trial of brace treatment for patellofemoral OA, and a trial of intraarticular steroids in knee OA. Both trials reported an improvement in pain following treatment. Participants rated pain on a 100‐mm visual analog scale (VAS), in the activity that caused them the most knee pain (VAS_NA), as well as completing questions on overall knee pain and the Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire. Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scores were also calculated from the KOOS. Standardized changes in each outcome were generated between treatment and control after 6 weeks intervention in the BRACE trial, and 1–2 weeks following intervention in the steroid trial.

Results

The VAS_NA produced standardized changes following treatment that were at least as large as other pain outcomes. In the BRACE trial, the between‐groups standardized change with the VAS_NA was −0.63, compared with the KOOS pain subscale change of −0.33, and pain in the last week VAS change of −0.56. In the steroid study, within‐group change following treatment in the VAS_NA was −0.60, compared to the last week VAS change of −0.51, and KOOS pain subscale change of −0.58.

Conclusion

Pain on nominated activity appears to be at least as, and in some cases more, sensitive to change than the KOOS/WOMAC questionnaire.

Patient-Reported Measures of Physical Function in Knee Osteoarthritis

Knee osteoarthritis is a common cause of an array of functional limitations in older adults, and the accurate assessment of such limitations is critical for practicing clinicians and scientists. Patient-reported measures are a valuable resource to track the type and severity of limitation, although the psychometric performance of each instrument should be thoroughly evaluated before adoption. This article reviews the validity, reliability, sensitivity to change, and responsiveness of 3 patient-reported measures of physical function: the Western Ontario and McMaster Universities Osteoarthritis Index, the Knee Injury and Osteoarthritis Outcome Score, and the Patient Reported Outcomes Measurement Information System Physical Function scale.

Development of an item list to assess the forgotten joint concept in shoulder patients

BACKGROUND

To generate an item list for the assessment of joint awareness in shoulder patients and to collect patient feedback on the comprehensibility of the items and the forgotten joint concept.

METHODS

Item content was generated on the basis of literature search and expert ratings following a stepwise refinement procedure, including final evaluation by an international expert board (n = 12) including members with various professional backgrounds. Items were translated from English to German and evaluated in 30 German-speaking shoulder patients in Switzerland and 30 shoulder patients in the UK.

RESULTS

Literature search identified 45 questionnaires covering 805 issues potentially relevant for the assessment of joint awareness. Stepwise item selection resulted in 97 items to be evaluated by the international expert board leaving 70 items for collecting patient feedback. The majority of patients indicated that the introductory text explaining the forgotten joint concept was easy or very easy to understand (79.3%) and that the items were clear (91.4%).

CONCLUSION

We developed a list of 70 questions for the assessment of joint awareness in shoulder patients and obtained positive patient feedback for these. In a next step, we will administer the items to a large international patient sample to obtain data for psychometric analysis and development of a measurement model, which is the basis for creation of computer-adaptive assessments or static short-forms.

Development and validation of the computer-administered animated activity questionnaire to measure physical functioning of patients with hip or knee osteoarthritis

BACKGROUND

Physical functioning of patients with hip or knee osteoarthritis is measured by self-report questionnaires and performance-based tests. However, performance-based tests often are not feasible.

OBJECTIVE

The aim of this study was to develop a computer-administered questionnaire (ie, the Animated Activity Questionnaire [AAQ]) to measure physical functioning in patients with hip or knee OA. By showing animations of activities, the influence of the patient's own reference frame is minimized. The AAQ measures the same aspects of physical functioning as performance-based tests do.

DESIGN

This was a development and preliminary validation (cross-sectional) study.

METHODS

A pilot version of the AAQ was developed using motion capture to analyze the movement of a person performing 7 daily activities. Different animations of the same activity were made with 2 to 5 levels of difficulty. For each activity, participants were asked to choose one animation that best corresponds to their own way of performing the activity. A preliminary validation study was performed to compare the AAQ with validated self-report questionnaires (Knee Injury and Osteoarthritis Outcome Score, Hip Disability and Osteoarthritis Outcome Score, and questionnaires on walking, stair climbing, and rising and sitting down) and performance-based tests (walking, Timed "Up & Go" Test, Timed Stair Test) in 33 patients with hip or knee osteoarthritis.

RESULTS

As expected, the AAQ showed a correlation above .70 (.79, 95% confidence interval=.61-.89) with the total score of the performance-based tests. On the subscore level, the results were partly as expected. Fifty-eight percent of the participants preferred the AAQ over self-report questionnaires and performance-based tests.

LIMITATIONS

The findings need to be replicated in larger samples of patients because the sample size of the study was rather small.

CONCLUSION

The AAQ might be a good alternative for measuring physical functioning of patients with hip or knee osteoarthritis. The AAQ can easily be adapted for use in other patient populations. However, further development and validation are needed.

Statistical considerations in the psychometric validation of outcome measures

The evaluation of the outcomes of total knee arthroplasty requires measurement tools that are valid, reliable, and responsive to change. However, the accuracy of any outcome measurement is determined by the validity and reliability of the instrument used. To ensure this accuracy, it is imperative that each instrument used in orthopaedics is free of biases leading to inaccurate estimates of treatment effects. WHERE ARE WE NOW?: Many patient-derived outcome instruments have been developed and tested through the application of the standard assessments that form the basis of classical test theory: validity, reliability, and responsiveness. These assessments determine if the instrument reliably measures what it is intended to measure, and if it captures differences among groups of patients or changes over time. WHERE DO WE NEED TO GO?: Thorough evaluation of the outcome instruments used in orthopaedics is a critical prerequisite for the continued improvement of effective patient care. Additional steps of psychometric testing that are sometimes overlooked include testing for differential item functioning (DIF) and the effects of the mode of administration of the outcome instrument. The use of suitable approaches to test for these potential sources of bias would facilitate the development of more robust outcome assessment in research and clinical practice. HOW DO WE GET THERE?: Testing for DIF, including the effects of mode of administration, may be performed using several analytical approaches. This will allow optimal application of each outcome instrument with respect to patient characteristics, time and mode of the administration, and modification, as necessary.

Development of a self-report physical function instrument for disability assessment: item pool construction and factor analysis

OBJECTIVES

To build a comprehensive item pool representing work-relevant physical functioning and to test the factor structure of the item pool. These developmental steps represent initial outcomes of a broader project to develop instruments for the assessment of function within the context of Social Security Administration (SSA) disability programs.

DESIGN

Comprehensive literature review; gap analysis; item generation with expert panel input; stakeholder interviews; cognitive interviews; cross-sectional survey administration; and exploratory and confirmatory factor analyses to assess item pool structure.

SETTING

In-person and semistructured interviews and Internet and telephone surveys.

PARTICIPANTS

Sample of SSA claimants (n=1017) and a normative sample of adults from the U.S. general population (n=999).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

Model fit statistics.

RESULTS

The final item pool consisted of 139 items. Within the claimant sample, 58.7% were white; 31.8% were black; 46.6% were women; and the mean age was 49.7 years. Initial factor analyses revealed a 4-factor solution, which included more items and allowed separate characterization of: (1) changing and maintaining body position, (2) whole body mobility, (3) upper body function, and (4) upper extremity fine motor. The final 4-factor model included 91 items. Confirmatory factor analyses for the 4-factor models for the claimant and the normative samples demonstrated very good fit. Fit statistics for claimant and normative samples, respectively, were: Comparative Fit Index=.93 and .98; Tucker-Lewis Index=.92 and .98; and root mean square error approximation=.05 and .04.

CONCLUSIONS

The factor structure of the physical function item pool closely resembled the hypothesized content model. The 4 scales relevant to work activities offer promise for providing reliable information about claimant physical functioning relevant to work disability.

Clinical outcomes assessment in clinical trials to assess treatment of femoroacetabular impingement: use of patient-reported outcome measures

Patient-reported outcome measures are an important component of outcomes assessment in clinical trials to assess the treatment of femoroacetabular impingement (FAI). This review of disease-specific measures and instruments used to assess the generic quality of life and physical activity levels of patients with FAI found no conclusive evidence to support a single disease-specific questionnaire. Using a systematic review of study methodology, the Copenhagen Hip and Groin Outcome Score and the 33-item International Hip Outcome Tool scored the best. Nevertheless, both of these instruments were developed recently and have not been established in the literature. Although currently used generic and activity-level measures have limitations, as well, they should be considered, depending on the specific goals of the study. Additional research is needed to assess the properties of these measures fully when used to evaluate patients with FAI.

Modern psychometrics applied in rheumatology--a systematic review

BACKGROUND

Although item response theory (IRT) appears to be increasingly used within health care research in general, a comprehensive overview of the frequency and characteristics of IRT analyses within the rheumatic field is lacking. An overview of the use and application of IRT in rheumatology to date may give insight into future research directions and highlight new possibilities for the improvement of outcome assessment in rheumatic conditions. Therefore, this study systematically reviewed the application of IRT to patient-reported and clinical outcome measures in rheumatology.

METHODS

Literature searches in PubMed, Scopus and Web of Science resulted in 99 original English-language articles which used some form of IRT-based analysis of patient-reported or clinical outcome data in patients with a rheumatic condition. Both general study information and IRT-specific information were assessed.

RESULTS

Most studies used Rasch modeling for developing or evaluating new or existing patient-reported outcomes in rheumatoid arthritis or osteoarthritis patients. Outcomes of principle interest were physical functioning and quality of life. Since the last decade, IRT has also been applied to clinical measures more frequently. IRT was mostly used for evaluating model fit, unidimensionality and differential item functioning, the distribution of items and persons along the underlying scale, and reliability. Less frequently used IRT applications were the evaluation of local independence, the threshold ordering of items, and the measurement precision along the scale.

CONCLUSION

IRT applications have markedly increased within rheumatology over the past decades. To date, IRT has primarily been applied to patient-reported outcomes, however, applications to clinical measures are gaining interest. Useful IRT applications not yet widely used within rheumatology include the cross-calibration of instrument scores and the development of computerized adaptive tests which may reduce the measurement burden for both the patient and the clinician. Also, the measurement precision of outcome measures along the scale was only evaluated occasionally. Performed IRT analyses should be adequately explained, justified, and reported. A global consensus about uniform guidelines should be reached concerning the minimum number of assumptions which should be met and best ways of testing these assumptions, in order to stimulate the quality appraisal of performed IRT analyses.

Reliability, validity and administrative burden of the community reintegration of injured service members computer adaptive test (CRIS-CAT)"

BACKGROUND

The Computer Adaptive Test version of the Community Reintegration of Injured Service Members measure (CRIS-CAT) consists of three scales measuring Extent of, Perceived Limitations in, and Satisfaction with community integration. The CRIS-CAT was developed using item response theory methods. The purposes of this study were to assess the reliability, concurrent, known group and predictive validity and respondent burden of the CRIS-CAT.The CRIS-CAT was developed using item response theory methods. The purposes of this study were to assess the reliability, concurrent, known group and predictive validity and respondent burden of the CRIS-CAT.

METHODS

This was a three-part study that included a 1) a cross-sectional field study of 517 homeless, employed, and Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans; who completed all items in the CRIS item set, 2) a cohort study with one year follow-up study of 135 OEF/OIF Veterans, and 3) a 50-person study of CRIS-CAT administration. Conditional reliability of simulated CAT scores was calculated from the field study data, and concurrent validity and known group validity were examined using Pearson product correlations and ANOVAs. Data from the cohort were used to examine the ability of the CRIS-CAT to predict key one year outcomes. Data from the CRIS-CAT administration study were used to calculate ICC (2,1) minimum detectable change (MDC), and average number of items used during CAT administration.

RESULTS

Reliability scores for all scales were above 0.75, but decreased at both ends of the score continuum. CRIS-CAT scores were correlated with concurrent validity indicators and differed significantly between the three Veteran groups (P < .001). The odds of having any Emergency Room visits were reduced for Veterans with better CRIS-CAT scores (Extent, Perceived Satisfaction respectively: OR = 0.94, 0.93, 0.95; P < .05). CRIS-CAT scores were predictive of SF-12 physical and mental health related quality of life scores at the 1 year follow-up. Scales had ICCs >0.9. MDCs were 5.9, 6.2, and 3.6, respectively for Extent, Perceived and Satisfaction subscales. Number of items (mn, SD) administered at Visit 1 were 14.6 (3.8) 10.9 (2.7) and 10.4 (1.7) respectively for Extent, Perceived and Satisfaction subscales.

CONCLUSION

The CRIS-CAT demonstrated sound measurement properties including reliability, construct, known group and predictive validity, and it was administered with minimal respondent burden. These findings support the use of this measure in assessing community reintegration.

Staging activity limitation and participation restriction in elderly community-dwelling persons according to difficulties in self-care and domestic life functioning

OBJECTIVE

This study aimed to describe the conceptual foundation and development of an activity limitation and participation restriction staging system for community-dwelling people 70 yrs or older according to the severity and types of self-care (activities of daily living [ADLs]) and domestic life (instrumental ADLs (IADLs)) limitations experienced.

DESIGN

Data from the second Longitudinal Study of Aging (N = 9447) were used to develop IADL stages through the analyses of self- and proxy-reported difficulties in performing IADLs. An analysis of activity limitation profiles identified hierarchical thresholds of difficulty that defined each stage. IADL stages are combined with ADL stages to profile status for independent living.

RESULTS

IADL stages define five ordered thresholds of increasing activity limitations and a ``not relevant'' stage for those who normally have someone else do those activities. Approximately 42% of the population experience IADL limitations. To achieve a stage, a person must meet or exceed stage-specific thresholds of retained functioning defined for each activity. Combined ADL and IADL stages define 29 patterns of activity limitations expressing the individual's potential for participating in life situations pertinent to self-care and independent community life.

CONCLUSIONS

ADL and IADL stages can serve to distinguish between groups of people according to both severity and the types of limitations experienced during home or outpatient assessments, in population surveillance, and in research.