A comparison of computerized adaptive testing and fixed-length short forms for the Prosthetic Limb Users Survey of Mobility (PLUS-MTM)

Computer Adaptive vs. Non-adaptive Medical Progress Testing: Feasibility, Test Performance, and Student Experiences

Background

Computerized adaptive testing tailors test items to students' abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students' performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing. Therefore, we assessed the correlation between CA-PT and conventional PT test performance and explored the feasibility and student experiences of CA-PT in a large medical cohort.

Methods

In this cross-over study medical students (n = 1432) of three Dutch medical schools participated in both a conventional PT and CA-PT. They were stratified to start with either a conventional PT or CA-PT to determine test performance. Student motivation, engagement and experiences were assessed by questionnaires in students from seven Dutch medical schools. Parallel-forms reliability was assessed using the Pearson correlation coefficient.

Results

A strong correlation was found (0.834) between conventional PT and CA-PT test performance. The CA-PT was administered without system performance issues and was completed in a median time of 83 minutes (67-102 minutes). Questionnaire response rate was 31.7% (526/1658). Despite a higher experienced difficulty, most students reported persistence, adequate task management and good focus during the CA-PT.

Conclusions

CA-PT provides a reliable estimation of students' ability level in less time than a conventional non-adaptive PT and is feasible in students throughout the entire medical curriculum. Despite the strong correlation between PT scores, students found the CA-PT more challenging.

Rasch validation of the German translation of the Prosthetic Limb Users Survey of Mobility short forms in people with lower limb amputation

BACKGROUND

Prostheses are a primary rehabilitative intervention for people after lower limb amputation. To appropriately measure the effectiveness of prosthetic interventions, valid and reliable measures of prosthetic mobility are required. The Prosthetic Limb Users Survey of Mobility (PLUS-M) is a promising instrument for measuring prosthesis users' mobility. However, German translations of the PLUS-M short forms have not yet been validated.

OBJECTIVES

Rasch validation of the German translation of the PLUS-M short forms in people with lower limb amputation.

STUDY DESIGN

This study is based on a cross-sectional survey of prosthesis-using lower limb amputees from a nation-wide cohort.

METHODS

PLUS-M data (the 7-item and the 12-item shortforms; PLUS-M-7 and PLUS-M-12, respectively) from 194 lower limb prosthesis users were subjected to Rasch analysis, an advanced statistical method for assessing if the measurement properties of a questionnaire comply with a wide spectrum of psychometric requirements.

RESULTS

Analysis showed appropriate rating scale functioning, good internal construct validity (item fit), unidimensionality, and good targeting of the PLUS-M-7 and PLUS-M-12 short forms. Moreover, the greater conditional measurement precision of PLUS-M-12 (regarding higher test information and lower standard error of mobility estimates) was quantified.

CONCLUSIONS

Rasch analysis of the German translation of both PLUS-M short forms showed good psychometric qualities. In addition, our study showed that test scores from the PLUS-M-12 are more accurate. Therefore, the PLUS-M-12 is recommended for individual-level clinical applications (e.g., classification or change assessment).

Computerized-adaptive testing versus short forms for pediatric inflammatory bowel disease patient-reported outcome assessment

Introduction

Computerized-adaptive testing (CAT) may increase reliability or reduce respondent burden for assessing patient-reported outcomes compared with static short forms (SFs). We compared CAT versus SF administration of the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Pediatric measures in pediatric inflammatory bowel disease (IBD).

Methods

Participants completed 4-item CAT, 5- or 6-item CAT, and 4-item SF versions of the PROMIS Pediatric measures. We compared average T-scores, intra-class correlations (ICCs), floor and ceiling effects, and standard error of measurement (SEM) across forms, along with mean effect sizes between active versus quiescent IBD disease activity groups.

Results

Average PROMIS T-scores across forms were <3 points (minimally important difference) of each other. All forms correlated highly with each other (ICCs ≥0.90) and had similar ceiling effects, but the CAT-5/6 had lower floor effects. The CAT-5/6 had lower SEM than the CAT-4 and SF-4, and the CAT-4 had a lower SEM than the SF-4. Mean effect sizes were similar across forms when contrasting disease activity groups.

Conclusions

The CAT and SF forms produced similar score results, but the CAT had better precision and lower floor effects. Researchers should consider PROMIS pediatric CAT if they anticipate that their sample will skew toward symptom extremes.

Development of a candidate item bank for measuring mobility of lower limb orthosis users

BACKGROUND

Orthoses are often prescribed to improve mobility of people with chronic health conditions that affect lower limb function. Patient-reported survey instruments can be used to measure aspects of mobility that cannot be easily assessed in clinical or research settings. A population-specific item bank could be designed to measure aspects of mobility that are most important to lower limb orthosis users and used to evaluate the effects of orthoses.

OBJECTIVE

To develop items for a new survey instrument to measure mobility of lower limb orthosis users.

DESIGN

Survey items were developed using a qualitative item review process.

SETTING

Focus groups were held by video conferencing. Cognitive interviews were conducted by telephone.

PARTICIPANTS

Focus group and cognitive interview participants were adults with at least 6 months of experience using a lower limb orthosis that extended from the foot to a level above the ankle.

METHODS

Research methods included focus groups with lower limb orthosis users, an item generation and reduction process that involved a stakeholder advisory panel, and cognitive interviews with target respondents.

RESULTS

A total of 1180 extant items were identified in a literature review. Focus group participants (n = 29) provided feedback that informed the suitability of a construct definition and conceptual model. An advisory panel contributed to the selection of 118 candidate items for measuring orthotic mobility. Feedback from cognitive interview participants (n = 30) informed removal or revision of problematic items, resulting in a candidate bank of 100 mobility items.

CONCLUSIONS

The rigorous qualitative methods applied here resulted in a large set of candidate items that spanned a range of situations relevant to moving with a lower limb orthosis. Next steps include administration of the candidate items to a large sample of lower limb orthosis users and calibration of the item bank.

Development of an item bank for measuring prosthetic mobility in people with lower limb amputation: The Prosthetic Limb Users Survey of Mobility (PLUS-M)

BACKGROUND

Achieving mobility with a prosthesis is a common post-amputation rehabilitation goal and primary outcome in prosthetic research studies. Patient-reported outcome measures (PROMs) available to measure prosthetic mobility have practical and psychometric limitations that inhibit their use in clinical care and research.

OBJECTIVE

To develop a brief, clinically meaningful, and psychometrically robust PROM to measure prosthetic mobility.

DESIGN

A cross-sectional study was conducted to administer previously developed candidate items to a national sample of lower limb prosthesis users. Items were calibrated to an item response theory model and two fixed-length short forms were created. Instruments were assessed for readability, effective range of measurement, agreement with the full item bank, ceiling and floor effects, convergent validity, and known groups validity.

SETTING

Participants were recruited using flyers posted in hospitals and prosthetics clinics across the United States, magazine advertisements, notices posted to consumer websites, and direct mailings.

PARTICIPANTS

Adult prosthesis users (N = 1091) with unilateral lower limb amputation due to traumatic or dysvascular causes.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Candidate items (N = 105) were administered along with the Patient Reported Outcome Measurement Information System Brief Profile, Prosthesis Evaluation Questionnaire - Mobility Subscale, and Activities-Specific Balance Confidence Scale, and questions created to characterize respondents.

RESULTS

A bank of 44 calibrated self-report items, termed the Prosthetic Limb Users Survey of Mobility (PLUS-M), was produced. Clinical and statistical criteria were used to select items for 7- and 12-item short forms. PLUS-M instruments had an 8th grade reading level, measured with precision across a wide range of respondents, exhibited little-to-no ceiling or floor effects, correlated expectedly with scores from existing PROMs, and differentiated between groups of respondents expected to have different levels of mobility.

CONCLUSION

The PLUS-M appears to be well suited to measuring prosthetic mobility in people with lower limb amputation. PLUS-M instruments are recommended for use in clinical and research settings.

Comparison of Patient-Reported Outcomes Measurement Information System Computerized Adaptive Testing Versus Fixed Short Forms in Juvenile Myositis

OBJECTIVE

Patient-Reported Outcomes Measurement Information System (PROMIS) measures can be administered via computerized adaptive testing (CAT) or fixed short forms (FSFs), but the empirical benefits of CAT versus FSFs are unknown in juvenile myositis (JM). The present study was undertaken to assess whether PROMIS CAT is feasible, precise, correlated with FSFs, and less prone to respondent burden and floor/ceiling effects than FSFs in JM.

METHODS

Patients 8-17 years of age (self-report and parent proxy) and parents of patients 5-7 years of age (only parent proxy) completed PROMIS fatigue, pain interference, upper extremity function, mobility, anxiety, and depressive symptoms measures. Pearson correlations, paired t-tests, and Cohen's d were calculated between PROMIS CAT and FSFs. McNemar's test assessed floor/ceiling effects between CAT and FSFs. Precision and respondent burden were examined across the T score range.

RESULTS

Data from 67 patient-parent dyads were analyzed. CAT and FSF mean scores did not significantly differ except in parent proxy anxiety and fatigue (effect size 0.23 and 0.19, respectively). CAT had less pronounced floor/ceiling effects at the less symptomatic extreme in all domains except self-report anxiety. Increased item burden and higher SEs were seen in less symptomatic scorers for CAT. Modified stopping rules limiting CAT item administration did not decrease precision.

CONCLUSION

PROMIS CAT appears to be feasible and correlated with FSFs. CAT had less pronounced floor/ceiling effects, allowing detection of individual differences in less symptomatic patients. Modified stopping rules for CAT may decrease respondent burden. CAT can be considered for long-term follow-up of JM patients.

From outcome measurement to improving health outcomes after lower limb amputation-A narrative review exploring outcome measurement from a clinical practice perspective

Outcome measurement is essential to understand the impact of clinical interventions and the performance of services. Despite national and professional body encouragement, and successful examples of system level outcome measurement within some health care settings, many barriers still exist preventing outcome measurement from becoming embedded in clinical practice. This paper presents a narrative review which aims to describe the state of the outcome measurement evidence base in prosthetic rehabilitation, as applied in clinical practice, with a view to identifying areas for future work aimed at making outcome measurement in prosthetic rehabilitation a meaningful reality. A literature search of four databases was undertaken, following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis principals appropriate to narrative reviews, and using the search terms outcome, measur*, tool, scale, instrument, prosthe*, amput* and limb loss. A total of 1116 papers were identified. Following screening 35 papers, focusing on four main themes, were included in the review. Themes were: 1) What outcome domains should be measured? 2) How can these outcome domains be measured? 3) What are the barriers to outcome measurement? and 4) What can be learnt from examples of ROM in prosthetic rehabilitation? Findings suggest that successful outcome measurement is multifaceted. Understanding and embedding value at every step appears to be key to success. Addressing the questions of 'what' outcome domains to measure and 'how' to measure them, may help establish consensus. Routine outcome measurement practice at the clinical level should ensure data collection is valuable to clinical practice, makes use of information technology solutions and has organisational engagement.

Japanese translation and linguistic validation of the Prosthetic Limb Users Survey of Mobility (PLUS-M)

BACKGROUND

Self-report survey instruments can be used to improve clinical care for lower-limb prosthesis users on a global scale by pooling comparable health outcomes data from multiple countries. The language translation process is critical to the quality and comparability of a translated survey instrument.

OBJECTIVES

The goals of this research were to translate the Prosthetic Limb Users Survey of Mobility (PLUS-M) item bank from English to Japanese using established guidelines, and linguistically validate the translated instrument by assessing its clarity, comprehension, and cultural applicability with Japanese prosthesis users.

STUDY DESIGN

Instrument translation and qualitative interviews.

METHODS

The translation process included two forward translations, reconciliation, backward translation, and review by clinical experts in Japan. Adult lower-limb prosthesis users participated in a linguistic validation study by responding to translated survey items and providing item-by-item feedback through cognitive interviews.

RESULTS

Following expert reviews, translated items were classified as grammatically and contextually unchanged (n = 21), minor revision required (n = 19), major revision required (n = 3), or removed (n = 1). Cognitive interviews with 10 participants indicated that additional revisions were required (n = 4). Items were revised and retested until it was determined that they were clear, well understood, and culturally applicable.

CONCLUSIONS

Use of a multistep translation and linguistic validation processes resulted in a linguistically comparable Japanese translation of the PLUS-M item bank. Japanese translations of the PLUS-M 7- and 12-item short forms were created for use in clinical practice and research. Availability of these linguistically comparable instruments is expected to facilitate opportunities for international comparison of prosthetic mobility data.

Validation and revision of the questionnaire to explore human factors and their technical potential for lower limb prosthetics

BACKGROUND

A variety of instruments exist to measure human factors for lower limb amputation and prosthesis research. Yet, there is no valid or reliable tool available that focuses on technical potentials.

OBJECTIVE

This study aimed to validate and revise the Questionnaire to Explore Human Factors and their Technical Potential.

STUDY DESIGN

Cross-sectional study METHODS:: A total of 150 persons with lower limb amputation from Germany participated in the study. Statistical properties, including Cronbach's alpha, item difficulty, item-total correlation, and distribution of missing values were calculated. Thresholds for acceptable psychometric properties were defined, unsuitable items were removed, and problematic items were reviewed regarding formulation.

RESULTS

The Cronbach's alpha for subscales within the Questionnaire to Explore Human Factors and their Technical Potential were between 0.72 and 0.89. A total of 56 items showed acceptability, and 14 items had problematic item property values. Four of those items were reformulated, five were excluded, six were left in the scale, and an additional one was added to the scale.

CONCLUSION

Evaluation of the Questionnaire to Explore Human Factors and their Technical Potential indicates it exhibits good internal consistency and acceptable psychometric properties. The scale was revised and is recommended to explore aspects of technical prosthesis development.

CLINICAL RELEVANCE

Our results show that the revised Questionnaire to Explore Human Factors and their Technical Potential may serve as a reliable and valid means, when designing prostheses, both during development and clinical evaluations and fittings, to assess the technical potential of lower limb prostheses directly according to the needs of users with lower-limb amputations.

Improving the utility of the European Health Literacy Survey Questionnaire: a computerized adaptive test for patients with stroke

PURPOSE

Health literacy among patients is crucial for effective stroke management. The European Health Literacy Survey Questionnaire is a theory-based measure that comprehensively captures 12 domains of health literacy. We aimed to develop a computerized adaptive test of the European Health Literacy Survey Questionnaire to efficiently assess health literacy among patients with stroke.

MATERIALS AND METHODS

The European Health Literacy Survey Questionnaire data of 311 patients and item parameters were retrieved from a Rasch validation study. Real data simulations were performed to develop a computerized adaptive test of the European Health Literacy Survey Questionnaire and explore its efficiency and reliability.

RESULTS

The computerized adaptive test of the European Health Literacy Survey Questionnaire displayed suitable reliability in all 12 domains (0.72-0.84) with a mean test length of 17 items (36.2% of the 47-item European Health Literacy Survey Questionnaire).

CONCLUSIONS

Our findings indicate that the computerized adaptive test of the European Health Literacy Survey Questionnaire, which assesses 12 domains of health literacy among patients with stroke in a timely and precise fashion, is efficient and reliable.Implications for rehabilitationThe computerized adaptive test of the European Health Literacy Survey Questionnaire assesses the subjective fit of personal health literacy competencies to environmental demands, providing insight into patient strengths and weaknesses when dealing with health tasks and interacting with health systems.The computerized adaptive test of the European Health Literacy Survey Questionnaire assesses the 12 domains of health literacy among patients with stroke in a timely and precise manner.Our study has demonstrated the utility of the computerized adaptive test of the European Health Literacy Survey Questionnaire in reducing the assessment burden of patients in clinical practice.Developing of the computerized adaptive test of the European Health Literacy Survey Questionnaire provided findings that may benefit researchers and clinicians interested in developing efficient outcome measures.

Psychometric Properties of Functional, Ambulatory, and Quality of Life Instruments in Lower Limb Amputees: A Systematic Review

OBJECTIVE

Summarize the psychometric properties of functional, ambulatory, and quality of life instruments among adult lower limb amputees, highlighting evidence deemed generalizable to the United States Medicare population.

DATA SOURCES

Six databases and existing systematic reviews through October 30, 2017. Searches included terms for lower limb amputation or prostheses and outcome measures in humans, without language restriction.

STUDY SELECTION

We included peer-reviewed studies of at least 20 adults (≥18 years) with lower limb amputation. Eligible studies reported on psychometric properties of functional, ambulatory, or quality of life instruments. Fifty-three of 425 retrieved articles (12%) met criteria.

DATA EXTRACTION

Study characteristics and psychometric property data (validity, reliability, responsiveness, minimum detectable change, minimal important difference, or floor or ceiling effect) were extracted into a customized form based on standardized criteria. All extracted data were confirmed by 2 experts in systematic review and rehabilitation outcome measurement. Instruments were categorized regarding having been validated and found reliable. Other reported psychometric properties were recorded. Studies were also assessed for applicability to the Medicare population based on age and amputation etiology (dysvascular).

DATA SYNTHESIS

Fifty-six studies (in 53 articles) reported psychometric properties of 50 instruments. There is evidence for both validity and reliability for 30 instruments, 17 of which have evidence that was deemed generalizable to the Medicare population. Most of the remaining instruments have evidence of either validity or reliability, but not both. Twelve instruments have been assessed specifically among lower limb amputees prior to prosthesis prescription. Thirteen instruments have been assessed regarding their predictive properties for future outcomes.

CONCLUSIONS

Numerous instruments assessing ambulation, function, quality of life, and other patient-centered outcomes have evidence of validity and reliability for adults with lower limb amputations. Researchers and clinicians should use validated, reliable instruments when feasible. Many existing and new instruments require validation for use with lower limb amputees.

Mobility Analysis of AmpuTees II: Comorbidities and Mobility in Lower Limb Prosthesis Users

Supplemental digital content is available in the text.

Objective

The aim of the study was to determine the impact of comorbidities on mobility in patients with lower limb prostheses.

Design

Cohort database chart review was conducted to examine mobility in lower limb prosthesis users grouped according to comorbidities. Regression models were used to determine significant predictor comorbidities for mobility. General linear univariate models were implemented to investigate differences in mobility among cohorts (N = 596).

Results

Patient age and history of stroke, peripheral vascular disease, and anxiety/panic disorders were predictors of decreased mobility. After adjusting for covariates, the differences in mobility reported by patients older than 65 yrs was compared with those younger than 65 yrs; in addition, we compared those with a history of peripheral vascular disease with those without. The comparative analyses for both categories did not satisfy the minimal clinically important difference. There were no significant differences when comparing overall comorbid health after adjusting for covariates.

Conclusions

Clinicians should consider patient age and history of stroke, peripheral vascular disease, or anxiety/panic disorders when optimizing a lower limb prosthesis users' mobility because these variables may be predictive of modest but clinically meaningful decreased prosthetic mobility. By contrast, common comorbid health conditions such as arthritis, chronic obstructive pulmonary disease, congestive heart failure, and diabetes do not seem predictive of decreased mobility among lower limb prosthesis users.