An application of computerised adaptive testing for measuring health status in patients with knee osteoarthritis

Application of the Rasch measurement model in rehabilitation research and practice: early developments, current practice, and future challenges

The application of the Rasch measurement model in rehabilitation is now well established. Both its dichotomous and polytomous forms provide for transforming ordinal scales into interval-level measures, consistent with the requirements of fundamental measurement. The growth of applying the model in rehabilitation spans 30 years, during which both the protocol has steadily developed and several software packages have emerged that provide for analysis, together with the "R" language that has an increasing set of codes for applying the model. This article reviews that development and highlights current practice requirements, including those for providing the relevant information for the methods, and what is expected of the analysis. In addition, this provides a worked example and looks at the remaining issues and current developments of its application.

Classification and Regression Tree and Computer Adaptive Testing in Cardiac Rehabilitation: Instrument Validation Study

Background

There is a need for shorter-length assessments that capture patient questionnaire data while attaining high data quality without an undue response burden on patients. Computerized adaptive testing (CAT) and classification and regression tree (CART) methods have the potential to meet these needs and can offer attractive options to shorten questionnaire lengths.

Objective

The objective of this study was to test whether CAT or CART was best suited to reduce the number of questionnaire items in multiple domains (eg, anxiety, depression, quality of life, and social support) used for a needs assessment procedure (NAP) within the field of cardiac rehabilitation (CR) without the loss of data quality.

Methods

NAP data of 2837 CR patients from a multicenter Cardiac Rehabilitation Decision Support System (CARDSS) Web-based program was used. Patients used a Web-based portal, MyCARDSS, to provide their data. CAT and CART were assessed based on their performances in shortening the NAP procedure and in terms of sensitivity and specificity.

Results

With CAT and CART, an overall reduction of 36% and 72% of NAP questionnaire length, respectively, was achieved, with a mean sensitivity and specificity of 0.765 and 0.817 for CAT, 0.777 and 0.877 for classification trees, and 0.743 and 0.40 for regression trees, respectively.

Conclusions

Both CAT and CART can be used to shorten the questionnaires of the NAP used within the field of CR. CART, however, showed the best performance, with a twice as large overall decrease in the number of questionnaire items of the NAP compared to CAT and the highest sensitivity and specificity. To our knowledge, our study is the first to assess the differences in performance between CAT and CART for shortening questionnaire lengths. Future research should consider administering varied assessments of patients over time to monitor their progress in multiple domains. For CR professionals, CART integrated with MyCARDSS would provide a feedback loop that informs the rehabilitation progress of their patients by providing real-time patient measurements.

World Health Organization disability assessment schedule 2.0: An international systematic review

PURPOSE

This systematic review examines research and practical applications of the World Health Organization Disability Assessment Schedule (WHODAS 2.0) as a basis for establishing specific criteria for evaluating relevant international scientific literature. The aims were to establish the extent of international dissemination and use of WHODAS 2.0 and analyze psychometric research on its various translations and adaptations. In particular, we wanted to highlight which psychometric features have been investigated, focusing on the factor structure, reliability, and validity of this instrument.

METHOD

Following Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology, we conducted a search for publications focused on "whodas" using the ProQuest, PubMed, and Google Scholar electronic databases.

RESULTS

We identified 810 studies from 94 countries published between 1999 and 2015. WHODAS 2.0 has been translated into 47 languages and dialects and used in 27 areas of research (40% in psychiatry).

CONCLUSIONS

The growing number of studies indicates increasing interest in the WHODAS 2.0 for assessing individual functioning and disability in different settings and individual health conditions. The WHODAS 2.0 shows strong correlations with several other measures of activity limitations; probably due to the fact that it shares the same disability latent variable with them. Implications for Rehabilitation WHODAS 2.0 seems to be a valid, reliable self-report instrument for the assessment of disability. The increasing interest in use of the WHODAS 2.0 extends to rehabilitation and life sciences rather than being limited to psychiatry. WHODAS 2.0 is suitable for assessing health status and disability in a variety of settings and populations. A critical issue for rehabilitation is that a single "minimal clinically important .difference" score for the WHODAS 2.0 has not yet been established.

Item Response Theory and Computerized Adaptive Testing for Orthopaedic Outcomes Measures

The use of patient-reported outcome measures (PROMs) to objectively and quantifiably assess patient symptomatology allows tracking of symptoms over time, measurement of the effect of healthcare interventions, and performance of cost-effectiveness analyses to assess and compare the value of treatment options. Many of the PROMs historically used had limited versatility because they were developed using classical test theory, which generates static tests that are not comparable with other measures assessing similar outcomes. Recently developed PROMs, however, were designed according to the principles of the newer item response theory (IRT), which allows for the creation of dynamic instruments deliverable in a variety of forms that are readily comparable with similar measures. IRT also enables computerized adaptive testing to decrease the burden of using PROMs by allowing rapid and complete data acquisition. IRT-based instruments are suitable for patient care and research and have been validated in a variety of populations, many of which are relevant to orthopaedic populations.

A simplified version of the maximum information per time unit method in computerized adaptive testing

In this article, we propose a simplified version of the maximum information per time unit method (MIT; Fan, Wang, Chang, & Douglas, Journal of Educational and Behavioral Statistics 37: 655-670, 2012), or MIT-S, for computerized adaptive testing. Unlike the original MIT method, the proposed MIT-S method does not require fitting a response time model to the individual-level response time data. It is also computationally efficient. The performance of the MIT-S method was compared against that of the maximum information (MI) method in terms of measurement precision, testing time saving, and item pool usage under various item response theory (IRT) models. The results indicated that when the underlying IRT model is the two- or three-parameter logistic model, the MIT-S method maintains measurement precision and saves testing time. It performs similarly to the MI method in exposure control; both result in highly skewed item exposure distributions, due to heavy reliance on the highly discriminating items. If the underlying model is the one-parameter logistic (1PL) model, the MIT-S method maintains the measurement precision and saves a considerable amount of testing time. However, its heavy reliance on time-saving items leads to a highly skewed item exposure distribution. This weakness can be ameliorated by using randomesque exposure control, which successfully balances the item pool usage. Overall, the MIT-S method with randomesque exposure control is recommended for achieving better testing efficiency while maintaining measurement precision and balanced item pool usage when the underlying IRT model is 1PL.

The assessment of outcome after total knee arthroplasty: are we there yet?

The routine use of patient reported outcome measures (PROMs) in evaluating the outcome after arthroplasty by healthcare organisations reflects a growing recognition of the importance of patients' perspectives in improving treatment. Although widely embraced in the NHS, there are concerns that PROMs are being used beyond their means due to a poor understanding of their limitations. This paper reviews some of the current challenges in using PROMs to evaluate total knee arthroplasty. It highlights alternative methods that have been used to improve the assessment of outcome.

Applying computerized adaptive testing to the Negative Acts Questionnaire-Revised: Rasch analysis of workplace bullying

Background

Workplace bullying is a prevalent problem in contemporary work places that has adverse effects on both the victims of bullying and organizations. With the rapid development of computer technology in recent years, there is an urgent need to prove whether item response theory–based computerized adaptive testing (CAT) can be applied to measure exposure to workplace bullying.

Objective

The purpose of this study was to evaluate the relative efficiency and measurement precision of a CAT-based test for hospital nurses compared to traditional nonadaptive testing (NAT). Under the preliminary conditions of a single domain derived from the scale, a CAT module bullying scale model with polytomously scored items is provided as an example for evaluation purposes.

Methods

A total of 300 nurses were recruited and responded to the 22-item Negative Acts Questionnaire-Revised (NAQ-R). All NAT (or CAT-selected) items were calibrated with the Rasch rating scale model and all respondents were randomly selected for a comparison of the advantages of CAT and NAT in efficiency and precision by paired t tests and the area under the receiver operating characteristic curve (AUROC).

Results

The NAQ-R is a unidimensional construct that can be applied to measure exposure to workplace bullying through CAT-based administration. Nursing measures derived from both tests (CAT and NAT) were highly correlated (r=.97) and their measurement precisions were not statistically different (P=.49) as expected. CAT required fewer items than NAT (an efficiency gain of 32%), suggesting a reduced burden for respondents. There were significant differences in work tenure between the 2 groups (bullied and nonbullied) at a cutoff point of 6 years at 1 worksite. An AUROC of 0.75 (95% CI 0.68-0.79) with logits greater than –4.2 (or >30 in summation) was defined as being highly likely bullied in a workplace.

Conclusions

With CAT-based administration of the NAQ-R for nurses, their burden was substantially reduced without compromising measurement precision.

Instrument of Professional Attitude for Student Nurses (IPASN): a confirmatory factor analytic study

The purpose of this study is to confirm the indentified factor structure of the Instrument of Professional Attitude for Student Nurses (IPASN) developed by Hisar et al. (2010). Confirmatory factor analysis (CFA) was performed using 1039 voluntary students from 23 schools which were randomly selected from institutions offering an undergraduate education in nursing in the 2009-10 academic year in Turkey. The CFA yielded high fit indices, thus confirming the factor structure of the IPASN with 28 items and 8 factors. The IPASN was found to be appropriate, both theoretically and statistically. Due to these results, it is recommended that the IPASN scale could be used in future studies in Turkey. It is also recommended that the scale could be adapted and used in other cultures.

A web-based computerized adaptive testing (CAT) to assess patient perception in hospitalization

BACKGROUND

Many hospitals have adopted mobile nursing carts that can be easily rolled up to a patient's bedside to access charts and help nurses perform their rounds. However, few papers have reported data regarding the use of wireless computers on wheels (COW) at patients' bedsides to collect questionnaire-based information of their perception of hospitalization on discharge from the hospital.

OBJECTIVE

The purpose of this study was to evaluate the relative efficiency of computerized adaptive testing (CAT) and the precision of CAT-based measures of perceptions of hospitalized patients, as compared with those of nonadaptive testing (NAT). An Excel module of our CAT multicategory assessment is provided as an example.

METHOD

A total of 200 patients who were discharged from the hospital responded to the CAT-based 18-item inpatient perception questionnaire on COW. The numbers of question administrated were recorded and the responses were calibrated using the Rasch model. They were compared with those from NAT to show the advantage of CAT over NAT.

RESULTS

Patient measures derived from CAT and NAT were highly correlated (r = 0.98) and their measurement precisions were not statistically different (P = .14). CAT required fewer questions than NAT (an efficiency gain of 42%), suggesting a reduced burden for patients. There were no significant differences between groups in terms of gender and other demographic characteristics.

CONCLUSIONS

CAT-based administration of surveys of patient perception substantially reduced patient burden without compromising the precision of measuring patients' perceptions of hospitalization. The Excel module of animation-CAT on the wireless COW that we developed is recommended for use in hospitals.