An item response theory-based pain item bank can enhance measurement precision

Application of the Rasch measurement model in rehabilitation research and practice: early developments, current practice, and future challenges

The application of the Rasch measurement model in rehabilitation is now well established. Both its dichotomous and polytomous forms provide for transforming ordinal scales into interval-level measures, consistent with the requirements of fundamental measurement. The growth of applying the model in rehabilitation spans 30 years, during which both the protocol has steadily developed and several software packages have emerged that provide for analysis, together with the "R" language that has an increasing set of codes for applying the model. This article reviews that development and highlights current practice requirements, including those for providing the relevant information for the methods, and what is expected of the analysis. In addition, this provides a worked example and looks at the remaining issues and current developments of its application.

Examining the validity and reliability of the revised developmental work personality scale (RDWPS)-traditional Chinese version

Background

Work personality was found to relate to successful work adjustment, job readiness, work motivation and job maintenance. The revised developmental work personality scale (RDWPS) is a self-reported assessment to evaluate the work personality of the examinee which further psychometric study is required and needs to be applied to different culture. The aim of this study was to examine the theoretical structure, validity, and reliability of the traditional Chinese version of the RDWPS.

Methods

The subjects were 113 university students with no known physical or mental illness who aged between 19 and 22. Cluster analysis was used to examine the theoretical structure. Rasch analysis was applied to examine the psychometric properties.

Results

We dropped the unfit three items and found the 11 items of traditional Chinese version of the RDWPS was adequate to fit the theoretical construct. The results of the Rasch analysis showed that the 11 items with a three-point rating scale of the traditional Chinese version of the RDWPS had acceptable internal consistency (α = 0.76), moderate reliability, and met the criteria of unidimensionality.

Conclusions

The results of this study provided initial evidence that the traditional Chinese version of the RDWPS can be readily applied to research related to work behavior for those whose primary language is traditional Chinese, with adequate reliability and validity.

Development of an Item Bank to Measure Medication Adherence: Systematic Review

Background

Medication adherence is important in managing the progression of chronic diseases. A promising approach to reduce cognitive burden when measuring medication adherence lies in the use of computer‐adaptive tests (CATs) or in the development of shorter patient-reported outcome measures (PROMs). However, the lack of an item bank currently hampers this progress.

Objective

We aim to develop an item bank to measure general medication adherence.

Methods

Using the preferred reporting items for systematic review and meta-analysis (PRISMA), articles published before October 2019 were retrieved from PubMed, Embase, CINAHL, the Cochrane Library, and Web of Science. Items from existing PROMs were classified and selected (“binned” and “winnowed”) according to standards published by the Patient-Reported Outcomes Measurement Information System (PROMIS) Cooperative Group.

Results

A total of 126 unique PROMs were identified from 213 studies in 48 countries. Items from the literature review (47 PROMs with 579 items for which permission has been obtained) underwent binning and winnowing. This resulted in 421 candidate items (77 extent of adherence and 344 reasons for adherence).

Conclusions

We developed an item bank for measuring general medication adherence using items from validated PROMs. This will allow researchers to create new PROMs from selected items and provide the foundation to develop CATs.

Reconsidering the minimally important difference: evidence of instability over time and across groups

BACKGROUND CONTEXT

Underlying cognitive factors have been found to influence patients' symptom experience. Current evidence suggests that concomitant changes in appraisal must be taken into account to accurately interpret change as measured by standard spine patient-reported outcomes (PROs).

PURPOSE

To investigate changes in patients' minimally important differences (MID) over recovery from spinal surgery; whether and how cognitive appraisal processes are implicated in the change trajectories.

STUDY DESIGN/SETTING

Longitudinal cohort study with up to 12 months follow-up.

PATIENT SAMPLE

Surgical patients (n = 167) with a diagnosis of disc herniation or spinal stenosis.

OUTCOME MEASURES

Standard spine patient-reported PROs were used (Rand-36, Oswestry Disability Index, Numerical Rating Scale for pain, PROMIS Pain Impact).

METHODS

This study was funded by the Feldberg Chair in Spinal Research, Sunnybrook Health Sciences Centre and the authors have no conflicts of interest. MID used an anchor technique and was computed by global assessment of change (GAC) grouping. Participants were binned into groups based on their GAC response patterns at all time points: Consistently better post-surgery, consistently worse post-surgery, and bouncers, whose GAC ratings fluctuate (ie, better-then-worse-then-better; or vice versa). Individuals' longitudinal quality of life (QOL) and appraisal slope scores were computed. QOL-appraisal slopes' correlations were computed by GAC group. Fisher's Z transformation tested the hypothesis that GAC groups differed in the QOL-appraisal relationship over time.

RESULTS

Moderate to large changes are recognized as clinically important in the early stages of recovery (ie, 6 weeks post-surgery), and over time smaller and smaller changes become important. The three pattern groups emphasized and deemphasized different standards of comparison over time, with the Better group emphasizing personal goals and the Worse and Bouncers deemphasizing doctors' input. These group differences translated to differential relationships between PRO change and appraisal changes over time.

CONCLUSIONS

The MID reflects increasingly subtle change over time in PROs. Appraisal may influence how patients experience the same (MID) change over time, with better outcomes associated with emphasizing long-term goals. PRO change seems to be driven by different standards of comparison. Potential avenues for clinical intervention are discussed.

Reliability and validity of the self-reported Activities of Daily Living Scale for people with mental illness

Background/objectives

Patient-reported outcome measures have been found to be an effective method of reflecting client perspectives on their personal health condition. The primary aim of this study was to determine the reliability and validity of the self-reported Activities of Daily Living Scale (sf-ADLS) using Rasch analysis in Taiwan.

Methods

A total of 455 people were included in this study; 224 were persons with mental illness and 231 were healthy adults. We applied Rasch analysis as the means of testing the psychometrics of the scale.

Results

The final version of the sf-ADLS used in this study included 14 items, with no differential item functioning being discernible on the gender variable. The scale was found to be of use in classifying the subjects into four levels of independence.

Conclusions

The revised sf-ADLS conforms to the Rasch measurement model in the formulation of a unidimensional scale. The scale can be used to measure the level of independence with acceptable reliability (internal consistency as 0.9) and validity.

Using item response theory to optimize measurement of chronic stress in pregnancy

OBJECTIVE

Utilize Rasch analysis to develop an optimized self-reported measure of stress in pregnancy and examine the association with patient demographics and biologic measures of stress.

STUDY DESIGN

Measured self-reported stress in pregnant women using 12 existing scales. Collected blood for biologic measures of stress (Epstein Barr Virus [EBV], C-Reactive Protein [CRP], Corticotropin Reactive Hormone [pCRH], and Adenocorticotropin Hormone [ACTH]). Used multidimensional scaling and Rasch analysis to produce an item reduced self-report measure.

RESULTS

Enrolled 112 women. Survey items reduced to two factors: perceived stressors and buffers of stress. Women with a domestic partner had lower perceived stress (p = 0.003). Caucasian women reported higher buffers of stress (p = 0.045), as did women with private insurance (p < 0.001), a planned pregnancy (p < 0.01), and a domestic partner (p < 0.001). Women with higher buffers of stress had lower levels of pCRH (adjusted p = 0.01).

CONCLUSION

Item reduced, optimized measures of stress were associated with significant differences in patient demographics and biologic measures of stress.

Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities

Patient-reported outcomes (PROs) are reports of the status of a patient's health condition that come directly from the patient. While PRO measures are a well-developed technology with robust standards in research, their use for informing healthcare decisions is still poorly understood. We review relevant examples of their application in the provision of healthcare and examine the challenges associated with implementing PROs in clinical settings. We evaluate evidence for their use and examine barriers to their uptake, and present an evidence-based framework for the successful implementation of PROs in clinical practice. We discuss current and future developments for the use of PROs in clinical practice, such as individualized measurement and computer-adaptive testing.

Development of a symptom index for patients with primary brain tumors

OBJECTIVES

This study's primary goals included identifying the highest priority symptoms of patients with advanced brain tumors on treatment, comparing patient priority ratings with those of oncology experts, and constructing a brief symptom index using combined input to assess these symptoms and concerns.

METHODS

Fifty patients with advanced primary brain tumors and 10 physician experts were recruited from the National Comprehensive Cancer Network institutions and community support agencies. By using a 40-item symptom checklist, patients first selected up to 10 of the most important symptoms/concerns to monitor when assessing the value of drug treatment for brain tumors, then nominated up to 5 of the very most important concerns, and finally generated additional symptoms/concerns. By using the same checklist as patients, physicians rated each symptom/concern as disease- or treatment-related.

RESULTS

By using the combined input, a 24-item National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy-Brain Symptom Index (NFBrSI-24) was developed. The NFBrSI-24 showed good internal consistency (α = 0.84), significantly differentiated patients with different levels of functional status (F2,47 = 8.21; P < .001), and demonstrated good convergent validity with the Functional Assessment of Cancer Therapy-General functional, physical, social, emotional, and brain tumor-specific concerns (ρ = 0.59, 0.57, 0.40, 0.35, and 0.50, respectively; Ps < 0.05).

CONCLUSIONS

The NFBrSI-24, an index of the symptoms in advanced brain tumors perceived as most important by both patients and clinicians, improves upon existing measures of brain tumor symptoms through better satisfaction of regulatory requirements for measure development. The findings suggest good reliability and validity, indicating that the NFBrSI-24 is a promising brief assessment of high-priority advanced brain tumor symptoms for research and clinical settings.

Examining the validity and reliability of the Taita symptom checklist using Rasch analysis

BACKGROUND/PURPOSE

The Taita symptom checklist (TSCL) is a standardized self-rating psychiatric symptom scale for outpatients with mental illness in Taiwan. This study aimed to examine the validity and reliability of the TSCL using Rasch analysis.

METHODS

The TSCL was given to 583 healthy people and 479 people with mental illness. Rasch analysis was used to examine the appropriateness of the rating scale, the unidimensionality of the scale, the differential item functioning across sex and diagnosis, and the Rasch cut-off score of the scale.

RESULTS

Rasch analysis confirmed that the revised 37 items with a three-point rating scale of the TSCL demonstrated good internal consistency and met criteria for unidimensionality. The person and item reliability indices were high. The TSCL could reliably measure healthy participants and patients with mental illness. Differential item functioning due to sex or psychiatric diagnosis was evident for three items. A Rasch cut-off score for TSCL was produced for detecting participants' psychiatric symptoms based on an eight-level classification.

CONCLUSION

The TSCL is a reliable and valid assessment to evaluate the participants' perceived disturbance of psychiatric symptoms based on Rasch analysis.

Development and evaluation of a computer adaptive test to assess anxiety in cardiovascular rehabilitation patients

OBJECTIVE

To develop and evaluate a computer adaptive test for the assessment of anxiety in cardiovascular rehabilitation patients (ACAT-cardio) that tailors an optimal test for each patient and enables precise and time-effective measurement.

DESIGN

Simulation study, validation study (against the anxiety subscale of the Hospital Anxiety and Depression Scale and the physical component summary scale of the 12-Item Short-Form Health Survey), and longitudinal study (beginning and end of rehabilitation).

SETTING

Cardiac rehabilitation centers.

PARTICIPANTS

Cardiovascular rehabilitation patients: simulation study sample (n=106; mean age, 57.8y; 25.5% women) and validation and longitudinal study sample (n=138; mean age, 58.6 and 57.9y, respectively; 16.7% and 12.1% women, respectively).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Hospital Anxiety and Depression Scale, 12-Item Short-Form Health Survey, and ACAT-cardio.

RESULTS

The mean number of items was 9.2 with an average processing time of 1:13 minutes when an SE ≤.50 was used as a stopping rule; with an SE ≤.32, there were 28 items and a processing time of 3:47 minutes. Validity could be confirmed via correlations between .68 and .81 concerning convergent validity (ACAT-cardio vs Hospital Anxiety and Depression Scale anxiety subscale) and correlations between -.47 and -.30 concerning discriminant validity (ACAT-cardio vs 12-Item Short-Form Health Survey physical component summary scale). Sensitivity to change was moderate to high with standardized response means between .45 and .82.

CONCLUSIONS

The ACAT-cardio shows good psychometric properties and provides the opportunity for an innovative and time-effective assessment of anxiety in cardiovascular rehabilitation. A more flexible stopping rule might further improve the ACAT-cardio. Additionally, testing in other cardiovascular populations would increase generalizability.

Pain assessment using the NIH Toolbox

OBJECTIVE

Pain is an important component of health and function, and chronic pain can be a problem in its own right. The purpose of this report is to review the considerations surrounding pain measurement in the NIH Toolbox, as well as to describe the measurement tools that were adopted for inclusion in the NIH Toolbox assessment battery.

METHODS

Instruments to measure pain in the NIH Toolbox were selected on the basis of scholarly input from a diverse group of experts, as well as review of existing instruments, which include verbal rating scales, numerical rating scales, and graphical scales.

RESULTS

Brief self-report measures of pain intensity and pain interference were selected for inclusion in the core NIH Toolbox for use with adults. A 0 to 10 numerical rating scale was recommended for measuring pain intensity, and a 6-item Patient Reported Outcome Measurement Information System (PROMIS) short form for measuring pain interference. The 8-item PROMIS Pediatric Pain Interference measure was recommended as a supplemental measure. No specific measure was recommended for measuring pain intensity in children.

CONCLUSIONS

Core and supplemental measures were recommended for the NIH Toolbox. Additional measures were reviewed for investigators who seek tools for measuring pain intensity in pediatric samples.

Improvements in measuring the health-related quality of life of cancer patients

There have been considerable improvements in the collection of health-related quality of life (HRQOL) data in oncology over the past 20 years. The facility with which HRQOL information can now be collected from patients has also been associated with an improved understanding of the meaning of HRQOL and the association between HRQOL and patients' treatment and care plans. Therefore the role that HRQOL data can play in routine clinical practice is increasingly being recognized. The appearance of computer-adaptive tests will herald a new era in HRQOL where questionnaires will be individually tailored to each patient utilizing details unique to that person and linked to clinical management systems to allow a comprehensive assessment of HRQOL status.

[Modern testing theory and its application in the field of health measurement]

This paper briefly introduces item response theory (IRT) as a typical representation of modern testing theory (MTT), and systematically reviews the processes and contents of the application of IRT in the area of health measurement, including, for example, item bank development, scale revision and computerized adaptive testing. The author presents the potential benefits and the notable problems during health measuring by IRT. Then, the author asserts the need for thorough assessment of feasibility when using the IRT in patient-reported outcome research. Further research based on IRT and computerized adaptive testing in health measurement will be carried out in the field of medical care including traditional Chinese medicine and integrative medicine.

Patient Perspective Workshop: Moving Towards OMERACT Guidelines for Choosing or Developing Instruments to Measure Patient-Reported Outcomes

The workshop Choosing or Developing Instruments held at the Outcome Measures in Rheumatology (OMERACT) 10 meeting was designed to help participants think about the underlying methods of instrument development. Conference pre-reading material and 3 brief introductory presentations elaborated the issues, and participants broke into discussion groups before reconvening to share insights, engage in a more general discussion of the issues, and vote on recommendations. Tradeoffs between using current imperfect measures and the long and complex process of developing new instruments were considered, together with the need for rigor in patient-reported outcome (PRO) instrument development. The main considerations for PRO instrument development were listed and a research agenda for action produced. As part of the agenda for action, it is recommended that researchers and patient partners work together to tackle these issues, and that OMERACT bring forward proposals for acceptable instrument development protocols that would meet an enhanced “Truth” statement in the OMERACT Filter.

Studies comparing Numerical Rating Scales, Verbal Rating Scales, and Visual Analogue Scales for assessment of pain intensity in adults: a systematic literature review

CONTEXT

The use of unidimensional pain scales such as the Numerical Rating Scale (NRS), Verbal Rating Scale (VRS), or Visual Analogue Scale (VAS) is recommended for assessment of pain intensity (PI). A literature review of studies specifically comparing the NRS, VRS, and/or VAS for unidimensional self-report of PI was performed as part of the work of the European Palliative Care Research Collaborative on pain assessment.

OBJECTIVES

To investigate the use and performance of unidimensional pain scales, with specific emphasis on the NRSs.

METHODS

A systematic search was performed, including citations through April 2010. All abstracts were evaluated by two persons according to specified criteria.

RESULTS

Fifty-four of 239 papers were included. Postoperative PI was most frequently studied; six studies were in cancer. Eight versions of the NRS (NRS-6 to NRS-101) were used in 37 studies; a total of 41 NRSs were tested. Twenty-four different descriptors (15 for the NRSs) were used to anchor the extremes. When compared with the VAS and VRS, NRSs had better compliance in 15 of 19 studies reporting this, and were the recommended tool in 11 studies on the basis of higher compliance rates, better responsiveness and ease of use, and good applicability relative to VAS/VRS. Twenty-nine studies gave no preference. Many studies showed wide distributions of NRS scores within each category of the VRSs. Overall, NRS and VAS scores corresponded, with a few exceptions of systematically higher VAS scores.

CONCLUSION

NRSs are applicable for unidimensional assessment of PI in most settings. Whether the variability in anchors and response options directly influences the numerical scores needs to be empirically tested. This will aid in the work toward a consensus-based, standardized measure.

Development of a PROMIS item bank to measure pain interference

This paper describes the psychometric properties of the PROMIS-pain interference (PROMIS-PI) bank. An initial candidate item pool (n=644) was developed and evaluated based on the review of existing instruments, interviews with patients, and consultation with pain experts. From this pool, a candidate item bank of 56 items was selected and responses to the items were collected from large community and clinical samples. A total of 14,848 participants responded to all or a subset of candidate items. The responses were calibrated using an item response theory (IRT) model. A final 41-item bank was evaluated with respect to IRT assumptions, model fit, differential item function (DIF), precision, and construct and concurrent validity. Items of the revised bank had good fit to the IRT model (CFI and NNFI/TLI ranged from 0.974 to 0.997), and the data were strongly unidimensional (e.g., ratio of first and second eigenvalue=35). Nine items exhibited statistically significant DIF. However, adjusting for DIF had little practical impact on score estimates and the items were retained without modifying scoring. Scores provided substantial information across levels of pain; for scores in the T-score range 50-80, the reliability was equivalent to 0.96-0.99. Patterns of correlations with other health outcomes supported the construct validity of the item bank. The scores discriminated among persons with different numbers of chronic conditions, disabling conditions, levels of self-reported health, and pain intensity (p<0.0001). The results indicated that the PROMIS-PI items constitute a psychometrically sound bank. Computerized adaptive testing and short forms are available.

Development and validation of a computer adaptive test for measuring dyspnea in heart failure

BACKGROUND

Dyspnea is a common symptom among patients with heart failure. Currently, there is no standardized, rapid, precise method to assess dyspnea.

METHODS AND RESULTS

From a review of the literature, we pooled questions from various questionnaires assessing dyspnea. A total of 201 patients with heart failure completed all questions in the preliminary item bank. Each item asks how much shortness of breath the patient had when doing an activity. Medical charts were reviewed for hospitalization within 1 or 3 months of completing the questions. We created a dyspnea item bank of 44 items. Computer adaptive tests (CAT) generated from this item bank can assess dyspnea by administering on average 10 questions. Simulation CAT scores were generated to compare with the item bank scores. The CAT scores had a correlation of 0.98 with item bank scores. Logistic regression models predicting the probability of being hospitalized from the dyspnea score were statistically significant (P < .05). A 5-point score increase was associated with a 32% increased odds of hospitalization in 1 month and a 20% increased odds of hospitalization in 3 months.

CONCLUSIONS

This computer-based tool for dyspnea assessment obtains similar precision to that of answering the entire dyspnea item bank with less patient burden.

Development of symptom assessments utilising item response theory and computer-adaptive testing--a practical method based on a systematic review

Assessment of individual patients' distress is a cornerstone of clinical care for advanced cancer. Patients' ability to fill out lengthy questionnaires is compromised by many factors. Computer-adaptive tests (CAT) offer a promising approach to developing tailored instruments, that administer only items relevant to the individual patient. A systematic review of the literature about CATs in medical databases was conducted. Based on the results, a method for developing a CAT was designed that requires nine steps: (1) build an item pool; (2) administer the items to a predefined sample in a calibration study; (3) eliminate inappropriate items; (4) examine whether all items are influenced by a single dominant trait; (5) calibrate the items to the best-fitting item response theory (IRT) model; (6) evaluate items' parameter equivalence across subgroups; (7) build an item bank with the calibrated items; (8) develop the CAT; and (9) pilot test the developed CAT. CAT offers the chance to extend the usefulness of patient-reported outcome (PRO) measurements from clinical studies to daily clinical practice.

Linking pain items from two studies onto a common scale using item response theory

This study examined two approaches to linking items from two pain surveys to form a single item bank with a common measurement scale. Secondary analysis of two independent surveys: Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials Survey with Main Survey (959 chronic pain patients; 42 pain items) and Pain Module (n=148; 36 pain items), and Center on Outcomes, Research and Education Survey (400 cancer patients; 43 pain items). There were common items included among the three data sets. In the first approach, all items were calibrated to an item response theory (IRT) model simultaneously, and in the second approach, items were calibrated separately and then the scales were transformed to a common metric. The two approaches produced similar linking results across the two sets of pain interference items because there was sufficient number of common items and large enough sample size. For pain intensity, simultaneous calibration yielded more stable results. Separated calibration yielded an unsatisfactory linking result for pain intensity because of a single common item with small sample size. The results suggested that a simultaneous IRT calibration method produces the more stable item parameters across independent samples, and hence, should be recommended for developing comprehensive item banks. Patient-reported health outcome surveys are often limited in sample sizes and the number of items owing to the difficulty of recruitment and the burden to the patients. As a result, the surveys either lack statistical power or are limited in scope. Using IRT methodology, survey data can be pooled to lend strength to each other to expand the scope and to increase the sample sizes.

Development and preliminary testing of a computerized adaptive assessment of chronic pain

The aim of this article is to report the development and preliminary testing of a prototype computerized adaptive test of chronic pain (CHRONIC PAIN-CAT) conducted in 2 stages: (1) evaluation of various item selection and stopping rules through real data-simulated administrations of CHRONIC PAIN-CAT; (2) a feasibility study of the actual prototype CHRONIC PAIN-CAT assessment system conducted in a pilot sample. Item calibrations developed from a US general population sample (N = 782) were used to program a pain severity and impact item bank (kappa = 45), and real data simulations were conducted to determine a CAT stopping rule. The CHRONIC PAIN-CAT was programmed on a tablet PC using QualityMetric's Dynamic Health Assessment (DYHNA) software and administered to a clinical sample of pain sufferers (n = 100). The CAT was completed in significantly less time than the static (full item bank) assessment (P < .001). On average, 5.6 items were dynamically administered by CAT to achieve a precise score. Scores estimated from the 2 assessments were highly correlated (r = .89), and both assessments discriminated across pain severity levels (P < .001, RV = .95). Patients' evaluations of the CHRONIC PAIN-CAT were favorable.

PERSPECTIVE

This report demonstrates that the CHRONIC PAIN-CAT is feasible for administration in a clinic. The application has the potential to improve pain assessment and help clinicians manage chronic pain.

Evaluating the dimensionality of perceived cognitive function

Decrements in cognitive function are common in cancer patients and other clinical populations. As direct neuropsychological testing is often not feasible or affordable, there is potential utility in screening for deficits that may warrant a more comprehensive neuropsychological assessment. Furthermore, some evidence suggests that perceived cognitive function (PCF) is independently associated with structural and functional changes on neuroimagery, and may precede more overt deficits. To appropriately measure PCF, one must understand its components and the underlying dimensional structure. The purpose of this study was to examine the dimensionality of PCF in people with cancer. The sample included 393 cancer patients from four clinical trials who completed a questionnaire consisting of the prioritized areas of concerns identified by patients and clinicians: self-reported mental acuity, concentration, memory, verbal fluency, and functional interference. Each area contained both negatively worded (i.e., deficit) and positively worded (i.e., capability) items. Data were analyzed by using Cronbach's alpha, item-total correlations, one-factor confirmatory factor analysis, and a bi-factor analysis model. Results indicated that perceived cognitive problem items are distinct from cognitive capability items, supporting a two-factor structure of PCF. Scoring of PCF based on these two factors should lead to improved assessment of PCF for people with cancer.

Classical test theory and item response theory/Rasch model to assess differences between patient-reported fatigue using 7-day and 4-week recall periods

OBJECTIVE

This study compared self-reported fatigue between 7-day and 4-week time frames and explored factors that affect patients' responses.

STUDY DESIGN AND SETTING

Two hundred and sixteen cancer patients completed either 7-day or 4-week version of the Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-F). Cochran-Mantel-Haenszel statistics and Cochran-Armitage trend tests were used to assess the association between time frame and item scores. Information function curves at both item and scale levels were depicted to evaluate the precision along the fatigue continuum. Differential item functioning (DIF) was used to examine the stability of the psychometric properties between time frames.

RESULTS

Time frame did not influence patients' item responses. Examination of information function curves at item level did not clearly favor either time frame. At the scale level, the 7-day time frame was slightly more precise overall than the 4-week time frame. No item demonstrated DIF between time frames. Neither gender nor fatigue severity had an impact on above results.

CONCLUSION

This study suggests 7-day and 4-week time frame are equally appropriate in measuring fatigue, preference might be given to the more informative 7-day time frame. However, substantive considerations regarding the appropriate time frame should outweigh statistical ones.

Computerized adaptive testing in back pain: validation of the CAT-5D-QOL

STUDY DESIGN

We have conducted an outcome instrument validation study.

OBJECTIVE

Our objective was to develop a computerized adaptive test (CAT) to measure 5 domains of health-related quality of life (HRQL) and assess its feasibility, reliability, validity, and efficiency.

SUMMARY OF BACKGROUND DATA

Kopec and colleagues have recently developed item response theory based item banks for 5 domains of HRQL relevant to back pain and suitable for CAT applications. The domains are Daily Activities (DAILY), Walking (WALK), Handling Objects (HAND), Pain or Discomfort (PAIN), and Feelings (FEEL).

METHODS

An adaptive algorithm was implemented in a web-based questionnaire administration system. The questionnaire included CAT-5D-QOL (5 scales), Modified Oswestry Disability Index (MODI), Roland-Morris Disability Questionnaire (RMDQ), SF-36 Health Survey, and standard clinical and demographic information. Participants were outpatients treated for mechanical back pain at a referral center in Vancouver, Canada.

RESULTS

A total of 215 patients completed the questionnaire and 84 completed a retest. On average, patients answered 5.2 items per CAT-5D-QOL scale. Reliability ranged from 0.83 (FEEL) to 0.92 (PAIN) and was 0.92 for the MODI, RMDQ, and Physical Component Summary (PCS-36). The ceiling effect was 0.5% for PAIN compared with 2% for MODI and 5% for RMQ. The CAT-5D-QOL scales correlated as anticipated with other measures of HRQL and discriminated well according to the level of satisfaction with current symptoms, duration of the last episode, sciatica, and disability compensation. The average relative discrimination index was 0.87 for PAIN, 0.67 for DAILY and 0.62 for WALK, compared with 0.89 for MODI, 0.80 for RMDQ, and 0.59 for PCS-36.

CONCLUSION

The CAT-5D-QOL is feasible, reliable, valid, and efficient in patients with back pain. This methodology can be recommended for use in back pain research and should improve outcome assessment, facilitate comparisons across studies, and reduce patient burden.

Evaluation of the psychometrics of the Social Impact Scale: a measure of stigmatization

As stigmatization has a large impact on patients, therapists need a measure of this impact to provide patients with adequate services. This study, therefore, examined the reliability and validity of the Social Impact Scale (SIS) when applied to three groups of individuals diagnosed with major depression, schizophrenia, or HIV/AIDS. The study sample (N=580) included 237 patients with depressive disorder, 119 with schizophrenia, and 224 with HIV/AIDS. Of these, 56% were men, 45.5% had an elementary school education or less, 48% were employed, and 56% were single. The Rasch measurement model, an item-response theory, was used to analyze the SIS structure and quality. The Rasch model solves several statistical problems of traditional measurement theory, such as misuse of ordinal data as interval data and sample dependence. Rasch analysis indicated that the 24 items of the SIS fit the measurement model. The match between item difficulties and person abilities was adequate. All items showed acceptable rating scale structure. The separation reliability of the scale reached 0.99. The SIS had acceptable psychometric qualities in terms of internal consistency, item validity, person validity, sensitivity, and concurrent validity when applied to patients with depression, schizophrenia, and HIV/AIDS in Taiwan.

Using Q-methodology to understand perceived fatigue reported by adolescents with cancer

Although fatigue is an unpleasant symptom commonly experienced by pediatric oncology patients, it also tends to be under-recognized and under-treated. A conceptual understanding of how patients perceive fatigue is beneficial for better identification and treatment planning. The purpose of this study was to understand how adolescents with cancer perceive their fatigue and to explore potential factors influencing their perceptions by using Q-methodology. The sample included 15 patients (ages 12-18 years) from the Chicago metropolitan area who were receiving cancer treatment. All participants completed a 37-statement Q-sort task. Data were analyzed by using PQMethod computer software. Results identified three descriptors of perceived fatigues reported by adolescents: energy and related capacity for physical functioning, psychosocial effects, and anemia-specific concerns. Appropriate referral for patients with the latter two descriptors of fatigue was recommended.

Measuring fatigue for children with cancer: development and validation of the pediatric Functional Assessment of Chronic Illness Therapy-Fatigue (pedsFACIT-F)

Fatigue is a major concern for cancer patients of all ages. The lack of an appropriate assessment tool has impeded our understanding of its prevalence and significance, specifically in the pediatric cancer population. This paper documents the reliability and validity of the pediatric Functional Assessment of Chronic Illness Therapy-Fatigue (pedsFACIT-F) in a comprehensive manner. The 11-item PedsFACIT-F was developed via literature review, feedback from patient/parent/clinician, and a face-to-face consensus meeting. Its reliability and validity were examined on the basis of data from 159 pediatric patients with cancer via classical test theory and Rasch analysis. Results showed that the pedsFACIT-F demonstrated good internal consistency (Cronbach alpha), acceptable item-total correlations, and met the unidimensionality assumption set by confirmatory factor analysis. All items had acceptable fit statistics in the Rasch analysis and demonstrated stable measurement properties by age, sex, and cancer type. Scores on the pedsFACIT-F significantly discriminated between patients with and without anemia and among patients with different functional status; clinically relevant minimally important differences were estimated accordingly. The pedsFACIT-F was significantly correlated to the PedsQL Multidimensional Fatigue Scale. In conclusion, the pedsFACIT-F demonstrates satisfactory reliability and validity and can be a useful tool in clinical trials and other research.

The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years

BACKGROUND

The National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) Roadmap initiative (www.nihpromis.org) is a 5-year cooperative group program of research designed to develop, validate, and standardize item banks to measure patient-reported outcomes (PROs) relevant across common medical conditions. In this article, we will summarize the organization and scientific activity of the PROMIS network during its first 2 years.

DESIGN

The network consists of 6 primary research sites (PRSs), a statistical coordinating center (SCC), and NIH research scientists. Governed by a steering committee, the network is organized into functional subcommittees and working groups. In the first year, we created an item library and activated 3 interacting protocols: Domain Mapping, Archival Data Analysis, and Qualitative Item Review (QIR). In the second year, we developed and initiated testing of item banks covering 5 broad domains of self-reported health.

RESULTS

The domain mapping process is built on the World Health Organization (WHO) framework of physical, mental, and social health. From this framework, pain, fatigue, emotional distress, physical functioning, social role participation, and global health perceptions were selected for the first wave of testing. Item response theory (IRT)-based analysis of 11 large datasets supplemented and informed item-level qualitative review of nearly 7000 items from available PRO measures in the item library. Items were selected for rewriting or creation with further detailed review before the first round of testing in the general population and target patient populations.

CONCLUSIONS

The NIH PROMIS network derived a consensus-based framework for self-reported health, systematically reviewed available instruments and datasets that address the initial PROMIS domains. Qualitative item research led to the first wave of network testing which began in the second year.

The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment

The use of item banks and computerized adaptive testing (CAT) begins with clear definitions of important outcomes, and references those definitions to specific questions gathered into large and well-studied pools, or "banks" of items. Items can be selected from the bank to form customized short scales, or can be administered in a sequence and length determined by a computer programmed for precision and clinical relevance. Although far from perfect, such item banks can form a common definition and understanding of human symptoms and functional problems such as fatigue, pain, depression, mobility, social function, sensory function, and many other health concepts that we can only measure by asking people directly. The support of the National Institutes of Health (NIH), as witnessed by its cooperative agreement with measurement experts through the NIH Roadmap Initiative known as PROMIS (www.nihpromis.org), is a big step in that direction. Our approach to item banking and CAT is practical; as focused on application as it is on science or theory. From a practical perspective, we frequently must decide whether to re-write and retest an item, add more items to fill gaps (often at the ceiling of the measure), re-test a bank after some modifications, or split up a bank into units that are more unidimensional, yet less clinically relevant or complete. These decisions are not easy, and yet they are rarely unforgiving. We encourage people to build practical tools that are capable of producing multiple short form measures and CAT administrations from common banks, and to further our understanding of these banks with various clinical populations and ages, so that with time the scores that emerge from these many activities begin to have not only a common metric and range, but a shared meaning and understanding across users. In this paper, we provide an overview of item banking and CAT, discuss our approach to item banking and its byproducts, describe testing options, discuss an example of CAT for fatigue, and discuss models for long term sustainability of an entity such as PROMIS. Some barriers to success include limitations in the methods themselves, controversies and disagreements across approaches, and end-user reluctance to move away from the familiar.

Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities

The current mechanism for monitoring toxicity symptoms in cancer trials depends on a complex paper-based process. Electronic collection of patient-reported outcomes (PROs) may be more efficient and accurate. An online PRO platform was created including a simple data entry interface, real-time report generation, and an alert system to e-mail clinicians when patients self-report serious toxicities. Feasibility assessment involving 180 chemotherapy patients demonstrated high levels of use at up to 40 follow-up clinic visits per patient over 16 months (85% of patients at any given visit), with high levels of patient and clinician acceptance and satisfaction (>95%). Alerts were used as the basis for delayed chemotherapy treatments, dose modifications, and scheduling changes. These results demonstrate that online patient-reporting is a feasible strategy for chemotherapy toxicity symptom monitoring, and may improve safety and satisfaction with care. Ongoing multi-center research will evaluate the impact of this approach on clinical and administrative outcomes.

The initial development of an item bank to assess and screen for psychological distress in cancer patients

Psychological distress is a common problem among cancer patients. Despite the large number of instruments that have been developed to assess distress, their utility remains disappointing. This study aimed to use Rasch models to develop an item-bank which would provide the basis for better means of assessing psychological distress in cancer patients.An item bank was developed from eight psychological distress questionnaires using Rasch analysis to link common items. Items from the questionnaires were added iteratively with common items as anchor points and misfitting items (infit mean square >1.3) removed, and unidimensionality assessed.A total of 4914 patients completed the questionnaires providing an initial pool of 83 items. Twenty items were removed resulting in a final pool of 63 items. Good fit was demonstrated and no additional factor structure was evident from the residuals. However, there was little overlap between item locations and person measures, since items mainly targeted higher levels of distress. The Rasch analysis allowed items to be pooled and generated a unidimensional instrument for measuring psychological distress in cancer patients. Additional items are required to more accurately assess patients across the whole continuum of psychological distress.

Assessment of health-related quality of life in arthritis: conceptualization and development of five item banks using item response theory

BACKGROUND

Modern psychometric methods based on item response theory (IRT) can be used to develop adaptive measures of health-related quality of life (HRQL). Adaptive assessment requires an item bank for each domain of HRQL. The purpose of this study was to develop item banks for five domains of HRQL relevant to arthritis.

METHODS

About 1,400 items were drawn from published questionnaires or developed from focus groups and individual interviews and classified into 19 domains of HRQL. We selected the following 5 domains relevant to arthritis and related conditions: Daily Activities, Walking, Handling Objects, Pain or Discomfort, and Feelings. Based on conceptual criteria and pilot testing, 219 items were selected for further testing. A questionnaire was mailed to patients from two hospital-based clinics and a stratified random community sample. Dimensionality of the domains was assessed through factor analysis. Items were analyzed with the Generalized Partial Credit Model as implemented in Parscale. We used graphical methods and a chi-square test to assess item fit. Differential item functioning was investigated using logistic regression.

RESULTS

Data were obtained from 888 individuals with arthritis. The five domains were sufficiently unidimensional for an IRT-based analysis. Thirty-one items were deleted due to lack of fit or differential item functioning. Daily Activities had the narrowest range for the item location parameter (-2.24 to 0.55) and Handling Objects had the widest range (-1.70 to 2.27). The mean (median) slope parameter for the items ranged from 1.15 (1.07) in Feelings to 1.73 (1.75) in Walking. The final item banks are comprised of 31-45 items each.

CONCLUSION

We have developed IRT-based item banks to measure HRQL in 5 domains relevant to arthritis. The items in the final item banks provide adequate psychometric information for a wide range of functional levels in each domain.