The Refractive Status and Vision Profile: Evaluation of psychometric properties and comparison of Rasch and summated Likert-scaling

Catquest-9SF patient outcomes questionnaire: nine-item short-form Rasch-scaled revision of the Catquest questionnaire

PURPOSE

To assess and optimize the Catquest questionnaire for measuring patient-reported outcomes of cataract surgery using Rasch analysis.

SETTING

Fifty-eight ophthalmic surgical units in Sweden.

METHODS

Catquest questionnaires (n = 21364) from the Swedish National Cataract Register were selected and randomized to 2 groups. Data from 10486 questionnaires were comprehensively Rasch analyzed using a 4-Andrich rating scale model in Winsteps software. A revised version of Catquest was developed (Catquest-9SF) and tested in 10886 patients for validity and responsiveness to cataract surgery.

RESULTS

Only the visual disability subscale formed a valid measurement scale. This could be enhanced through the addition of the 2 global assessment items; however, the symptoms and frequency of performing the activities items did not contribute to the measurement. The 9-item short-form version (Catquest-9SF) had ordered response thresholds and good person separation (2.65) and was largely free from differential item functioning. All items fit a single overall construct (infit range, 0.75 to 1.29; outfit range, 0.70 to 1.39) and unidimensional by principal components analysis. The items were well targeted to the preoperative participants (0.34 logit difference in means). The score correlated with visual acuity (r = 0.43 preoperatively; r = 0.48 postoperatively) and was highly responsive to cataract surgery (preoperatively -0.32 +/- 2.15 logits; postoperatively -3.21 +/- 2.50 logits (P<.0001).

CONCLUSIONS

The 9-item Rasch-scaled Catquest-9SF was highly valid in measuring visual disability outcomes of cataract surgery. Its brevity makes it suited to routine clinical use, and a raw-data to Rasch-measure conversion simplifies application.

An evaluation of the 10-item vision core measure 1 (VCM1) scale (the Core Module of the Vision-Related Quality of Life scale) using Rasch analysis

PURPOSE

To assess and re-engineer the Vision Core Measure 1 (VCM1) questionnaire in low vision (LV) and cataract participants using Rasch analysis.

METHODS

295 participants drawn from a low vision clinic and 181 from a cataract surgery waiting list completed the 10-item VCM1. Unidimensionality, item fit to the model, response category performance, differential item functioning (DIF) and targeting of items to patients were assessed. Category collapsing and item removal were considered to improve the questionnaire.

RESULTS

The initial fit of the VCM1 (combined populations) to the Rasch model showed lack of fit (chi2 = 83.3, df = 50, p = 0.002). There was evidence of DIF between the two populations which could not be resolved. Consequently, each population was assessed separately. Irrespective of the population, disordering of response category thresholds was evident. However, collapsing categories produced ordered thresholds and resulted in fit to the Rasch model for the LV (Total chi2 = 41.6, df = 30; p = 0.08) and cataract population (Total chi2 = 17.9, df = 20, p = 0.59). Overall, the VCM1 behaved as a unidimensional scale for each population and no item showed evidence of DIF. Item targeting to patients was however sub-optimal particularly for the cataract population.

CONCLUSION

The VCM1 questionnaire could be improved by shortening the response scale, although different response categories are required for cataract and LV populations. Calibration of items also differed across populations. While the VCM1 performs well within the Rasch model, in line with its initial purpose, it requires the addition of items to satisfactorily target low vision and cataract populations.

Literacy affected ability to adequately discriminate among categories in multipoint Likert Scales

OBJECTIVES

To assess the effect of inability to read on a five-point Likert scale, using the WHOQOL-BREF response scale. It is hypothesized that inability to read is related to loss of discriminant power among the five-category response scale.

STUDY DESIGN AND SETTING

In a cross-sectional design, nonreaders (n=141) and subjects educated at a graduate level (n-907) were assessed. Statistical analyses combined classic and modern psychometric approaches (Confirmatory factor analysis and Rasch analysis).

RESULTS

There is a remarkable difference in the psychometric performance of items across the two subgroups. Fit indexes proved to be closer to the ideal for the graduate group, but not for the nonreader group. Reducing the number of response categories improved the model exclusively for the nonreader sample. Nonreaders do not interpret the scale as a true five-category scale, but exhibit a response pattern indicative that only three categories are in fact considered.

CONCLUSION

These results confirm the hypothesis that a multiple-category response scale is not suitable for nonreaders. They suggest that a simpler response scale should be adopted to achieve a more reliable and satisfactory psychometric performance in this population. This effect seems to be stronger when more abstract and subjective constructs are involved.

Applying multilevel item response theory to vision-related quality of life in Dutch visually impaired elderly

PURPOSE

Instead of applying the usual longitudinal methods to assess the outcome of low-vision rehabilitation services in terms of vision-related quality of life, a three-level Item Response Theory (IRT) method was proposed.

METHODS

The translated Vision-Related Quality of Life Core Measure (VCM1) and Low Vision Quality Of Life (LVQOL) questionnaires were used in a nonrandomized follow-up study among elderly patients (n = 296) referred to two different low-vision rehabilitation services in the Netherlands. Factor analysis was performed on the matrix of polychoric correlations to investigate (uni-)dimensionality and to prepare both questionnaires for the multilevel IRT analyses. A statistical model, which was characterized by a graded response model for rating scales, was developed. Threshold and item difficulty parameters and group by time-specific mean fixed effects were estimated. Random individual effects were predicted. Measurement invariance across occasions was tested.

RESULTS

The VCM1 and the LVQOL "reading and fine work" dimension showed item parameter drift. In the multidisciplinary rehabilitation center patients, deterioration was found on the "mobility" dimension after 1 year and improvement was found on "adjustment" and "visual (motor) skills" after 5 months (p < 0.05). Patients in both low-vision services showed improvement on the "reading small print" subscale at both follow-up time points (p < 0.05).

CONCLUSIONS

Except for improvement in "reading small print," low-vision rehabilitation services did not seem to contribute substantially to any other dimensions of vision-related quality of life. The results showed a change in only a limited number of individual patients. However, with regard to the field of low-vision rehabilitation, the proposed IRT method seemed to be successful in the follow-up of individuals. IRT specific software was unnecessary. The data did not have to be complete and the use of cumulative logits made the proposed IRT method an economical and efficient approach. Because of item parameter drift, the VCM1 was difficult to interpret. The use of multilevel IRT models with longitudinal data and dependent observations is recommended.

The Development, Assessment, and Selection of Questionnaires

Patient-reported outcome measurement has become accepted as an important component of comprehensive outcomes research. Researchers wishing to use a patient-reported measure must either develop their own questionnaire (called an instrument in the research literature) or choose from the myriad of instruments previously reported. This article summarizes how previously developed instruments are best assessed using a systematic process and we propose a system of quality assessment so that clinicians and researchers can determine whether there exists an appropriately developed and validated instrument that matches their particular needs. These quality assessment criteria may also be useful to guide new instrument development and refinement. We welcome debate over the appropriateness of these criteria as this will lead to the evolution of better quality assessment criteria and in turn better assessment of patient-reported outcomes.

Content Development of the Optometric Patient Anxiety Scale

PURPOSE

Patient anxiety has been shown to be detrimental to many aspects of healthcare outcomes. To date, there is no method of evaluating anxiety in optometric practice. Therefore, the purpose of this study was the content development of a questionnaire to measure optometric patient anxiety. Such a tool will have both clinical and research application; allowing the identification of anxious patients in practice and as a method to establish the success of anxiety reducing interventions.

METHODS

Selection of initial items was based on patient interviews, literature review, and focus group feedback. The initial 30-item Optometric Patient Anxiety Scale was piloted on 148 patients in optometric practice. Rasch analysis was used to analyze response category operation and to facilitate item removal to ensure a valid and unidimensional scale. Test-retest reliability (test-retest time, 2 weeks) was measured on 59 young adults to test the stability of the measure with time.

RESULTS

Rasch analysis identified disordering of category thresholds and underutilization of the end-response category. Therefore, categories were merged to a three response solution. Item reduction was principally driven by infit and outfit statistics. The items in the final 10-item scale all had good infit and outfit values (infit: 0.80-1.20, outfit: 0.7-1.3), good person separation (>2) and high person and item reliability coefficients, 0.84 and 0.88, respectively. Test-retest reliability also demonstrated good stability of the measure with time (intraclass correlation; ICC = 0.85).

CONCLUSIONS

The Optometric Patient Anxiety Scale is the first questionnaire to measure patient anxiety specific to optometric practice. The scale was developed using Rasch analysis to ensure that all the items work together to form a valid unidimensional interval scale.

Why Measurement Matters for Measuring Patient Vision Outcomes

Optometrists, by definition, care deeply about measurement. This brief review article considers the essential features of measurement that make many optometric instruments so useful and how patient-centered survey instruments such as vision-related quality of life questionnaires, can be analyzed using contemporary psychometric methods, so that they also conform to these essential features of measurement. These features include unidimensionality, hierarchical order, and equal interval scaling. Optometrists demand these features because they need to make meaningful comparisons both between patients and over time. Questionnaires about visual function or health-related quality of life, typically involve a series of rating scale type items that are added up to produce a total raw score. Yet total raw scores, which are ordinal, do not exhibit the essential properties of measurement. The Rasch Model, developed by Georg Rasch in 1956, converts ordinal-level raw score data into interval measures that demonstrate the essential features of measurement. Under the Rasch model any obtained score (response) is conceptualized as the difference between the amount of a trait reflected in an item, i.e., how "hard" the item is, and the ability of the person responding to the item. The Rasch model estimates the log odds probability (logit) for any response by any person. Logits are equal interval, representing equal amounts of the construct being measured across the entire range of the construct. Logits define the hierarchical order of items, how hard or easy items are, and the Rasch model specifies that this order of items must be invariant for all persons, that is, must be unidimensional. There are numerous software packages available for applying the Rasch model, all provide methods for evaluating how well data demonstrate unidimensionality, hierarchical order, and equal interval scaling. These can be used in the development, assessment or revision of questionnaires to optimize measurement.

An Interval-Scaled Scoring Algorithm for Visual Function Questionnaires

PURPOSE

The primary purpose of the study is to present and test a simple algorithm for scoring visual function questionnaires (VFQs) that approximates person measure estimates from Rasch analysis, does not introduce nonlinearities at extreme scores, and is insensitive to missing data. A secondary purpose is to test the hypothesis that all VFQs measure the same visual ability variable and can be calibrated to a common measurement scale.

METHODS

Each of 407 consecutively recruited low vision patients were administered two of four visual function questionnaires: Activities of Daily Living Scale (ADVS), National Eye Institute Visual Functioning Questionnaire (NEI VFQ), 14-item Visual Functioning Index (VF-14), Visual Activities Questionnaire (VAQ). Separate Rasch analyses, using the Andrich rating scale model, were performed on responses to each of the four VFQs and again on the merged data of all instruments. An approximation of visual ability, based on average functional reserve and an inverse hyperbolic tangent transformation, is presented and tested by comparing visual ability estimates from the Rasch analyses to corresponding estimates from the approximations.

RESULTS

Relative to person measure estimates from Rasch analysis, the approximations were observed to be linear and highly reliable (intraclass correlations ranged from 0.97 to 0.997). The measurement scale of each of the four instruments was observed to be a linear transformation of the measurement scale estimated from the merged responses of all four instruments. The approximation algorithm transforms rating scale responses for each instrument to a common measurement scale. By randomly censuring item responses for each subject, it was demonstrated that the approximation algorithm is robust and insensitive to missing data.

CONCLUSIONS

A simple scoring algorithm based on an inverse hyperbolic tangent transformation of average functional reserve produces highly reliable approximations of visual ability estimated from Rasch analysis for the ADVS, NEI VFQ, VAQ, and VF-14. All four instruments measure the same visual ability variable in units that can be calibrated to a common measurement scale.

Are Standard Instruments Valid for the Assessment of Quality of Life and Symptoms in Glaucoma?

PURPOSE

To determine if the impact of Visual Impairment Instrument (IVI) and Glaucoma Symptom Scale (GSS) are valid instruments to assess participation in daily living and ophthalmic complaints, respectively, in a glaucoma population.

METHODS

Patients with glaucoma were recruited from private and public clinics and completed the IVI and GSS questionnaires. The two scales were assessed for fit to the Rasch model. Unidimensionality, individual item and person fit to the model, response category performance (how respondents differentiate between the response options), differential item functioning (how subgroups, despite equal levels of the underlying trait, respond differently to an individual item), and targeting of items to patients (good spread of items across the full range of patients' scores) were assessed.

RESULTS

One hundred seventy-five participants (mean age = 71 year) were recruited. The majority (65%) had primary open angle glaucoma and good presenting visual acuity >or=6/9 in the better eye (87%). Only one-third of the participants had severe visual field loss in both eyes. Disordered thresholds were evident across all GSS items, indicating that the categories were difficult to discriminate and required category collapsing (5 to 3 categories). There was no evidence of person and item misfit, differential item functioning, and multidimensionality. However, both scales displayed ineffective person-item targeting as a large number of participants demonstrated little difficulty with the most difficult items.

CONCLUSIONS

Because of unsatisfactory targeting, The IVI and GSS are suboptimal scales to assess patients with glaucoma but relatively good vision. It is likely that items could be added to optimize the performance of both instruments. There may however be a need to develop a glaucoma-specific instrument to assess Quality of Life in this population.

Impact of Cataract Surgery on Quality of Life in Patients with Early Age-Related Macular Degeneration

PURPOSE

To investigate if cataract surgery improves overall and specific areas of quality of life (QoL) in patients with early age-related macular degeneration (AMD) using the impact of vision impairment (IVI) questionnaire.

METHODS

Patients with visually significant cataract and early AMD, who were being considered for cataract surgery in the study eye, were recruited. Eligible patients were randomized to either "early surgery" or "standard surgery" (standard cataract surgery waiting time of 6 months) groups. The IVI, sociodemographic, and clinical data were collected. Rasch analysis was used to estimate QoL person measures at baseline and follow-up. The data were analyzed using repeated measures ANOVA. Effect sizes were calculated using Cohen's d coefficient.

RESULTS

Fifty six patients (mean age = 78.5 years and visual acuity = 6/15) had one eye randomly allocated to either the early surgery (n = 29) or standard surgery (n = 27) groups. At follow-up, significant interaction effects were found for the overall IVI score [F(1,54) = 17.7; p < 0.001], the emotional well-being [F(1,54) = 13.4; p = 0.001], mobility and independence [F(1,54) = 13.4; p = 0.001], and reading and accessing information subscales [F(1,54) = 13.1; p = 0.001]. The standard surgery group systematically recorded worse scores at 6 months on all QoL measures whereas the early surgery group recorded significant gains (p < 0.001; Cohen's d = 0.66 to 0.91) on all of them. Visual acuity in the study eye significantly improved in the early surgery group only (Cohen's d = 1.1; p < 0.05) and improvement in log MAR lines read was identified as the single independent predictor of enhanced QoL explaining between 26 and 34% of the variance in the IVI scores.

CONCLUSIONS

Cataract surgery is justified in patients with early AMD. It brings significant improvements in visual acuity, aspects of daily living, and overall QoL.

Development of a near activity visual questionnaire to assess accommodating intraocular lenses

PURPOSE

To develop a questionnaire that subjectively assesses near visual function in patients with 'accommodating' intraocular lenses (IOLs).

METHODS

A literature search of existing vision-related quality-of-life instruments identified all questions relating to near visual tasks. Questions were combined if repeated in multiple instruments. Further relevant questions were added and item interpretation confirmed through multidisciplinary consultation and focus groups. A preliminary 19-item questionnaire was presented to 22 subjects at their 4-week visit post first eye phacoemulsification with 'accommodative' IOL implantation, and again 6 and 12 weeks post-operatively. Rasch Analysis, Frequency of Endorsement, and tests of normality (skew and kurtosis) were used to reduce the instrument. Cronbach's alpha and test-retest reliability (intraclass correlation coefficient, ICC) were determined for the final questionnaire. Construct validity was obtained by Pearson's product moment correlation (PPMC) of questionnaire scores to reading acuity (RA) and to Critical Print Size (CPS) reading speed. Criterion validity was obtained by receiver operating characteristic (ROC) curve analysis and dimensionality of the questionnaire was assessed by factor analysis.

RESULTS

Rasch Analysis eliminated nine items due to poor fit statistics. The final items have good separation (2.55), internal consistency (Cronbach's alpha=0.97) and test-retest reliability (ICC=0.66). PPMC of questionnaire scores with RA was 0.33, and with CPS reading speed was 0.08. Area under the ROC curve was 0.88 and Factor Analysis revealed one principal factor.

CONCLUSION

The pilot data indicates the questionnaire to be internally consistent, reliable and a valid instrument that could be useful for assessing near visual function in patients with 'accommodating' IOLS. The questionnaire will now be expanded to include other types of presbyopic correction.

Patient-centred measurement in ophthalmology--a paradigm shift

Ophthalmologists and researchers in ophthalmology understand what a rapidly evolving field ophthalmology is, and that to conduct good research it is essential to use the latest and best methods. In outcomes research, one modern initiative has been to conduct holistic measurement of outcomes inclusive of the patient's point of view; patient-centred outcome. This, of course, means including a questionnaire. However, the irony of trying to improve outcomes research by being inclusive of many measures is that the researcher may not be expert in all measures used. Certainly, few people conducting outcomes research in ophthalmology would claim to be questionnaire experts. Most tend to be experts in their ophthalmic subspecialty and probably simply choose a popular questionnaire that appears to fit their needs and think little more about it. Perhaps, unlike our own field, we assume that the field of questionnaire research is relatively stable. This is far from the case. The measurement of patient-centred outcomes with questionnaires is a rapidly evolving field. Indeed, over the last few years a paradigm shift has occurred in patient-centred measurement.