Analysis of the psychometric properties of the Multiple Sclerosis Impact Scale-29 (MSIS-29) in relapsing-remitting multiple sclerosis using classical and modern test theory

Establishing clinically meaningful within-individual improvement thresholds for eight patient-reported outcome measures in people with relapsing-remitting multiple sclerosis

BACKGROUND

As disease-modifying therapies do not reverse the course of multiple sclerosis (MS), assessment of therapeutic success involves documenting patient-reported outcomes (PROs) concerning health-related quality of life, disease and treatment-related symptoms, and the impact of symptoms on function. Interpreting PRO data involves going beyond statistical significance to calculate within-patient meaningful change scores. These thresholds are needed for each PRO in order to fully interpret the PRO data. This analysis of PRO data from the PROMiS AUBAGIO study, which utilized 8 PRO instruments in teriflunomide-treated relapsing-remitting MS (RRMS) patients, was designed to estimate clinically meaningful within-individual improvement thresholds in the same manner, for 8 PRO instruments.

RESULTS

The analytical approach followed a triangulation exercise that considered results from anchor- and distribution-based methods and graphical representations of empirical cumulative distribution functions in PRO scores in groups defined by anchor variables. Data from 8 PRO instruments (MSIS-29 v2, FSMC, MSPS, MSNQ, TSQM v1.4, PDDS, HRPQ-MS v2, and HADS) were assessed from 434 RRMS patients. For MSIS-29 v2, FSMC, MSPS, and MSNQ total scores, available anchor variables enabled both anchor- and distribution-based methods to be applied. For instruments with no appropriate anchor available, distribution-based methods were applied. A recommended value for meaningful within-individual improvement was defined by comparing mean change in PRO scores between participants showing improvement of one or two categories in the anchor variable or those showing no change. A "lower bound" estimate was calculated using distribution-based methods. An improvement greater than the lower-bound estimate was considered "clinically meaningful".

CONCLUSION

This analysis produced estimates for assessing meaningful within-individual improvements for 8 PRO instruments used in MS studies. These estimates should be useful for interpreting scores and communicating study results and should facilitate decision-making by regulatory and healthcare authorities where these 8 PROs are commonly employed.

The MSIS-29 and SF-36 as outcomes in secondary progressive MS trials

Background:

Patient-reported outcome measures (PROMs) are often used in clinical research, but little is known about their performance as longitudinal outcomes.

Methods:

We used data from ASCEND, a large SPMS trial (n = 889), to investigate changes on the Short Form Health Survey 36 (SF-36 v2) and the Multiple Sclerosis Impact Scale (MSIS-29) over 2 years of follow-up.

Results:

PROM scores changed little over the 2 years of follow-up. In contrast to physical disability measures, there was no consistent trend in PROM change: significant worsening occurred about as often as improvement. Using a 6-month confirmation reduced the number of both worsening and improvement events without altering their relative balance. There was no clear difference in worsening events in groups based on population characteristics, nor was there a noticeable effect using different thresholds for clinically significant change.

Conclusion:

We found little consistent change in MSIS-29 and SF-36 over 2 years of follow-up in people with SPMS. Our findings show a disconnect between disability worsening and PROM change in this population. Our findings raise caution about the use of these PROMs as primary outcome measures in SPMS trials and call for a critical reappraisal of the longitudinal use of these measures in SPMS trials.

Group cognitive rehabilitation to reduce the psychological impact of multiple sclerosis on quality of life: the CRAMMS RCT

BACKGROUND

People with multiple sclerosis have problems with memory and attention. The effectiveness of cognitive rehabilitation has not been established.

OBJECTIVES

The objectives were to assess the clinical effectiveness and cost-effectiveness of a cognitive rehabilitation programme for people with multiple sclerosis.

DESIGN

This was a multicentre, randomised controlled trial in which participants were randomised in a ratio of 6 : 5 to receive cognitive rehabilitation plus usual care or usual care alone. Participants were assessed at 6 and 12 months after randomisation.

SETTING

The trial was set in hospital neurology clinics and community services.

PARTICIPANTS

Participants were people with multiple sclerosis who had cognitive problems, were aged 18-69 years, could travel to attend group sessions and gave informed consent.

INTERVENTION

The intervention was a group cognitive rehabilitation programme delivered weekly by an assistant psychologist to between four and six participants for 10 weeks.

MAIN OUTCOME MEASURES

The primary outcome was the Multiple Sclerosis Impact Scale - Psychological subscale at 12 months. Secondary outcomes included results from the Everyday Memory Questionnaire, the 30-Item General Health Questionnaire, the EuroQol-5 Dimensions, five-level version and a service use questionnaire from participants, and the Everyday Memory Questionnaire - relative version and the Modified Carer Strain Index from a relative or friend of the participant.

RESULTS

Of the 449 participants randomised, 245 were allocated to cognitive rehabilitation (intervention group) and 204 were allocated to usual care (control group). Of these, 214 in the intervention group and 173 in the control group were included in the primary analysis. There was no clinically important difference in the Multiple Sclerosis Impact Scale - Psychological subscale score between the two groups at the 12-month follow-up (adjusted difference in means -0.6, 95% confidence interval -1.5 to 0.3; p = 0.20). There were no important differences between the groups in relation to cognitive abilities, fatigue, employment, or carer strain at follow-up. However, there were differences, although small, between the groups in the Multiple Sclerosis Impact Scale - Psychological subscale score at 6 months (adjusted difference in means -0.9, 95% confidence interval -1.7 to -0.1; p = 0.03) and in everyday memory on the Everyday Memory Questionnaire as reported by participants at 6 (adjusted difference in means -5.3, 95% confidence interval -8.7 to -1.9) and 12 months (adjusted difference in means -4.4, 95% confidence interval -7.8 to -0.9) and by relatives at 6 (adjusted difference in means -5.4, 95% confidence interval -9.1 to -1.7) and 12 months (adjusted difference in means -5.5, 95% confidence interval -9.6 to -1.5) in favour of the cognitive rehabilitation group. There were also differences in mood on the 30-Item General Health Questionnaire at 6 (adjusted difference in means -3.4, 95% confidence interval -5.9 to -0.8) and 12 months (adjusted difference in means -3.4, 95% confidence interval -6.2 to -0.6) in favour of the cognitive rehabilitation group. A qualitative analysis indicated perceived benefits of the intervention. There was no evidence of a difference in costs (adjusted difference in means -£574.93, 95% confidence interval -£1878.93 to £729.07) or quality-adjusted life-year gain (adjusted difference in means 0.00, 95% confidence interval -0.02 to 0.02). No safety concerns were raised and no deaths were reported.

LIMITATIONS

The trial included a sample of participants who had relatively severe cognitive problems in daily life. The trial was not powered to perform subgroup analyses. Participants could not be blinded to treatment allocation.

CONCLUSIONS

This cognitive rehabilitation programme had no long-term benefits on quality of life for people with multiple sclerosis.

FUTURE WORK

Future research should evaluate the selection of those who may benefit from cognitive rehabilitation.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN09697576.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 4. See the National Institute for Health Research Journals Library website for further project information.

Development and Validation of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS)

Importance

A new outcome measure for overall disability level with improved responsiveness is needed for amyotrophic lateral sclerosis (ALS) clinical trials.

Objective

To describe the creation and development of a new self-reported ALS disability scale with improved item targeting and psychometric properties that used a mathematically rigorous Rasch methodology.

Design, Setting, and Participants

A preliminary ALS disability questionnaire with 119 questions was created based on literature review, clinical judgement of an expert panel, and patient input. Patients with ALS were recruited from January 2017 to June 2019 from the Emory University and Atlanta VA Medical Center ALS clinics, both in Atlanta, Georgia, during regularly scheduled clinic appointments to complete the draft questionnaire and standard ALS outcome measures. All consecutive patients seen at the Emory University and Atlanta VA Medical Center ALS clinics during the recruitment period with a diagnosis of ALS who were able to provide informed consent were invited to participate in the study. Rasch analyses were performed, and items were systematically removed based on missing data, model fit, disordered thresholds, item bias, and clinical judgment. A total of 509 patients with ALS were seen at the 2 sites during the recruitment period, and 264 patients provided informed consent.

Interventions

Participants completed the draft Rasch questionnaire and the revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R).

Main Outcomes and Measures

Rasch analyses and standard scale metrics were performed to create the new scale, and Rasch analyses were performed on the ALSFRS-R for comparison.

Results

Overall, 243 participants with ALS completed the draft questionnaire, and 230 participants were included for Rasch analyses. The mean (SD) age for study participants was 61.9 (11.1) years, 146 (60.1%) were men, and site of onset was 23.0% bulbar (n = 56), 36.2% upper extremity (n = 88), and 39.5% lower extremity (n = 96). A 28-question Rasch-Built Overall ALS Disability Scale (ROADS) was constructed with each item scored 0, 1, or 2. The ROADS fulfilled Rasch model requirements, demonstrated improved item targeting compared with the ALSFRS-R, and had test-retest reliability of 0.97. Individual question fit statistics demonstrated infit values from 0.68 to 1.37 and outfit values from 0.66 to 1.43. The difference between the empirical variance explained by the measures and the modeled variance was 0.1%. The ALSFRS-R violated Rasch model expectations and demonstrated disordered thresholds for 9 of 12 questions; 13 of 48 answer choices on the ALSFRS-R were never the most probable answer choice for any overall disability level.

Conclusions and Relevance

In this study, the 28-question, self-reported ROADS, which is linearly weighted, had improved item targeting compared with the ALSFRS-R, had high test-retest reliability, and was validated. ROADS may serve as a valuable and easily accessible outcome measure for use in ALS trials and in the clinic with improved responsiveness compared with the ALSFRS-R.