[Standardization of the Test Your Memory and evaluation of their concordance with the outcome of the psychometric examination]

Correlation or Limits of Agreement? Applying the Bland-Altman Approach to the Comparison of Cognitive Screening Instruments

BACKGROUND/AIMS

Calculation of correlation coefficients is often undertaken as a way of comparing different cognitive screening instruments (CSIs). However, test scores may correlate but not agree, and high correlation may mask lack of agreement between scores. The aim of this study was to use the methodology of Bland and Altman to calculate limits of agreement between the scores of selected CSIs and contrast the findings with Pearson's product moment correlation coefficients between the test scores of the same instruments.

METHODS

Datasets from three pragmatic diagnostic accuracy studies which examined the Mini-Mental State Examination (MMSE) vs. the Montreal Cognitive Assessment (MoCA), the MMSE vs. the Mini-Addenbrooke's Cognitive Examination (M-ACE), and the M-ACE vs. the MoCA were analysed to calculate correlation coefficients and limits of agreement between test scores.

RESULTS

Although test scores were highly correlated (all >0.8), calculated limits of agreement were broad (all >10 points), and in one case, MMSE vs. M-ACE, was >15 points.

CONCLUSION

Correlation is not agreement. Highly correlated test scores may conceal broad limits of agreement, consistent with the different emphases of different tests with respect to the cognitive domains examined. Routine incorporation of limits of agreement into diagnostic accuracy studies which compare different tests merits consideration, to enable clinicians to judge whether or not their agreement is close.

Test Your Memory is sensitive to cognitive change but lacks prospective validity

OBJECTIVE

To determine the prospective validity of Test Your Memory (TYM) and its sensitivity to change in cognitive state.

METHODS

This longitudinal prospective study followed 71 patients with subjective cognitive symptoms and 48 with mild cognitive impairment for a mean time period of 35.2 ± 15 months. Subjects did not have dementia or depression at the beginning of follow-up and each participant was given the TYM at least two times. A psychometric threshold was established to determine presence of a cognitive deficit (z-score ≤ 1.5 on at least one cognitive domain) and the Disability Assessment for Dementia scale was used to ensure full functional ability. The criterion for deterioration was a change in the stage on the Global Deterioration Scale.

RESULTS

Sixty-one patients remained cognitively stable and 58 worsened. There were no differences between them with respect to sex, educational attainment, the initial stage on the GDS, or the score on the first TYM. Subjects who worsened were older than those who did not. The TYM increased an average of 0.04 points per month in patients who remained stable or improved (95% CI, -0.01 to 0.08) and decreased an average of 0.14 points per month in those whose condition worsened (95% CI, -0.19 to -0.09). Subjects with mild cognitive impairment who worsened displayed a sharper loss of TYM points than did subjects with subjective cognitive symptoms.

CONCLUSIONS

While the TYM lacks prospective validity, it is sensitive to changes in cognitive state.