Reliability of cognitive tests used in Alzheimer's disease

Using the Guttman scale to define and estimate measurement error in items over time: the case of cognitive decline and the meaning of "points lost"

We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as "points lost" in studies of cognitive decline or interventions. To capture this meaning of "point loss", over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of "true score", which are two key consequences of the definition of "measurement error"--and thereby, reliability--under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of "cognitive loss" was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.

Repeated amphetamine administration in rats revealed consistency across days and a complete dissociation between locomotor and hypothalamic-pituitary-adrenal axis effects of the drug

RATIONALE

Most drugs of abuse stimulate both locomotor activity and the hypothalamic-pituitary-adrenal (HPA) axis, but the relationship between the two responses within the same subjects and their reliabilities has been scarcely studied. Our objectives were to study: (1) the consistency and stability across time of locomotor and HPA activation induced by repeated d-amphetamine (AMPH); (2) the relationship between locomotor and hormonal responses to AMPH; and (3) the relationship between novelty-induced activity and both types of responses to the drug.

METHODS

Male adult rats were exposed to a novel environment to study the locomotor response. Later, they were injected with AMPH (2 mg/kg, sc) for 5 days. In Experiment 1, Plasma adrenocorticotropin (ACTH) and corticosterone levels in response to AMPH were studied on days 1, 3, and 5, and locomotor response on days 2 and 4. In Experiment 2, ACTH and corticosterone responses were studied on days 2 and 4, and locomotor response on days 1, 3, and 5.

RESULTS

Across days, both locomotor and HPA responses to the drug were consistent, but independent measures, unrelated to the reactivity to novelty. As measured by the area under the curve, the HPA response to AMPH desensitized with the repeated injection, whereas the initial locomotor response to the drug increased.

CONCLUSIONS

Dissociation exists between HPA and locomotor activation induced by AMPH, which seemed to be both reliable individual traits. Locomotor reactivity to novelty was related neither to HPA nor to locomotor responses to AMPH.

Bayesian statistics in medicine: a 25 year review

This review examines the state of Bayesian thinking as Statistics in Medicine was launched in 1982, reflecting particularly on its applicability and uses in medical research. It then looks at each subsequent five-year epoch, with a focus on papers appearing in Statistics in Medicine, putting these in the context of major developments in Bayesian thinking and computation with reference to important books, landmark meetings and seminal papers. It charts the growth of Bayesian statistics as it is applied to medicine and makes predictions for the future. From sparse beginnings, where Bayesian statistics was barely mentioned, Bayesian statistics has now permeated all the major areas of medical statistics, including clinical trials, epidemiology, meta-analyses and evidence synthesis, spatial modelling, longitudinal modelling, survival modelling, molecular genetics and decision-making in respect of new technologies.

Challenges in the clinical diagnosis of Alzheimer's disease: influence of "family coaching" on the mini-mental state examination

The Mini-Mental State Examination (MMSE) is a commonly used clinical tool for evaluating the cognitive aspects of mental function. In this study, 40 consecutive patients presenting to a memory disorder clinic and their caregivers were evaluated for coaching of the patient with respect to MMSE content over the 24 hours prior to clinical evaluation. Caregivers completed a questionnaire concerning MMSE practice sessions with the patient prior to the physician encounter; then, the patients were asked to spell the word "WORM" backwards instead of "WORLD" during the MMSE test. Some or all of the MMSE content was reviewed with the patient prior to the interview by 42.5 percent of the caregivers; 17.5 percent of patients spelled or attempted to spell "WORLD" backwards instead of "WORM". These results demonstrate that coaching of patients prior to administration of the MMSE is not uncommon, and that this needs to be taken into consideration when forming therapeutic decisions based on MMSE results.

Progress in clinical neurosciences: cognitive markers of progression in Alzheimer's disease

The objective of this review is to summarize the literature on Alzheimer's disease progression utilizing cognitive batteries to track change over time. Studies published in English and obtained through PubMed searches (1983-2004) were included (i) if they had a longitudinal design and followed probable Alzheimer's patients diagnosed by National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association or Diagnostic and Statistical Manual III/IV criteria, and (ii) if the techniques used for serial assessment were well-established in terms of validity and reliability. Longitudinal studies examining Alzheimer's disease progression report highly variable annual rates of change in decline rate. It remains unclear if this reflects disease subgroups or stage-related rate of decline. In conclusion a combination of stage-appropriate cognitive tests such as the Mattis Dementia Rating Scale and the Severe Impairment Battery, along with appropriate statistical methods to account for individual variability in decline rates, can capture the progression of Alzheimer disease and may be useful in further investigation.

Analyzing reliability of change in depression among persons with rheumatoid arthritis

OBJECTIVE

To examine several methods of determining reliability of change constructs in depressive symptoms in patients with rheumatoid arthritis (RA) and to demonstrate the strengths, weaknesses, and uses of each method.

METHODS

Data were analyzed from a cohort of 54 persons with RA who participated in a combined behavioral/pharmacologic intervention of 15 months duration. These longitudinal data were used to examine 3 methodologies for assessing the reliability of change for various measures of depression. The specific methodologies involved the calculations of reliable change, sensitivity to change, and reliability of the change score.

RESULTS

The analyses demonstrated differences in reliability of change performance across the various depression measures, which suggest that no single measure of depression for persons with RA should be considered superior in all contexts.

CONCLUSION

The findings highlight the value of utilizing reliability of change constructs when examining changes in depressive symptoms over time.

Evaluation and correction for a 'training effect' in the cognitive assessment of older adults

BACKGROUND

Repeated administration of psychometric instruments frequently results in a higher score on retesting, the so-called training effect. Yet, a training effect has been poorly considered in longitudinal studies of cognitive changes in older persons.

METHODS

We investigated the presence, magnitude and potential adjustments for training effect in the older participants of the Systolic Hypertension in the Elderly Program (SHEP). SHEP evaluated the cognitive status effects of a diuretic-based treatment of isolated systolic hypertension versus placebo. Changes in the short Comprehensive Assessment and Referral Evaluation (short-CARE) questionnaire score, from baseline through 4 years of follow-up, were assessed in 4,718 participants. In this study, we used two regression techniques to adjust data for the training effect.

RESULTS

In both study groups, a training effect was evident as a progressive improvement in the short-CARE score throughout year 1. Thereafter, cognitive scores tended to deteriorate, more in the placebo than in the active treatment group (p = 0.055). When follow-up scores were adjusted based upon baseline data, the difference between the study groups reached statistical significance (p = 0.019), but the apparent overall trend towards deterioration in cognitive score was no longer observed. Adjustment of baseline data preserved this apparent temporal course, but did not improve the discrimination between the two study groups (p = 0.165).

CONCLUSIONS

In SHEP, repeated cognitive assessments were likely biased by a training effect which could be only partially corrected by statistical techniques. Studies of changes in the cognitive status of older persons should be designed appropriately to estimate and minimize the consequences of a training effect in follow-up data.

Modeling the time-course of Alzheimer dementia

Alzheimer's disease (AD) progresses from a preclinical period, through a middle phase of cognitive deterioration, to a late, profound state. The temporal progression of disability can be modeled with a horologic (time-based) function using "time-index" (TI) intervals (day- or year-units) to quantify an individual's disability across multiple cognitive and functional domains relative to a reference AD population. Clinicians and researchers can use TI quantification to assess dementia severity and initial therapy benefits. Rate of progression and confidence intervals require at least two successive measurements. Rate of progression measures can be used to support diagnosis and to investigate disease-course-modifying therapies.