Multivariate normative comparisons using an aggregated database

An online platform and a dynamic database for neuropsychological assessment in Indonesia

Proper use of neuropsychological tests in Indonesia is hindered by a lack of properly adapted neurocognitive tests as well as an absence of normative data. In 2016, we started adapting ten of these tests for use in Indonesia and collected data from healthy participants in Java. Here we introduce and propose a system that will facilitate the proper usage and interpretation of test scores: an online platform and a dynamic database. Newly collected data (492 healthy adults) of the Indonesian version of the Boston Naming Test (I-BNT) were used to illustrate the usefulness of the two functions. Analysis of variances, post-hoc tests, and a simulation study revealed the effects of age and education on the I-BNT, indicating that it is imperative to fine-tune the reference group based on these demographic factors. Putative inadequate sample size issues for obtaining reliable normative scores were overcome by employing regression analyses and the prediction of normative scores. It can be concluded that a flexible online platform is available for the calculation of normative scores either based on the whole population, on fine-tuned reference groups, or on predicted scores. The dynamic database's growth will allow to obtain even more fine-tuned and more reliable reference data as well as more accurate predictions. Fine-tuned reference data are badly needed for the heterogenous Indonesian population.

One size does not fit all: An individualized approach to understand heterogeneous cognitive performance in autistic adults

Cognitive performances of autistic people vary widely. Therefore, previous group-based comparisons on cognitive aging in autistic adults might have overlooked those autistic adults that are particularly vulnerable for cognitive decline. Multivariate normative comparisons (MNC) statistically assess individual cognitive differences on the entire cognitive profile. Cognitive deviancy as indicated by MNC accurately predicts future cognitive decline, and is therefore sensitive in detecting meaningful cognitive differences. The current study aimed to (1) investigate the applicability of MNC to assess cognitive performance in autism individually, and (2) understand heterogeneous cognitive performance in autistic adults. As pre-registered, we performed MNC in a sample of 254 non-autistic adults, and two independent samples of respectively 118, and 86 autistic adults (20-85 years, mean: 50 years). Cognitive performance was measured on 11 outcomes in six domains (verbal/visual memory, working memory, verbal fluency, Theory of Mind, and psychomotor speed). Using MNC, about twice as many autistic individuals had a deviant cognitive profile (i.e., deviated statistically from the multivariate normspace) as compared to non-autistic individuals. Importantly, most autistic individuals (>80%) did not have a deviant cognitive profile. Having a deviant profile was significantly associated with higher levels of psychological distress in autistic adults specifically, showing the clinical relevance of this method. Therefore, MNC seem a useful tool to individually detect meaningful cognitive differences in autism. These results are consistent with previous cognitive studies suggesting that most autistic adults show fairly similar cognitive profiles to non-autistic adults, yet highlight the necessity for approaches reflecting the heterogeneity observed in autistic people.

Implementation of a pilot novel objective peer comparison evaluation system in an emergency medicine residency program

OBJECTIVES

Emergency medicine (EM) residents are currently evaluated via The Milestones, which have been shown to be imperfect and subjective. There is also a need for residents to achieve competency in patient safety and quality improvement processes, which can be accomplished through provision of peer comparison metrics. This pilot study aimed to evaluate the implementation of an objective peer comparison system for metrics that quantified aspects of quality and safety, efficiency and throughput, and utilization.

METHODS

This pilot study took place at an academic, tertiary care center with a 3-year residency and 14 residents per postgraduate year (PGY) class. Metrics were compared within each PGY class using Wilcoxon signed-rank and rank-order analyses.

RESULTS

Significant changes were seen in the majority of the metrics for all PGY classes. PGY3s accounted for the significant change in EKG and X-ray reads, while PGY1s and PGY2s accounted for the significant change in disposition to final note share. Physician evaluation to disposition decision was the only metric that did not reach significance in any class.

CONCLUSIONS

These preliminary data suggest that providing objective metrics is possible. Peer comparison metrics could provide an effective objective addition to the milestone evaluation system currently in use.

An Operational Definition of 'Abnormal Cognition' to Optimize the Prediction of Progression to Dementia: What Are Optimal Cut-Off Points for Univariate and Multivariate Normative Comparisons?

Background:

In neuropsychology and neurology, there is no consensus on the definition of abnormal cognition.

Objective:

To operationally define ‘abnormal cognition’ for optimally predicting progression to dementia in a memory clinic sample, and to test whether multivariate profile analysis of cognitive test results improves this prediction compared to standard clinical evaluation.

Methods:

We used longitudinal data from 835 non-demented patients of the Amsterdam Dementia Cohort. For 10 cognitive measures at baseline, we determined which number of abnormal tests and which magnitude of score deviations best predicted progression.

Results:

Predictive ability for progression to dementia of one, two, and three abnormal test scores out of 10 is highly similar (Cox hazard ratios: 3.7–4.1) provided cut-off values are adapted appropriately. Cut-offs have to be less stringent if the number of abnormal tests required increases: the optimal cut-off is z < –1.45 when one deviating score is required, z < –1.15 when two abnormal tests are required, and z < –0.70 when three abnormal tests are required. The profile analysis has similar predictive ability at the cut-off of p < 0.22 (hazard ratio 3.8). A likelihood ratio test showed that this analysis improves prediction of progression to dementia when added to standard clinical evaluation (p < 0.001).

Conclusion:

Abnormal cognition may be defined as one, two, or three abnormal test scores out of 10 if the magnitude of score deviations is adapted accordingly. An abnormal score profile predicts decline to dementia equally well, and improves the prediction when used complimentary to standard clinical evaluation.

Development of a computerised neurocognitive battery for children and adolescents with HIV in Botswana: study design and protocol for the Ntemoga study

INTRODUCTION

Neurodevelopmental delays and cognitive impairments are common in youth living with HIV. Unfortunately, in resource-limited settings, where HIV infection impacts millions of children, cognitive and neurodevelopmental disorders commonly go undetected because of a lack of appropriate assessment instruments and local expertise. Here, we present a protocol to culturally adapt and validate the Penn Computerized Neurocognitive Battery (PennCNB) and examine its validity for detecting both advanced and subtle neurodevelopmental problems among school-aged children affected by HIV in resource-limited settings.

METHODS AND ANALYSIS

This is a prospective, observational cohort study. The venue for this study is Gaborone, Botswana, a resource-limited setting with high rates of perinatal exposure to HIV and limited neurocognitive assessment tools and expertise. We aim to validate the PennCNB in this setting by culturally adapting and then administering the adapted version of the battery to 200 HIV-infected, 200 HIV-exposed uninfected and 240 HIV-unexposed uninfected children. A series of analyses will be conducted to examine the reliability and construct validity of the PennCNB in these populations.

ETHICS AND DISSEMINATION

This project received ethical approval from local and university Institutional Review Boards and involved extensive input from local stakeholders. If successful, the proposed tools will provide practical screening and streamlined, comprehensive assessments that could be implemented in resource-limited settings to identify children with cognitive deficits within programmes focused on the care and treatment of children affected by HIV. The utility of such assessments could also extend beyond children affected by HIV, increasing general access to paediatric cognitive assessments in resource-limited settings.

Objective cognitive performance and subjective complaints in patients with chronic Q fever or Q fever fatigue syndrome

BACKGROUND

Primary aim of this study was to compare cognitive performance of patients with chronic Q fever or Q fever fatigue syndrome (QFS) to matched controls from the general population, while taking performance validity into account. Second, we investigated whether objective cognitive performance was related to subjective cognitive complaints or psychological wellbeing.

METHODS

Cognitive functioning was assessed with a neuropsychological test battery measuring the domains of processing speed, episodic memory, working memory and executive functioning. Tests for performance validity and premorbid intelligence were also included. Validated questionnaires were administered to assess self-reported fatigue, depressive symptoms and cognitive complaints.

RESULTS

In total, 30 patients with chronic Q fever, 32 with QFS and 35 controls were included. A high percentage of chronic Q fever patients showed poor performance validity (38%) compared to controls (14%, p = 0.066). After exclusion of participants showing poor performance validity, no significant differences between patients and controls were found in the cognitive domains. QFS patients reported a high level of cognitive complaints compared to controls (41.2 vs 30.4, p = 0.023). Cognitive complaints were not significantly related to cognitive performance in any of the domains for this patient group.

CONCLUSIONS

The high level of self-reported cognitive complaints in QFS patients does not indicate cognitive impairment. A large proportion of the chronic Q fever patients showed suboptimal mental effort during neuropsychological assessment. More research into the underlying explanations is needed. Our findings stress the importance of assessing cognitive functioning by neuropsychological examination including performance validity, rather than only measuring subjective cognitive complaints.

Cognitive impairments in patients with persistent symptoms attributed to Lyme disease

Background

Persistent symptoms attributed to Lyme borreliosis often include self-reported cognitive impairment. However, it remains unclear whether these symptoms can be substantiated by objective cognitive testing.

Methods

For this observational study, cognitive performance was assessed in 280 adults with persistent symptoms attributed to Lyme borreliosis (as part of baseline data collected for the Dutch PLEASE study). Cognitive testing covered the five major domains: episodic memory, working memory / attention, verbal fluency, information-processing speed and executive function. Patients’ profiles of test scores were compared to a large age-, education- and sex-adjusted normative sample using multivariate normative comparison. Performance validity was assessed to detect suboptimal effort, and questionnaires were administered to measure self-reported cognitive complaints, fatigue, anxiety, depressive symptoms and several other psychological factors.

Results

Of 280 patients, one was excluded as the test battery could not be completed. Of the remaining 279 patients, 239 (85.4%) displayed sufficient performance validity. Patients with insufficient performance validity felt significantly more helpless and physically fatigued, and less orientated. Furthermore, they had a lower education level and less often paid work. Of the total study cohort 5.7% (n = 16) performed in the impaired range. Among the 239 patients who displayed sufficient performance validity, 2.9% (n = 7) were classified as cognitively impaired. No association between subjective cognitive symptoms and objective impairment was found.

Conclusions

Only a small percentage of patients with borreliosis-attributed persistent symptoms have objective cognitive impairment. Performance validity should be taken into account in neuropsychological examinations of these patients. Self-report questionnaires are insufficiently valid to diagnose cognitive impairment.

Trial registration

ClinicalTrials.gov NCT01207739. Registered 23 September 2010.

Predicting Progression to Parkinson's Disease Dementia Using Multivariate Normative Comparisons

OBJECTIVE

Parkinson's disease with mild cognitive impairment (PD-MCI) is a risk factor for progression to PD dementia (PDD) at a later stage of the disease. The consensus criteria of PD-MCI use a traditional test-by-test normative comparison. The aim of this study was to investigate whether a new multivariate statistical method provides a more sensitive tool for predicting dementia status at 3- and 5-year follow-ups. This method allows a formal evaluation of a patient's profile of test scores given a large aggregated database with regression-based norms.

METHOD

The cognitive test results of 123 newly diagnosed PD patients from a previously published longitudinal study were analyzed with three different methods. First, the PD-MCI criteria were applied in the traditional way. Second, the PD-MCI criteria were applied using the large aggregated normative database. Last, multivariate normative comparisons (MNCs) were made using the same aggregated normative database. The outcome variable was progression to dementia within 3 and 5 years.

RESULTS

The MNC was characterized by higher sensitivity and higher specificity in predicting progression to PDD at follow-up than the two PD-MCI criteria methods, although the difference in classification accuracy did not reach statistical significance.

CONCLUSION

We conclude that MNCs could allow for a more accurate prediction of PDD than the traditional PD-MCI criteria, because there are encouraging trends in both increased sensitivity and increased specificity. (JINS, 2019, 25, 678-687).

Effect of prolonged antibiotic treatment on cognition in patients with Lyme borreliosis

Objective

To investigate whether longer-term antibiotic treatment improves cognitive performance in patients with persistent symptoms attributed to Lyme borreliosis.

Methods

Data were collected during the Persistent Lyme Empiric Antibiotic Study Europe (PLEASE) trial, a randomized, placebo-controlled study. Study participants passed performance-validity testing (measure for detecting suboptimal effort) and had persistent symptoms attributed to Lyme borreliosis. All patients received a 2-week open-label regimen of intravenous ceftriaxone before the 12-week blinded oral regimen (doxycycline, clarithromycin/hydroxychloroquine, or placebo). Cognitive performance was assessed at baseline and after 14, 26, and 40 weeks with neuropsychological tests covering the cognitive domains of episodic memory, attention/working memory, verbal fluency, speed of information processing, and executive function.

Results

Baseline characteristics of patients enrolled (n = 239) were comparable in all treatment groups. After 14 weeks, performance on none of the cognitive domains differed significantly between the treatment arms (p = 0.49–0.82). At follow-up, no additional treatment effect (p = 0.35–0.98) or difference between groups (p = 0.37–0.93) was found at any time point. Patients performed significantly better in several cognitive domains at weeks 14, 26, and 40 compared to baseline, but this was not specific to a treatment group.

Conclusions

A 2-week treatment with ceftriaxone followed by a 12-week regimen of doxycycline or clarithromycin/hydroxychloroquine did not lead to better cognitive performance compared to a 2-week regimen of ceftriaxone in patients with Lyme disease–attributed persistent symptoms.

ClinicalTrials.gov identifier

NCT01207739.

Classification of evidence

This study provides Class II evidence that longer-term antibiotics in patients with borreliosis-attributed persistent symptoms does not increase cognitive performance compared to shorter-term antibiotics.

HIV-Associated Neurocognitive Disorders: A Global Perspective

The present review on HIV-associated neurocognitive disorders (HAND) provides a worldwide overview of studies that have investigated the rate and neuropsychological (NP) profile of HAND research since the inception of the 2007 HAND diagnostic nomenclature. In the first part, the review highlights some of the current controversies around HAND prevalence rates. In the second part, the review critically assesses some solutions to move the field forward. In the third part, we present the cross-sectional NP profile in non-Western HIV+ cohorts and in relation to Western cohorts' findings. The adopted global perspective highlights the successful expansion of NP studies in HIV infection to culturally diverse low- to medium-income countries with high HIV burden. These studies have produced interestingly similar rates of HAND whether patients were naïve or treated and/or virally suppressed compared to the rich income countries where the NP research in NeuroHIV has originated. The perspective also demonstrates that globally, the group which is the most representative of the HIV epidemic, and thus at risk for HAND are persons with chronic HIV infection and survivors of past immunosuppression, while in relative terms, those who have been treated early with long-term viral suppression represent a minority. In the last part, we present a review of the naturalistic longitudinal NP global studies in HIV+cohorts, discuss the role of longitudinal design in solving issues around the question of asymptomatic neurocognitive impairment, and the question of biomarker discovery. Finally, we conclude by calling for greater methods and data harmonization at a global level. (JINS, 2017, 23, 860-869).

Univariate comparisons given aggregated normative data

OBJECTIVE

Normative comparison is a method to compare an individual to a norm group. It is commonly used in neuropsychological assessment to determine if a patient's cognitive capacities deviate from those of a healthy population. Neuropsychological assessment often involves multiple testing, which might increase the familywise error rate (FWER). Recently, several correction methods have been proposed to reduce the FWER. However these methods require that multivariate normative data are available, which is often not the case. We propose to obtain these data by merging the control group data of existing studies into an aggregated database. In this paper, we study how the correction methods fare given such an aggregated normative database.

METHODS

In a simulation study mimicking the aggregated database situation, we compared applying no correction, the Bonferroni correction, a maximum distribution approach and a stepwise approach on their FWER and their power to detect genuine deviations.

RESULTS

If the aggregated database contained data on all neuropsychological tests, the stepwise approach outperformed the other methods with respect to the FWER and power. However, if data were missing, the Bonferroni correction produced the lowest FWER.

DISCUSSION

Overall, the stepwise approach appears to be the most suitable normative comparison method for use in neuropsychological assessment. When the norm data contained large amounts of missing data, the Bonferroni correction proved best. Advice of which method to use in different situations is provided.