Continuing challenges for computer-based neuropsychological tests

A Flexible and Integrated System for the Remote Acquisition of Neuropsychological Data in Stroke Research

BACKGROUND

Neuropsychological testing is a central aspect of stroke research because it provides critical information about the cognitive-behavioral status of stroke survivors, as well as the diagnosis and treatment of stroke-related disorders. Standard neuropsychological methods rely upon face-to-face interactions between a patient and researcher, which creates geographic and logistical barriers that impede research progress and treatment advances.

INTRODUCTION

To overcome these barriers, we created a flexible and integrated system for the remote acquisition of neuropsychological data (RAND). The system we developed has a secure architecture that permits collaborative videoconferencing. The system supports shared audiovisual feeds that can provide continuous virtual interaction between a participant and researcher throughout a testing session. Shared presentation and computing controls can be used to deliver auditory and visual test items adapted from standard face-to-face materials or execute computer-based assessments. Spoken and manual responses can be acquired, and the components of the session can be recorded for offline data analysis.

MATERIALS AND METHODS

To evaluate its feasibility, our RAND system was used to administer a speech-language test battery to 16 stroke survivors with a variety of communication, sensory, and motor impairments. The sessions were initiated virtually without prior face-to-face instruction in the RAND technology or test battery.

RESULTS

Neuropsychological data were successfully acquired from all participants, including those with limited technology experience, and those with a communication, sensory, or motor impairment. Furthermore, participants indicated a high level of satisfaction with the RAND system and the remote assessment that it permits.

CONCLUSIONS

The results indicate the feasibility of using the RAND system for virtual home-based neuropsychological assessment without prior face-to-face contact between a participant and researcher. Because our RAND system architecture uses off-the-shelf technology and software, it can be duplicated without specialized expertise or equipment. In sum, our RAND system offers a readily available and promising alternative to face-to-face neuropsychological assessment in stroke research.

Simple reaction time in 8-9-year old children environmentally exposed to PCBs

Simple reaction time (SRT) has been studied in children exposed to polychlorinated biphenyls (PCBs), with variable results. In the current work we examined SRT in 146 boys and 161 girls, aged 8.53 ± 0.65 years (mean ± SD), exposed to PCBs in the environment of eastern Slovakia. We divided the children into tertiles with regard to increasing PCB serum concentration. The mean ± SEM serum concentration of the sum of 15 PCB congeners was 191.15 ± 5.39, 419.23 ± 8.47, and 1315.12 ± 92.57 ng/g lipids in children of the first, second, and third tertiles, respectively. We created probability distribution plots for each child from their multiple trials of the SRT testing. We fitted response time distributions from all valid trials with the ex-Gaussian function, a convolution of a normal and an additional exponential function, providing estimates of three independent parameters μ, σ, and τ. μ is the mean of the normal component, σ is the standard deviation of the normal component, and τ is the mean of the exponential component. Group response time distributions were calculated using the Vincent averaging technique. A Q-Q plot comparing probability distribution of the first vs. third tertile indicated that deviation of the quantiles of the latter tertile from those of the former begins at the 40th percentile and does not show a positive acceleration. This was confirmed in comparison of the ex-Gaussian parameters of these two tertiles adjusted for sex, age, Raven IQ of the child, mother's and father's education, behavior at home and school, and BMI: the results showed that the parameters μ and τ significantly (p ≤ 0.05) increased with PCB exposure. Similar increases of the ex-Gaussian parameter τ in children suffering from ADHD have been previously reported and interpreted as intermittent attentional lapses, but were not seen in our cohort. Our study has confirmed that environmental exposure of children to PCBs is associated with prolongation of simple reaction time reflecting impairment of cognitive functions.

I just want my research credit: frequency of suboptimal effort in a non-clinical healthy undergraduate sample

Although performance validity testing is becoming fairly routine in clinical settings, research protocols involving neuropsychological tests infrequently include assessments of performance validity. The current study utilized an embedded measure of effort over two administrations of CNS Vital Signs to determine the frequency of poor effort in non-clinical healthy undergraduate students participating in a research study for course credit. Results indicate that more than 1 in 10 college students participating in a cognitive test battery for research showed test scores consistent with inadequate effort, which was associated with poor performance on testing across many domains. This conclusion was supported by poor performance on many other subtests. Healthy college students with suboptimal effort (n = 11) had an overall score in the 15th percentile on average compared to the 48th percentile in the rest of the students (n = 66). Those who failed validity indicators on the baseline administration were more likely to fail validity indicators on the repeat administration. Those who were tested in the morning were also more likely to fail validity indicators. The current study provides evidence for the potential limitations of conducting research using neuropsychological tests with healthy college student volunteers in the absence of performance validity testing. Revised college-level cutoffs are proposed.

Effects of occupational exposure to chlorpyrifos on neuropsychological function: a prospective longitudinal study

BACKGROUND

Exposure to chlorpyrifos (CPF), an organophosphorus (OP) anticholinesterase insecticide, occurs typically in settings where multiple agents are present (e.g., agriculture) and quantitative dose measures may be absent (e.g., pesticide application). Such exposures allow few opportunities to study potential neurobehavioral effects of CPF alone. We studied the relationship between CPF exposure and behavioral function among CPF manufacturing workers, which allowed identification, measurement, and estimation of exposure and important non-exposure variables that potentially could affect study findings.

METHODS

A prospective longitudinal study design was used to compare neurobehavioral function over a one-year period among 53 CPF workers and 60 referent workers. Quantitative and qualitative measures were used, and potential confounders were identified and tested for possible inclusion in our statistical models. Neurobehavioral function was assessed by neuropsychological tests covering various behavioral domains that may be adversely affected by exposure to CPF in sufficient amount.

RESULTS

CPF workers had significantly greater CPF exposures during the study period than did referents at levels where physiologic effects on plasma butyrylcholinesterase (BuChE) activity were apparent and with higher 3,5,6-trichloro-2-pyridinol (TCPy/Cr) urinary excretion (p<0.0001) and lower average BuChE activity (p<0.01). No evidence for impaired neurobehavioral domains by either group of workers was observed at baseline, on repeat examination, or between examinations. CPF workers scored higher than referent workers on the verbal memory domain score (p=0.03) at baseline, but there were no significant changes in verbal memory over time and no significant group-by-time interactions.

CONCLUSIONS

The study provides important information about CPF exposure in the workplace by not supporting our working hypothesis that CPF exposure associated with various aspects of the manufacturing process would be accompanied by adverse neurobehavioral effects detectable by quantitative neurobehavioral testing. Some aspects making this workplace site attractive for study and also present limitations for the generalization of results to other situations that might have exposures that vary widely between and within different facilities and locations. For example, these results might not apply to occupations such as applicators with higher exposure or to workers with low educational levels.

The Human-Computer Interface in Computer-Based Concussion Assessment

Recent progress in technology has allowed for the development and validation of computer-based adaptations of existing pencil-and-paper neuropsychological measures and comprehensive cognitive test batteries. These computer-based assessments are frequently implemented in the field of clinical sports psychology to evaluate athletes’ functioning postconcussion. These tests provide practical and psychometric advantages over their pencil-and-paper counterparts in this setting; however, these tests also provide clinicians with unique challenges absent in paper-and-pencil testing. The purpose of this article is to present advantages and disadvantages of computer-based testing, generally, as well as considerations for the use of computer-based assessments for the evaluation of concussion among athletes. Furthermore, the paper provides suggestions for further development of computerized assessment of sports concussion given the limitations of the current technology.

Factors affecting hand tremor and postural sway in children

OBJECTIVES

It is crucial to consider covariates relevant for outcome variables in developing dose-effect relations of environmental hazardous toxins. The aim of this study was to clarify the covariates affecting hand tremor and postural sway in children.

METHODS

Hand tremor and postural sway, as well as hair mercury concentrations, were measured in 155 boys and 148 girls at age 7 years.

RESULTS

Current mercury concentrations in child hair ranged from 0.35 to 6.32 μg/g (geometric mean, 1.71 μg/g for boys and 1.58 μg/g for girls), and were not significantly correlated with the neuromotor parameters. All hand tremor and postural sway parameters, except for tremor intensity at 1-6 Hz with non-dominant hand, were significantly larger in the boys than in the girls. Using multiple regression analysis, some postural sway parameters were related negatively to age in the boys and girls (p<0.05), and positively to height (p<0.05). Similarly, hand tremor parameters were positively related to age, height and heart rate either in the boys or in the girls (p<0.05). Also, there were positive relationships between tremor intensity at 1-6 Hz and transversal and sagittal sways at 1-2 Hz and 2-4 Hz (p<0.05).

CONCLUSIONS

Heart rate and postural sway, together with age, sex, and height, should be considered in interpreting hand tremor in children. Hand tremor or postural sway may not be so sensitive or specific to methylmercury exposures at levels of less than 7 μg/g in hair.

Evaluation of reliability of computerized neurobehavioral tests in Korean children

Neurobehavioral tests are among the most efficient methods of identifying the adverse health effects of neurotoxicants. The reliability of neurobehavioral tests may be affected by racial or cultural backgrounds, but the widely used computerized neurobehavioral tests have been developed and standardized for Western children. It is thus necessary to assess the reliability of the existing computerized neurobehavioral tests for Korean children. For this reason, 254 healthy 7- to 8-year-old Korean children completed a neurobehavioral test-retest, with the test and retest held two months apart. Six neurobehavioral test items adapted from Korean Computerized Neurobehavioral Tests (KCNT) and modified to match the children's ability levels: Simple Reaction Time, Choice Reaction Time, Color Word Vigilance, Addition, Symbol Digit, and Finger Tapping Speed. The test reliability was assessed using the Pearson product-moment correlation coefficient (r) and the intraclass correlation coefficient (ICC). The ICCs ranged from 0.46 to 0.84 and were very similar to the Pearson coefficients. High reliability was detected in Symbol Digit (r=0.84, ICC=0.83), followed by the Finger Tapping Speed of the dominant hand (r=0.67, ICC=0.67) and of the non-dominant hand (r=0.65, ICC=0.65). The study findings suggest that the reliability of most computerized neurobehavioral tests is appropriate for epidemiological researches on Korean children, and that Symbol Digit and Finger Tapping Speed are more satisfactory bases for the periodic examination of neurobehavioral performance. These findings can also be useful in the future assembly of a neurobehavioral test battery, by providing more stable neurobehavioral test items for Korean children.

A brief history of INA and ICOH SCNP: International Neurotoxicology Association and International Congress on Occupational Health Scientific Committee on Neurotoxicology and Psychophysiology

Two international scientific societies dedicated to research in neurotoxicology and neurobehavioral toxicology are the International Neurotoxicology Association (INA) and the International Congress on Occupational Health International Scientific Committee on Neurotoxicology and Psychophysiology (ICOH SCNP). From June 5-10, 2011 these two societies held a joint conference in Xi'an China entitled the Xi'an International Neurotoxicology Conference, Neurotoxicity and Neurodegeneration: Local Effect and Global Impact. At the conference two featured talks presented a brief history of the two societies. This article is a synthesis and expansion of those two presentations. The history of INA and ICOH SCNP is described in relation to the antecedent events leading to the formation of the two societies, their parallel developments, the nature of the societies and their scientific conferences, and a brief description of some of their accomplishments. Together, the historical development of these two societies is an important component of the development of the scientific discipline of neurotoxicology.

Computerized analysis of speech and language to identify psycholinguistic correlates of frontotemporal lobar degeneration

OBJECTIVE

To evaluate the use of a semiautomated computerized system for measuring speech and language characteristics in patients with frontotemporal lobar degeneration (FTLD).

BACKGROUND

FTLD is a heterogeneous disorder comprising at least 3 variants. Computerized assessment of spontaneous verbal descriptions by patients with FTLD offers a detailed and reproducible view of the underlying cognitive deficits.

METHODS

Audiorecorded speech samples of 38 patients from 3 participating medical centers were elicited using the Cookie Theft stimulus. Each patient underwent a battery of neuropsychologic tests. The audio was analyzed by the computerized system to measure 15 speech and language variables. Analysis of variance was used to identify characteristics with significant differences in means between FTLD variants. Factor analysis was used to examine the implicit relations between subsets of the variables.

RESULTS

Semiautomated measurements of pause-to-word ratio and pronoun-to-noun ratio were able to discriminate between some of the FTLD variants. Principal component analysis of all 14 variables suggested 4 subjectively defined components (length, hesitancy, empty content, grammaticality) corresponding to the phenomenology of FTLD variants.

CONCLUSION

Semiautomated language and speech analysis is a promising novel approach to neuropsychologic assessment that offers a valuable contribution to the toolbox of researchers in dementia and other neurodegenerative disorders.

Hänninen lecture: Problems and prevention: Research in developing countries and immigrant populations from developing countries

The tradition of this excellent Triennial International Symposium, now in its Ninth gathering, has focused first on methods and later on effects in neurotoxicology. I respectfully suggest that it is time to add prevention to our agenda in order to focus not just on finding problems, but also on solving them. Otherwise our research runs the risk of making conditions worse for the participants because we inform them they have a problem but do not show them how to prevent that problem. Many barriers have been surmounted since the early days of this Symposium. Methods developed in industrialized nations have been modified and used in populations with limited education. Now, the primary barriers to effective collaborations are institutional. Perhaps the most challenging are companies that can bar a research study, or may welcome the opportunity to safeguard their workforce. A focus on prevention can increase the willingness of institutions, including companies, to accept and support research that examines neurotoxic effects of chemical exposures. This is because we are offering positive value, not simply uncovering a problem for them to fix. The cTRAIN software program is an illustration of a computer-based training method developed by neurotoxicologists that has been implemented in a variety of populations. This includes those from a developing country with limited education and those with college degrees. Training can provide prevention, can easily be appended to any neurotoxicology research study and is well within the skill set of the neurotoxicology community of scientists.

A comparison of MicroCog and the Wechsler Memory Scale (3rd ed.) in older adults

In recent years, computerized assessment of cognitive functions such as memory has increased in popularity. Here we contrast the performance of 33 community-dwelling older adults (mean age 71.2 years, SD = 7.62) on the computer-based MicroCog with the conventional Wechsler Memory Scale-3rd Edition (WMS-III). Participants were screened for possible depression using the Geriatric Depression Scale and completed both memory tests in counterbalanced order. WMS-III General Memory correlated moderately with both the MicroCog Memory index and the General Cognitive Functioning index. Correlations between the visual memory measures of the 2 tests were not statistically significant. Agreement between the tests on the classification of participants as lying within the average, below average, or above average ranges was fair at best. We conclude that the correspondence between the 2 measures is not sufficient to substitute 1 for the other for clinical decision making as to the memory functioning of older adults.

Neurobehavioral science in hazard identification and risk assessment of neurotoxic agents--what are the requirements for further development?

OBJECTIVES

Modern neurobehavioral methods find useful application in research into the early effects of exposure to neurotoxic agents in the environment. This paper briefly describes the history and evolution of neurobehavioral toxicology, reviews some current trends in research in this specific discipline and identifies the most important needs and challenges to be addressed in future studies.

METHODS

All published literature was considered, including ad hoc meeting reports. Further information was obtained directly from experts in the field.

RESULTS

The number of studies (including those in occupational, environmental and pediatric exposure) using neurobehavioral evaluation is constantly increasing. Regulatory agencies are using scientific data obtained through neurobehavioral assessment, which includes other areas such as neurosensory toxicology. However, further development of this discipline is facing a number of problems and issues. Three major areas that deserve further attention have been identified: (1) specific technical issues regarding testing development, (2) epidemiological issues regarding the study design, including the need for meta-analysis/multi-center studies and for longitudinal observation, and statistical issues regarding the most adequate models for the analysis and treatment of complex neurobehavioral datasets, and (3) the need for scientific consensus on the significance of adverse effects identified with neurobehavioral methods.

CONCLUSIONS

The importance of neurobehavioral toxicology in the evaluation of mechanisms of action and for preventive purposes is progressively growing. Further development is needed for the advancement of this discipline through collaboration between experts from different fields.

Further standardization of some NES3 tests

NES3 is a computer-based neurobehavioral testing system designed for use in investigating potential cognitive impairment. Data from NES3 tests employed in a study of epilepsy self-management were analyzed to estimate test-retest reliability for NES3 tests on a large sample and to estimate the effect of some common covariates of test performance. A total of 319 participants in an epilepsy self-management study were examined on three occasions (baseline, 3-month follow-up, and 6-month follow-up) with a set of psychological questionnaires and seven neuropsychological tests from NES3. Test-retest correlations were calculated between measures obtained at baseline and at 3 months. Principal components analysis was performed on the baseline data. The potential effects of covariates (age, education, reading test scores, depression status, and examiner) were investigated in regression models. Three-month test-retest correlations were excellent for Adult Reading Test (ART) (r=0.95), strong for Digit-Symbol (r=0.82), Sequence B (r=0.79), and Sequence A (r=0.76); and modest (r's between 0.56 and 0.67) for Digit Span Forward and Backward, Visual Span Forward and Backward, and Pattern Memory. Alternate-forms correlations were strong for HVLT (r's between 0.71 and 0.82). Principal components analysis yielded four interpretable components. Age and reading score were significant covariates of virtually all of the test summary measures, while education, gender, race, and depression were not generally significant covariates. Changes to the method of calculation of some summary measures, changes to the initial instructions to the subjects, and addition of correctional feedback to subjects during the tests appeared to improve the reliability of some NES3 tests. Implementation of the HVLT and ART in computer-assisted format added breadth of coverage to the battery. NES3 tests may provide reliable, efficient data for use in epidemiologic studies of potential cognitive effects of occupational and environmental exposures.