Importance of experimenter-blind procedure in neurotoxicology

Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording

Observer bias and other "experimenter effects" occur when researchers' expectations influence study outcome. These biases are strongest when researchers expect a particular result, are measuring subjective variables, and have an incentive to produce data that confirm predictions. To minimize bias, it is good practice to work "blind," meaning that experimenters are unaware of the identity or treatment group of their subjects while conducting research. Here, using text mining and a literature review, we find evidence that blind protocols are uncommon in the life sciences and that nonblind studies tend to report higher effect sizes and more significant p-values. We discuss methods to minimize bias and urge researchers, editors, and peer reviewers to keep blind protocols in mind.

Long-term perchloroethylene exposure: a meta-analysis of neurobehavioral deficits in occupationally and residentially exposed groups

The literature concerning the neurobehavioral and neurophysiological effects of long-term exposure to perchloroethylene (PERC) in humans was meta-analyzed to provide a quantitative review and synthesis in the form of dose-effect curves. The useable database from this literature comprised studies reporting effects of long-term exposure to PERC, effects that included slowed reaction times, cognitive deficits, impaired color vision, and reduced visual contrast sensitivity. For the meta-analyses, dose was defined as the product of the concentration inhaled PERC and the duration of exposure, expressed in unites of ppm-h/1000 (for numerical convenience). Dose-related results were highly variable across studies. Reports involving low exposure concentrations characteristic of nonoccupational exposures consistently produced effects of a magnitude that were comparable to those reported for higher concentration occupational studies. If this finding is reliable and general, studies of occupationally exposed persons may underestimate the magnitude of effects of PERC and other chemicals in the total population. Given the limited scope of the available data for PERC and its methodological and reporting problems (small sample sizes, testers were not blind to the subjects' exposure conditions, and the timing and location of testing were insufficiently documented), it seems important to test this conclusion with a well-documented study of two groups (occupational and nonoccupational exposure) in which subjects are evaluated in randomized order, using the same procedures and with the testers kept blind to the status of the subjects.

Meta-analysis — a systematic and quantitative review of animal experiments to maximise the information derived

Meta-analysis provides a tool to statistically aggregate data from existing randomised controlled animal experiments. The results can then be summarised across a range of conditions and an increased pool of experimental data can be subjected to statistical analysis. New information can be derived, but most frequently the results are a refinement of existing knowledge. By designing experiments and reporting protocols, so that they have the capability of being useful to meta-analyses, maximum benefit can be derived from individual randomised controlled experiments, which may individually have little statistical power, and new avenues for productive research identified. The methodology for meta-analysis is derived from clinical trials in the medical sciences. Now that there is substantial output from animal science experiments, there is an opportunity to apply the technique to these and reduce the need for further experimentation. This paper describes the contribution of meta-analysis to the reduction of animals in research and provides details on data collection, analysis, the models used, and on interpreting and reporting the results. Three applications of meta-analysis to the field of animal science are also briefly described. First, the impact of undernutrition on the production and composition of milk from dairy cows confirmed existing knowledge about partitioning scarce nutrients to milk yield and live weight. Second, increased absorption of cadmium — a widespread toxic element — from organic sources was detected in sheep, which was previously untested. Third, no significant relationships were found between common indicators of undernutrition and weight, and condition score in cattle suggesting that the common indicators used are not suitable as evidence of long term undernutrition. This paper concludes that opportunities exist to increase the information gained from animal experiments by subjecting the results to meta-analysis, particularly if this can be anticipated in advance of study protocols being constructed.

Human neurobehavioral effects of long-term exposure to styrene: a meta-analysis

Many reports in the literature suggest that long-term exposure to styrene may exert a variety of effects on the nervous system, including increased choice reaction time and decreased performance of color discrimination and color arrangement tasks. Sufficient information exists to perform a meta-analysis of these observations quantifying the relationships between exposure (estimated from biomarkers) and effects on two measures of central nervous system function: reaction time and color vision. To perform the meta-analysis, we pooled data into a single database for each end point. End-point data were transformed to a common metric of effect magnitude (percentage of baseline). We estimated styrene concentration from biomarkers of exposure and fitted linear least-squares equations to the pooled data to produce dose-effect relationships. Statistically significant relationships were demonstrated between cumulative styrene exposure and increased choice reaction time as well as increased color confusion index. Eight work-years of exposure to 20 ppm styrene was estimated to produce a 6.5% increase in choice reaction time, which has been shown to significantly increase the probability of automobile accidents. The same exposure history was predicted to increase the color confusion index as much as 1.7 additional years of age in men.

Memory one month after acute carbon monoxide intoxication: a prospective study

BACKGROUND

Serious delayed neuropsychological sequelae may complicate carbon monoxide intoxication. The existence of minor manifestations, especially memory disturbances, is not well documented.

AIMS

To study several memory functions after carbon monoxide intoxication.

METHODS

In a prospective study, 32 poisoned patients without risk factors for cognitive disturbances were compared to 32 paired control subjects one month after acute carbon monoxide intoxication (blood carbon monoxide concentration at least 1.0 mmol/l), who had been treated with standard conventional therapy. Psychometric tests included Buschke's verbal memory testing, verbal digit span, Corsi's visuospatial span, reaction times, Stroop's colours decoding test, and verbal fluency test.

RESULTS

(1) Memory functions in poisoned subjects were not worse than in the control group and were even better in some areas: learning, word recall, and quality of learning by Buschke's verbal memory testing. Attention was also better in the patients, in whom visual reaction time was shorter than in controls. (2) Results of several memory functions-quality of learning and immediate visual memory-were positively correlated with the initial carbon monoxide level.

CONCLUSIONS

In a highly selected subset of patients devoid of risk factors for memory impairment, memory, objectively evaluated by psychometric testing, was not worse one month after carbon monoxide intoxication in patients undergoing standard treatment than in paired control subjects.

Carbon monoxide and the nervous system

Carbon monoxide (CO) is a colorless, tasteless, odorless, and non-irritating gas formed when carbon in fuel is not burned completely. It enters the bloodstream through the lungs and attaches to hemoglobin (Hb), the body's oxygen carrier, forming carboxyhemoglobin (COHb) and thereby reducing oxygen (O(2)) delivery to the body's organs and tissues. High COHb concentrations are poisonous. Central nervous system (CNS) effects in individuals suffering acute CO poisoning cover a wide range, depending on severity of exposure: headache, dizziness, weakness, nausea, vomiting, disorientation, confusion, collapse, and coma. At lower concentrations, CNS effects include reduction in visual perception, manual dexterity, learning, driving performance, and attention level. Earlier work is frequently cited to justify the statement that CO exposure sufficient to produce COHb levels of ca. 5% would be sufficient to produce visual sensitivity reduction and various neurobehavioral performance deficits. In a recent literature re-evaluation, however, the best estimate was that [COHb] would have to rise to 15-20% before a 10% reduction in any behavioral or visual measurement could be observed. This conclusion was based on (1) critical review of the literature on behavioral and sensory effects, (2) review and interpretation of the physiological effects of COHb on the CNS, (3) extrapolation from the effects of hypoxic hypoxia to the effects of CO hypoxia, and (4) extrapolation from rat behavioral effects of CO to humans. Also covered in this review article are effects of chronic CO exposure, the discovery of neuroglobin, a summary of the relatively new role for endogenous CO in neurotransmission and vascular homeostasis, groups which might be especially sensitive to CO, and recommendations on further research. The interested reader is directed to other published reviews of the literature on CO and historically seminal references that form our understanding of this ubiquitous gas.

Do "we just know"? Masked assessors' ability to accurately identify children with prenatal cocaine exposure

This study evaluated perceptions of masked assessors to determine whether there are subtle differences in cocaine-exposed and unexposed children who might be identified by those interacting with children. As part of a longitudinal study, developmental assessors were masked to 163 4-year-old children's actual in utero cocaine exposure status and developmental history. After each battery, assessors documented their guesses of the child's cocaine exposure. Thirty-seven percent of the children who were exposed were misclassified as unexposed, whereas 74% of those unexposed were incorrectly classified as exposed. Although the sample did not differ on assessment scores when results were analyzed by actual cocaine exposure status ( >.3), children who did less well on assessments were more likely to be labeled by assessors as cocaine-exposed ( <.001). Results highlight the potential of stereotyping and negative attributions that might distort observations, both in unmasked studies of prenatal cocaine exposure and in clinical settings.

Neurotoxicologic examination of rats exposed to 1,1,2,2-tetrachloroethylene (perchloroethylene) vapor for 13 weeks

Large evoked potential and EEG changes occurred in a pilot study in Fischer 344 rats during exposure to 800 ppm of 1,1,2,2-tetrachloroethylene [perchloroethylene (Perc)], a cleaning solvent with anesthetic properties. In the main study, rats were evaluated for persistent nervous system effects the week following exposure to 0, 50, 200, or 800 ppm Perc for 6 h/day, 5 days/week, for 13 weeks. The only effect related to treatment was in the flash evoked potential (FEP-V), recorded from the visual cortex. The longer latency potentials (N3) of the FEP-V had a greater amplitude in the 800 ppm Perc group. The FEP-Vs were of normal shape and latency. Although mild neurotoxicity could not be ruled out completely, amplitude changes in N3 can occur for a variety of psychophysiological reasons other than neurotoxicity. Consequently, as a stand-alone finding, the toxicologic significance of the larger FEP in the 800 ppm exposure group was unknown. Other data did not support a diagnosis of neurotoxicity. No treatment-related alterations were noted in expanded clinical observations, in the FEP recorded from the cerebellum (as opposed to visual cortex FEP-V), or in auditory, somatosensory, or caudal nerve evoked potentials. No treatment-related lesions were noted during histopathologic examination of eyes, optic nerves, optic tract, or multiple sections of brain, spinal cord, peripheral nerves, or limb muscles. The no-observed-effect-level (NOEL) was 200 ppm, based on increased amplitude of the longer latency potentials of the FEP at 800 ppm.

A few considerations in the design and analysis of experiments in neurotoxicology

To help minimize errors and confusion in the scientific literature, a few principles can be followed. The original intent and hypothesis of each study should be made clear, and deviations from the initial purpose should be stated. The hypothesis-testing portions of the research need to be clearly differentiated from the hypothesis generating sections. If data were selected and reanalyzed in unintended ways, the analyses should be clearly identified as hypothesis generating, and no conclusions should be drawn from such data. The data should be unassailable. The control group should be adequate. The data analysis should be consistent with the experimental design. Given that the larger the number of P-values, the greater the rate of false declarations, the total number of derived P-values should be reasonable and should be reported. The publication should unveil all details necessary to understand and replicate the research project, including the data analysis. Alternate etiologies should be seriously considered in light of potential confounding factors. It is important to be critical of the results of a study, especially when data confirm a preconceived hypothesis.

The neuroepidemiology of styrene: a critical review of representative literature

Because exposure to styrene occurs commonly in some industries and styrene is highly lipid soluble, it is reasonable to be concerned about the possibility that styrene is neurotoxic. Styrene, like many other solvents, volatile anesthetics, and drugs, does, at certain concentrations, produce acute changes in consciousness with consequent alterations of feelings, cognition, and psychomotor functioning. Such acute actions do not imply that styrene also would produce reversible or irreversible damage to the nervous system; the evaluation of long-term exposures to styrene also is necessary to draw conclusions about the full range of neural effects that styrene might produce. To that end, several studies of workers exposed to styrene for up to 30 years have been undertaken in factories in many parts of the world. Epidemiologists have suggested that neuropsychological deficits such as slowing of reaction time, loss of color vision, and vestibulooculomotor dysfunction are reliably induced by styrene at levels near or below current exposure standards, which range from 20 to 50 ppm in most of the world. However, the workers so studied always were described as healthy, and the effects noted were considered to be subclinical. A detailed evaluation of much of the neuroepidemiological literature on styrene (38 papers and related literature), however, indicated that the findings were, almost universally, false positive outcomes due to (1) type I statistical error, (2) the action of some factor other than styrene, and (3) misinterpretation of data. Despite the study of workers exposed for many years, no indications of persisting damage to the nervous system were evident from this review. The conclusions of this review of the neuroepidemiology of styrene are consistent with those based on critical reviews of the solvent literature in general, with specific reference to the probable absence of such an entity as the "painter's syndrome" or "chronic toxic encephalopathy". Because the results on styrene neurotoxicity that provide an inclination to lower the current threshold limit values (TLVs) are false positive findings, there is no scientific basis for a reduction in the current TLV.