Repeatability of paired counts

Defining ELISpot cut-offs from unreplicated test and control wells

In the absence of replication of wells, empirical criteria for enzyme-linked immunospot (ELISpot) positivity use fixed differences or ratios between spot forming units (SFU) counts between test and control. We propose an alternative approach which first identifies the optimally variance-stabilizing transformation of the SFU counts, based on the Bland-Altman plot of the test and control wells. The second step is to derive a positivity threshold from the difference in between-plate distribution functions of the transformed test and control SFU counts. This method is illustrated using 1309 assay results from a cohort study of influenza in Vietnam in which some, but not all, of the peptide pools have clear tendencies for SFU counts to be higher in test than control wells.

Review: analysis of parasite and other skewed counts

OBJECTIVE

To review methods for the statistical analysis of parasite and other skewed count data.

METHODS

Statistical methods for skewed count data are described and compared, with reference to a 10-year period of Tropical Medicine and International Health (TMIH). Two parasitological datasets are used for illustration.

RESULTS

The review of TMIH found 90 articles, of which 89 used descriptive methods and 60 used inferential analysis. A lack of clarity is noted in identifying the measures of location, in particular the Williams and geometric means. The different measures are compared, emphasising the legitimacy of the arithmetic mean for the skewed data. In the published articles, the t test and related methods were often used on untransformed data, which is likely to be invalid. Several approaches to inferential analysis are described, emphasising (1) non-parametric methods, while noting that they are not simply comparisons of medians, and (2) generalised linear modelling, in particular with the negative binomial distribution. Additional methods, such as the bootstrap, with potential for greater use are described.

CONCLUSIONS

Clarity is recommended when describing transformations and measures of location. It is suggested that non-parametric methods and generalised linear models are likely to be sufficient for most analyses.

Assessing agreement between malaria slide density readings

Background

Several criteria have been used to assess agreement between replicate slide readings of malaria parasite density. Such criteria may be based on percent difference, or absolute difference, or a combination. Neither the rationale for choosing between these types of criteria, nor that for choosing the magnitude of difference which defines acceptable agreement, are clear. The current paper seeks a procedure which avoids the disadvantages of these current options and whose parameter values are more clearly justified.

Methods and Results

Variation of parasite density within a slide is expected, even when it has been prepared from a homogeneous sample. This places lower limits on sensitivity and observer agreement, quantified by the Poisson distribution. This means that, if a criterion of fixed percent difference criterion is used for satisfactory agreement, the number of discrepant readings is over-estimated at low parasite densities. With a criterion of fixed absolute difference, the same happens at high parasite densities. For an ideal slide, following the Poisson distribution, a criterion based on a constant difference in square root counts would apply for all densities. This can be back-transformed to a difference in absolute counts, which, as expected, gives a wider range of acceptable agreement at higher average densities. In an example dataset from Tanzania, observed differences in square root counts correspond to a 95% limits of agreement of -2,800 and +2,500 parasites/μl at average density of 2,000 parasites/μl, and -6,200 and +5,700 parasites/μl at 10,000 parasites/μl. However, there were more outliers beyond those ranges at higher densities, meaning that actual coverage of these ranges was not a constant 95%, but decreased with density. In a second study, a trial of microscopist training, the corresponding ranges of agreement are wider and asymmetrical: -8,600 to +5,200/μl, and -19,200 to +11,700/μl, respectively. By comparison, the optimal limits of agreement, corresponding to Poisson variation, are ± 780 and ± 1,800 parasites/μl, respectively. The focus of this approach on the volume of blood read leads to other conclusions. For example, no matter how large a volume of blood is read, some densities are too low to be reliably detected, which in turn means that disagreements on slide positivity may simply result from within-slide variation, rather than reading errors.

Conclusions

The proposed method defines limits of acceptable agreement in a way which allows for the natural increase in variability with parasite density. This includes defining the levels of between-reader variability, which are consistent with random variation: disagreements within these limits should not trigger additional readings. This approach merits investigation in other settings, in order to determine both the extent of its applicability, and appropriate numerical values for limits of agreement.

What not to do in medical statistics

There have been major efforts to improve the application of statistical methods in medical research, although some errors and misconceptions persist. In this paper I will review some of the topics which most often cause problems: a) comparison of two methods of clinical measurement; b) comparison of baseline values between arms of a randomized trial; c) absence of evidence as opposed to evidence of absence; and d) regression to the mean. I will also revisit a statistical error in one of my own publications. I review some causes of the continuing misuse of statistics, and make some suggestions for modifying the education of statistical and non-statistical medical researchers in order to alleviate this.