Statistical methods for establishing equivalency of a sampling device to the OSHA standard

A Primer on the Use of Equivalence Testing for Evaluating Measurement Agreement

PURPOSE

Statistical equivalence testing is more appropriate than conventional tests of difference to assess the validity of physical activity (PA) measures. This article presents the underlying principles of equivalence testing and gives three examples from PA and fitness assessment research.

METHODS

The three examples illustrate different uses of equivalence tests. Example 1 uses PA data to evaluate an activity monitor's equivalence to a known criterion. Example 2 illustrates the equivalence of two field-based measures of physical fitness with no known reference method. Example 3 uses regression to evaluate an activity monitor's equivalence across a suite of 23 activities.

RESULTS

The examples illustrate the appropriate reporting and interpretation of results from equivalence tests. In the first example, the mean criterion measure is significantly within ±15% of the mean PA monitor. The mean difference is 0.18 METs and the 90% confidence interval of -0.15 to 0.52 is inside the equivalence region of -0.65 to 0.65. In the second example, we chose to define equivalence for these two measures as a ratio of mean values between 0.98 and 1.02. The estimated ratio of mean V˙O2 values is 0.99, which is significantly (P = 0.007) inside the equivalence region. In the third example, the PA monitor is not equivalent to the criterion across the suite of activities. The estimated regression intercept and slope are -1.23 and 1.06. Neither confidence interval is within the suggested regression equivalence regions.

CONCLUSIONS

When the study goal is to show similarity between methods, equivalence testing is more appropriate than traditional statistical tests of differences (e.g., ANOVA and t-tests).

Comparing the means and variances of a bivariate log-normal distribution

For a bivariate log-normal distribution, a confidence interval is developed for the ratio of the means. The generalized confidence interval approach is used for this purpose, and the procedure is applicable regardless of the sample size. It is also noted that the same approach can be used to obtain a confidence interval for the ratio of the variances. A modified signed log-likelihood ratio procedure is also described for computing confidence intervals. The coverage probabilities of the proposed confidence intervals are estimated by Monte Carlo, and the generalized confidence intervals are found to exhibit satisfactory performance even for small sample sizes. Numerical results also show that the corresponding test procedures provide larger power compared with the modified signed log-likelihood ratio test. Two examples are given: one dealing with the comparison of the means and variances of health-care costs and the other dealing with testing mean equivalence in quantitative assays.

Statistical methods for establishing equivalency of several sampling devices

The problem of comparing several alternate sampling devices to the Occupational Safety and Health Administration (OSHA) standard, or the comparison of several sampling devices among themselves, is considered. A test based on the OSHA criterion that states that "90% of the readings of the sampling device should be within +/- 25% of the readings obtained by the standard" is developed. Type I error rates and powers of the test are studied using Monte Carlo simulation. The study indicates that the proposed test is quite satisfactory for applications. The method is illustrated using a simulated data set.