One-Sided Tolerance Limits in Balanced and Unbalanced One-Way Random Models Based on Generalized Confidence Intervals

Introducing Robust Statistics in the Uncertainty Quantification of Nuclear Safeguards Measurements

The monitoring of nuclear safeguards measurements consists of verifying the coherence between the operator declarations and the corresponding inspector measurements on the same nuclear items. Significant deviations may be present in the data, as consequence of problems with the operator and/or inspector measurement systems. However, they could also be the result of data falsification. In both cases, quantitative analysis and statistical outcomes may be negatively affected by their presence unless robust statistical methods are used. This article aims to investigate the benefits deriving from the introduction of robust procedures in the nuclear safeguards context. In particular, we will introduce a robust estimator for the estimation of the uncertainty components of the measurement error model. The analysis will prove the capacity of robust procedures to limit the bias in simulated and empirical contexts to provide more sounding statistical outcomes. For these reasons, the introduction of robust procedures may represent a step forward in the still ongoing development of reliable uncertainty quantification methods for error variance estimation.

Capturing the underlying distribution in meta-analysis: Credibility and tolerance intervals

Tolerance intervals provide a bracket intended to contain a percentage (e.g., 80%) of a population distribution given sample estimates of the mean and variance. In random-effects meta-analysis, tolerance intervals should contain researcher-specified proportions of underlying population effect sizes. Using Monte Carlo simulation, we investigated the coverage for five relevant tolerance interval estimators: the Schmidt-Hunter credibility intervals, a prediction interval, two content tolerance intervals adapted to meta-analysis, and a bootstrap tolerance interval. None of the intervals contained the desired percentage of coverage at the nominal rates in all conditions. However, the prediction worked well unless the number of primary studies was small (<30), and one of the content tolerance intervals approached nominal levels with small numbers (<20) of primary studies. The bootstrap tolerance interval achieved near nominal coverage if there were sufficient numbers of primary studies (30+) and large enough sample sizes (N ≅ 70) in the included primary studies, although it slightly exceeded nominal coverage with large numbers of large-sample primary studies. Next, we showed the results of applying the intervals to real data using a set of previously published analyses and provided suggestions for practice. Tolerance intervals incorporate error of estimation into the construction of proper brackets for fractions of population true effects. In many contexts, such intervals approach the desired nominal levels of coverage.

Confidence, prediction, and tolerance in linear mixed models

The literature about Prediction Interval (PI) and Tolerance Interval (TI) in linear mixed models is usually developed for specific designs, which is a main limitation to their use. This paper proposes to reformulate the two‐sided PI to be generalizable under a wide variety of designs (one random factor, nested and crossed designs for multiple random factors, and balanced or unbalanced designs). This new methodology is based on the Hessian matrix, namely, the inverse of (observed) Fisher Information matrix, and is built with a cell mean model. The degrees of freedom for the total variance are calculated with the generalized Satterthwaite method and compared to the Kenward‐Roger's degrees of freedom for fixed effects. Construction of two‐sided TIs are also detailed with one random factor, and two nested and two crossed random variables. An extensive simulation study is carried out to compare the widths and coverage probabilities of Confidence Intervals (CI), PIs, and TIs to their nominal levels. It shows excellent coverage whatever the design and the sample size are. Finally, these CIs, PIs, and TIs are applied to two real data sets: one from orthopedic surgery study (intralesional resection risk) and the other from assay validation study during vaccine development.

Setting Alarm Thresholds in Measurements with Systematic and Random Errors

For statistical evaluations that involve within-group and between-group variance components (denoted σ W 2 and σ B 2 , respectively), there is sometimes a need to monitor for a shift in the mean of time-ordered data. Uncertainty in the estimates σ ^ W 2 and σ ^ B 2 should be accounted for when setting alarm thresholds to check for a mean shift as both σ W 2 and σ B 2 must be estimated. One-way random effects analysis of variance (ANOVA) is the main tool for analysing such grouped data. Nearly all of the ANOVA applications assume that both the within-group and between-group components are normally distributed. However, depending on the application, the within-group and/or between-group probability distributions might not be well approximated by a normal distribution. This review paper uses the same example throughout to illustrate the possible approaches to setting alarm limits in grouped data, depending on what is assumed about the within-group and between-group probability distributions. The example involves measurement data, for which systematic errors are assumed to remain constant within a group, and to change between groups. The false alarm probability depends on the assumed measurement error model and its within-group and between-group error variances, which are estimated while using historical data, usually with ample within-group data, but with a small number of groups (three to 10 typically). This paper illustrates the parametric, semi-parametric, and non-parametric options to setting alarm thresholds in such grouped data.

Immunogenicity assay cut point determination using nonparametric tolerance limit

The newly released FDA guidance on immunogenicity assay development and validation recommends use of a lower confidence limit of the percentile of the negative subject population as the cut point in order to guarantee a pre-specified false positive rate with high confidence. The limit is, in essence, a lower tolerance limit. Although in literature several methods are available for determining the tolerance limit, they either fail to take into account the repeated measurement of the data from a typical immunogenicity assay quantification/validation experiment or rely heavily on normality assumption of the data, which is rarely correct. As a result, the methods may result in biased estimates of the cut point, causing the false positive rate to be either lower or higher than expected. To overcome this drawback, we propose two non-parametric methods under repeated measure data structure and without normal distribution assumption. Simulation studies were carried to compare the performance of the two non-parametric approaches with the current methods. The results of the simulation studies show that one of the two nonparametric methods outperforms all the other methods and provides a satisfactory coverage probability even with moderate sample sizes. In addition, it is simple and straightforward to implement. Therefore, it is a preferred method for immunogenicity assay cut point determination.

A simple method for assessing occupational exposure via the one-way random effects model

A one-way random effects model is postulated for the log-transformed shift-long personal exposure measurements, where the random effect in the model represents an effect due to the worker. Simple closed-form confidence intervals are proposed for the relevant parameters of interest using the method of variance estimates recovery (MOVER). The performance of the confidence bounds is evaluated and compared with those based on the generalized confidence interval approach. Comparison studies indicate that the proposed MOVER confidence bounds are better than the generalized confidence bounds for the overall mean exposure and an upper percentile of the exposure distribution. The proposed methods are illustrated using a few examples involving industrial hygiene data.

Respirable crystalline silica exposures during asphalt pavement milling at eleven highway construction sites

Asphalt pavement milling machines use a rotating cutter drum to remove the deteriorated road surface for recycling. The removal of the road surface has the potential to release respirable crystalline silica, to which workers can be exposed. This article describes an evaluation of respirable crystalline silica exposures to the operator and ground worker from two different half-lane and larger asphalt pavement milling machines that had ventilation dust controls and water-sprays designed and installed by the manufacturers. Manufacturer A completed milling for 11 days at 4 highway construction sites in Wisconsin, and Manufacturer B completed milling for 10 days at 7 highway construction sites in Indiana. To evaluate the dust controls, full-shift personal breathing zone air samples were collected from an operator and ground worker during the course of normal employee work activities of asphalt pavement milling at 11 different sites. Forty-two personal breathing zone air samples were collected over 21 days (sampling on an operator and ground worker each day). All samples were below 50 µg/m(3) for respirable crystalline silica, the National Institute for Occupational Safety and Health recommended exposure limit. The geometric mean personal breathing zone air sample was 6.2 µg/m(3) for the operator and 6.1 µg/m(3) for the ground worker for the Manufacturer A milling machine. The geometric mean personal breathing zone air sample was 4.2 µg/m(3) for the operator and 9.0 µg/m(3) for the ground worker for the Manufacturer B milling machine. In addition, upper 95% confidence limits for the mean exposure for each occupation were well below 50 µg/m(3) for both studies. The silica content in the bulk asphalt material being milled ranged from 7-23% silica for roads milled by Manufacturer A and from 5-12% silica for roads milled by Manufacturer B. The results indicate that engineering controls consisting of ventilation controls in combination with water-sprays are capable of controlling occupational exposures to respirable crystalline silica generated by asphalt pavement milling machines on highway construction sites.

The symmetric-range accuracy under a one-way random model with balanced or unbalanced data

The symmetric-range accuracy A of a sampler is defined as the fractional range, symmetric about the true concentration, that includes a specified proportion of sampler measurements. In this article, we give an explicit expression for A assuming that the sampler measurements follow a one-way random model so as to capture different components of variability, for example, variabilities among and within different laboratories or variabilities among and within exposed workers. We derive an upper confidence limit for A based on the concept of a 'generalized confidence interval'. A convenient approximation is also provided for computing the upper confidence limit. Both balanced and unbalanced data situations are investigated. Monte Carlo evaluation indicates that the proposed upper confidence limit is satisfactory even for small samples. The statistical procedures are illustrated using an example.

Tolerance limits for a ratio of normal random variables

The problem of deriving an upper tolerance limit for a ratio of two normally distributed random variables is addressed, when the random variables follow a bivariate normal distribution, or when they are independent normal. The derivation uses the fact that an upper tolerance limit for a random variable can be derived from a lower confidence limit for the cumulative distribution function (cdf) of the random variable. The concept of a generalized confidence interval is used to derive the required lower confidence limit for the cdf. In the bivariate normal case, a suitable representation of the cdf of the ratio of the marginal normal random variables is also used, coupled with the generalized confidence interval idea. In addition, a simplified derivation is presented in the situation when one of the random variables has a small coefficient of variation. The problem is motivated by an application from a reverse transcriptase assay. Such an example is used to illustrate our results. Numerical results are also reported regarding the performance of the proposed tolerance limit.

Upper limits for exceedance probabilities under the one-way random effects model

In this article, we propose statistical methods for setting upper limits on (i) the probability that the mean exposure of an individual worker exceeds the occupational exposure limit (OEL) and (ii) the probability that the exposure of a worker exceeds the OEL. The proposed method for (i) is obtained using the generalized variable approach, and the one for (ii) is based on an approximate method for constructing one-sided tolerance limits in the one-way random effects model. Even though tolerance limits can be used to assess the proportion of exposure measurements exceeding the OEL, the upper limits on these probabilities are more informative than tolerance limits. The methods are conceptually as well as computationally simple. Two data sets involving industrial exposure data are used to illustrate the methods.

Generalized P-values and confidence intervals: a novel approach for analyzing lognormally distributed exposure data

The problem of assessing occupational exposure using the mean of a lognormal distribution is addressed. The novel concepts of generalized p-values and generalized confidence intervals are applied for testing hypotheses and computing confidence intervals for a lognormal mean. The proposed methods perform well, they are applicable to small sample sizes, and they are easy to implement. Power studies and sample size calculation are also discussed. Computational details and a source for the computer program are given. The procedures are also extended to compare two lognormal means and to make inference about a lognormal variance. In fact, our approach based on generalized p-values and generalized confidence intervals is easily adapted to deal with any parametric function involving one or two lognormal distributions. Several examples involving industrial exposure data are used to illustrate the methods. An added advantage of the generalized variables approach is the ease of computation and implementation. In fact, the procedures can be easily coded in a programming language for implementation. Furthermore, extensive numerical computations by the authors show that the results based on the generalized p-value approach are essentially equivalent to those based on the Land's method. We want to draw the attention of the industrial hygiene community to this accurate and unified methodology to deal with any parameter associated with the lognormal distribution.

Double bootstrapping a tolerance limit

We consider the problem of constructing tolerance limits in the context of a one-way random effects model The usual parametric method for calculating tolerance intervals is based on the normality assumption. However, in practice, we frequently observe nonnormally distributed data. We propose the use of the double bootstrap (or nested bootstrap) method to estimate tolerance limits, which allows us to relax the normality assumption.