A Tolerance Interval Approach for Assessment of Agreement with Left Censored Data

Distinguishability and agreement with continuous data

The analysis of concordance among repeated measures has received a huge amount of attention in the statistical literature leading to a range of different approaches. However, because all the approaches are able to assess the closeness among the readings taken on the same subject, the conclusions about the degree of concordance should be similar regardless the approach applied. Here, two indices to assess the concordance among continuous repeated measures, the intraclass correlation coefficient and the total deviation index, are applied and compared in two case examples. The first example concerns the repeatability of individual nutrient allocation strategy assessed by stable isotope analysis. The second example dealt with the assessment of the concordance of functional magnetic resonance imaging data that shows spatial correlation. The results differ depending upon the approach applied leading to contradictory conclusions about the degree of concordance. The reason behind these results is discussed reaching the conclusion that the total deviation index is just assessing agreement among repeated measurements, whereas the intraclass correlation coefficient assesses the concept of distinguishability among subjects that involves agreement among repeated measurements and spread of subjects at once. Therefore, the best way to select the right approach is to understand the right question behind the research hypothesis.

Marginal and conditional approaches to multivariate variables subject to limit of detection

A marginal approach and a variance-component mixed effect model approach (here called a conditional approach) are commonly used to analyze variables that are subject to limit of detection. We examine the theoretical relationship and investigate the numerical performance of these two approaches. We make some recommendations based on our results. The marginal approach is recommended for bivariate normal variables, and the variance-component mixed effect model is preferable for other multivariate analysis in most circumstances. Two approaches are illustrated through one case study from a preclinical experiment.

The total deviation index estimated by tolerance intervals to evaluate the concordance of measurement devices

BACKGROUND

In an agreement assay, it is of interest to evaluate the degree of agreement between the different methods (devices, instruments or observers) used to measure the same characteristic. We propose in this study a technical simplification for inference about the total deviation index (TDI) estimate to assess agreement between two devices of normally-distributed measurements and describe its utility to evaluate inter- and intra-rater agreement if more than one reading per subject is available for each device.

METHODS

We propose to estimate the TDI by constructing a probability interval of the difference in paired measurements between devices, and thereafter, we derive a tolerance interval (TI) procedure as a natural way to make inferences about probability limit estimates. We also describe how the proposed method can be used to compute bounds of the coverage probability.

RESULTS

The approach is illustrated in a real case example where the agreement between two instruments, a handle mercury sphygmomanometer device and an OMRON 711 automatic device, is assessed in a sample of 384 subjects where measures of systolic blood pressure were taken twice by each device. A simulation study procedure is implemented to evaluate and compare the accuracy of the approach to two already established methods, showing that the TI approximation produces accurate empirical confidence levels which are reasonably close to the nominal confidence level.

CONCLUSIONS

The method proposed is straightforward since the TDI estimate is derived directly from a probability interval of a normally-distributed variable in its original scale, without further transformations. Thereafter, a natural way of making inferences about this estimate is to derive the appropriate TI. Constructions of TI based on normal populations are implemented in most standard statistical packages, thus making it simpler for any practitioner to implement our proposal to assess agreement.

Estimating variance parameters from multivariate normal variables subject to limit of detection: MLE, REML, or Bayesian approaches?

Likelihood-based approaches, which naturally incorporate left censoring due to limit of detection, are commonly utilized to analyze censored multivariate normal data. However, the maximum likelihood estimator (MLE) typically underestimates variance parameters. The restricted maximum likelihood estimator (REML), which corrects the underestimation of variance parameters, cannot be easily extended to analyze censored multivariate normal data. In the light of the connection between the REML and a Bayesian approach discovered in 1974 by Dr Harville, this paper describes a Bayesian approach to censored multivariate normal data. This Bayesian approach is justified through its link to the REML via Laplace's approximation and its performance is evaluated through a simulation study. We consider the Bayesian approach as a valuable alternative because it yields less biased variance parameter estimates than the MLE, and because a solid REML is technically difficult when data are left censored.

An Overview on Assessing Agreement with Continuous Measurements

Reliable and accurate measurements serve as the basis for evaluation in many scientific disciplines. Issues related to reliable and accurate measurement have evolved over many decades, dating back to the nineteenth century and the pioneering work of Galton (1886), Pearson (1896, 1899, 1901), and Fisher (1925). Requiring a new measurement to be identical to the truth is often impractical, either because (1) we are willing to accept a measurement up to some tolerable (or acceptable) error, or (2) the truth is simply not available to us, either because it is not measurable or is only measurable with some degree of error. To deal with issues related to both (1) and (2), a number of concepts, methods, and theories have been developed in various disciplines. Some of these concepts have been used across disciplines, while others have been limited to a particular field but may have potential uses in other disciplines. In this paper, we elucidate and contrast fundamental concepts employed in different disciplines and unite these concepts into one common theme: assessing closeness (agreement) of observations. We focus on assessing agreement with continuous measurements and classify different statistical approaches as (1) descriptive tools; (2) unscaled summary indices based on absolute differences of measurements; and (3) scaled summary indices attaining values between -1 and 1 for various data structures, and for cases with and without a reference. We also identify gaps that require further research and discuss future directions in assessing agreement.