Intermediate, indeterminate, and uninterpretable diagnostic test results

Advances in statistical methodology for diagnostic medicine in the 1980's

Research on diagnostic medicine has been directed at a number of topics in the past decade. Issues which have received a lot of attention are ROC analysis and the identification and correction of various analytic biases. Other topics of widespread interest include the use of expert systems, the relationship of such systems to statistical data-based systems, and the evaluation of tests using cost-effectiveness analysis. Increasingly there is a sentiment that well-designed, prospective trials are required to provide credible information on the accuracy of diagnostic technologies, and so a consensus on methodological standards is needed, paralleling the earlier development of such standards in clinical trials and epidemiology.

Decision-making studies in patient management: twenty years later

This article surveys the contribution of decision analysis to diagnosis and treatment in medicine, beginning with a review of Dr. Lee B. Lusted's 1971 paper, "Decision-making studies in patient management." From this beginning methods of decision analysis were developed that were particularly appropriate to medical problems. The author reviews the effects of Dr. Lusted's interests in the field on subsequent research and publications, and critiques prophetic statements found in the early paper.

Likelihood ratios with confidence: sample size estimation for diagnostic test studies

Confidence intervals are important summary measures that provide useful information from clinical investigations, especially when comparing data from different populations or sites. Studies of a diagnostic test should include both point estimates and confidence intervals for the tests' sensitivity and specificity. Equally important measures of a test's efficiency are likelihood ratios at each test outcome level. We present a method for calculating likelihood ratio confidence intervals for tests that have positive or negative results, tests with non-positive/non-negative results, and tests reported on an ordinal outcome scale. In addition, we demonstrate a sample size estimation procedure for diagnostic test studies based on the desired likelihood ratio confidence interval. The renewed interest in confidence intervals in the medical literature is important, and should be extended to studies analyzing diagnostic tests.

Some methodological questions concerning receiver operating characteristic (ROC) analysis as a method for assessing image quality in radiology

This paper raises five methodological questions concerning receiver operating characteristic (ROC) analysis: (1) can the ROC "confidence criterion" be applied in a valid, reliable way?; (2) can ROC deal with ambiguous findings?; (3) can ROC deal effectively with false-negative findings?; (4) are ROC curves susceptible to valid statistical testing?; and (5) are ROC results useful in choosing among alternative imaging modalities? A review of the evidence leads to six conclusions. First, using ROC, all radiological findings must be unambiguously scored as true-positive, true-negative, false-positive, or false-negative, often forcing arbitrary, procrustean choices on readers and evaluators. Second, ROC requires radiologists to report findings by confidence level on a consistent, reliable basis throughout a ROC experiment; something that seems unrealistic, given what is known about human performance in almost all perceptual tasks of comparable complexity. Third, as gathered during the typical experiment, ROC data are probably nominal, but treated as if ordinal (or even interval) data, leading to distorted results. Fourth, ROC does not deal effectively with false-negatives, despite their importance. Fifth, there is no satisfactory method for statistically testing the significance of observed differences between two ROC curves if they are based on nominal data. Finally, the artificial tasks required of radiologists in a ROC evaluation limit the usefulness of ROC results in choosing among the imaging modalities.

A Bayesian method for evaluating medical test operating characteristics when some patients' conditions fail to be diagnosed by the reference standard

The evaluation of a diagnostic test when the reference standard fails to establish a diagnosis in some patients is a common and difficult analytical problem. Conventional operating characteristics, derived from a 2 x 2 matrix, require that tests have only positive or negative results, and that disease status be designated definitively as present or absent. Results can be displayed in a 2 x 3 matrix, with an additional column for undiagnosed patients, when it is not possible always to ascertain the disease status definitively. The authors approach this problem using a Bayesian method for evaluating the 2 x 3 matrix in which test operating characteristics are described by a joint probability density function. They show that one can derive this joint probability density function of sensitivity and specificity empirically by applying a sampling algorithm. The three-dimensional histogram resulting from this sampling procedure approximates the true joint probability density function for sensitivity and specificity. Using a clinical example, the authors illustrate the method and demonstrate that the joint probability density function for sensitivity and specificity can be influenced by assumptions used to interpret test results in undiagnosed patients. This Bayesian method represents a flexible and practical solution to the problem of evaluating test sensitivity and specificity when the study group includes patients whose disease could not be diagnosed by the reference standard.

Beyond sensitivity, specificity and statistical independence

Sensitivity and specificity have clear definitions when there is a single test for one disease, and the test is either positive or negative. This paper presents a unified approach for obtaining posterior probabilities (predictive values) when there are more than two test outcomes and/or more than one disease state. In these cases, sensitivity and specificity do not have clear definitions. Three examples from the literature demonstrate how this approach simplifies the presentation of Bayesian revision of prior probabilities. Use of proper care in data collection for the purpose of estimating conditional probabilities can avoid assumptions of statistical independence.

Quantitating bedside diagnosis: clinical evaluation of ascites

The authors prospectively evaluated the operating characteristics of the history and physical examination for ascites in a broad spectrum of hospitalized patients. The overall clinical evaluation produced a positive likelihood ratio = 37.7-83.3 when suggestive of ascites, a likelihood ratio = 2.23-3.42 when intermediate, and a negative likelihood ratio = 0.77-0.90 when not suggestive of ascites. Patients' perceptions of increased abdominal girth (positive likelihood ratio = 4.16) or recent weight gain (positive likelihood ratio = 3.20) increased the likelihood of ascites. The absence of subjective ankle swelling (negative likelihood ratio = 0.10) or increased abdominal girth (negative likelihood ratio = 0.17) decreased the likelihood of ascites. The positive likelihood ratios for a fluid wave = 9.6 and shifting dullness = 5.76 favored ascites, while the absence of bulging flanks (negative likelihood ratio = 0.12) or peripheral edema (negative likelihood ratio = 0.17) favored ascites the least. Thus, a routine history and physical examination are quantitatively useful in the clinical evaluation of ascites.