Understanding the effects of conditional dependence in research studies involving imperfect diagnostic tests

Examining the correlation between treatment effects in clinical trials and economic modelling

INTRODUCTION

Many diseases have a sequential treatment pathway. Compared with patients without previous treatment, patients who fail initial treatment may have lower success rates with a second treatment. This phenomenon can be explained by a correlation between treatment effects.

METHODS

We developed a statistical model of covariance for the underlying unobserved correlation between treatments and established a mathematical expression for the magnitude of the latent correlation term. We conducted a simulation study of clinical trials to investigate the correlation between two treatments and explored clinical examples based on published literature to illustrate the identification and evaluation of these correlations.

RESULTS

Our simulation study confirmed that a treatment correlation reduces the probability of success for the second treatment, compared with no correlation. We found that treatment correlations may be observable in clinical trials, such as for depression and lung cancer, and the magnitude of correlation may be estimated. We illustrated that treatment correlations can be incorporated into an economic model, with possible impacts on cost-effectiveness results. Additional applications of correlation concepts are also discussed.

CONCLUSIONS

We evaluated the correlation between treatment effects and our approach can be applied to clinical trial design and economic modelling of sequential clinical treatment pathways.

Developing Economic Models for Assessing the Cost-Effectiveness of Multiple Diagnostic Tests: Methods and Applications

Background

Clinical pathways with multiple diagnostic tests are complex to model, but problematic and simplistic approaches are often used in economic evaluations.

Methods

We analyzed statistical methods of handling multiple diagnostic tests and provided guidance on applying these methods in economic modeling. We first introduced a statistical model to quantify the correlations between 2 tests and how those correlations can be incorporated within an economic model. We also presented the general form of conditional dependence among multiple tests. We then introduced net reclassification improvement (NRI), a measure that evaluates the added value of a new risk factor (e.g., biomarker) for risk prediction. We further provided 2 examples to illustrate the application of these methods.

Results

Our first example illustrated how to model an add-on test to an existing test, in the absence of a perfect reference standard. After accounting for the imperfect nature of both tests and the conditional dependence between tests, the potential health benefits from the additional test were reduced. This led to differential cost-effectiveness results when comparing models using the perfect test and conditional independence assumptions. The second example illustrated how to evaluate the added value of a new risk factor using the NRI measure. Using the new risk classification provides greater precision in risk prediction, and in the example, the strategy using the new risk classification with treatment for selected individuals led to more favorable cost-effectiveness results.

Conclusions

These innovative methods for handling multiple diagnostic tests have improved the methodology within the field and should be adopted to provide more accurate estimates within cost-effectiveness analyses.

Highlights

Economic evaluations of multiple diagnostic tests often apply problematic simplistic approaches, such as ignoring conditional dependence between 2 tests or assuming a perfect final test in the diagnostic pathway. We provided guidance on how to apply improved methods for economic modeling. We introduced methods to model conditional dependence between 2 imperfect tests. We used an example to illustrate how assumptions about perfect diagnostic test accuracy and conditional independence between tests affect cost-effectiveness. Compared with the results of the area under the receiver-operating-characteristic curve, net reclassification improvement has distinct advantages in measuring the added value of a new risk factor for model-based economic evaluation. Economic evaluations that appropriately account for the complexities of diagnostic test pathways can help decision makers ensure efficient use of resources.

Performance of bovine genital campylobacteriosis diagnostic tests in bulls from Uruguay: a Bayesian latent class model approach

The sensitivity (Se) and specificity (Sp) of three diagnostic tests for the detection of Campylobacter fetus venerealis (Cfv) using field samples were estimated using a Bayesian latent class model (BLCM), accounting for the absence of a gold standard. The tests included in this study were direct immunofluorescence antibody test (IFAT), polymerase chain reaction (PCR), and real-time PCR (RT-PCR). Twelve farms from two different populations were selected and bull prepuce samples were collected. The IFAT was performed according to the OIE Manual. The conventional PCR was performed as multiplex, targeting the gene nahE for C. fetus species identification and insertion element ISCfe1 for Cfv identification. The RT-PCR was performed as uniplex: one targeting the gene nahE for C. fetus and the other targeting the insertion ISCfe1 (ISC2) for Cfv. Results from the BLCM showed a median Se of 11.7% (Bayesian credibility interval (BCI): 1.93-29.79%), 53.7% (BCI: 23.1-95.0%), and 36.1% (BCI: 14.5-71.7%) for IFAT, PCR, and RT-PCR respectively. The Sp were 94.5% (BCI: 90.1-97.9%), 97.0% (BCI: 92.9-99.3%), and 98.4% (BCI: 95.3-99.7%) for IFAT, PCR, and RT-PCR respectively. The correlation between PCR and RT-PCR was positive and low in samples from both sampled population (0.63% vs 8.47%). These results suggest that diagnostic sensitivity of the studied tests is lower using field samples than using pure Cfv strains.

Estimation of diagnostic test accuracy: A "Rule of Three" for data with repeated observations but without a gold standard

This article considers how to estimate the accuracy of a diagnostic test when there are repeated observations, but without the availability of a gold standard or reference test. We identify conditions under which the structure of the observed data is rich enough to provide sufficient degrees of freedom, such that a suitable latent class model can be fitted with identifiable accuracy parameters. We show that a Rule of Three applies, specifying that accuracy can be evaluated as long as there are at least three observations per individual with the given test. This rule also applies if the three observations arise from combinations of different test methods, or from a sequential design in which individuals are tested for a maximum number of times with the same test but stopping if a positive (or negative) result occurs. The rule pertains to tests having an arbitrary number of response categories. Accuracy is evaluated by parameters reflecting rates of misclassification among the response categories, and the model also provides estimates of the underlying distribution of the true disease state. These ideas are illustrated by data from two medical studies. Issues discussed include the advantages and disadvantages of analyzing the response variable as binary or multinomial, as well as the feasibility of testing goodness of fit when the model incorporates a large number of parameters. Comparisons are possible between models that do or do not assume equal accuracy rates for the observations, and between models where certain misclassification parameters are or are not assumed to be zero.

Interpreting ambiguous 'trace' results in Schistosoma mansoni CCA Tests: Estimating sensitivity and specificity of ambiguous results with no gold standard

BACKGROUND

The development of new diagnostics is an important tool in the fight against disease. Latent Class Analysis (LCA) is used to estimate the sensitivity and specificity of tests in the absence of a gold standard. The main field diagnostic for Schistosoma mansoni infection, Kato-Katz (KK), is not very sensitive at low infection intensities. A point-of-care circulating cathodic antigen (CCA) test has been shown to be more sensitive than KK. However, CCA can return an ambiguous 'trace' result between 'positive' and 'negative', and much debate has focused on interpretation of traces results.

METHODOLOGY/PRINCIPLE FINDINGS

We show how LCA can be extended to include ambiguous trace results and analyse S. mansoni studies from both Côte d'Ivoire (CdI) and Uganda. We compare the diagnostic performance of KK and CCA and the observed results by each test to the estimated infection prevalence in the population. Prevalence by KK was higher in CdI (13.4%) than in Uganda (6.1%), but prevalence by CCA was similar between countries, both when trace was assumed to be negative (CCAtn: 11.7% in CdI and 9.7% in Uganda) and positive (CCAtp: 20.1% in CdI and 22.5% in Uganda). The estimated sensitivity of CCA was more consistent between countries than the estimated sensitivity of KK, and estimated infection prevalence did not significantly differ between CdI (20.5%) and Uganda (19.1%). The prevalence by CCA with trace as positive did not differ significantly from estimates of infection prevalence in either country, whereas both KK and CCA with trace as negative significantly underestimated infection prevalence in both countries.

CONCLUSIONS

Incorporation of ambiguous results into an LCA enables the effect of different treatment thresholds to be directly assessed and is applicable in many fields. Our results showed that CCA with trace as positive most accurately estimated infection prevalence.

Modeling conditional dependence among multiple diagnostic tests

When multiple imperfect dichotomous diagnostic tests are applied to an individual, it is possible that some or all of their results remain dependent even after conditioning on the true disease status. The estimates could be biased if this conditional dependence is ignored when using the test results to infer about the prevalence of a disease or the accuracies of the diagnostic tests. However, statistical methods correcting for this bias by modelling higher-order conditional dependence terms between multiple diagnostic tests are not well addressed in the literature. This paper extends a Bayesian fixed effects model for 2 diagnostic tests with pairwise correlation to cases with 3 or more diagnostic tests with higher order correlations. Simulation results show that the proposed fixed effects model works well both in the case when the tests are highly correlated and in the case when the tests are truly conditionally independent, provided adequate external information is available in the form of fixed constraints or prior distributions. A data set on the diagnosis of childhood pulmonary tuberculosis is used to illustrate the proposed model.

Taking Costs and Diagnostic Test Accuracy into Account When Designing Prevalence Studies: An Application to Childhood Tuberculosis Prevalence

BACKGROUND

When planning a study to estimate disease prevalence to a pre-specified precision, it is of interest to minimize total testing cost. This is particularly challenging in the absence of a perfect reference test for the disease because different combinations of imperfect tests need to be considered. We illustrate the problem and a solution by designing a study to estimate the prevalence of childhood tuberculosis in a hospital setting.

METHODS

All possible combinations of 3 commonly used tuberculosis tests, including chest X-ray, tuberculin skin test, and a sputum-based test, either culture or Xpert, are considered. For each of the 11 possible test combinations, 3 Bayesian sample size criteria, including average coverage criterion, average length criterion and modified worst outcome criterion, are used to determine the required sample size and total testing cost, taking into consideration prior knowledge about the accuracy of the tests.

RESULTS

In some cases, the required sample sizes and total testing costs were both reduced when more tests were used, whereas, in other examples, lower costs are achieved with fewer tests.

CONCLUSION

Total testing cost should be formally considered when designing a prevalence study.