The use of imperfect diagnostic tests had an impact on prevalence estimation

Estimates of sensitivity and specificity of serological tests for SARS-CoV-2 specific antibodies using a Bayesian latent class model approach

OBJECTIVES

Assessing the accuracy of serological tests for SARS-CoV-2 was challenging due to the lack of a gold standard. This study aimed to estimate the accuracy of SARS-CoV-2-specific serological tests using Bayesian latent class models (BLCM) and compare methods with and without a gold standard.

STUDY DESIGN AND SETTING

In this study, we analyzed 356 samples-254 positives, ie, from individuals with a previous SARS-CoV-2 infection diagnosis, and 102 negatives, ie, prepandemic samples-using six different rapid serological tests and one laboratory assay. A BLCM was employed to concurrently estimate the sensitivity and specificity of all serological tests for the immunoglobulin (Ig) M and IgG antibodies specific for SARS-CoV-2. Noninformative priors were used. A sensitivity analysis was conducted considering three methods: 1) reverse transcription-polymerase chain reaction test (RT-PCR) as the gold standard, 2) BLCM with RT-PCR as an imperfect gold standard, and 3) frequentist latent class model (LCM). All analyses used software R version 4.3.0, and BLCM were fitted using package runjags using the software JAGS (Just Another Gibbs Sampler).

RESULTS

The BLCM-derived sensitivity for IgM varied from 10.7% [95% credibility interval (CrI):1.9-24.6] to 96.9% (95% CrI: 91.0-100.0), with specificities ranging from 48.3% (95% CrI: 39.0-57.6) to 98.9% (95% CrI: 96.2-100.0). Sensitivity for IgG varied between 76.9% (95% CrI: 68.2-84.7) and 99.1% (95% CrI: 96.1-100.0), and specificity ranged from 49.9% (95% CrI: 19.4-95.8) to 99.3% (95% CrI: 97.2-100.0). LCM results were comparable to BLCM. Considering the RT-PCR as a gold standard underestimated the tests' sensitivity, particularly for IgM.

CONCLUSION

BLCM-derived results deviated from those using a gold standard, which underestimated the tests' characteristics, particularly sensitivity. Although Bayesian and frequentist LCM approaches yielded comparable results, BLCM had the benefit of enabling credibility interval computation even when sample power is limited.

Different latent class models were used and evaluated for assessing the accuracy of campylobacter diagnostic tests: overcoming imperfect reference standards?

In the absence of perfect reference standard, classical techniques result in biased diagnostic accuracy and prevalence estimates. By statistically defining the true disease status, latent class models (LCM) constitute a promising alternative. However, LCM is a complex method which relies on parametric assumptions, including usually a conditional independence between tests and might suffer from data sparseness. We carefully applied LCMs to assess new campylobacter infection detection tests for which bacteriological culture is an imperfect reference standard. Five diagnostic tests (culture, polymerase chain reaction and three immunoenzymatic tests) of campylobacter infection were collected in 623 patients from Bordeaux and Lyon Hospitals, France. Their diagnostic accuracy were estimated with standard and extended LCMs with a thorough examination of models goodness-of-fit. The model including a residual dependence specific to the immunoenzymatic tests best complied with LCM assumptions. Asymptotic results of goodness-of-fit statistics were substantially impaired by data sparseness and empirical distributions were preferred. Results confirmed moderate sensitivity of the culture and high performances of immunoenzymatic tests. LCMs can be used to estimate diagnostic tests accuracy in the absence of perfect reference standard. However, their implementation and assessment require specific attention due to data sparseness and limitations of existing software.

The Right Tool for the Job: Detection of Soil-Transmitted Helminths in Areas Co-endemic for Other Helminths

Background

Due to the recent increased use of the McMaster (MM) fecal egg counting method for assessing benzimidazole drug efficacy for treating soil-transmitted helminth (STH) infections, the aim of the current study was to determine the operational value of including the MM method alongside the Kato-Katz (KK) fecal thick smear to increase the diagnostic sensitivity when STHs are co-endemic with trematode helminths (e.g., Schistosoma mansoni).

Methods

A cross-sectional study was conducted in school-aged children aged 4-18 years in the northeastern region of the State of Minas Gerais (Brazil), where Necator americanus, Ascaris lumbricoides, Trichuris trichiura, and S. mansoni are co-endemic. One fecal sample from each participant was collected and transported to the field laboratory for analysis. Coprological diagnosis was performed on each fecal sample by three different methods: Formalin-Ether Sedimentation (FES), KK and the MM technique. The diagnostic sensitivity and negative predictive value (NPV) of each technique was calculated using the combination of all three techniques as the composite standard. In order to determine the agreement between the three techniques Fleiss´ kappa was used. Both the Cure Rate (CR) and the Fecal Egg Count Reduction (FECR) were calculated using the two quantification techniques (i.e., the MM and KK).

Results

Fecal samples from 1260 children were analyzed. The KK had higher diagnostic sensitivity than the MM for the detection of both A. lumbricoides (KK 97.3%, MM 69.5%) and hookworm (KK 95.1%, MM 80.8%). The CR of a single dose of mebendazole varied significantly between the KK and MM for both A. lumbricoides (p = 0.016) and hookworm (p = 0.000), with lower rates obtained with the KK. On the other hand, the FECR was very similar between both techniques for both A. lumbricoides and hookworm.

Conclusion

The MM did not add any diagnostic value over the KK in areas where both STHs and trematodes were co-endemic. The lower sensitivity of the MM would have an important impact on the administration of selective school-based treatment in this area since if only the MM were used, 36 (13.9%) children diagnosed with A. lumbricoides would have gone untreated.

Diagnosis of intestinal helminths and Schistosoma mansoni infections is based on the detection of eggs in feces. There are many techniques available for both detection and quantification of infection. For the quantification of helminth infections, the methods traditionally used are the Kato-Katz (KK) fecal think smear in humans, and the McMaster (MM) counting method in animals. Recently, the MM has been used for assessing the efficacy of benzimidazole drugs for treating soil-transmitted helminth (STH) infections in humans. In most parts of the world, however, STHs occur simultaneously with other helminth species, and the MM does not detect other helminth eggs. Therefore, in this study we sought to determine if the use of the MM in an area of Brazil were both STHs and S. mansoni are co-endemic, added any value to the current standard of diagnosis using the KK.

Sensitivity of diagnostic tests for human soil-transmitted helminth infections: a meta-analysis in the absence of a true gold standard

•

A Bayesian latent class meta-analysis of diagnostic tests for soil-transmitted helminths was performed.

•

Overall sensitivity of evaluated diagnostic tests was low.

•

Test performance was strongly influenced by intensity of infection.

•

FLOTAC method sensitivity was highest overall and in both intensity groups.

•

The performance of the Kato-Katz method in high intensity settings was acceptable.

Reliable, sensitive and practical diagnostic tests are an essential tool in disease control programmes for mapping, impact evaluation and surveillance. To provide a robust global assessment of the relative performance of available diagnostic tools for the detection of soil-transmitted helminths, we conducted a meta-analysis comparing the sensitivities and the quantitative performance of the most commonly used copro-microscopic diagnostic methods for soil-transmitted helminths, namely Kato-Katz, direct microscopy, formol-ether concentration, McMaster, FLOTAC and Mini-FLOTAC. In the absence of a perfect reference standard, we employed a Bayesian latent class analysis to estimate the true, unobserved sensitivity of compared diagnostic tests for each of the soil-transmitted helminth species Ascaris lumbricoides, Trichuris trichiura and the hookworms. To investigate the influence of varying transmission settings we subsequently stratified the analysis by intensity of infection. Overall, sensitivity estimates varied between the different methods, ranging from 42.8% for direct microscopy to 92.7% for FLOTAC. The widely used double slide Kato-Katz method had a sensitivity of 74–95% for the three soil-transmitted helminth species at high infection intensity, however sensitivity dropped to 53–80% in low intensity settings, being lowest for hookworm and A. lumbricoides. The highest sensitivity, overall and in both intensity groups, was observed for the FLOTAC method, whereas the sensitivity of the Mini-FLOTAC method was comparable with the Kato-Katz method. FLOTAC average egg count estimates were significantly lower compared with Kato-Katz, while the compared McMaster counts varied. In conclusion, we demonstrate that the Kato-Katz and Mini-FLOTAC methods had comparable sensitivities. We further show that test sensitivity of the Kato-Katz method is reduced in low transmission settings.

Evaluation of chemiluminescence, toluidine blue and histopathology for detection of high risk oral precancerous lesions: A cross-sectional study

Background

Early detection holds the key to an effective control of cancers in general and of oral cancers in particular. However, screening procedures for oral cancer are not straightforward due to procedural requirements as well as feasibility issues, especially in resource-limited countries.

Methods

We conducted a cross-sectional study to compare the performance of chemiluminescence, toluidine blue and histopathology for detection of high-risk precancerous oral lesions. We evaluated 99 lesions from 55 patients who underwent chemiluminescence and toluidine blue tests along with biopsy and histopathological examination. We studied inter-as well as intra-rater agreement in the histopathological evaluation and then using latent class modeling, we estimated the operating characteristics of these tests in the absence of a reference standard test.

Results

There was a weak inter-rater agreement (kappa < 0.15) as well as a weak intra-rater reproducibility (Pearson's r = 0.28, intra-class correlation rho = 0.03) in the histopathological evaluation of potentially high-risk precancerous lesions. When compared to histopathology, chemiluminescence and toluidine blue retention had a sensitivity of 1.00 and 0.59, respectively and a specificity of 0.01 and 0.79, respectively. However, latent class analysis indicated a low sensitivity (0.37) and high specificity (0.90) of histopathological evaluation. Toluidine blue had a near perfect high sensitivity and specificity for detection of high-risk lesions.

Conclusion

In our study, there was variability in the histopathological evaluation of oral precancerous lesions. Our results indicate that toluidine blue retention test may be better suited than chemiluminescence to detect high-risk oral precancerous lesions in a high-prevalence and low-resource setting like India.

Designing 2-phase prevalence studies in the absence of a "gold standard" test

A population survey for estimating prevalence is challenging when a disease or condition is difficult to diagnose. If clinical diagnosis is expensive, a 2-phase study, in which less expensive but less accurate tests are administered to all study subjects in the first phase (screening phase) and a more accurate but expensive or time-consuming test is administered to only a subset of the subjects in the second phase, is an attractive approach. Published research has discussed ways of maximizing precision of the prevalence estimate from a 2-phase study with a "gold standard" second-phase test. For many psychiatric disorders, even the best diagnostic tests are not of gold standard quality. In this paper, the authors propose a quasi-optimal design for 2-phase prevalence studies without a gold standard test; random-effects latent class analysis facilitates the estimation of prevalence and appropriately addresses the issue of dependent errors among the diagnostic tests. The authors show that the quasi-optimal design is efficient compared with the balanced and random designs when there is strong inter-test dependence caused by additional factors, apart from disease status, and highlight the importance of collecting data on those subjects testing negative in the first phase.