Bayesian latent class models to estimate diagnostic test accuracies of COVID-19 tests

Estimates of sensitivity and specificity of serological tests for SARS-CoV-2 specific antibodies using a Bayesian latent class model approach

OBJECTIVES

Assessing the accuracy of serological tests for SARS-CoV-2 was challenging due to the lack of a gold standard. This study aimed to estimate the accuracy of SARS-CoV-2-specific serological tests using Bayesian latent class models (BLCM) and compare methods with and without a gold standard.

STUDY DESIGN AND SETTING

In this study, we analyzed 356 samples-254 positives, ie, from individuals with a previous SARS-CoV-2 infection diagnosis, and 102 negatives, ie, prepandemic samples-using six different rapid serological tests and one laboratory assay. A BLCM was employed to concurrently estimate the sensitivity and specificity of all serological tests for the immunoglobulin (Ig) M and IgG antibodies specific for SARS-CoV-2. Noninformative priors were used. A sensitivity analysis was conducted considering three methods: 1) reverse transcription-polymerase chain reaction test (RT-PCR) as the gold standard, 2) BLCM with RT-PCR as an imperfect gold standard, and 3) frequentist latent class model (LCM). All analyses used software R version 4.3.0, and BLCM were fitted using package runjags using the software JAGS (Just Another Gibbs Sampler).

RESULTS

The BLCM-derived sensitivity for IgM varied from 10.7% [95% credibility interval (CrI):1.9-24.6] to 96.9% (95% CrI: 91.0-100.0), with specificities ranging from 48.3% (95% CrI: 39.0-57.6) to 98.9% (95% CrI: 96.2-100.0). Sensitivity for IgG varied between 76.9% (95% CrI: 68.2-84.7) and 99.1% (95% CrI: 96.1-100.0), and specificity ranged from 49.9% (95% CrI: 19.4-95.8) to 99.3% (95% CrI: 97.2-100.0). LCM results were comparable to BLCM. Considering the RT-PCR as a gold standard underestimated the tests' sensitivity, particularly for IgM.

CONCLUSION

BLCM-derived results deviated from those using a gold standard, which underestimated the tests' characteristics, particularly sensitivity. Although Bayesian and frequentist LCM approaches yielded comparable results, BLCM had the benefit of enabling credibility interval computation even when sample power is limited.

Two-phase Bayesian latent class analysis to assess diagnostic test performance in the absence of a gold standard: COVID-19 serological assays as a proof of concept

BACKGROUND AND OBJECTIVES

In this proof-of-concept study, which included blood donor samples, we aimed to demonstrate how Bayesian latent class models (BLCMs) could be used to estimate SARS-CoV-2 seroprevalence in the absence of a gold standard assay under a two-phase sampling design.

MATERIALS AND METHODS

To this end, 6810 plasma samples from blood donors who resided in Québec (Canada) were collected from May to July 2020 and tested for anti-SARS-CoV-2 antibodies using seven serological assays (five commercial and two non-commercial).

RESULTS

SARS-CoV-2 seroprevalence was estimated at 0.71% (95% credible interval [CrI] = 0.53%-0.92%). The cPass assay had the lowest sensitivity estimate (88.7%; 95% CrI = 80.6%-94.7%), while the Héma-Québec assay had the highest (98.7%; 95% CrI = 97.0%-99.6%).

CONCLUSION

The estimated low seroprevalence (which indicates a relatively limited spread of SARS-CoV-2 in Quebec) might change rapidly-and this tool, developed using blood donors, could enable a rapid update of the prevalence estimate in the absence of a gold standard. Further, the present analysis illustrates how a two-stage BLCM sampling design, along with blood donor samples, can be used to estimate the performance of new diagnostic tests and inform public health decisions regarding a new or emerging disease for which a perfect reference standard does not exist.

Estimating the number of probable new SARS-CoV-2 infections among tested subjects from the number of confirmed cases

OBJECTIVES

In most African countries, confirmed COVID-19 case counts underestimate the number of new SARS-CoV-2 infection cases. We propose a multiplying factor to approximate the number of biologically probable new infections from the number of confirmed cases.

METHODS

Each of the first thousand suspect (or alert) cases recorded in South Kivu (DRC) between 29 March and 29 November 2020 underwent a RT-PCR test and an IgM and IgG serology. A latent class model and a Bayesian inference method were used to estimate (i) the incidence proportion of SARS-CoV-2 infection using RT-PCR and IgM test results, (ii) the prevalence using RT-PCR, IgM and IgG test results; and, (iii) the multiplying factor (ratio of the incidence proportion on the proportion of confirmed -RT-PCR+- cases).

RESULTS

Among 933 alert cases with complete data, 218 (23%) were RT-PCR+; 434 (47%) IgM+; 464 (~ 50%) RT-PCR+, IgM+, or both; and 647 (69%) either IgG + or IgM+. The incidence proportion of SARS-CoV-2 infection was estimated at 58% (95% credibility interval: 51.8-64), its prevalence at 72.83% (65.68-77.89), and the multiplying factor at 2.42 (1.95-3.01).

CONCLUSIONS

In monitoring the pandemic dynamics, the number of biologically probable cases is also useful. The multiplying factor helps approximating it.

Evaluation of diagnostic test procedures for SARS-CoV-2 using latent class models

Polymerase chain reaction (PCR) and antigen tests have been used extensively for screening during the severe acute respiratory syndrome coronavirus 2 pandemics. However, the real‐world sensitivity and specificity of the two testing procedures in the field have not yet been estimated without assuming that the PCR constitutes a gold standard test. We use latent class models to estimate the in situ performance of both tests using data from the Danish national registries. We find that the specificity of both tests is very high (>99.7%), while the sensitivities are 95.7% (95% confidence interval [CI]: 92.8%–98.4%) and 53.8% (95% CI: 49.8%–57.9%) for the PCR and antigen tests, respectively. These findings have implications for the use of confirmatory PCR tests following a positive antigen test result: we estimate that serial testing is counterproductive at higher prevalence levels.

Performance of Three Tests for SARS-CoV-2 on a University Campus Estimated Jointly with Bayesian Latent Class Modeling

Accurate tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have been critical in efforts to control its spread. The accuracy of tests for SARS-CoV-2 has been assessed numerous times, usually in reference to a gold standard diagnosis. One major disadvantage of that approach is the possibility of error due to inaccuracy of the gold standard, which is especially problematic for evaluating testing in a real-world surveillance context. We used an alternative approach known as Bayesian latent class modeling (BLCM), which circumvents the need to designate a gold standard by simultaneously estimating the accuracy of multiple tests. We applied this technique to a collection of 1,716 tests of three types applied to 853 individuals on a university campus during a 1-week period in October 2020. We found that reverse transcriptase PCR (RT-PCR) testing of saliva samples performed at a campus facility had higher sensitivity (median, 92.3%; 95% credible interval [CrI], 73.2 to 99.6%) than RT-PCR testing of nasal samples performed at a commercial facility (median, 85.9%; 95% CrI, 54.7 to 99.4%). The reverse was true for specificity, although the specificity of saliva testing was still very high (median, 99.3%; 95% CrI, 98.3 to 99.9%). An antigen test was less sensitive and specific than both of the RT-PCR tests, although the sample sizes with this test were small and the statistical uncertainty was high. These results suggest that RT-PCR testing of saliva samples at a campus facility can be an effective basis for surveillance screening to prevent SARS-CoV-2 transmission in a university setting. IMPORTANCE Testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has been vitally important during the COVID-19 pandemic. There are a variety of methods for testing for this virus, and it is important to understand their accuracy in choosing which one might be best suited for a given application. To estimate the accuracy of three different testing methods, we used a data set collected at a university that involved testing the same samples with multiple tests. Unlike most other estimates of test accuracy, we did not assume that one test was perfect but instead allowed for some degree of inaccuracy in all testing methods. We found that molecular tests performed on saliva samples at a university facility were similarly accurate as molecular tests performed on nasal samples at a commercial facility. An antigen test appeared somewhat less accurate than the molecular tests, but there was high uncertainty about that.

Diagnostic accuracy of three commercially available one step RT-PCR assays for the detection of SARS-CoV-2 in resource limited settings

Background

COVID-19 is an ongoing public health pandemic regardless of the countless efforts made by various actors. Quality diagnostic tests are important for early detection and control. Notably, several commercially available one step RT-PCR based assays have been recommended by the WHO. Yet, their analytic and diagnostic performances have not been well documented in resource-limited settings. Hence, this study aimed to evaluate the diagnostic sensitivities and specificities of three commercially available one step reverse transcriptase-polymerase chain reaction (RT-PCR) assays in Ethiopia in clinical setting.

Methods

A cross-sectional study was conducted from April to June, 2021 on 279 respiratory swabs originating from community surveillance, contact cases and suspect cases. RNA was extracted using manual extraction method. Master-mix preparation, amplification and result interpretation was done as per the respective manufacturer. Agreements between RT-PCRs were analyzed using kappa values. Bayesian latent class models (BLCM) were fitted to obtain reliable estimates of diagnostic sensitivities, specificities of the three assays and prevalence in the absence of a true gold standard.

Results

Among the 279 respiratory samples, 50(18%), 59(21.2%), and 69(24.7%) were tested positive by TIB, Da An, and BGI assays, respectively. Moderate to substantial level of agreement was reported among the three assays with kappa value between 0 .55 and 0.72. Based on the BLCM relatively high specificities (95% CI) of 0.991(0.973–1.000), 0.961(0.930–0.991) and 0.916(0.875–0.952) and considerably lower sensitivities with 0.813(0.658–0.938), 0.836(0.712–0.940) and 0.810(0.687–0.920) for TIB MOLBIOL, Da An and BGI respectively were found.

Conclusions

While all the three RT-PCR assays displayed comparable sensitivities, the specificities of TIB MOLBIOL and Da An were considerably higher than BGI. These results help adjust the apparent prevalence determined by the three RT-PCRs and thus support public health decisions in resource limited settings and consider alternatives as per their prioritization matrix.

Die Erforschung der Dynamik der Corona-Pandemie in Deutschland: Survey-Konzepte und eine exemplarische Umsetzung mit dem Sozio-oekonomischen Panel (SOEP)

Die Weltgesundheitsorganisation (WHO) hat im Frühjahr 2020 Richtlinien für Bevölkerungsstichproben veröffentlicht, die Basisdaten für gesundheitspolitische Entscheidungen im Pandemiefall liefern können. Diese Richtlinien umzusetzen ist keineswegs trivial. In diesem Beitrag schildern wir die Herausforderungen einer entsprechenden statistischen Erfassung der Corona Pandemie. Hierbei gehen wir im ersten Teil auf die Erfassung der Dunkelziffer bei der Meldung von Corona Infektionen, die Messung von Krankheitsverläufen im außerklinischen Bereich, die Messung von Risikomerkmalen sowie die Erfassung von zeitlichen und regionalen Veränderungen der Pandemie-Intensität ein. Wir diskutieren verschiedene Möglichkeiten, aber auch praktische Grenzen der Survey-Statistik, den vielfältigen Herausforderungen durch eine geeignete Anlage der Stichprobe und des Survey-Designs zu begegnen. Ein zentraler Punkt ist die schwierige Koppelung medizinischer Tests mit bevölkerungsrepräsentativen Umfragen, wobei bei einer personalisierten Rückmeldung der Testergebnisse das Statistik-Geheimnis eine besondere Herausforderung darstellt.

Im zweiten Teil berichten wir wie eine der großen Wiederholungsbefragungen in Deutschland, das Sozio-oekonomische Panel (SOEP), für eine WHO-konforme Covid-19-Erhebung genutzt wird, die im Rahmen einer Kooperation des Robert-Koch-Instituts (RKI) mit dem SOEP als „RKI-SOEP Stichprobe“ im September 2020 gestartet wurde. Erste Ergebnisse zum Rücklauf dieser Studie, die ab Oktober 2021 mit einer zweiten Erhebungswelle bei denselben Personen fortgesetzt werden wird, werden vorgestellt. Es zeigt sich, dass knapp fünf Prozent der bereits in der Vergangenheit erfolgreich Befragten aufgrund der Anfrage zwei Tests zu machen die weitere Teilnahme an der SOEP-Studie verweigern. Berücksichtigt man alle in der Studie erhobenen Informationen (IgG-Antikörper-Tests, PCR-Tests und Fragebögen) ergibt eine erste Schätzung, dass sich bis November 2020 nur etwa zwei Prozent der in Privathaushalten lebenden Erwachsenen in Deutschland mit SARS-CoV‑2 infiziert hatten. Damit war die Zahl der Infektionen etwa doppelt so hoch wie die offiziell gemeldeten Infektionszahlen.