Dual composite reference standards (dCRS) in molecular diagnostic research: A new approach to reduce bias in the presence of Imperfect reference

Comparison of commercial assays and two-step approach to detect Clostridioides difficile in South Africa

Background

Clostridioides difficile is the number one cause of hospital-acquired diarrhoea. Accurate diagnosis of C. difficile is of utmost importance as it guides patient management and infection control practices. Studies evaluating the performance of commercially available nucleic acid amplification tests (NAATs) versus algorithms are lacking in resource-limited settings.

Objective

This study assessed the performance of three commercially available tests and a two-step approach for the diagnosis of C. difficile infection using toxigenic culture (TC) as the gold standard.

Methods

Two hundred and twenty-three non-duplicate loose stool samples were submitted to the National Health Laboratory Service Microbiology Laboratory at Tygerberg Hospital, Cape Town, South Africa, from October 2017 to October 2018. The samples were tested in parallel using the C. DIFF QUIK CHEK COMPLETE enzyme immunoassay (EIA) and two NAATs (Xpert C. difficile and BD MAX Cdiff), and the results were compared to TC. The performance of a two-step approach consisting of the C. DIFF QUIK CHEK COMPLETE followed by the Xpert C. difficile was also determined.

Results

Of 223 faecal specimens tested, 37 (16.6%) were TC-positive. The sensitivity and specificity of the C. DIFF QUIK CHEK COMPLETE were 54.1% and 98.9%; Xpert C. difficile, 86.4% and 96.8%; BD MAX Cdiff, 89.2% and 96.8%; and two-step approach, 89.2% and 96.2%.

Conclusion

The C. DIFF QUIK CHEK COMPLETE, in a two-step approach with the Xpert C. difficile, performed similarly to the NAATs on their own and offer advantages in terms of cost and workflow in low-resource settings.

Nasopharyngeal Swabs vs. Nasal Aspirates for Respiratory Virus Detection: A Systematic Review

Nasal pathogen detection sensitivities can be as low as 70% despite advances in molecular diagnostics. This may be linked to the choice of sampling method. A diagnostic test accuracy review for sensitivity was undertaken to compare sensitivity of swabbing to the nasopharynx and extracting nasal aspirates, using the PRISMA protocol, Cochrane rapid review methodology, and QUADAS-2 risk of bias tools, with meta-analysis of included studies. Sensitivities were calculated by a consensus standard of positivity by either method as the ‘gold standard.’ Insufficient sampling methodology, cross sectional study designs, and studies pooling samples across anatomical sites were excluded. Of 13 subsequently eligible studies, 8 had ‘high’ risk of bias, and 5 had ‘high’ applicability concerns. There were no statistical differences in overall sensitivities between collection methods for eight different viruses, and this did not differ with use of PCR, immunofluorescence, or culture. In one study alone, Influenza H1N1(2009) favored nasopharyngeal swabs, with aspirates having 93.3% of the sensitivity of swabs (p > 0.001). Similarly equivocal sensitivities were noted in reports detecting bacteria. The chain of sampling, from anatomical site to laboratory results, features different potential foci along which sensitivity may be lost. A fair body of evidence exists that use of a different sampling method will not yield more respiratory pathogens.

Saliva Testing Is Accurate for Early-Stage and Presymptomatic COVID-19

Although nasopharyngeal samples have been considered the gold standard for COVID-19 testing, variability in viral load across different anatomical sites could cause nasopharyngeal samples to be less sensitive than saliva or nasal samples in certain cases. Self-collected samples have logistical advantages over nasopharyngeal samples, making them amenable to population-scale screening. To evaluate sampling alternatives for population screening, we collected nasopharyngeal, saliva, and nasal samples from two cohorts with varied levels and types of symptoms. In a mixed cohort of 60 symptomatic and asymptomatic participants, we found that saliva had 88% concordance with nasopharyngeal samples when tested in the same testing lab (n = 41) and 68% concordance when tested in different testing labs (n = 19). In a second cohort of 20 participants hospitalized for COVID-19, saliva had 74% concordance with nasopharyngeal samples tested in the same testing lab but detected virus in two participants that tested negative with nasopharyngeal samples on the same day. Medical record review showed that the saliva-based testing sensitivity was related to the timing of symptom onset and disease stage. We find that no sample site will be perfectly sensitive for COVID-19 testing in all situations, and the significance of negative results will always need to be determined in the context of clinical signs and symptoms. Saliva retained high clinical sensitivity for early-stage and presymptomatic COVID-19 while allowing easier collection, minimizing the exposure of health care workers, and need for personal protective equipment and making it a viable option for population-scale testing.

IMPORTANCE Methods for COVID-19 detection are necessary for public health efforts to monitor the spread of disease. Nasopharyngeal samples have been considered the best approach for COVID-19 testing. However, alternative samples like self-collected saliva offer advantages for population-scale screening. Meta-analyses of recent studies suggest that saliva is useful for detecting SARS-CoV-2; however, differences in disease prevalence, sample collection, and analysis methods still confound strong conclusions on the utility of saliva compared to nasopharyngeal samples. Here, we find that the sensitivity of saliva testing is related to both the timing of the sample collection relative to symptom onset and the disease stage. Importantly, several clinical vignettes in our cohorts highlight the challenges of medical decision making with limited knowledge of the associations between laboratory test data and the natural biology of infection.

Performance of Xpert MTB/RIF Ultra for tuberculosis diagnosis in the context of passive and active case finding

We present a field evaluation of the diagnostic accuracy of Xpert MTB/RIF (Xpert) and Xpert MTB/RIF Ultra (Ultra), using two cohorts in a high TB/HIV burden setting in Southern Mozambique. Single respiratory specimens from symptomatic adults accessing health care services (passive case finding (PCF) cohort), and from household and community close contacts (active case finding (ACF) cohort), were tested by smear microscopy, culture, Xpert and Ultra. Liquid and solid culture served as a composite reference standard. We explored trace results' impact on specificity via their recategorisation to negative (in all and just among those previously treated individuals) A total of 1419 and 252 participants were enrolled in the PCF and ACF cohorts, respectively. For the PCF cohort, Ultra showed higher sensitivity than Xpert overall (0.95 (95% CI: 0.90, 0.98) versus 0.88 (0.82, 0.93); p<0.001) and among smear negative patients (0.63 (0.48, 0.76) and 0.84 (0.71, 0.93). Ultra's specificity was lower than Xpert's (0.98 (0.97, 0.99) versus 0.96 (0.95, 0.97); p=0.008). For ACF, sensitivities were the same (0.67 (95% CI: 0.22,0.96) for both tests), although Ultra detected a higher number of microbiologically confirmed samples than Xpert (4.7% (12/252) versus 2.7% (7/252)). Conditional recategorisation of trace results among previously treated participants maintained differences in specificity in the PCF cohort. These results add evidence on the improved sensitivity of Ultra and support its use in different case finding scenarios.

Sensitivity of ID NOW and RT-PCR for detection of SARS-CoV-2 in an ambulatory population

Diagnosis of SARS-CoV-2 (COVID-19) requires confirmation by reverse transcription-polymerase chain reaction (RT-PCR). Abbott ID NOW provides fast results but has been criticized for low sensitivity. Here we determine the sensitivity of ID NOW in an ambulatory population presented for testing. The study enrolled 785 symptomatic patients, of whom 21 were positive by both ID NOW and RT-PCR, and 2 only by RT-PCR. All 189 asymptomatic patients tested negative. The positive percent agreement between the ID NOW assay and the RT-PCR assay was 91.3%, and negative percent agreement was 100%. The results from the current study were included into a larger systematic review of literature where at least 20 subjects were simultaneously tested using ID NOW and RT-PCR. The overall sensitivity for ID NOW assay was calculated at 84% (95% confidence interval 55-96%) and had the highest correlation to RT-PCR at viral loads most likely to be associated with transmissible infections.

Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update

OBJECTIVE

To systematically review methods developed and employed to evaluate the diagnostic accuracy of medical test when there is a missing or no gold standard.

STUDY DESIGN AND SETTINGS

Articles that proposed or applied any methods to evaluate the diagnostic accuracy of medical test(s) in the absence of gold standard were reviewed. The protocol for this review was registered in PROSPERO (CRD42018089349).

RESULTS

Identified methods were classified into four main groups: methods employed when there is a missing gold standard; correction methods (which make adjustment for an imperfect reference standard with known diagnostic accuracy measures); methods employed to evaluate a medical test using multiple imperfect reference standards; and other methods, like agreement studies, and a mixed group of alternative study designs. Fifty-one statistical methods were identified from the review that were developed to evaluate medical test(s) when the true disease status of some participants is unverified with the gold standard. Seven correction methods were identified and four methods were identified to evaluate medical test(s) using multiple imperfect reference standards. Flow-diagrams were developed to guide the selection of appropriate methods.

CONCLUSION

Various methods have been proposed to evaluate medical test(s) in the absence of a gold standard for some or all participants in a diagnostic accuracy study. These methods depend on the availability of the gold standard, its' application to the participants in the study and the availability of alternative reference standard(s). The clinical application of some of these methods, especially methods developed when there is missing gold standard is however limited. This may be due to the complexity of these methods and/or a disconnection between the fields of expertise of those who develop (e.g. mathematicians) and those who employ the methods (e.g. clinical researchers). This review aims to help close this gap with our classification and guidance tools.

Combining population-based administrative health records and electronic medical records for disease surveillance

BACKGROUND

Administrative health records (AHRs) and electronic medical records (EMRs) are two key sources of population-based data for disease surveillance, but misclassification errors in the data can bias disease estimates. Methods that combine information from error-prone data sources can build on the strengths of AHRs and EMRs. We compared bias and error for four data-combining methods and applied them to estimate hypertension prevalence.

METHODS

Our study included rule-based OR and AND methods that identify disease cases from either or both data sources, respectively, rule-based sensitivity-specificity adjusted (RSSA) method that corrects for inaccuracies using a deterministic rule, and probabilistic-based sensitivity-specificity adjusted (PSSA) method that corrects for error using a statistical model. Computer simulation was used to estimate relative bias (RB) and mean square error (MSE) under varying conditions of population disease prevalence, correlation amongst data sources, and amount of misclassification error. AHRs and EMRs for Manitoba, Canada were used to estimate hypertension prevalence using validated case definitions and multiple disease markers.

RESULTS

The OR method had the lowest RB and MSE when population disease prevalence was 10%, and the RSSA method had the lowest RB and MSE when population prevalence increased to 20%. As the correlation between data sources increased, the OR method resulted in the lowest RB and MSE. Estimates of hypertension prevalence for AHRs and EMRs alone were 30.9% (95% CI: 30.6-31.2) and 24.9% (95% CI: 24.6-25.2), respectively. The estimates were 21.4% (95% CI: 21.1-21.7), for the AND method, 34.4% (95% CI: 34.1-34.8) for the OR method, 32.2% (95% CI: 31.8-32.6) for the RSSA method, and ranged from 34.3% (95% CI: 34.1-34.5) to 35.9% (95% CI, 35.7-36.1) for the PSSA method, depending on the statistical model.

CONCLUSIONS

The OR and AND methods are influenced by correlation amongst the data sources, while the RSSA method is dependent on the accuracy of prior sensitivity and specificity estimates. The PSSA method performed well when population prevalence was high and average correlations amongst disease markers was low. This study will guide researchers to select a data-combining method that best suits their data characteristics.