Evaluation of bias in diagnostic-test sensitivity and specificity estimates computed by discrepant analysis

Salivary testing for SARS-CoV-2 in the pediatric population: a diagnostic accuracy study

BACKGROUND

Accurate and timely testing for SARS-CoV-2 in the pediatric population is crucial to control the COVID-19 pandemic; saliva testing has been proposed as a less invasive alternative to nasopharyngeal swabs. We sought to compare the detection of SARS-CoV-2 using saliva versus nasopharyngeal swab in the pediatric population, and to determine the optimum time of testing for SARS-CoV-2 using saliva.

METHODS

We conducted a longitudinal diagnostic study in Ottawa, Canada, from Jan. 19 to Mar. 26, 2021. Children aged 3-17 years were eligible if they exhibited symptoms of COVID-19, had been identified as a high-risk or close contact to someone confirmed positive for SARS-CoV-2 or had travelled outside Canada in the previous 14 days. Participants provided both nasopharyngeal swab and saliva samples. Saliva was collected using a self-collection kit (DNA Genotek, OM-505) or a sponge-based kit (DNA Genotek, ORE-100) if they could not provide a saliva sample into a tube.

RESULTS

Among 1580 paired nasopharyngeal and saliva tests, 60 paired samples were positive for SARS-CoV-2. Forty-four (73.3%) were concordant-positive results and 16 (26.6%) were discordant, among which 8 were positive only on nasopharyngeal swab and 8 were positive only on saliva testing. The sensitivity of saliva was 84.6% (95% confidence interval 71.9%-93.1%).

INTERPRETATION

Salivary testing for SARS-CoV-2 in the pediatric population is less invasive and shows similar detection of SARS-CoV-2 to nasopharyngeal swabs. It may therefore provide a feasible alternative for diagnosis of SARS-CoV-2 infection in children.

The evaluation of the utility of the GENECUBE HQ SARS-CoV-2 for anterior nasal samples and saliva samples with a new rapid examination protocol

INTRODUCTION

GENECUBE® is a rapid molecular identification system, and previous studies demonstrated that GENECUBE® HQ SARS-CoV-2 showed excellent analytical performance for the detection of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) with nasopharyngeal samples. However, other respiratory samples have not been evaluated.

METHODS

This prospective comparison between GENECUBE® HQ SARS-CoV-2 and reference real-time reverse transcriptase polymerase chain reaction (RT-PCR) was performed for the detection of SARS-CoV-2 using anterior nasal samples and saliva samples. Additionally, we evaluated a new rapid examination protocol using GENECUBE® HQ SARS-CoV-2 for the detection of SARS-CoV-2 with saliva samples. For the rapid protocol, in the preparation of saliva samples, purification and extraction processes were adjusted, and the total process time was shortened to approximately 35 minutes.

RESULTS

For 359 anterior nasal samples, the total-, positive-, and negative concordance of the two assays was 99.7% (358/359), 98.1% (51/52), and 100% (307/307), respectively. For saliva samples, the total-, positive-, and negative concordance of the two assays was 99.6% (239/240), 100% (56/56), and 99.5% (183/184), respectively. With the new protocol, total-, positive-, and negative concordance of the two assays was 98.8% (237/240), 100% (56/56), and 98.4% (181/184), respectively. In all discordance cases, SARS-CoV-2 was detected by additional molecular examinations.

CONCLUSION

GENECUBE® HQ SARS-CoV-2 provided high analytical performance for the detection of SARS-CoV-2 in anterior nasal samples and saliva samples.

Direct Detection of Conserved Viral Sequences and Other Nucleic Acid Motifs with Solid-State Nanopores

The rapid and reliable recognition of nucleic acid sequences is essential to a broad range of fields including genotyping, gene expression analysis, and pathogen screening. For viral detection in particular, the capability is critical for optimal therapeutic response and preventing disease transmission. Here, we report an approach for detecting identifying sequence motifs within genome-scale single-strand DNA and RNA based on solid-state nanopores. By designing DNA oligonucleotide probes with complementarity to target sequences within a target genome, we establish a protocol to yield affinity-tagged duplex molecules the same length as the probe only if the target is present. The product can subsequently be bound to a protein chaperone and analyzed quantitatively with a selective solid-state nanopore assay. We first use a model DNA genome (M13mp18) to validate the approach, showing the successful isolation and detection of multiple target sequences simultaneously. We then demonstrate the protocol for the detection of RNA viruses by identifying and targeting a highly conserved sequence within human immunodeficiency virus (HIV-1B).

The effect of uncertainty in patient classification on diagnostic performance estimations

Background

The performance of a new diagnostic test is typically evaluated against a comparator which is assumed to correspond closely to some true state of interest. Judgments about the new test’s performance are based on the differences between the outputs of the test and comparator. It is commonly assumed that a small amount of uncertainty in the comparator’s classifications will negligibly affect the measured performance of a diagnostic test.

Methods

Simulated datasets were generated to represent typical diagnostic scenarios. Comparator noise was introduced in the form of random misclassifications, and the effect on the apparent performance of the diagnostic test was determined. An actual dataset from a clinical trial on a new diagnostic test for sepsis was also analyzed.

Results

We demonstrate that as little as 5% misclassification of patients by the comparator can be enough to statistically invalidate performance estimates such as sensitivity, specificity and area under the receiver operating characteristic curve, if this uncertainty is not measured and taken into account. This distortion effect is found to increase non-linearly with comparator uncertainty, under some common diagnostic scenarios. For clinical populations exhibiting high degrees of classification uncertainty, failure to measure and account for this effect will introduce significant risks of drawing false conclusions. The effect of classification uncertainty is magnified further for high performing tests that would otherwise reach near-perfection in diagnostic evaluation trials. A requirement of very high diagnostic performance for clinical adoption, such as a 99% sensitivity, can be rendered nearly unachievable even for a perfect test, if the comparator diagnosis contains even small amounts of uncertainty. This paper and an accompanying online simulation tool demonstrate the effect of classification uncertainty on the apparent performance of tests across a range of typical diagnostic scenarios. Both simulated and real datasets are used to show the degradation of apparent test performance as comparator uncertainty increases.

Conclusions

Overall, a 5% or greater misclassification rate by the comparator can lead to significant underestimation of true test performance. An online simulation tool allows researchers to explore this effect using their own trial parameters (https://imperfect-gold-standard.shinyapps.io/classification-noise/) and the source code is freely available (https://github.com/ksny/Imperfect-Gold-Standard).

Seeking a valid gold standard for an innovative, dialect-neutral language test

PURPOSE In this study, the authors explored alternative gold standards to validate an innovative, dialect-neutral language assessment. METHOD Participants were 78 African American children, ages 5;0 (years;months) to 6;11. Twenty participants had previously been identified as having language impairment. The Diagnostic Evaluation of Language Variation-Norm Referenced (DELV-NR; Seymour, Roeper, & J. de Villiers, 2005) was administered, and concurrent language samples (LSs) were collected. Using LS profiles as the gold standard, sensitivity, specificity, and other measures of diagnostic accuracy were compared for diagnoses made from the DELV-NR and participants' clinical status prior to recruitment. In a second analysis, the authors used results from the first analysis to make evidence-based adjustments in the estimates of DELV-NR diagnostic accuracy. RESULTS Accuracy of the DELV-NR relative to LS profiles was greater than that of prior diagnoses, indicating that the DELV-NR was an improvement over preexisting diagnoses for this group. Specificity met conventional standards, but sensitivity was somewhat low. Reanalysis using the positive and negative predictive power of the preexisting diagnosis in a discrepant-resolution procedure revealed that estimates for sensitivity and specificity for the DELV-NR were .85 and .93, respectively. CONCLUSION The authors found that, even after making allowances for the imperfection of available gold standards, clinical decisions made with the DELV-NR achieved high values on conventional measures of diagnostic accuracy.

Detection of Streptococcus pyogenes by use of Illumigene group A Streptococcus assay

The performance of the Illumigene group A Streptococcus assay was evaluated by comparing it to culture using 437 consecutive throat swabs. The Illumigene assay was also directly compared to PCR with 161 samples. This Illumigene assay is rapid and easy to perform. The assay also has high sensitivity (100%) compared to culture or PCR and high specificity (99.2%) compared to PCR. A total of 8.8% of the isolates were erythromycin resistant, and 6.9% were clindamycin resistant.

Nucleic acid amplification tests for diagnosis of Neisseria gonorrhoeae and Chlamydia trachomatis rectal infections

It is uncertain which methods for the diagnosis of rectal gonococcal and chlamydial infection are optimal. This study evaluated the performance of culture and nucleic acid amplification tests (NAATs) for rectal chlamydial and gonococcal diagnosis. From July 2003 until February 2007, 441 rectal test sets were collected from individuals attending a sexually transmitted disease clinic and three HIV clinics who gave a history of anal intercourse or were women at high risk for Neisseria gonorrhoeae or Chlamydia trachomatis infections. Rectal swab specimens were tested using culture and commercial NAATs employing transcription-mediated amplification (TMA), strand displacement amplification (SDA), and PCR amplification. Test performance was evaluated using a rotating standard by which patients were classified as infected if either two or three comparator tests were positive. Test sensitivities for the detection of N. gonorrhoeae ranged from 66.7% to 71.9% for culture to 100% for TMA. Specificities were 99.7% to 100% for culture and greater than 95.5% for all three NAATs. Test sensitivities for C. trachomatis ranged from 36.1% to 45.7% for culture and among NAATS from 91.4% to 95.8% for PCR to 100% for TMA. Specificities of the NAATs ranged from 95.6% to 98.5% (two-of-three standard) and from 88.8% to 91.8% (three-of-three standard). Over 60% and 80% of gonococcal and chlamydial infections, respectively, among men who have sex with men and over 20% of chlamydial infections in women would have been missed if the rectal site had not been tested. Currently available NAATs are more sensitive for the detection of chlamydial and gonococcal infection at the rectal site than is culture.

Performance of three nucleic acid amplification tests for detection of Chlamydia trachomatis and Neisseria gonorrhoeae by use of self-collected vaginal swabs obtained via an Internet-based screening program

Use of self-obtained vaginal specimens processed by nucleic acid amplification tests (NAATs) has significantly increased the utilization of nontraditional locations for Chlamydia trachomatis and Neisseria gonorrhoeae screening programs. One important emerging source of such venues includes home-based self-sampling kits available via the Internet. The objective of our study was to evaluate the performance of three commercially available NAATs (Becton-Dickinson ProbeTec SDA, Gen-Probe Aptima Combo2 TMA, and Roche Amplicor PCR) for detection of C. trachomatis and N. gonorrhoeae in vaginal samples obtained via an Internet-based screening program. From July 2004 to August 2005, 500 self-collected vaginal swabs were tested for C. trachomatis and N. gonorrhoeae by using all three NAATs. Another 500 samples were collected between August 2005 and November 2007 and tested by ProbeTec and Combo2; PCR testing was discontinued due to low specificity for N. gonorrhoeae. All tests were conducted according to the manufacturers' procedures; the "gold standard" for an infected C. trachomatis or N. gonorrhoeae patient was defined as > or = 2 positive NAAT results. Of the first 500 swabs submitted, 46 were C. trachomatis infected (9.2%) and 5 were N. gonorrhoeae infected (1.0%), and 3 of these were coinfected (0.6%). All C. trachomatis and N. gonorrhoeae Combo2-positive/ProbeTec-negative samples were confirmed as true positives by an alternative NAAT. For C. trachomatis, ProbeTec, Combo2, and PCR had sensitivities of 82.6%, 100%, and 100%, with specificities of 100%, 100%, and 99.3%, respectively. For N. gonorrhoeae, ProbeTec, Combo2, and PCR had sensitivities of 80%, 100%, and 100%, with specificities of 100%, 100%, and 98.8%, respectively. Of the total 1,000 swabs submitted, 92 were C. trachomatis infected (9.2%) and 15 were N. gonorrhoeae infected (1.5%), and 7 of these were coinfected (0.7%). There were no ProbeTec-positive/Combo2-negative samples. For C. trachomatis, ProbeTec and Combo2 had sensitivities of 81.5% and 100%, with specificities of 100% and 100%, respectively. For N. gonorrhoeae, ProbeTec and Combo2 had sensitivities of 80% and 100%, with specificities of 100% and 100%, respectively. Overall, ProbeTec had 17 C. trachomatis false-negative results (1.7%) and 3 N. gonorrhoeae false-negative results (0.3%), while Combo2 had none. Our results were consistent with the sensitivities and specificities stated by the manufacturers. NAATs perform well for detection of chlamydia and gonorrhea with self-obtained vaginal swabs shipped in a dry state to a laboratory. For 1,000 self-collected vaginal swabs tested by NAATs, the sensitivities for C. trachomatis and N. gonorrhoeae for Combo2 were 100% and 100%, while they were 81.5% and 80%, respectively, for ProbeTec. For 500 PCR samples, the C. trachomatis sensitivity was 100% and the N. gonorrhoeae sensitivity was 100%, with specificities of 99.3% and 98.8%, respectively.

Estimating diagnostic accuracy of multiple binary tests with an imperfect reference standard

The goal in diagnostic medicine is often to estimate the diagnostic accuracy of multiple experimental tests relative to a gold standard reference. When a gold standard reference is not available, investigators commonly use an imperfect reference standard. This paper proposes methodology for estimating the diagnostic accuracy of multiple binary tests with an imperfect reference standard when information about the diagnostic accuracy of the imperfect test is available from external data sources. We propose alternative joint models for characterizing the dependence between the experimental tests and discuss the use of these models for estimating individual-test sensitivity and specificity as well as prevalence and multivariate post-test probabilities (predictive values). We show using analytical and simulation techniques that, as long as the sensitivity and specificity of the imperfect test are high, inferences on diagnostic accuracy are robust to misspecification of the joint model. The methodology is demonstrated with a study examining the diagnostic accuracy of various HIV-antibody tests for HIV.

Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis

Numerous evaluations of the clinical sensitivity and specificity of PCR and serologic assays for Bordetella pertussis have been hampered by the low sensitivity of culture, the gold standard test, which leads to biased accuracy estimates. The bias can be reduced by using statistical approaches such as the composite reference standard (CRS) (e.g., positive if culture or serology positive; negative otherwise) or latent class analysis (LCA), an internal reference standard based on a statistical model. We illustrated the benefits of the CRS and LCA approaches by reanalyzing data from a 1995 to 1996 study of cough illness among 212 patients. The accuracy of PCR in this study was evaluated using three reference standards: culture, CRS, and LCA. Using specimens obtained 0 to 34 days after cough onset, estimates of the sensitivity of PCR obtained using CRS (47%) and LCA (34%) were lower than the culture-based estimate (62%). The CRS and LCA approaches, which utilized more than one diagnostic marker of pertussis, likely produced more accurate reference standards than culture alone. In general, the CRS approach is simple, with a well-defined disease status. LCA requires statistical modeling but incorporates more indicators of disease than CRS. When three or more indicators of pertussis are available, these approaches should be used in evaluations of pertussis diagnostic tests.

Utility of real-time PCR for diagnosis of Legionnaires' disease in routine clinical practice

The main aim of our study was to determine the added value of PCR for the diagnosis of Legionnaires' disease (LD) in routine clinical practice. The specimens were samples submitted for routine diagnosis of pneumonia from December 2002 to November 2005. Patients were evaluated if, in addition to PCR, the results of at least one of the following diagnostic tests were available: (i) culture for Legionella spp. on buffered charcoal yeast extract agar or (ii) detection of Legionella pneumophila antigen in urine specimens. Of the 151 evaluated patients, 37 (25%) fulfilled the European Working Group on Legionella Infections criteria for a confirmed case of LD (the "gold standard"). An estimated sensitivity, specificity, and overall percent agreement of 86% (32 of 37; 95% confidence interval [CI] = 72 to 95%), 95% (107 of 112; 95% CI = 90 to 98%), and 93% (139 of 149), respectively, were found for 16S rRNA-based PCR, and corresponding values of 92% (34 of 37; 95% CI = 78 to 98%), 98% (110 of 112; 95% CI = 93 to 100%), and 97% (144 of 149), respectively, were found for the mip gene-based PCR. A total of 35 patients were diagnosed by using the urinary antigen test, and 34 were diagnosed by the 16S rRNA-based PCR. With the mip gene PCR one more case of LD (n = 36; not significant) was detected. By combining urinary antigen test and the mip gene PCR, LD was diagnosed in an additional 4 (11%) patients versus the use of the urinary antigen test alone. The addition of a L. pneumophila-specific mip gene PCR to a urinary antigen test is useful in patients with suspected LD who produce sputum and might allow the early detection of a significant number of additional patients.

The use of imperfect diagnostic tests had an impact on prevalence estimation

OBJECTIVE

Problems arising with the estimation of sensitivity and specificity when two imperfect diagnostic tests are applied are widely discussed. Effects on the estimation of prevalence may be of importance as well. Different methods of dealing with two or more imperfect tests and unknown reference standard are contrasted with regard to their implications on prevalence estimation: discrepant analysis, composite reference standards, and latent class models.

STUDY DESIGN AND SETTING

Prospective epidemiological multicenter study to determine the prevalence of respiratory syncytial virus in children with lower respiratory tract infections. A subsample of 1,003 patients from a hospital population and from a practice population is considered. Virus isolation, polymerase chain reaction, and rapid antigen test had been applied.

RESULTS

Prevalence estimates obtained under various assumptions ranged from 0.263 to 0.386 in the hospital population and from 0.214 to 0.277 in the practice population.

CONCLUSION

Estimation procedures involving a resolver test applied to some but not all cells are at risk of introducing a serious bias in prevalence estimation as well as in the estimation of test accuracy parameters. Estimation via latent class modeling may be more useful, but care should be taken regarding the underlying assumptions.

Performance of the gen-probe transcription-mediated [corrected] amplification research assay compared to that of a multitarget real-time PCR for Mycoplasma genitalium detection

Mycoplasma genitalium (MG) can cause nongonococcal urethritis and is potentially associated with urethritis, endometritis, and cervicitis. Several assays have been developed to detect MG. Molecular amplification assays for organism detection can be problematic due to the potential for false-positive and false-negative results. Confirmatory testing is often required in these situations, requiring additional time and resources. Use of multigene targets could integrate both detection and verification at lower cost. Utilizing two targets, the MgPa adhesion gene and the 16S rRNA gene, a multitarget real-time (MTRT) PCR for the detection of MG was developed. Samples from patients attending sexually transmitted disease clinics were collected in duplicate. Urine samples from males (n = 286) and self-collected vaginal swabs from females (n = 321) were analyzed by MTRT PCR for MG and the Gen-Probe transcription-mediated [corrected] amplification (TMA) assay, which targets MG rRNA for detection (TMA-MG research use only). Utilizing the criteria of any two targets being positively amplified, the MTRT PCR had a sensitivity and specificity of 91.8% (101 positive samples/110 samples tested) and 99.5% (495/497), respectively, with a positive predictive value (PPV) of 98.1% (101/103) and a negative predictive value (NPV) of 98.2% (495/504). The Gen-Probe TMA-MG assay had a sensitivity, specificity, PPV, and NPV of 98.1% (108/110), 98.1% (488/497), 92.3% (108/117), and 99.5% (488/490), respectively. Comparison between the MTRT PCR and TMA-MG assay by kappa statistic analysis indicated that an overall kappa value was 0.941 (95% confidence interval, 0.907 and 0.976). Both assays demonstrated accuracy in the detection of MG from urine samples from male patients and self-collected vaginal swabs from female patients.

Comparison between the Gen-Probe transcription-mediated amplification Trichomonas vaginalis research assay and real-time PCR for Trichomonas vaginalis detection using a Roche LightCycler instrument with female self-obtained vaginal swab samples and male urine samples

This study compared two assays for Trichomonas vaginalis detection, Gen-Probe's transcription-mediated amplification (TMA) assay for Trichomonas vaginalis and BTUB FRET PCR, using self-obtained clinical samples from 611 patients. Infection status was defined as two positive results by two different tests. The initial TMA assay sensitivity was 96.7%; specificity was 97.5%. The TMA assay was comparable to BTUB FRET PCR.

Development and evaluation of a real-time PCR assay targeting the type III secretion system of Burkholderia pseudomallei

Here we report on the development of a discriminatory real-time assay for the rapid identification of Burkholderia pseudomallei isolates and the evaluation of this assay for sensitivity against related species and detection in spiked human blood samples. The assay targets a 115-base-pair region within orf2 of the B. pseudomallei type III secretion system gene cluster and distinguishes B. pseudomallei from other microbial species. Assay performance was evaluated with 224 geographically, temporally, and clinically diverse B. pseudomallei isolates from the Centers for Disease Control and Prevention strain collection. This represents the first real-time PCR for rapid and sensitive identification of B. pseudomallei that has been tested for cross-reactivity with 23 Burkholderia mallei, 5 Burkholderia thailandensis, and 35 Burkholderia and 76 non-Burkholderia organisms which have historically presented diagnostic challenges. The assay performed with 100% specificity. The limit of detection was found to be 76 femtograms of DNA (equivalent to 5.2 x 10(3) genome equivalents per ml) in a single PCR. In spiked human blood, the assay could detect as few as 8.4 x 10(3) CFU per ml. This rapid assay is a valuable tool for identification of B. pseudomallei and may improve diagnosis in regions endemic for melioidosis.

Nucleic acid amplification testing for Neisseria gonorrhoeae: an ongoing challenge

Nucleic acid amplification tests (NAATs) for the detection of Neisseria gonorrhoeae became available in the early 1990s. Although offering several advantages over traditional detection methods, N. gonorrhoeae NAATs do have some limitations. These include cost, risk of carryover contamination, inhibition, and inability to provide antibiotic resistance data. In addition, there are sequence-related limitations that are unique to N. gonorrhoeae NAATs. In particular, false-positive results are a major consideration. These primarily stem from the frequent horizontal genetic exchange occurring within the Neisseria genus, leading to commensal Neisseria species acquiring N. gonorrhoeae genes. Furthermore, some N. gonorrhoeae subtypes may lack specific sequences targeted by a particular NAAT. Therefore, NAAT false-negative results because of sequence variation may occur in some gonococcal populations. Overall, the N. gonorrhoeae species continues to present a considerable challenge for molecular diagnostics. The need to evaluate N. gonorrhoeae NAATs before their use in any new patient population and to educate physicians on the limitations of these tests is emphasized in this review.

Evaluation of Nucleic Acid Amplification Tests in the Absence of a Perfect Gold-Standard Test

During the past 10 years, medical diagnostic testing for sexually transmitted infections (STIs) has changed markedly as a result of the rapid expansion and marketing of nucleic acid amplification tests (NAATs). Among such new DNA/RNA-amplification techniques are the polymerase chain reaction (PCR), the ligase chain reaction (LCR), and the transcription-mediated amplification (TMA) tests. Regrettably, the test evaluation process undergone by these tests has not always been rigorous or scientifically sound. Here, we review the controversy surrounding the statistical evaluation of these NAATs. We also review some of the traditional and recent statistical methods developed to estimate test sensitivity and specificity parameters in the absence of reliable gold-standard tests. In particular, we review the traditional latent class modeling approach that requires the assumption of independence between diagnostic tests conditional on the true disease status, and the more recent procedures that relax the conditional independence assumption. Finally, we apply some of these statistical modeling techniques to real data to estimate the sensitivity and specificity of a NAAT for Chlamydia trachomatis. On the basis of the latent class modeling approach with a pessimistic prior for culture sensitivity, the NAAT specificity estimate was 97.6% and, on the basis of an optimistic prior, the specificity was 95.3%. Similarly, the sensitivity estimates ranged from 88.1% to 89.6%.

Diagnostic Tests for Chlamydial and Gonorrheal Infections

Chlamydia trachomatis and Neisseria gonorrhoeae are the two most prevalent bacterial sexually transmitted infections reported in the United States. The purpose of this article is to review the various tests available for diagnosing C. trachomatis and N. gonorrhoeae. The nucleic acid amplification tests are preferred owing to their high sensitivity and specificity and use on specimens obtained noninvasively. Non-amplified non-culture tests, such as the DNA probe test, remain a choice for settings in which nucleic acid amplification tests are not available or affordable. Culture is recommended for use on conjunctival, oropharyngeal, and rectal specimens and in cases of suspected abuse. In addition, for gonorrhea, culture is recommended when screening in populations with a low prevalence of infection. Point-of-care tests are limited by low sensitivity and should be used for only populations unlikely to return for follow-up.

Use of multiple nucleic acid amplification tests to define the infected-patient "gold standard" in clinical trials of new diagnostic tests for Chlamydia trachomatis infections

Nucleic acid amplification tests (NAATs) can be used to define the infected-patient "gold standard" for the purpose of designing studies of the performance of Chlamydia trachomatis diagnostic tests. It is unclear how many test results run by different NAATs and what combinations of specimens comprise the best infected-patient gold standard. We approached this question with data from a large study of the performance of a new NAAT. Data were available from three endocervical swabs and a urine specimen collected from each of 1,412 women and tested by three different NAATs. Results from all three assays were used equally in a rotating fashion to define the infected-patient gold standard. Multiple different infected-patient gold standards for estimating swab and urine specimen sensitivity and specificity for one NAAT method were created by varying the number and combinations of swab and urine comparator results with two different NAATs, The effect of changing the infected-patient gold standard definition was determined by constructing receiver-operator-like curves with calculated sensitivities and specificities for each test. The one-positive-of-two-results or two-positive-of-two-results (same or two different assays) infected-patient gold standard definitions produced low sensitivity and low specificity estimates, respectively. If four comparator NAAT results were used, the any-three-positive-of-four-results definition or the at-least-one-specimen-positive-by-each-of-two-comparator-assays definition appeared to provide better combinations of sensitivity and specificity estimates. The any-two-positive-out-of-three-results definition resulted in estimates that were as good as produced with the former two definitions. This analytic approach provides a means of clearly visualizing the effects of changing NAAT-based infected-patient gold standards and should be helpful in designing future studies of new C. trachomatis diagnostic tests.

Genetic tests and their evaluation: Can we answer the key questions?

The rapid pace of research in the field of genetics has already yielded many benefits. The development of new genetic tests is one such example. Before there can be widespread uptake of these tests they need to be evaluated to confirm the benefits of their use. The authors review some of the key features of the evaluation of diagnostic tests focusing on analytical and clinical validity. Test properties such as sensitivity, specificity, likelihood ratios, positive and negative predictive values, and how they relate to molecular genetic testing are discussed. Associated issues such as the concepts of disease definition, imperfect reference standards, and false positives are also explored. The authors suggest possible approaches to addressing some of the problems identified.

Diagnosis and assessment of trachoma

Trachoma is caused by Chlamydia trachomatis. Clinical grading with the WHO simplified system can be highly repeatable provided graders are adequately trained and standardized. At the community level, rapid assessments are useful for confirming the absence of trachoma but do not determine the magnitude of the problem in communities where trachoma is present. New rapid assessment protocols incorporating techniques for obtaining representative population samples (without census preparation) may give better estimates of the prevalence of clinical trachoma. Clinical findings do not necessarily indicate the presence or absence of C. trachomatis infection, particularly as disease prevalence falls. The prevalence of ocular C. trachomatis infection (at the community level) is important because it is infection that is targeted when antibiotics are distributed in trachoma control campaigns. Methods to estimate infection prevalence are required. While culture is a sensitive test for the presence of viable organisms and nucleic acid amplification tests are sensitive and specific tools for the presence of chlamydial nucleic acids, the commercial assays presently available are all too expensive, too complex, or too unreliable for use in national programs. There is an urgent need for a rapid, reliable test for C. trachomatis to assist in measuring progress towards the elimination of trachoma.

Comparison of multiplex PCR assay with culture for detection of genital mycoplasmas

Ureaplasma, spp. Mycoplasma genitalium, and Mycoplasma hominis are associated with infection of the genitourinary tract, reproductive failure, and neonatal morbidity and mortality. We have developed a multiplex PCR for the detection of these Mycoplasmatales in a single amplification reaction. The analytical sensitivities of this assay were 10.8, 10.8, and 8.8 CFU for each organism, respectively. This multiplex PCR was compared to culture on 26 cervical swabs, 2 vaginal swabs, 4 female urine specimens, 49 semen samples, 2 male urine specimens, and 1 nonspecified sample. A total of 21 specimens were culture positive (25%); 17 of these were PCR positive. An additional 11 specimens were PCR positive but culture negative. Of the 21 culture-positive specimens, 17 (81%) grew Ureaplasma spp. and 4 (19%) grew Mycoplasma spp. Of the 28 PCR-positive specimens, Ureaplasma spp. was detected in 23 (82%), M. hominis was detected in 3 (11%), and both were detected in 2 (7%). In a confirmatory analysis, all samples were tested by amplification of a second target of the ureaplasma genome. True-positive cases were defined as a positive result by culture or by both amplification assays. The multiplex PCR detected organisms in 26 of the 30 true-positive specimens, as well as in 2 other specimens. Based on a 36% prevalence of infection, the sensitivity, specificity, and positive and negative predictive values of multiplex PCR analyses were 87, 96, 94, and 93%, respectively. Multiplex PCR offers a rapid, sensitive, and easy method to detect genital mycoplasmas.

Comparison of three nucleic acid amplification tests for detection of Chlamydia trachomatis in urine specimens

Traditionally, culture and immunoassays have been performed for the detection of sexually transmitted diseases, including Chlamydia trachomatis. However, these assays may often require invasive specimen collection methods, such as female cervical and male urethral swabs. Recently, nucleic acid amplification tests (NAATs) have been approved for testing for the presence of C. trachomatis in urine samples. Our objective was to compare the sensitivities and specificities of C. trachomatis detection in urine samples with three NAATs: the Abbott LCx (LCx), BD ProbeTec ET (ProbeTec), and Gen-Probe APTIMA Combo 2 (AC2). Urine specimens (n = 506) were collected from both symptomatic and asymptomatic males and females from various high school health clinics. Specimens were tested for C. trachomatis with the three NAATs, and a true-positive result was defined as any two positive NAATs. The C. trachomatis prevalence was 14.8% (75 of 506 samples). Of the 75 urine samples defined as true positives, LCx detected 72, ProbeTec 72, and AC2 detected 75. The sensitivities of LCx, ProbeTec, and AC2 for C. trachomatis detection were 96.0, 96.0, and 100%, and the specificities were 99.1, 100, and 98.8%, respectively. Four of five samples that were positive with AC2 and negative with LCx and ProbeTec were found to be positive with an alternative target TMA-based NAAT, APTIMA C. trachomatis, suggesting that they may have been true positives. Two of four uniquely positive LCx samples available for subsequent testing were both found to be positive by Roche PCR. We found that the LCx, ProbeTec, and AC2 NAATs are highly sensitive and specific methods for the detection of C. trachomatis in urine specimens and can be recommended for noninvasive screening of C. trachomatis in urine.

PCR-based diagnostics for infectious diseases: uses, limitations, and future applications in acute-care settings

Molecular diagnostics are revolutionising the clinical practice of infectious disease. Their effects will be significant in acute-care settings where timely and accurate diagnostic tools are critical for patient treatment decisions and outcomes. PCR is the most well-developed molecular technique up to now, and has a wide range of already fulfilled, and potential, clinical applications, including specific or broad-spectrum pathogen detection, evaluation of emerging novel infections, surveillance, early detection of biothreat agents, and antimicrobial resistance profiling. PCR-based methods may also be cost effective relative to traditional testing procedures. Further advancement of technology is needed to improve automation, optimise detection sensitivity and specificity, and expand the capacity to detect multiple targets simultaneously (multiplexing). This review provides an up-to-date look at the general principles, diagnostic value, and limitations of the most current PCR-based platforms as they evolve from bench to bedside.

Use of the Roche LightCycler instrument in a real-time PCR for Trichomonas vaginalis in urine samples from females and males

Trichomonas vaginalis is the agent of a highly prevalent sexually transmitted infection (STI) that can result in vaginitis, urethritis, and preterm birth. Traditional methods of diagnosis, including wet preparation, can be unreliable. In this study, we describe the adaptation of an existing PCR method for specific detection of T. vaginalis DNA into a rapid real-time PCR assay based on fluorescence resonance energy transfer (FRET) probe chemistry. The FRET-based assay described demonstrated high sensitivity with a detection limit of 1.06 organisms, as well as a high specificity. A total of 253 urine samples collected prospectively from both men and women were tested for T. vaginalis DNA with both the FRET-based assay and a previously validated PCR assay. When the validated PCR assay was used as the "gold standard" and after discrepancies had been resolved, our FRET-based assay demonstrated an analytical sensitivity and specificity of 90.1 and 100%, respectively. Overall results suggest that FRET-based assays can provide rapid, accurate, and high-throughput detection of T. vaginalis and may prove useful in clinical settings and for large-scale screening programs.

Confirmation by 16S rRNA PCR of the COBAS AMPLICOR CT/NG test for diagnosis of Neisseria gonorrhoeae infection in a low-prevalence population

The COBAS AMPLICOR CT/NG test is widely used for the diagnosis of Neisseria gonorrhoeae infection using genital swabs or urine samples. Although highly specific, cross-reactivity occurs with some nonpathogenic strains of Neisseria and Lactobacillus species. In low-prevalence populations, even highly specific assays may require confirmatory testing of positive results. We assessed the positive predictive value (PPV) of this test in a low-prevalence (0.5%) setting. Genital and urine specimens testing positive using the COBAS AMPLICOR NG test were retested using an investigational 16S rRNA PCR assay. Additionally, 737 specimens were tested in parallel by both culture and the above PCR protocol. Of 9,772 specimens tested in-house, 168 were positive by the AMPLICOR test; in addition, 62 AMPLICOR-positive specimens were referred to our laboratory for confirmatory testing, yielding 230 positive specimens. Of these, 72 were confirmed positive by 16S rRNA PCR, yielding a specificity of 98.7% and a PPV of 31.3%. Specificity was similar for all specimen types, whereas PPV varied with prevalence: specimens from males, females, urine specimens, and genital swabs had PPVs of 70.8, 13.3, 51.9, and 20.1%, respectively. The PPV was higher when the initial AMPLICOR optical density (OD) was > or =3.5 versus initial and repeat OD readings in an equivocal zone of > or =0.2 to <3.5 (65.1 versus 10.1%; P < 0.001). On repeat testing of specimens with ODs in the equivocal zone, 54 gave ODs of > or =0.2 and <2.0, 35 gave ODs of > or =2.0 and <3.5, and 12 gave ODs of > or =3.5, with 3.7, 20, and 33.3% confirmed positive, respectively (P = 0.004). Comparing PCR to culture as the "gold standard," specificity increased from 96.8 to 99.9% when 16S rRNA PCR was performed on specimens positive by the COBAS AMPLICOR NG test. Confirmatory testing with a more specific method such as 16S rRNA PCR should be considered in low-prevalence populations, especially for specimens with an OD in the equivocal zone.

Head-to-head multicenter comparison of DNA probe and nucleic acid amplification tests for Chlamydia trachomatis infection in women performed with an improved reference standard

Few evaluations of tests for Chlamydia trachomatis have compared nucleic acid amplification tests (NAATs) with diagnostic tests other than those by culture. In a five-city study of 3,551 women, we compared the results of commercial ligase chain reaction (LCR) and PCR tests performed on cervical swabs and urine with the results of PACE 2 tests performed on cervical swabs, using independent reference standards that included both cervical swabs and urethral swab-urine specimens. Using cervical culture as a standard, the sensitivities of PACE 2, LCR, and PCR tests with cervical specimens were 78.1, 96.9, and 89.9%, respectively, and the specificities were 99.3, 97.5, and 98.2%, respectively. Using either cervical swab or urine LCR-positive tests as the standard decreased sensitivities to 60.8% for PACE 2 and to 75.8 and 74.9% for PCR with cervical swabs and urine, respectively. Specificities increased to 99.7% for PACE 2 and to 99.7 and 99.4% for PCR with cervical swabs and urine, respectively. Sensitivities with a cervical swab-urine PCR standard were 61.9% for PACE 2 and 85.5 and 80.8% for LCR with cervical swabs and urine, respectively. Specificities were 99.6% for PACE 2 and 99.0 and 98.9% for LCR with cervical swabs and urine, respectively. Cervical swab versus urine differences were significant only for PCR specificities (P = 0.034). Overall, LCR sensitivity exceeded that of PCR, and sensitivities obtained with cervical swabs exceeded those obtained with urine specimens by small amounts. These data have substantiated, using a large multicenter sample and a patient standard, that LCR and PCR tests performed on endocervical swabs and urine are superior to PACE 2 tests for screening C. trachomatis infections in women. In our study, NAATs improved the detection of infected women by 17 to 38% compared to PACE 2.

Appropriate use of clinical microbiology tests

Laboratory medicine lacks the tools necessary to define appropriate test use; nonetheless, existing laboratory test characteristics, although inadequate, provide a common starting point for developing definitions of appropriate test use. As Lundberg [1], Szczepura [15], and van Walraven and Naylor [2] have emphasized, developing a process for defining appropriate laboratory use should receive a high priority. There is a particular need to develop methods for standardizing studies [59]. Laboratory medicine does not, however, lack the tools necessary to change test use. Although past interventions were largely ineffective, there is growing evidence that test use can be changed by use of a variety of approaches. By using the existing tools there is much that can be done to change inappropriate test use, such as minimizing redundant testing or the use of tests that have no clinical relevance. The real opportunities will come when there are scientifically and medically sound definitions of appropriate test use that can be used to change test use and improve patient care in a cost-effective way.

Relative accuracy of nucleic acid amplification tests and culture in detecting Chlamydia in asymptomatic men

Published estimates of the sensitivity and specificity of PCR and ligase chain reaction (LCR) for detecting Chlamydia trachomatis are potentially biased because of study design limitations (confirmation of test results was limited to subjects who were PCR or LCR positive but culture negative). Relative measures of test accuracy are less prone to bias in incomplete study designs. We estimated the relative sensitivity (RSN) and relative false-positive rate (RFP) for PCR and LCR versus cell culture among 1,138 asymptomatic men and evaluated the potential bias of RSN and RFP estimates. PCR and LCR testing in urine were compared to culture of urethral specimens. Discordant results (PCR or LCR positive, but culture negative) were confirmed by using a sequence including the other DNA amplification test, direct fluorescent antibody testing, and a DNA amplification test to detect chlamydial major outer membrane protein. The RSN estimates for PCR and LCR were 1.45 (95% confidence interval [CI] = 1.3 to 1.7) and 1.49 (95% CI = 1.3 to 1.7), respectively, indicating that both methods are more sensitive than culture. Very few false-positive results were found, indicating that the specificity levels of PCR, LCR, and culture are high. The potential bias in RSN and RFP estimates were <5 and <20%, respectively. The estimation of bias is based on the most likely and probably conservative parameter settings. If the sensitivity of culture is between 60 and 65%, then the true sensitivity of PCR and LCR is between 90 and 97%. Our findings indicate that PCR and LCR are significantly more sensitive than culture, while the three tests have similar specificities.

Some issues in resolution of diagnostic tests using an imperfect gold standard

As a subject's true disease status is seldom known with certainty, it is necessary to compare the performance of new diagnostic tests with those of a currently accepted but imperfect 'gold standard'. Errors made by the gold standard mean that the sensitivity and specificity calculated for the new test are biased, and do not correctly estimate the new method's sensitivity and specificity. The traditional approach to this problem was 'discrepant resolution', in which the subjects for whom the two methods disagreed were subjected to a third 'resolver' test. Recent work has pointed out that this does not automatically solve the problem. A sounder approach goes beyond the discordant test results and tests at least some of the subjects with concordant results with the resolver also. This leaves some issues unresolved. One is the basic question of the direction of biases in various estimators. We point out that this question does not have a simple universal answer. Another issue, if one is to test a sample of the subjects with concordant results rather than all cases, is how to compute estimates and standard errors of the measures of test performance, notably sensitivity and specificity of the test method relative to the resolver. Expressions for these standard errors are given and illustrated with a numeric example. It is shown that using just a sample of subjects with concordant results may lead to great savings in assays. The design issue of how many concordant cells to test depends on the numbers of concordants and discordants. The formulae given show how to evaluate impact of different choices for these numbers and hence settle on a design that gives the required precision of estimates.

Evaluation of a modified sanitary napkin as a sample self-collection device for the detection of genital chlamydial infection in women

A modified sanitary napkin was compared with endocervical swab and urine specimens for the detection of urogenital Chlamydia trachomatis infection. Endocervical swabs and/or first-catch urine were collected from 510 women at medical or community settings in Quebec City. Participants were also asked to wear a modified sanitary napkin (Ezy-Detek) during 4 consecutive hours and to bring it back to the clinic or mail it to the laboratory. Endocervical and urine specimens were tested using the Cobas Amplicor CT/NG assay (Roche Diagnostic Systems) according to the manufacturer's instructions, as were specimens collected with the napkin after adequate preparation. If the PCR test result was positive on the endocervical sample or on any two samples, a woman was considered to be infected. PCR testing results on paired samples were identical for 493 (96.6%) of 510 women. According to the definition given above, 58 (11.3%; 95% confidence interval [CI], 8.7 to 14.5%) women were infected with C. trachomatis. The sensitivity and specificity of PCR testing on modified sanitary napkin specimens were, respectively, 93.1% (54 of 58; 95% CI, 83.3 to 98.1%) and 98.9% (447 of 452; 95% CI, 97.4 to 99.6%) compared to 81.0% (47 of 58; 95% CI, 68.6 to 90.1%) and 100% (451 of 451; 95% CI, 99.2 to 100%) for urine specimens. The positive and negative predictive values were, respectively, 91.5% (54 of 59) and 99.1% (447 of 451) for the sanitary napkin specimens compared to 100% (47 of 47) and 97.6% (451 of 462) for urine samples. These results suggest that a modified sanitary napkin represents an effective noninvasive device for self-collection of specimens to detect urogenital C. trachomatis infection.

Detection of Chlamydia trachomatis and Neisseria gonorrhoeae by enzyme immunoassay, culture, and three nucleic acid amplification tests

The purpose of this study was to evaluate and compare three commercially available nucleic acid amplification tests (NAATs) for the detection of Neisseria gonorrhoeae and Chlamydia trachomatis. Roche PCR and Becton Dickinson strand displacement amplification (SDA) were performed on 733 endocervical swab specimens from commercial sex workers. Abbott ligase chain reaction (LCR) was performed on a subset of 396 samples. Endocervical specimens from all women were also tested by culture for N. gonorrhoeae and by Syva MicroTrak enzyme immunoassay (EIA) for C. trachomatis. A positive N. gonorrhoeae result was defined as a positive result by culture or by two NAATs, and a positive C. trachomatis result was defined as a positive result by two tests. According to these definitions, the sensitivities and specificities for the subsample of 396 specimens of N. gonorrhoeae culture, PCR, SDA, and LCR were 69.8, 95.2, 88.9, and 88.9% and 100, 99.4, 100, and 99.1%, respectively; the sensitivities and specificities of C. trachomatis EIA, PCR, SDA, and LCR were 42.0, 98.0, 94.0, and 90.0% and 100, 98.0, 100, and 98.6%, respectively. The performance characteristics of N. gonorrhoeae culture, PCR, and SDA and C. trachomatis EIA, PCR, and SDA for all 733 specimens were defined without inclusion of LCR results and by discrepant analysis after resolution of discordant N. gonorrhoeae PCR results and of discordant C. trachomatis EIA and PCR results by LCR testing. The sensitivities of N. gonorrhoeae culture, PCR, and SDA before and after LCR resolution were 67.8, 95.7, and 93.9% and 65, 95.8, and 90.0%, respectively. The sensitivities of C. trachomatis EIA, PCR, and SDA decreased from 39.4, 100, and 100% to 38.7, 98.7, and 94.7%, respectively. All three NAATs proved to be superior to N. gonorrhoeae culture and to C. trachomatis EIA. The accuracies of the different NAATs were quite similar. SDA was the only amplification assay with 100% specificity for detection of both N. gonorrhoeae and C. trachomatis in endocervical specimens.

Screening for chlamydial infection

OBJECTIVES

To examine data on the effectiveness of screening for chlamydial infection by a physician or other health care professional. Specifically, we examine the evidence that early treatment of chlamydial infection improves health outcomes, as well as evidence of the effectiveness of screening strategies in nonpregnant women, pregnant women, and men, and the accuracy of tests used for screening. This review updates the literature since the last recommendation of the U.S. Preventive Services Task Force published in 1996.

SEARCH STRATEGY

We searched the topic of chlamydia in the MEDLINE, HealthSTAR, and Cochrane Library databases from January 1994 to July 2000, supplemented by reference lists of relevant articles and from experts in the field. Articles published prior to 1994 and research abstracts were cited if particularly important to the key questions or to the interpretation of included articles.

SELECTION CRITERIA

A single reader reviewed all English abstracts. Articles were selected for full review if they were about Chlamydia trachomatis genitourinary infections in nonpregnant women, pregnant women, or men and were relevant to key questions in the analytic framework. Investigators read the full-text version of the retrieved articles and applied additional eligibility criteria. For all topics, we excluded articles if they did not provide sufficient information to determine the methods for selecting subjects and for analyzing data.

DATA COLLECTION AND ANALYSIS

We systematically reviewed three types of studies about screening in nonpregnant women that relate to three key questions: (1) studies about the effectiveness of screening programs in reducing prevalence rates of infection, (2) studies about risk factors for chlamydial infection in women, and (3) studies about chlamydial screening tests in women. Our search found too few studies on pregnant women to systematically review, although pertinent studies are described. We systematically reviewed two types of studies about screening in men: (1) studies about prevalence rates and risk factors for chlamydial infection in men and (2) studies about chlamydial screening tests in men.

MAIN RESULTS

Nonpregnant women. The results of a randomized controlled trial conducted in a large health maintenance organization indicate that screening women selected by a set of risk factors reduces the incidence of pelvic inflammatory disease (PID) over a 1-year period. Changes in population prevalence rates have not been well documented because few studies have employed a representative population sample. Age continues to be the best predictor of chlamydial infection in women, with most studies evaluating cut-offs at age younger than 25 years. Other risk factors may be useful predictors, but these are likely to be population specific. To determine the accuracy of screening tests for women, we retrieved and critically reviewed 34 articles on test performance. Results indicate that endocervical swab specimens and first-void urine specimens have similar performance when using DNA amplification tests and have better sensitivity than endocervical culture. Recurrent chlamydial infections in women have been associated with increased risks for PID and ectopic pregnancies. Pregnant women. The Second Task Force recommendations for screening pregnant women were based on two major studies demonstrating improved pregnancy outcomes following treatment of chlamydial infection. We identified no recent studies on this topic in our literature search. Very few studies describe risk factors for chlamydial infection in pregnant women. Nonculture testing techniques appear to perform well in pregnant women, although studies are limited. Men. No studies described the effectiveness of screening or early treatment for men in reducing transmission to women or in preventing acute infections or complications in men. Studies of prevalence rates and risk factors for chlamydial infection in men are limited. Age lower than 25 years is the strongest known risk factor cited so far. Results of urethral swab specimens compared to first-void urine specimens were similar for DNA amplification tests. DNA amplification techniques are more sensitive than culture.

CONCLUSIONS

Screening women for Chlamydia trachomatis reduces the incidence of PID, and it is associated with reductions in prevalence of infection in uncontrolled studies. No studies were found to determine whether screening asymptomatic men would reduce transmission or prevent acute infections or complications. Age is the strongest risk factor for men and women. A variety of tests can detect chlamydial infection with acceptable sensitivity and specificity, including new DNA amplification tests that use either endocervical swabs in women, urethral swabs in men, or first-void urine specimens from men and women.

Evaluation of nucleic acid amplification tests as reference tests for Chlamydia trachomatis infections in asymptomatic men

Urine ligase chain reaction (LCR) and PCR tests and urethral swab culture were compared for their abilities to detect Chlamydia trachomatis infection in 3,639 asymptomatic men by using one-, two-, and three-test reference standards. Frozen urine at four of five participating centers was also tested by a transcription-mediated amplification assay which was used as a reference test. LCR increased the yield of positive results by 27% and PCR increased the yield of positive results by 26% over the yield of positive results by culture (n = 295). LCR and PCR sensitivities were similar, ranging from 80.4 to 93.5%, depending on the reference standard. Culture sensitivity was substantially less. A multiple-test standard yielded LCR, PCR, and culture specificities of 99.6%, with or without discrepant analysis. Test performance varied among centers partly due to different interpretations of the testing protocols. The study confirms that urine LCR and PCR for the detection of C. trachomatis have substantially improved sensitivities over that of urethral swab culture for testing of asymptomatic men, enabling screening of this important target group. These tests, perhaps in combination, are also candidate reference tests for the conduct of test evaluation studies.

Inference about misclassification probabilities from repeated binary responses

Repeated binary responses provide efficient information for two purposes: (1) estimating two misclassification (false-positive and false-negative error) probabilities and (2) testing the hypothesis that either is zero in a reliability study. We focus on the assessment of reliability of a diagnostic test when there is no gold standard. This paper uses a latent class model and illustrates some of its properties. In addition, application to data containing variation among individuals is considered. We apply this model to the serological data on the MNSs blood group of atomic bomb survivors and their children. The results provide valuable information for examining measurement reliability.

Multicenter evaluation of the AMPLICOR and automated COBAS AMPLICOR CT/NG tests for detection of Chlamydia trachomatis

The fully automated COBAS AMPLICOR CT/NG and semiautomated AMPLICOR CT/NG tests were evaluated in a multicenter trial for the ability to detect Chlamydia trachomatis infections. Test performance compared to that of culture was evaluated for 2,236 matched endocervical swab and urine specimens obtained from women and for 1,940 matched urethral swab and urine specimens obtained from men. Culture-negative, PCR-positive specimens that tested positive in a direct fluorescent-antibody test or in a confirmatory PCR test for an alternative target sequence were resolved as true positives. The overall prevalences of chlamydia were 2.4% in women and 7.2% in men. The COBAS AMPLICOR and AMPLICOR formats yielded concordant results for 98.1% of the specimens. With the infected patient as the reference standard, the resolved sensitivities of COBAS AMPLICOR were 89.7% for endocervical swab specimens, 89.2% for female urine specimens, 88.6% for male urethral swab specimens, and 90.3% for male urine specimens. When results were analyzed as if only a single test had been performed on a single specimen type, the resolved sensitivity was always higher. The resolved specificities of PCR were 99.4% for endocervical swab specimens, 99.0% for female urine specimens, 98.7% for male urethral swab specimens, and 98.4% for male urine specimens. The internal control revealed that 2.4% of the specimens were inhibitory when initially tested. Nevertheless, valid results were obtained for 98.6% of the specimens because 59.1% of the inhibitory specimens were not inhibitory when a second aliquot was tested. The COBAS AMPLICOR and AMPLICOR CT/NG tests for C. trachomatis exhibited equally high sensitivity and specificity with both urogenital swab and urine specimens and thus are well suited for screening for C. trachomatis infection.

Discrepant analysis: a biased and an unscientific method for estimating test sensitivity and specificity

Discrepant analysis is a widely used technique for estimating test performance indices (sensitivity, specificity, etc.) of DNA-amplification tests for detecting infectious diseases. It has recently been claimed that the discrepant analysis-based estimates of specificity are typically less biased than those based on culture and that the discrepant analysis-based specificity shows little appreciable bias. In this article, I show that those conclusions are incorrect. Using a typical example from the published literature, I show that the discrepant analysis-based estimates of sensitivity and specificity can generate a significant and clinically important overestimation of the true sensitivity and specificity values. Moreover, I demonstrate that the concept of discrepant analysis is profoundly flawed and unscientific. It violates a fundamental principle of diagnostic testing-the principle that the new test should not be used to determine the true disease status. Thus, the major problem with discrepant analysis is not only that it is biased but that it is unscientific. Therefore, discrepant analysis should not be adopted for the evaluation of any diagnostic or screening test.

Using a combination of reference tests to assess the accuracy of a new diagnostic test

Often the accuracy of a new diagnostic test must be assessed when a perfect gold standard does not exist. Use of an imperfect reference test biases accuracy estimates of the new test. This paper reviews existing approaches to this problem including discrepant resolution and latent class analysis. Deficiencies with these approaches are identified. A new approach is proposed that combines the results of several imperfect reference tests to define a better reference standard. We call this the composite reference standard (CRS). Using the CRS, accuracy can be assessed using multi-stage sampling designs. Maximum likelihood estimates of accuracy and expressions for the variance of sensitivity and specificity are provided. Data from clinical literature on the detection of Chlamydia trachomatis are used to illustrate and compare the different approaches. Advantages of the CRS relative to other approaches include that the CRS is explicitly defined, does not depend on the results of the new test under investigation, and is easy to interpret.

Detection of Neisseria gonorrhoeae infection by ligase chain reaction testing of urine among adolescent women with and without Chlamydia trachomatis infection

BACKGROUND AND OBJECTIVES

Culture, the conventional method for detection of Neisseria gonorrhoeae, requires invasive sampling and stringent specimen transport conditions. The recently developed ligase chain reaction test (LCR; Abbott Laboratories; North Chicago, IL) allows noninvasive sampling and stable transport conditions, but has not been evaluated with specimens from adolescent populations.

GOAL OF THIS STUDY

To perform a comparative evaluation of a commercial LCR test and culture for the diagnosis of N. gonorrhoeae in adolescent women.

STUDY DESIGN

Urine and endocervical swab specimens from 330 teenage women seen in two public health adolescent clinics were tested by LCR and culture. For resolution of discordant results, a polymerase chain reaction (PCR) test was developed that directly amplifies N. gonorrhoeae DNA from urine samples processed for LCR.

RESULTS

Thirty-one of 330 (9.4%) cervical specimens were culture-positive for N. gonorrhoeae, and 30 of 330 (9.1%) urine specimens were positive by LCR. After resolution of 13 discordant results, the sensitivity, specificity, and positive and negative predictive values of LCR for urine were 88.2%, 100%, 100%, 98.7%, respectively, and for culture of endocervical specimens were 82.3%, 98.9%, 90.3% and 98%, respectively.

CONCLUSIONS

Although more expensive than culture, LCR offers a sensitive means for the detection of N. gonorrhoeae in urine samples and may be useful for this purpose in settings where pelvic examinations are difficult to perform and simultaneous detection of N. gonorrhoeae and Chlamydia trachomatis is advantageous.

Evaluation of the Biostar Chlamydia OIA assay with specimens from women attending a sexually transmitted disease clinic

Chlamydia trachomatis infections are the most prevalent sexually transmitted diseases (STDs) in the United States. In acute-care settings such as clinics and emergency rooms, a desirable chlamydia screening assay should exhibit good sensitivity and good specificity and should provide test results while the patient is still present. The Biostar Chlamydia OIA (Biostar, Inc., Boulder, Colo.) is an optical immunoassay (OIA) that provides test results in less than 30 min and that uses a test format that allows office-based testing. This assay is performed entirely at room temperature without the need for rotators or other specialized equipment. The goal of this study was to compare the performance of the Biostar Chlamydia OIA for the detection of C. trachomatis with the performance of cell culture, direct fluorescent-antibody (DFA) assay (Syva MicroTrak; Syva Co., Palo Alto, Calif.), and PCR (Roche Amplicor Chlamydia trachomatis; Roche, Branchburg, N.J.) for the detection of C. trachomatis infections in women attending an urban STD clinic. For calculations of relative test performance (sensitivity, specificity, and positive and negative predictive values), patient specimens that yielded positive results by two or more of the four assays (cell culture, DFA assay, PCR, and OIA) were classified as "true infections." By these criteria, 42 of 306 total specimens were classified as positive for C. trachomatis (positive prevalence, 13.7%), 11 (3.6%; 10 by PCR and 1 by DFA assay) were positive by a single assay, and 253 (82.7%) were negative by all four tests. All culture-positive specimens were also positive by at least one other assay. Among the culture-negative specimens, 14(5%) specimens were positive by two of the three non-culture-based assays used. By using the criterion that positivity by at least two of the tests indicated a true infection,the relative sensitivities were as follows: culture and PCR, 92.9% each; Biostar Chlamydia OIA, 73.8%; and DFA assay, 59.5%.

Performance of a commercial polymerase chain reaction test for endocervical Chlamydia trachomatis infection in a university hospital population

OBJECTIVES

To examine the accuracy of a commercial polymerase chain reaction (PCR) test (Amplicor CTR, Roche Diagnostic Systems, Branchburg NJ) for identification of endocervical chlamydial infections through both laboratory evaluation and among a diverse teaching hospital patient population.

METHODS

Testing of reliable threshold inocula and reproducibility were carried out using laboratory stock organisms. Paired endocervical samples from patients with a wide range of indications were tested by PCR and an established culture procedure, and discrepant pairs were further analyzed to determine true results.

RESULTS

Laboratory evaluation suggested that one copy of target DNA from a viable organism consistently yielded a positive result, and test reproducibility was very good, with an overall coefficient of variation of 15%. Compared to true results in 1,588 paired clinical samples from 1,489 women with a 10% prevalence of infection, the PCR test and culture yielded respective sensitivities of 87.4% and 78.0%, and negative predictive values of 98.6% and 97.6%. Specificity and positive predictive value for both tests were 100%. Cost per specimen was nearly identical at $18.84 and $18.88 respectively. Polymerase inhibitors and organisms lacking target DNA were not found in false-negative PCR samples.

CONCLUSION

This commercial PCR test is accurate, cost-competitive, and much faster than culture for diagnosis of endocervical chlamydia infections in our population of intermediate prevalence of chlamydial infection.

Performance of a commercial polymerase chain reaction test for endocervical Chlamydia trachomatis infection in a university hospital population

OBJECTIVES

To examine the accuracy of a commercial polymerase chain reaction (PCR) test (Amplicor CTR, Roche Diagnostic Systems, Branchburg NJ) for identification of endocervical chlamydial infections through both laboratory evaluation and among a diverse teaching hospital patient population.

METHODS

Testing of reliable threshold inocula and reproducibility were carried out using laboratory stock organisms. Paired endocervical samples from patients with a wide range of indications were tested by PCR and an established culture procedure, and discrepant pairs were further analyzed to determine true results.

RESULTS

Laboratory evaluation suggested that one copy of target DNA from a viable organism consistently yielded a positive result, and test reproducibility was very good, with an overall coefficient of variation of 15%. Compared to true results in 1,588 paired clinical samples from 1,489 women with a 10% prevalence of infection, the PCR test and culture yielded respective sensitivities of 87.4% and 78.0%, and negative predictive values of 98.6% and 97.6%. Specificity and positive predictive value for both tests were 100%. Cost per specimen was nearly identical at $18.84 and $18.88 respectively. Polymerase inhibitors and organisms lacking target DNA were not found in false-negative PCR samples.

CONCLUSION

This commercial PCR test is accurate, cost-competitive, and much faster than culture for diagnosis of endocervical chlamydia infections in our population of intermediate prevalence of chlamydial infection.