Bias in discrepant analysis: when two wrongs don't make a right

The evaluation of the utility of the GENECUBE HQ SARS-CoV-2 for anterior nasal samples and saliva samples with a new rapid examination protocol

INTRODUCTION

GENECUBE® is a rapid molecular identification system, and previous studies demonstrated that GENECUBE® HQ SARS-CoV-2 showed excellent analytical performance for the detection of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) with nasopharyngeal samples. However, other respiratory samples have not been evaluated.

METHODS

This prospective comparison between GENECUBE® HQ SARS-CoV-2 and reference real-time reverse transcriptase polymerase chain reaction (RT-PCR) was performed for the detection of SARS-CoV-2 using anterior nasal samples and saliva samples. Additionally, we evaluated a new rapid examination protocol using GENECUBE® HQ SARS-CoV-2 for the detection of SARS-CoV-2 with saliva samples. For the rapid protocol, in the preparation of saliva samples, purification and extraction processes were adjusted, and the total process time was shortened to approximately 35 minutes.

RESULTS

For 359 anterior nasal samples, the total-, positive-, and negative concordance of the two assays was 99.7% (358/359), 98.1% (51/52), and 100% (307/307), respectively. For saliva samples, the total-, positive-, and negative concordance of the two assays was 99.6% (239/240), 100% (56/56), and 99.5% (183/184), respectively. With the new protocol, total-, positive-, and negative concordance of the two assays was 98.8% (237/240), 100% (56/56), and 98.4% (181/184), respectively. In all discordance cases, SARS-CoV-2 was detected by additional molecular examinations.

CONCLUSION

GENECUBE® HQ SARS-CoV-2 provided high analytical performance for the detection of SARS-CoV-2 in anterior nasal samples and saliva samples.

Simultaneous Evaluation of Diagnostic Assays for Pharyngeal and Rectal Neisseria gonorrhoeae and Chlamydia trachomatis Using a Master Protocol

Background

Pharyngeal and rectal Neisseria gonorrhoeae and Chlamydia trachomatis play important roles in infection and antibacterial resistance transmission, but no US Food and Drug Administration (FDA)–cleared assays for detection at these sites existed prior to this study. The objective was to estimate performance of assays to detect those infections in pharyngeal and rectal specimens to support regulatory submission.

Methods

We performed a cross-sectional, single-visit study of adults seeking sexually transmitted infection testing at 9 clinics in 7 states. We collected pharyngeal and rectal swabs from participants. The primary outcome was positive and negative percent agreement for detection of N. gonorrhoeae and C. trachomatis for 3 investigational assays compared to a composite reference. Secondary outcomes included positivity as well as positive and negative predictive values and likelihood ratios. Subgroup analyses included outcomes by symptom status and sex.

Results

A total of 2598 participants (79% male) underwent testing. We observed N. gonorrhoeae positivity of 8.1% in the pharynx and 7.9% in the rectum and C. trachomatis positivity of 2.0% in the pharynx and 8.7% in the rectum. Positive percent agreement ranged from 84.8% to 96.5% for different anatomic site infection combinations, whereas negative percent agreement was 98.8% to 99.6%.

Conclusions

This study utilized a Master Protocol to generate diagnostic performance data for multiple assays from different manufacturers in a single study population, which ultimately supported first-in-class FDA clearance for extragenital assays. We observed very good positive percent agreement when compared to a composite reference method for the detection of both pharyngeal and rectal N. gonorrhoeae and C. trachomatis.

Clinical Trials Registration

NCT02870101.

Clinical evaluation of the APAS® Independence: Automated imaging and interpretation of urine cultures using artificial intelligence with composite reference standard discrepant resolution

BACKGROUND

This study reports the outcome of the first evaluation of the APAS® Independence for automated reading and preliminary interpretation of urine cultures in the routine clinical microbiology laboratory. In a 2-stage evaluation involving 3000 urine samples, two objectives were assessed; 1) the sensitivity and specificity of the APAS® Independence compared to microbiologists using colony enumeration as the primary determinant, and 2) the variability between microbiologists in enumerating bacterial cultures using traditional culture reading techniques, performed independently to APAS® Independence interpretation.

METHODS

Routine urine samples received into the laboratory were processed and culture plates were interpreted by standard methodology and with the APAS® Independence. Results were compared using typical discrepant result resolution and with a composite reference standard, which provided an alternative assessment of performance.

RESULTS

The significant growth sensitivity of the APAS® Independence was determined to be 0.919 with a 95% confidence interval of (0.879, 0.948), and the growth specificity was 0.877 with a 95% confidence interval of (0.827, 0.916). Variability between microbiologists was demonstrated with microbiologist bi-plate enumerations in agreement with the consensus 88.6% of the time.

CONCLUSION

The APAS® Independence appears to offer microbiology laboratories a mechanism to standardise the processing and assessment of urine cultures whilst augmenting the skills of specialist microbiology staff.

Diagnostic test accuracy of the BioFire® FilmArray® meningitis/encephalitis panel: a systematic review and meta-analysis

BACKGROUND

The FilmArray® meningitis/encephalitis (ME) panel is a multiplex PCR assay which can detect the most commonly identified pathogens in central nervous system infections. It significantly decreases the time to diagnosis of ME and data has yielded several positive outcomes. However, in part, reports of both false positive and false negative detections have resulted in concerns about adoption.

OBJECTIVES

The aim was to evaluate the ME panel in a diagnostic test accuracy review.

DATA SOURCES

The PubMed and EMBASE databases were systematically searched through May 2019.

STUDY ELIGIBILITY CRITERIA

Eligible studies were those providing sensitivity and specificity data for the ME panel compared with a reference standard. Studies providing details on false positive and false negative results of the panel as well as further investigation (adjudication) of the discordant results between the panel and comparator assays were included and assessed separately.

PARTICIPANTS

Patients with suspected ME for whom a panel was ordered were included.

METHODS

The ME panel was compared to reference standard methods for diagnosing community-acquired ME. We performed a meta-analysis and calculated the summary sensitivity and specificity of the ME panel. Moreover, we evaluated the false positive and false negative results of the panel.

RESULTS

Thirteen studies (3764 patients) were included in the review and 8 of them (3059 patients) were pooled in a meta-analysis. The summary estimates of sensitivity and specificity with 95% confidence intervals (CI) was 90% (95% CI 86-93%) and 97% (95% CI 94-99%), respectively. When we looked specifically at studies that assessed further the false positive and false negative results, false positive detections were 11.4% and 4% before and after adjudication, respectively. The highest proportion of false positive was observed for Streptococcus pneumoniae followed by Streptococcus agalactiae. False negative isolates were 2.2% and 1.5% before and after adjudication, respectively. Herpes simplex virus 1 and 2, enterovirus and Cryptococcus neoformans/gattii had the highest proportions of false negative determinations. False negative C. neoformans/gattii were mostly patients with positive antigen titres, on treatment or cleared disease.

CONCLUSIONS

The currently available literature suggests that the ME panel has high diagnostic accuracy. However, the decision for implementation should be individualized based on the needs of the patient population, the capabilities of the laboratory, and the knowledge of the healthcare providers that will utilize the test.

Alternate Nucleic Acid Targets Can Be Used To Create a Composite Standard To Evaluate Clinical Performance of Nucleic Acid Amplification Tests

Evaluating the clinical performance of a new nucleic acid amplification test (NAAT) for Mycoplasma genitalium, B. Kirkconnell, B. Weinbaum, K. Santos, T. Le Nguyen, et al. (J Clin Microbiol 57:e00264-19, 2019, https://doi.org/10.1128/JCM.00264-19) created 3 alternate NAATs that detected other unique M. genitalium gene targets. Lacking a reference standard, they used the consensus of results with those 3 NAATs as the comparator. This approach could be a new paradigm to evaluate new NAATs when there is no previously defined reference standard.

Dual composite reference standards (dCRS) in molecular diagnostic research: A new approach to reduce bias in the presence of Imperfect reference

A main challenge in molecular diagnostic research is to accurately evaluate the performance of a new nucleic acid amplification test when the reference standard is imperfect. Several approaches, such as discrepant analysis, composite reference standard (CRS) method, or latent class analysis (LCA), are commonly applied for this purpose by combining multiple imperfect (reference) test results. In discrepant analysis or LCA, test results from the new assay are often involved in the construction of a new pseudo-reference standard, which results in the potential risk of overestimating the parameters of interest. On the contrary, the CRS methods only combine the results of reference tests, which is more preferable in practice. In this article, we study the properties of two extreme CRS methods, i.e., combining multiple reference test results by the "any positive" rule or by the "all-positive" rule, and propose a new approach "dual composite reference standards (dCRS)" based on these two extreme methods to reduce the biases of the estimates. Simulations are performed for various scenarios and the proposed approach is applied to two real datasets. The results demonstrate that our approach outperforms other commonly used approaches and therefore is recommended for future applications.

Guidelines for Validation of Next-Generation Sequencing-Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists

Next-generation sequencing (NGS) methods for cancer testing have been rapidly adopted by clinical laboratories. To establish analytical validation best practice guidelines for NGS gene panel testing of somatic variants, a working group was convened by the Association of Molecular Pathology with liaison representation from the College of American Pathologists. These joint consensus recommendations address NGS test development, optimization, and validation, including recommendations on panel content selection and rationale for optimization and familiarization phase conducted before test validation; utilization of reference cell lines and reference materials for evaluation of assay performance; determining of positive percentage agreement and positive predictive value for each variant type; and requirements for minimal depth of coverage and minimum number of samples that should be used to establish test performance characteristics. The recommendations emphasize the role of laboratory director in using an error-based approach that identifies potential sources of errors that may occur throughout the analytical process and addressing these potential errors through test design, method validation, or quality controls so that no harm comes to the patient. The recommendations contained herein are intended to assist clinical laboratories with the validation and ongoing monitoring of NGS testing for detection of somatic variants and to ensure high quality of sequencing results.

The natural history of Chlamydia trachomatis infection in women: a multi-parameter evidence synthesis

BACKGROUND AND OBJECTIVES

The evidence base supporting the National Chlamydia Screening Programme, initiated in 2003, has been questioned repeatedly, with little consensus on modelling assumptions, parameter values or evidence sources to be used in cost-effectiveness analyses. The purpose of this project was to assemble all available evidence on the prevalence and incidence of Chlamydia trachomatis (CT) in the UK and its sequelae, pelvic inflammatory disease (PID), ectopic pregnancy (EP) and tubal factor infertility (TFI) to review the evidence base in its entirety, assess its consistency and, if possible, arrive at a coherent set of estimates consistent with all the evidence.

METHODS

Evidence was identified using 'high-yield' strategies. Bayesian Multi-Parameter Evidence Synthesis models were constructed for separate subparts of the clinical and population epidemiology of CT. Where possible, different types of data sources were statistically combined to derive coherent estimates. Where evidence was inconsistent, evidence sources were re-interpreted and new estimates derived on a post-hoc basis.

RESULTS

An internally coherent set of estimates was generated, consistent with a multifaceted evidence base, fertility surveys and routine UK statistics on PID and EP. Among the key findings were that the risk of PID (symptomatic or asymptomatic) following an untreated CT infection is 17.1% [95% credible interval (CrI) 6% to 29%] and the risk of salpingitis is 7.3% (95% CrI 2.2% to 14.0%). In women aged 16-24 years, screened at annual intervals, at best, 61% (95% CrI 55% to 67%) of CT-related PID and 22% (95% CrI 7% to 43%) of all PID could be directly prevented. For women aged 16-44 years, the proportions of PID, EP and TFI that are attributable to CT are estimated to be 20% (95% CrI 6% to 38%), 4.9% (95% CrI 1.2% to 12%) and 29% (95% CrI 9% to 56%), respectively. The prevalence of TFI in the UK in women at the end of their reproductive lives is 1.1%: this is consistent with all PID carrying a relatively high risk of reproductive damage, whether diagnosed or not. Every 1000 CT infections in women aged 16-44 years, on average, gives rise to approximately 171 episodes of PID and 73 of salpingitis, 2.0 EPs and 5.1 women with TFI at age 44 years.

CONCLUSIONS AND RESEARCH RECOMMENDATIONS

The study establishes a set of interpretations of the major studies and study designs, under which a coherent set of estimates can be generated. CT is a significant cause of PID and TFI. CT screening is of benefit to the individual, but detection and treatment of incident infection may be more beneficial. Women with lower abdominal pain need better advice on when to seek early medical attention to avoid risk of reproductive damage. The study provides new insights into the reproductive risks of PID and the role of CT. Further research is required on the proportions of PID, EP and TFI attributable to CT to confirm predictions made in this report, and to improve the precision of key estimates. The cost-effectiveness of screening should be re-evaluated using the findings of this report.

FUNDING

The Medical Research Council grant G0801947.

Partial verification bias and incorporation bias affected accuracy estimates of diagnostic studies for biomarkers that were part of an existing composite gold standard

OBJECTIVE

To investigate how choice of gold standard biases estimates of sensitivity and specificity in studies reassessing the diagnostic accuracy of biomarkers that are already part of a lifetime composite gold standard (CGS).

STUDY DESIGN AND SETTING

We performed a simulation study based on the real-life example of the biomarker "protein 14-3-3" used for diagnosing Creutzfeldt-Jakob disease. Three different types of gold standard were compared: perfect gold standard "autopsy" (available in a small fraction only; prone to partial verification bias), lifetime CGS (including the biomarker under investigation; prone to incorporation bias), and "best available" gold standard (autopsy if available, otherwise CGS).

RESULTS

Sensitivity was unbiased when comparing 14-3-3 with autopsy but overestimated when using CGS or "best available" gold standard. Specificity of 14-3-3 was underestimated in scenarios comparing 14-3-3 with autopsy (up to 24%). In contrast, overestimation (up to 20%) was observed for specificity compared with CGS; this could be reduced to 0-10% when using the "best available" gold standard.

CONCLUSION

Choice of gold standard affects considerably estimates of diagnostic accuracy. Using the "best available" gold standard (autopsy where available, otherwise CGS) leads to valid estimates of specificity, whereas sensitivity is estimated best when tested against autopsy alone.

Advances in sampling and screening for chlamydia

Chlamydia is the most commonly diagnosed bacterial sexually transmitted infection in the developed world, with diagnosis rates continuing to increase each year. As chlamydia is largely asymptomatic, screening and treatment is the main way to detect cases and reduce transmission. Recent advances in self-collected specimens and laboratory tests has made chlamydia screening easier to implement as well as possible in nonclinical settings. This review will discuss new approaches to specimen collection and how these have expanded opportunities for reaching target populations for chlamydia screening. Furthermore, it will discuss how advanced molecular microbiological methods can be used with self-collected specimens to further our knowledge of the epidemiology of chlamydia and the dynamics of transmission.

Investigation into low-level anti-rubella virus IgG results reported by commercial immunoassays

Since the 1980s, commercial anti-rubella virus IgG assays have been calibrated against a WHO International Standard and results have been reported in international units per milliliter (IU/ml). Laboratories testing routine patients' samples collected 100 samples that gave anti-rubella virus IgG results of 40 IU/ml or less from each of five different commercial immunoassays (CIA). The total of 500 quantitative results obtained from 100 samples from each CIA were compared with results obtained from an in-house enzyme immunoassay (IH-EIA) calibrated using the WHO standard. All 500 samples were screened using a hemagglutination inhibition assay (HAI). Any sample having an HAI titer of 1:8 or less was assigned a negative anti-rubella virus antibody status. If the HAI titer was greater than 1:8, the sample was tested in an immunoblot (IB) assay. If the IB result was negative, the sample was assigned a negative anti-rubella virus IgG status; otherwise, the sample was assigned a positive status. Concordance between the CIA qualitative results and the assigned negative status ranged from 50.0 to 93.8% and 74.5 to 97.8% for the assigned positive status. Using a receiver operating characteristic analysis with the cutoff set at 10 IU/ml, the estimated sensitivity and specificity ranged from 70.2 to 91.2% and 65.9 to 100%, respectively. There was poor correlation between the quantitative CIA results and those obtained by the IH-EIA, with the coefficient of determination (R(2)) ranging from 0.002 to 0.413. Although CIAs have been calibrated with the same international standard for more than 2 decades, the level of standardization continues to be poor. It may be time for the scientific community to reevaluate the relevance of quantification of anti-rubella virus IgG.

Accuracy and trust of self-testing for bacterial vaginosis

PURPOSE

Two point-of-care tests are available to detect bacterial vaginosis (BV), a common vaginal condition. This study aimed to (1) compare the accuracy of two self-performed BV tests with clinician-performed BV tests and with clinical diagnosis of BV; and (2) compare trust of results for self-performed BV testing with clinician-performed BV testing.

METHODS

Participants (14-22 years old) in a study assessing self-testing for Trichomonas vaginalis were also asked to perform a self-test for BV (using a pH or sialidase test). Results were compared with clinician-performed tests and with clinical diagnosis of BV (defined by modified Amsel criteria). A two-item subscale from a larger acceptability scale was used to assess trust at baseline, after testing, and after discussion of results.

RESULTS

All 131 women performed self-BV testing correctly. Agreement between self- and clinician-performed tests was good (κ: .5-.7) Compared with clinical diagnosis of BV, self-pH was 73% sensitive and 67% specific, and self-sialidase was 40% sensitive and 90% specific. Trust in self-performed BV testing was lower than trust in clinician-performed BV testing at baseline, but increased after testing and discussion of results.

CONCLUSIONS

Young women can perform self-tests for BV with reasonable accuracy, which could increase testing when pelvic examinations are not feasible. Trust in self-testing increased after experience and after discussion of test results. Although the pH test is available over the counter, young women may continue to rely on clinicians for testing.

How much tubal factor infertility is caused by Chlamydia? Estimates based on serological evidence corrected for sensitivity and specificity

OBJECTIVES

To estimate the proportion of tubal factor infertility (TFI) caused by Chlamydia trachomatis (CT), the etiologic fraction, from a retrospective study of CT antibody prevalence in TFI cases and controls, adjusted for sensitivity and specificity.

METHODS

We use published data on sensitivity and specificity to estimate the performance of assays in (a) women with a previous CT infection without sequelae and (b) women with TFI caused by CT. A model was developed and applied to antibody prevalence in TFI cases and controls from 1 published case-control study to estimate the proportion of TFI caused by CT.

RESULTS

The proportion of TFI episodes that were due to CT infection was estimated to be 45% (credible intervals: 28%, 62%). Models which assume that test sensitivity is higher in women with CT-related TFI than women with previous infection and no sequelae fit the data significantly better than models that assume the same sensitivity in all those previously infected.

CONCLUSIONS

Greater attention needs to be paid to methods for characterizing the performance of CT antibody tests. Serological studies could be given a greater role both in CT etiology and in monitoring the effects of prevention and control programmes.

Methods and recommendations for evaluating and reporting a new diagnostic test

No standardized guidelines exist for the biostatistical methods appropriate for studies evaluating diagnostic tests. Publication recommendations such as the STARD statement provide guidance for the analysis of data, but biostatistical advice is minimal and application is inconsistent. This article aims to provide a self-contained, accessible resource on the biostatistical aspects of study design and reporting for investigators. For all dichotomous diagnostic tests, estimates of sensitivity and specificity should be reported with confidence intervals. Power calculations are strongly recommended to ensure that investigators achieve desired levels of precision. In the absence of a gold standard reference test, the composite reference standard method is recommended for improving estimates of the sensitivity and specificity of the test under evaluation.

A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard

OBJECTIVE

In diagnostic accuracy studies, the reference standard may be imperfect or not available in all patients. We systematically reviewed the proposed solutions for these situations and generated methodological guidance.

STUDY DESIGN AND SETTING

Review of methodological articles.

RESULTS

We categorized the solutions into four main groups. The first group includes methods that impute or adjust for missing data on the reference standard. The second group consists of methods that correct estimates of accuracy obtained with an imperfect reference standard. In the third group a reference standard is constructed by combining multiple test results through a predefined rule, based on a consensus procedure, or through statistical modeling. In the fourth group, the diagnostic accuracy paradigm is abandoned in favor of validation studies that relate index test results to relevant clinical data, such as history, future clinical events, and response to therapy.

CONCLUSION

Most of the methods try to impute, adjust, or construct a reference standard. In situations that deviate only marginally from the classical diagnostic accuracy paradigm, these are valuable methods. In cases where an acceptable reference standard does not exist, the concept of clinical test validation may provide an alternative paradigm to evaluate a diagnostic test.

Estimating diagnostic accuracy of multiple binary tests with an imperfect reference standard

The goal in diagnostic medicine is often to estimate the diagnostic accuracy of multiple experimental tests relative to a gold standard reference. When a gold standard reference is not available, investigators commonly use an imperfect reference standard. This paper proposes methodology for estimating the diagnostic accuracy of multiple binary tests with an imperfect reference standard when information about the diagnostic accuracy of the imperfect test is available from external data sources. We propose alternative joint models for characterizing the dependence between the experimental tests and discuss the use of these models for estimating individual-test sensitivity and specificity as well as prevalence and multivariate post-test probabilities (predictive values). We show using analytical and simulation techniques that, as long as the sensitivity and specificity of the imperfect test are high, inferences on diagnostic accuracy are robust to misspecification of the joint model. The methodology is demonstrated with a study examining the diagnostic accuracy of various HIV-antibody tests for HIV.

Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis

BACKGROUND

Several studies and systematic reviews have reported results that indicate that sensitivity and specificity may vary with prevalence.

STUDY DESIGN AND SETTING

We identify and explore mechanisms that may be responsible for sensitivity and specificity varying with prevalence and illustrate them with examples from the literature.

RESULTS

Clinical and artefactual variability may be responsible for changes in prevalence and accompanying changes in sensitivity and specificity. Clinical variability refers to differences in the clinical situation that may cause sensitivity and specificity to vary with prevalence. For example, a patient population with a higher disease prevalence may include more severely diseased patients, therefore, the test performs better in this population. Artefactual variability refers to effects on prevalence and accuracy associated with study design, for example, the verification of index test results by a reference standard. Changes in prevalence influence the extent of overestimation due to imperfect reference standard classification.

CONCLUSIONS

Sensitivity and specificity may vary in different clinical populations, and prevalence is a marker for such differences. Clinicians are advised to base their decisions on studies that most closely match their own clinical situation, using prevalence to guide the detection of differences in study population or study design.

Comparison of serological and real-time PCR assays to diagnose Bordetella pertussis infection in 2007

Bacterial culture for diagnosing pertussis infection has high specificity but poor sensitivity and is slow. Highly sensitive real-time PCR assays and single-serum pertussis serology have been developed to overcome these limitations, but there are few data available on the relative sensitivities and specificities of such assays for pertussis diagnosis. Using data on 195 participants (>or=7 years old) from an epidemiological study, we assessed the sensitivity, specificity, and performance (Youden index) for pertussis diagnosis of the pertussis toxin enzyme-linked immunosorbent assay (using single and paired serology) and of real-time PCR assays (using the IS481 and ptxA-Pr targets). All available diagnostic information (clinical and laboratory) was pooled to serve as the gold standard. Single serology was the most efficient diagnostic test (Youden index, 0.57 to 0.58), with relatively high sensitivity (>64%) and high specificity (>90%), independent of the cutoff level. IS481 PCR performance was superior to that of ptxA-Pr PCR, and it was the second-most-efficient tool (Youden index, 0.30). Performing both ptxA-Pr and IS481 PCRs did not improve diagnostic performance. The greatest test efficiency (Youden index, 0.69 to 0.74) was achieved when single-serum serology was used in combination with IS481 or ptxA-Pr PCR or paired serology. Combining single serology with one PCR or paired serology increased the sensitivity with an associated limited decrease in specificity. The most specific tests for diagnosis of pertussis were single serology and ptxA-Pr PCR, and the most sensitive diagnostic tool was the combination of IS481 PCR with single serology.

Using pseudogold standards and latent-class analysis in combination to evaluate the accuracy of three diagnostic tests

We previously reported our use of latent-class models to estimate the sensitivity (Se) and specificity (Sp) for each of three tests used to monitor farmed salmon for infectious salmon anaemia virus (ISAv). Those tests were reverse-transcriptase polymerase chain reaction (RT-PCR), virus isolation (VI), and an indirect immunofluorescent-antibody test (IFAT). We used tissues from 403 salmon from four populations presumed to have different prevalence of ISAv. However, no formal evaluation of the assumptions of conditional independence and constant accuracy had been carried out. In our present study, we "adjusted" that and used two pseudogold standards (a composite reference standard and a study pseudogold), as indicative of the true health status of each fish. The assumption of constant accuracy across populations was evaluated using separate random-effects logistic-regression models for fish classified as D+ or D- (disease positive or negative, according to the pseudogold standards) with study population included in the model to determine if it affected the probability of a positive test result. Where there was evidence of variation in test accuracy across populations, the issue was further investigated using separate latent-class models with informative priors for each study population. Our results suggested that only one PCR test had an accuracy that varied across populations. The assumption of conditional independence among tests was first evaluated using log-linear models of D+ and D- fish with significant interaction between test results indicative of conditional dependence. Latent-class models which incorporated up to two pairs of between-test dependencies were also fit using Bayesian methods. The two approaches showed considerable evidence of dependence between IFAT and VI and some evidence of dependence between one PCR and IFAT. Results obtained from both maximum-likelihood models and from Bayesian analyses of models allowing for conditional dependence between two pairs of tests were consistent with those obtained with the pseudogold standards. The results suggest that pseudogold standards can help in choosing a correct dependence structure and should be used in combination with latent-class models.

Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis

Numerous evaluations of the clinical sensitivity and specificity of PCR and serologic assays for Bordetella pertussis have been hampered by the low sensitivity of culture, the gold standard test, which leads to biased accuracy estimates. The bias can be reduced by using statistical approaches such as the composite reference standard (CRS) (e.g., positive if culture or serology positive; negative otherwise) or latent class analysis (LCA), an internal reference standard based on a statistical model. We illustrated the benefits of the CRS and LCA approaches by reanalyzing data from a 1995 to 1996 study of cough illness among 212 patients. The accuracy of PCR in this study was evaluated using three reference standards: culture, CRS, and LCA. Using specimens obtained 0 to 34 days after cough onset, estimates of the sensitivity of PCR obtained using CRS (47%) and LCA (34%) were lower than the culture-based estimate (62%). The CRS and LCA approaches, which utilized more than one diagnostic marker of pertussis, likely produced more accurate reference standards than culture alone. In general, the CRS approach is simple, with a well-defined disease status. LCA requires statistical modeling but incorporates more indicators of disease than CRS. When three or more indicators of pertussis are available, these approaches should be used in evaluations of pertussis diagnostic tests.

Foot-and-mouth disease non-structural protein serology in cattle: use of a Bayesian framework to estimate diagnostic sensitivity and specificity of six ELISA tests and true prevalence in the field

The diagnostic performance of six foot-and-mouth disease (FMD) assays for detection of antibodies to the non-structural proteins (NSP) of the FMD virus (FMDV) was estimated using a Bayesian analysis on field sera from cattle of unknown infection status originating from post-FMDV outbreak situations in Israel and Zimbabwe. Estimations of the disease prevalence in both populations were also obtained. The diagnostic sensitivity estimates did not differ between both field studies, although overall Bayesian estimates were markedly higher than those previously reported based on sera from comparable experimentally infected (vaccinated) cattle populations. All NSP-based assays demonstrated a lower diagnostic specificity when applied to the Zimbabwean sera compared to both published specificities and similar Bayesian specificity estimates derived for the Israeli dataset. In Israel, the disease prevalence was estimated at 23.9% (95% credibility interval: 19.5-28.8%), whereas 65.4% (59.0-72.5%) was found in Zimbabwe. The need for reliable diagnostic test performance estimates and the benefits of Bayesian analysis in obtaining them are also addressed.

The use of imperfect diagnostic tests had an impact on prevalence estimation

OBJECTIVE

Problems arising with the estimation of sensitivity and specificity when two imperfect diagnostic tests are applied are widely discussed. Effects on the estimation of prevalence may be of importance as well. Different methods of dealing with two or more imperfect tests and unknown reference standard are contrasted with regard to their implications on prevalence estimation: discrepant analysis, composite reference standards, and latent class models.

STUDY DESIGN AND SETTING

Prospective epidemiological multicenter study to determine the prevalence of respiratory syncytial virus in children with lower respiratory tract infections. A subsample of 1,003 patients from a hospital population and from a practice population is considered. Virus isolation, polymerase chain reaction, and rapid antigen test had been applied.

RESULTS

Prevalence estimates obtained under various assumptions ranged from 0.263 to 0.386 in the hospital population and from 0.214 to 0.277 in the practice population.

CONCLUSION

Estimation procedures involving a resolver test applied to some but not all cells are at risk of introducing a serious bias in prevalence estimation as well as in the estimation of test accuracy parameters. Estimation via latent class modeling may be more useful, but care should be taken regarding the underlying assumptions.

Role of the JAK2 mutation in the diagnosis of chronic myeloproliferative disorders in splanchnic vein thrombosis

The diagnosis of an underlying chronic myeloproliferative disorder (CMPD) is often problematic in patients with primary extrahepatic portal vein obstruction (EHPVO) or Budd-Chiari syndrome (BCS); indeed, conventional clinical and hematological parameters usually yield insufficient information. To assess the diagnostic contribution of the gain-of-function mutation V617F of the JAK2 gene, 93 patients with EHPVO or BCS were investigated. JAK2 V617F was identified in 35.6% of 73 patients with EHPVO and in 40% of 20 patients with BCS. Taking the JAK2 mutation as a test with the highest positive predictive value for the diagnosis of CMPD, conventional clinical-hematological parameters had a sensitivity for CMPD lower than 48%. Bone marrow (BM) histology provided a diagnosis of CMPD in 41/74 (55.4%) patients, with a sensitivity of 93.5%. Clonality of hematopoiesis as assessed by granulocyte X-chromosome inactivation was present in 65.1% of 43 informative female patients, with a sensitivity of 86.6%. By resolving the sensitivity bias of the JAK2 mutation with the results of BM histology and clonality assay, CMPD was diagnosed in 53% of patients with EHPVO or BCS. In conclusion, CMPD is the major cause of primary EHPVO or BCS. JAK2 V617F is a very reliable and noninvasive molecular marker for CMPD and should be used as a first test for diagnosis.

Performance of the gen-probe transcription-mediated [corrected] amplification research assay compared to that of a multitarget real-time PCR for Mycoplasma genitalium detection

Mycoplasma genitalium (MG) can cause nongonococcal urethritis and is potentially associated with urethritis, endometritis, and cervicitis. Several assays have been developed to detect MG. Molecular amplification assays for organism detection can be problematic due to the potential for false-positive and false-negative results. Confirmatory testing is often required in these situations, requiring additional time and resources. Use of multigene targets could integrate both detection and verification at lower cost. Utilizing two targets, the MgPa adhesion gene and the 16S rRNA gene, a multitarget real-time (MTRT) PCR for the detection of MG was developed. Samples from patients attending sexually transmitted disease clinics were collected in duplicate. Urine samples from males (n = 286) and self-collected vaginal swabs from females (n = 321) were analyzed by MTRT PCR for MG and the Gen-Probe transcription-mediated [corrected] amplification (TMA) assay, which targets MG rRNA for detection (TMA-MG research use only). Utilizing the criteria of any two targets being positively amplified, the MTRT PCR had a sensitivity and specificity of 91.8% (101 positive samples/110 samples tested) and 99.5% (495/497), respectively, with a positive predictive value (PPV) of 98.1% (101/103) and a negative predictive value (NPV) of 98.2% (495/504). The Gen-Probe TMA-MG assay had a sensitivity, specificity, PPV, and NPV of 98.1% (108/110), 98.1% (488/497), 92.3% (108/117), and 99.5% (488/490), respectively. Comparison between the MTRT PCR and TMA-MG assay by kappa statistic analysis indicated that an overall kappa value was 0.941 (95% confidence interval, 0.907 and 0.976). Both assays demonstrated accuracy in the detection of MG from urine samples from male patients and self-collected vaginal swabs from female patients.

Comparison between the Gen-Probe transcription-mediated amplification Trichomonas vaginalis research assay and real-time PCR for Trichomonas vaginalis detection using a Roche LightCycler instrument with female self-obtained vaginal swab samples and male urine samples

This study compared two assays for Trichomonas vaginalis detection, Gen-Probe's transcription-mediated amplification (TMA) assay for Trichomonas vaginalis and BTUB FRET PCR, using self-obtained clinical samples from 611 patients. Infection status was defined as two positive results by two different tests. The initial TMA assay sensitivity was 96.7%; specificity was 97.5%. The TMA assay was comparable to BTUB FRET PCR.

Real-Time Polymerase Chain Reaction Assay for the Rapid Detection and Characterization of Chloroquine-Resistant Plasmodium falciparum Malaria in Returned Travelers

BACKGROUND

Imported drug-resistant malaria is a growing problem in industrialized countries. Rapid and accurate diagnosis is essential to prevent malaria-associated mortality in returned travelers. However, outside of a limited number of specialized centers, the microscopic diagnosis of malaria is slow, unreliable, and provides little information about drug resistance. Molecular diagnostics have the potential to overcome these limitations.

OBJECTIVE

We developed and evaluated a rapid, real-time polymerase chain reaction (PCR) assay to detect Plasmodium falciparum malaria and chloroquine (CQ)-resistance determinants in returned travelers who are febrile.

METHODS

A real-time PCR assay based on detection of the K76T mutation in PfCRT (K76T) of P. falciparum was developed on a LightCycler platform (Roche). The performance characteristics of the real-time assay were compared with those of the nested PCR-restriction fragment-length polymorphism (RFLP) and the sequence analyses of samples obtained from 200 febrile returned travelers, who included 125 infected with P. falciparum (48 of whom were infected CQ-susceptible [K76] and 77 of whom were CQ-resistant [T76] P. falciparum), 22 infected with Plasmodium vivax, 10 infected with Plasmodium ovale, 3 infected with Plasmodium malariae malaria, and 40 infected with other febrile syndromes. All patient samples were coded, and all analyses were performed blindly.

RESULTS

The real-time PCR assay detected multiple pfcrt haplotypes associated with CQ resistance in geographically diverse malaria isolates acquired by travelers. Compared with nested-PCR RFLP (the reference standard), the real-time assay was 100% sensitive and 96.2% specific for detection of the P. falciparum K76T mutation.

CONCLUSION

This assay is rapid, sensitive, and specific for the detection and characterization of CQ-resistant P. falciparum malaria in returned travelers. This assay is automated, standardized, and suitable for routine use in clinical diagnostic laboratories.

Design Issues in Studies Evaluating Diagnostic Tests for Aspergillosis

Although invasive aspergillosis is recognized as an increased concern for immunosuppressed patients, establishing an early diagnosis remains a challenge. Current methods for diagnosis, including sensitive radiography and invasive procedures to identify organisms in affected tissues, provide an early diagnosis for only a small proportion of patients infected with Aspergillus species. The availability of a standardized assay to detect circulating Aspergillus galactomannan, the BioRad Platelia Aspergillus galactomannan enzyme immunoassay, should be considered as a valuable tool. However, at present, clinical utility is hampered by controversies surrounding the performance of the test and an apparent lack of definitive data. These controversies emphasize the complexity inherent in analyzing diagnostic tests for aspergillosis. The widespread availability of a standardized assay may now allow us to address the multiple sources of variability and bias via large, cooperative studies.

Evaluation of Nucleic Acid Amplification Tests in the Absence of a Perfect Gold-Standard Test

During the past 10 years, medical diagnostic testing for sexually transmitted infections (STIs) has changed markedly as a result of the rapid expansion and marketing of nucleic acid amplification tests (NAATs). Among such new DNA/RNA-amplification techniques are the polymerase chain reaction (PCR), the ligase chain reaction (LCR), and the transcription-mediated amplification (TMA) tests. Regrettably, the test evaluation process undergone by these tests has not always been rigorous or scientifically sound. Here, we review the controversy surrounding the statistical evaluation of these NAATs. We also review some of the traditional and recent statistical methods developed to estimate test sensitivity and specificity parameters in the absence of reliable gold-standard tests. In particular, we review the traditional latent class modeling approach that requires the assumption of independence between diagnostic tests conditional on the true disease status, and the more recent procedures that relax the conditional independence assumption. Finally, we apply some of these statistical modeling techniques to real data to estimate the sensitivity and specificity of a NAAT for Chlamydia trachomatis. On the basis of the latent class modeling approach with a pessimistic prior for culture sensitivity, the NAAT specificity estimate was 97.6% and, on the basis of an optimistic prior, the specificity was 95.3%. Similarly, the sensitivity estimates ranged from 88.1% to 89.6%.

Enhanced enzyme immunoassay with negative-gray-zone testing compared to a single nucleic Acid amplification technique for community-based chlamydial screening of men

We evaluated a low-cost diagnostic strategy for detecting Chlamydia trachomatis in a low-prevalence population. We used an amplified enzyme immunoassay (EIA) with a reduced-cutoff "negative gray zone" to identify reactive specimens for confirmation by a nucleic acid amplification test. As part of the Chlamydia Screening Studies project, men provided a first-pass urine specimen, which they returned by post for testing. We tested 1,003 specimens by IDEIA PCE EIA (Dako) and Cobas PCR (Roche). There were 32 (3.2%) true positive specimens according to a combined standard using an algorithm requiring concordant results from at least two independent tests. All of these were positive by Cobas PCR and 24 were confirmed to be positive by PCE EIA, including 2 that gave results in the negative gray zone. There were 971 true negative specimens, 2 of which were positive by Cobas PCR and 19 of which were initially inhibitory for PCR. The relative sensitivity, specificity, positive predictive value, and negative predictive value of PCE EIA with PCR confirmation were 75.0% (95% confidence interval [CI], 56.6 to 88.5%), 100% (95% CI, 99.7 to 100%), 100% (95% CI, 88.3 to 100%), and 99.2% (95% CI, 98.4 to 99.6%), respectively. The corresponding values for Cobas PCR were 100% (95% CI, 89.1 to 100%), 99.8% (95% CI, 99.3 to 100%), 94.1% (95% CI, 76.9 to 98.2%), and 100% (95% CI, 99.6 to 100%), respectively, with 1.9% (19/1003) of the samples being initially indeterminate. When the prevalence of C. trachomatis is low, the use of an amplified EIA on urine specimens, with confirmation of results in the negative gray zone by use of a nucleic acid amplification technique, is not suitable for screening asymptomatic men. In addition, positive nucleic acid amplification test results should be confirmed and an inhibition control should be used.

Use of the Roche LightCycler instrument in a real-time PCR for Trichomonas vaginalis in urine samples from females and males

Trichomonas vaginalis is the agent of a highly prevalent sexually transmitted infection (STI) that can result in vaginitis, urethritis, and preterm birth. Traditional methods of diagnosis, including wet preparation, can be unreliable. In this study, we describe the adaptation of an existing PCR method for specific detection of T. vaginalis DNA into a rapid real-time PCR assay based on fluorescence resonance energy transfer (FRET) probe chemistry. The FRET-based assay described demonstrated high sensitivity with a detection limit of 1.06 organisms, as well as a high specificity. A total of 253 urine samples collected prospectively from both men and women were tested for T. vaginalis DNA with both the FRET-based assay and a previously validated PCR assay. When the validated PCR assay was used as the "gold standard" and after discrepancies had been resolved, our FRET-based assay demonstrated an analytical sensitivity and specificity of 90.1 and 100%, respectively. Overall results suggest that FRET-based assays can provide rapid, accurate, and high-throughput detection of T. vaginalis and may prove useful in clinical settings and for large-scale screening programs.

Diagnosis of acute pulmonary embolism in outpatients: comparison of thin-collimation multi-detector row spiral CT and planar ventilation-perfusion scintigraphy

PURPOSE

To compare multi-detector row computed tomography (CT) and ventilation-perfusion (V-P) scintigraphy in the diagnosis of acute pulmonary embolism (PE) in outpatients who were cared for in the emergency department.

MATERIALS AND METHODS

Ninety-four nonconsecutive patients, in whom acute PE was suspected, underwent thin-collimation multi-detector row CT (collimation, 4 x 1 mm; pitch, 1.25; scanning time, 0.5 second) and V-P scintigraphy. Concordance between CT and scintigraphic images was used in the diagnosis of PE. Pulmonary angiography was performed within 24 hours if interpretations of V-P and spiral CT images were inconclusive or discordant. Sensitivity and specificity values were calculated for V-P scintigrams and CT scans of the lungs. The rates of conclusive results for scintigraphy and CT were compared.

RESULTS

The sensitivity of thin-collimation multi-detector row CT and V-P scintigraphy for the detection of PE was 96% (27 of 28; CI: 82%, 99%) and 98% (65 of 66; CI: 92%, 99%), respectively. The specificity of CT and V-P scintigraphy was 86% (24 of 28; CI: 67%, 96%) and 88% (58 of 66; CI: 77%, 94%), respectively. Seven V-P scintigrams were of intermediate probability, and one spiral CT study was indeterminate. Examinations with spiral CT yielded conclusive results more often than examinations with planar V-P scintigraphy (P <.05). Five V-P scintigrams and spiral CT scans were discordant. Twelve pulmonary angiographic examinations were performed. Angiographic findings were concordant in 10 (91%) of 11 patients with conclusive CT scans in whom pulmonary angiography was attempted. CT was used to establish an alternative diagnosis in 19 (29%) of 66 patients in whom PE was excluded.

CONCLUSION

Thin-collimation multi-detector row CT is more accurate than V-P scintigraphy in the diagnosis of acute PE in outpatients. Furthermore, CT provides alternative diagnoses for patients without PE on high-quality transverse or near-isotropic reformatted images.

Rescreening in cervical cytology for quality control. When bad data is worse than no data or what works, what doesn't, and why

The currently mandated methods to measure the sensitivity of Papanicolaou smear interpretation (including the 5-year look back and review of 10% of negative smears) are misleading. They do not allow one to measure the true sensitivity of the test and allow only a small fraction of errors to be detected and corrected. Rapid prescreening and automated screening are the only methods that seem practically feasible, and will allow the sensitivity of the method to be measured on a routine basis, and, thereby allow a reduction in overall errors. Although emerging technologies, such as HPV testing, may allow alternative methods of measuring sensitivity, the most appropriate way to use and interpret these data in this context are not yet fully developed. Unfortunately, at present there seems to be little desire to change the way things are done, and, therefore, to accurately measure sensitivity in cervical cytology. The first task that needs to be undertaken in fixing a problem is to admit that one exists. At present, most laboratory directors believe that their own laboratory is performing satisfactorily. They may well be, although the laboratories lack an analytic method to demonstrate this, and, therefore, the need for better QA methods does not seem to be acute. There is some educational value to the currently practiced and mandated performance measures, the 5-year look back and review of 10% of negative smears. Most laboratory directors seem to be happy with their QA methods and are not concerned that the data that are derived from the 10% review of negative slides does not reflect their actual sensitivity of screening in any meaningful way. Unfortunately, the forces that are currently in place in the United States ensure that accurate measures of the sensitivity of cervical cytology interpretation are unlikely to be implemented beyond the level of individual experimentation. As long as the expectation of cytologists is that the error rates are significantly less than they actually are; as long as there is significant legal and financial risk to actually measuring the true sensitivity; and as long as fictitious measures of performance are not only advocated but mandated; the confluence of incentives will ensure that the true sensitivity of the test will never be measured on a routine basis. Despite all of this, it is possible that cervical cytology screening may, in fact, already be performing at an optimal level. Being able to measure this operating performance may not effect any improvement to the overall process. The ultimate arbiter in this debate will always be the demonstration of a reduction of cervical cancer morbidity and mortality with any new measure implemented. At the present time, there is only one solution to the quality control issue; a force from outside the system must change the balance of the aforementioned incentives. The promise of data from European experiences with rapid rescreening may show that this method is effective and accurate. Such data might make the current methods that are in use in the United States more open to change. So yes, the answer is that "bad" data may be worse than no data at all. The bad data that we have been collecting for more than a decade is as effective a trap as anyone could have devised to ensure that actually measuring the performance of cervical smear interpretation does not happen. The only question that remains is, "How we will be able to escape?"

Validation of a urine-based PCR-enzyme-linked immunosorbent assay for use in clinical research settings to detect Trichomonas vaginalis in men

Trichomonas vaginalis infection is highly prevalent worldwide and is associated with urethritis, prostatitis, and urethral strictures in men. However, the natural history and importance of T. vaginalis in men are poorly understood, in part because of difficulties in diagnosing infection. Traditional detection methods rely on culture and wet-mount microscopy, which can be insensitive and time consuming. Urethral swabs are commonly used to detect T. vaginalis in men, but discomfort from specimen collection is a barrier to large studies. One thousand two hundred twenty-five Malawian men attending sexually transmitted disease and dermatology clinics were enrolled in this cross-sectional study to validate detection by urine-based PCR-enzyme-linked immunosorbent assay (ELISA) with urine and urethral swab culture as the reference standard. This assay for detection of amplified T. vaginalis DNA in first-catch urine (< or = 30 ml) performed with a sensitivity of 92.7%, a specificity of 88.6%, and an adjusted specificity of 95.2% compared to culture of urethral swabs or urine sediment. For clinical research settings in which urethral swabs are not available and culture is not feasible, the urine-based PCR-ELISA may be useful for detection of trichomoniasis in men.

Evaluating qualitative assays using sensitivity and specificity

Sensitivity and specificity are two important indices of performance of qualitative assays. Evaluating these indices usually requires one to identify the true disease state of each subject involved in a study. This implies that a perfect test, a "gold standard," is needed to test each subject. However, a gold standard test cannot always be performed on all subjects, whether because of cost or adverse effect on a subject's welfare. In these situations, a common practice is to apply both a currently used assay and an investigational assay to the same specimen. If the testing results are discordant, a gold standard test is applied. This approach has been criticized by many and, in fact, the statistics based on this approach usually overestimate sensitivity and specificity. This paper proposes two alternative methods to estimate sensitivity and specificity. Simulation results show that these methods perform better than the commonly used existing ones. This paper proposes new acceptance criteria and designs to specific topics for the evaluation of blood related assays as well. To evaluate a qualitative assay related to blood specimens, one must also perform studies of storage conditions, interfering substances, and other related factors, in order to establish the equivalency of the assay under standard and various other conditions. To conduct these studies, true negative blood donor specimens are used as a sample from a nondiseased population; and blood donor specimens with spiked analyte are used to represent a sample from a diseased population. Currently, the target-spiking ranges and sample sizes are determined subjectively. This paper presents new acceptance criteria on acceptable conditions and objective standards for selecting the target-spiking range and sample size.

Head-to-head multicenter comparison of DNA probe and nucleic acid amplification tests for Chlamydia trachomatis infection in women performed with an improved reference standard

Few evaluations of tests for Chlamydia trachomatis have compared nucleic acid amplification tests (NAATs) with diagnostic tests other than those by culture. In a five-city study of 3,551 women, we compared the results of commercial ligase chain reaction (LCR) and PCR tests performed on cervical swabs and urine with the results of PACE 2 tests performed on cervical swabs, using independent reference standards that included both cervical swabs and urethral swab-urine specimens. Using cervical culture as a standard, the sensitivities of PACE 2, LCR, and PCR tests with cervical specimens were 78.1, 96.9, and 89.9%, respectively, and the specificities were 99.3, 97.5, and 98.2%, respectively. Using either cervical swab or urine LCR-positive tests as the standard decreased sensitivities to 60.8% for PACE 2 and to 75.8 and 74.9% for PCR with cervical swabs and urine, respectively. Specificities increased to 99.7% for PACE 2 and to 99.7 and 99.4% for PCR with cervical swabs and urine, respectively. Sensitivities with a cervical swab-urine PCR standard were 61.9% for PACE 2 and 85.5 and 80.8% for LCR with cervical swabs and urine, respectively. Specificities were 99.6% for PACE 2 and 99.0 and 98.9% for LCR with cervical swabs and urine, respectively. Cervical swab versus urine differences were significant only for PCR specificities (P = 0.034). Overall, LCR sensitivity exceeded that of PCR, and sensitivities obtained with cervical swabs exceeded those obtained with urine specimens by small amounts. These data have substantiated, using a large multicenter sample and a patient standard, that LCR and PCR tests performed on endocervical swabs and urine are superior to PACE 2 tests for screening C. trachomatis infections in women. In our study, NAATs improved the detection of infected women by 17 to 38% compared to PACE 2.

Measuring sensitivity in gynecologic cytology: a review

BACKGROUND

The sensitivity of gynecologic cytology has been measured in several different ways. The current review summarizes the major sources of bias and the results of these efforts.

METHODS

In the current study, a review of the literature was conducted.

RESULTS

The major sources of bias in measuring the sensitivity of gynecologic cytology are a lack of reproducibility, bias in the review process, bias in case selection, selection and correction of the gold standard, and the value of surrogate markers. Despite these sources of variation, the sensitivity of the Papanicolaou (Pap) smear is relatively stable, ranging from 50-75% if a single consistent threshold is used, to up to 94% if either Autopap-directed rescreening or thin-layer methods are used to diagnose high-grade squamous intraepithelial lesions using a threshold of atypical squamous cells of undetermined significance. Methods for the routine evaluation of sensitivity currently are not available and may be difficult to devise.

CONCLUSIONS

The sensitivity of the Pap smear in study situations is well known. Whether these reflect performance in real life is not known, and methods to compare performance in real life are not available.

Appropriate use of clinical microbiology tests

Laboratory medicine lacks the tools necessary to define appropriate test use; nonetheless, existing laboratory test characteristics, although inadequate, provide a common starting point for developing definitions of appropriate test use. As Lundberg [1], Szczepura [15], and van Walraven and Naylor [2] have emphasized, developing a process for defining appropriate laboratory use should receive a high priority. There is a particular need to develop methods for standardizing studies [59]. Laboratory medicine does not, however, lack the tools necessary to change test use. Although past interventions were largely ineffective, there is growing evidence that test use can be changed by use of a variety of approaches. By using the existing tools there is much that can be done to change inappropriate test use, such as minimizing redundant testing or the use of tests that have no clinical relevance. The real opportunities will come when there are scientifically and medically sound definitions of appropriate test use that can be used to change test use and improve patient care in a cost-effective way.

Liquid-based cytology: is this the way forward for cervical screening?

Liquid-based cytology (LBC) is currently being marketed as an alternative methodology to replace the conventional PAP smear in cervical cytology. A substantial body of literature exists in support of LBC, some of which is at least partially sponsored by product manufacturers. The majority of published literature in support of LBC employs Bethesda reporting terminology. In this study we have analysed published raw data and presented this in NHSCSP terminology. Claims relating to sensitivity, specificity and smear adequacy have then been considered with reference to this data. Our analysis of existing data does not support the nationwide implementation of LBC at present. Further studies are recommended in order to evaluate the place of this technology within the NHSCSP.