Taking Costs and Diagnostic Test Accuracy into Account When Designing Prevalence Studies: An Application to Childhood Tuberculosis Prevalence

BACKGROUND

When planning a study to estimate disease prevalence to a pre-specified precision, it is of interest to minimize total testing cost. This is particularly challenging in the absence of a perfect reference test for the disease because different combinations of imperfect tests need to be considered. We illustrate the problem and a solution by designing a study to estimate the prevalence of childhood tuberculosis in a hospital setting.

METHODS

All possible combinations of 3 commonly used tuberculosis tests, including chest X-ray, tuberculin skin test, and a sputum-based test, either culture or Xpert, are considered. For each of the 11 possible test combinations, 3 Bayesian sample size criteria, including average coverage criterion, average length criterion and modified worst outcome criterion, are used to determine the required sample size and total testing cost, taking into consideration prior knowledge about the accuracy of the tests.

RESULTS

In some cases, the required sample sizes and total testing costs were both reduced when more tests were used, whereas, in other examples, lower costs are achieved with fewer tests.

CONCLUSION

Total testing cost should be formally considered when designing a prevalence study.

Evaluating the accuracy and economic value of a new test in the absence of a perfect reference test

BACKGROUND

Streptococcus pneumoniae (SP) pneumonia is often treated empirically as diagnosis is challenging because of the lack of a perfect test. Using BinaxNOW-SP, a urinary antigen test, as an add-on to standard cultures may not only increase diagnostic yield but also increase costs.

OBJECTIVE

To estimate the sensitivity and specificity of BinaxNOW-SP and subsequently estimate the cost-effectiveness of adding BinaxNOW-SP to the diagnostic work-up.

DESIGN

We fit a Bayesian latent-class meta-analysis model to obtain estimates of BinaxNOW-SP accuracy that adjust for the imperfect accuracy of culture. Meta-analysis results were combined with information on prevalence of SP pneumonia to estimate the number of patients who are correctly classified under competing diagnostic strategies. Taking into consideration the cost of antibiotics, we determined the incremental cost of adding BinaxNOW-SP to the work-up per case correctly diagnosed.

RESULTS

The BinaxNOW-SP test had a pooled sensitivity of 0.74 (95% credible interval [CrI], 0.67-0.83) and a pooled specificity of 0.96 (95% CrI, 0.92-0.99). An overall increase in diagnostic accuracy of 6.2% due to the addition of BinaxNOW-SP corresponded to an incremental cost per case correctly classified of $582 Canadian dollars.

CONCLUSIONS

The methods we have described allow us to evaluate the accuracy and economic value of a new test in the absence of a perfect reference test using an evidence-based approach.

Risk of acute myocardial infarction with NSAIDs in real world use: bayesian meta-analysis of individual patient data

Objective To characterise the determinants, time course, and risks of acute myocardial infarction associated with use of oral non-steroidal anti-inflammatory drugs (NSAIDs).Design Systematic review followed by a one stage bayesian individual patient data meta-analysis.Data sources Studies from Canadian and European healthcare databases.Review methods Eligible studies were sourced from computerised drug prescription or medical databases, conducted in the general or an elderly population, documented acute myocardial infarction as specific outcome, studied selective cyclo-oxygenase-2 inhibitors (including rofecoxib) and traditional NSAIDs, compared risk of acute myocardial infarction in NSAID users with non-users, allowed for time dependent analyses, and minimised effects of confounding and misclassification bias. Exposure and outcomes Drug exposure was modelled as an indicator variable incorporating the specific NSAID, its recency, duration of use, and dose. The outcome measures were the summary adjusted odds ratios of first acute myocardial infarction after study entry for each category of NSAID use at index date (date of acute myocardial infarction for cases, matched date for controls) versus non-use in the preceding year and the posterior probability of acute myocardial infarction.Results A cohort of 446 763 individuals including 61 460 with acute myocardial infarction was acquired. Taking any dose of NSAIDs for one week, one month, or more than a month was associated with an increased risk of myocardial infarction. With use for one to seven days the probability of increased myocardial infarction risk (posterior probability of odds ratio >1.0) was 92% for celecoxib, 97% for ibuprofen, and 99% for diclofenac, naproxen, and rofecoxib. The corresponding odds ratios (95% credible intervals) were 1.24 (0.91 to 1.82) for celecoxib, 1.48 (1.00 to 2.26) for ibuprofen, 1.50 (1.06 to 2.04) for diclofenac, 1.53 (1.07 to 2.33) for naproxen, and 1.58 (1.07 to 2.17) for rofecoxib. Greater risk of myocardial infarction was documented for higher dose of NSAIDs. With use for longer than one month, risks did not appear to exceed those associated with shorter durations.Conclusions All NSAIDs, including naproxen, were found to be associated with an increased risk of acute myocardial infarction. Risk of myocardial infarction with celecoxib was comparable to that of traditional NSAIDS and was lower than for rofecoxib. Risk was greatest during the first month of NSAID use and with higher doses.

STARD-BLCM: Standards for the Reporting of Diagnostic accuracy studies that use Bayesian Latent Class Models

The Standards for the Reporting of Diagnostic Accuracy (STARD) statement, which was recently updated to the STARD2015 statement, was developed to encourage complete and transparent reporting of test accuracy studies. Although STARD principles apply broadly, the checklist is limited to studies designed to evaluate the accuracy of tests when the disease status is determined from a perfect reference procedure or an imperfect one with known measures of test accuracy. However, a reference standard does not always exist, especially in the case of infectious diseases with a long latent period. In such cases, a valid alternative to classical test evaluation involves the use of latent class models that do not require a priori knowledge of disease status. Latent class models have been successfully implemented in a Bayesian framework for over 20 years. The objective of this work was to identify the STARD items that require modification and develop a modified version of STARD for studies that use Bayesian latent class analysis to estimate diagnostic test accuracy in the absence of a reference standard. Examples and elaborations for each of the modified items are provided. The new guidelines, termed STARD-BLCM (Standards for Reporting of Diagnostic accuracy studies that use Bayesian Latent Class Models), will facilitate improved quality of reporting on the design, conduct and results of diagnostic accuracy studies that use Bayesian latent class models.

Evaluating the impact of the multiplex respiratory virus panel polymerase chain reaction test on the clinical management of suspected respiratory viral infections in adult patients in a hospital setting

A retrospective cohort design was used to study the impact of a multiplex respiratory virus panel polymerase chain reaction test in 186 adult patients with suspected influenza-like illness. Decisions regarding continuation of empirical antiviral therapy appear to be impacted by the test. However, the impact on reducing antibiotic use remains unclear.

Diagnostic Test Accuracy in Childhood Pulmonary Tuberculosis: A Bayesian Latent Class Analysis

Evaluation of tests for the diagnosis of childhood pulmonary tuberculosis (CPTB) is complicated by the absence of an accurate reference test. We present a Bayesian latent class analysis in which we evaluated the accuracy of 5 diagnostic tests for CPTB. We used data from a study of 749 hospitalized South African children suspected to have CPTB from 2009 to 2014. The following tests were used: mycobacterial culture, smear microscopy, Xpert MTB/RIF (Cepheid Inc.), tuberculin skin test (TST), and chest radiography. We estimated the prevalence of CPTB to be 27% (95% credible interval (CrI): 21, 35). The sensitivities of culture, Xpert, and smear microscopy were estimated to be 60% (95% CrI: 46, 76), 49% (95% CrI: 38, 62), and 22% (95% CrI: 16, 30), respectively; specificities of these tests were estimated in accordance with prior information and were close to 100%. Chest radiography was estimated to have a sensitivity of 64% (95% CrI: 55, 73) and a specificity of 78% (95% CrI: 73, 83). Sensitivity of the TST was estimated to be 75% (95% CrI: 61, 84), and it decreased substantially among children who were malnourished and infected with human immunodeficiency virus (56%). The specificity of the TST was 69% (95% CrI: 63%, 76%). Furthermore, it was estimated that 46% (95% CrI: 42, 49) of CPTB-negative cases and 93% (95% CrI: 82; 98) of CPTB-positive cases received antituberculosis treatment, which indicates substantial overtreatment and limited undertreatment.

Understanding the effects of conditional dependence in research studies involving imperfect diagnostic tests

When two imperfect diagnostic tests are carried out on the same subject, their results may be correlated even after conditioning on the true disease status. While past work has focused on the consequences of ignoring conditional dependence, the degree to which conditional dependence can be induced has not been systematically studied. We examine this issue in detail by introducing a hypothetical missing covariate that affects the sensitivities of two imperfect dichotomous tests. We consider four forms for this covariate, normal, uniform, dichotomous and trichotomous. In the case of a dichotomous covariate, we derive an expression showing that the conditional covariance is a function of the product of the changes in test sensitivities (or specificities) between the subgroups defined by the covariate. The maximum possible covariance is induced by a dichotomous covariate with a very strong effect on both tests. Through simulations, we evaluate the extent to which fitting a latent class model ignoring each type of covariate but including a general covariance term can adjust for the correlation induced by the covariate. We compare the results to when the conditional dependence is ignored. We find that the bias because of ignoring conditional dependence is generally small even for moderate covariate effects, and when bias is present, a model including a covariance term works well. We illustrate our methods by analyzing data from a childhood tuberculosis study. Copyright © 2016 John Wiley & Sons, Ltd.

Lactobacillus probiotics in the prevention of diarrhea associated with Clostridium difficile: a systematic review and Bayesian hierarchical meta-analysis

BACKGROUND

Recent meta-analyses of the efficacy of probiotics for preventing diarrhea associated with Clostridium difficile have concluded there is a large effect favouring probiotics. We reexamined this evidence, which contradicts the results of a more recent large randomized controlled trial that found no benefit of Lactobacillus probiotics for preventing C. difficile-associated diarrhea.

METHODS

We performed a systematic review of the efficacy of treatment with Lactobacillus probiotics for preventing nosocomial C. difficile-associated diarrhea in adults and carried out a meta-analysis using a Bayesian hierarchical model. We used credibility analysis and meta-regression to characterize the heterogeneity between studies.

RESULTS

Ten studies met our inclusion criteria. The pooled risk ratio was highly statistically significant, at 0.25 (95% credible interval 0.08-0.47). However, the 95% prediction interval for the risk ratio in a future study, 0.02-1.34, was wider than the credible interval, owing to heterogeneity between studies. Furthermore, a credibility analysis showed that the strength of the evidence was weaker than the observed number of cases of C. difficile-associated diarrhea across studies would suggest. Meta-regression suggested that the beneficial effect of probiotics was more likely to be reported in studies with an increased risk of C. difficile-associated diarrhea in the control group, although this association was not statistically significant.

INTERPRETATION

Accounting for between-study heterogeneity showed that there is considerable uncertainty regarding the apparently large efficacy estimate associated with Lactobacillus probiotic treatment in preventing C. difficile-associated diarrhea. Most studies to date have been carried out in populations with a low risk of C. difficile-associated diarrhea, such that the evidence is inconclusive and inadequate to support a policy concerning routine use of probiotics in to prevent this condition.

Systematic review of the accuracy of antibody tests used to screen asymptomatic adults for hepatitis C infection

BACKGROUND

Several expert groups, including the United States Preventive Services Task Force and the Canadian Task Force on Preventive Health Care, have recently examined or are currently examining whether primary care physicians should screen asymptomatic adults for hepatitis C virus (HCV) infection. To inform decision-making on HCV screening, we performed a systematic review of the accuracy of antibody tests compared with other immunoassays and RNA detection for screening asymptomatic adults for HCV infection in Canada.

METHODS

MEDLINE and Embase databases were searched from 1990 to 2016; resulting citations were uploaded into DistillerSR and independently screened by 2 reviewers. Original research studies, systematic reviews and meta-analyses were eligible for inclusion. At least 80% of the study population had to be asymptomatic, nonpregnant, treatment-naïve adults with unknown liver enzyme values and unknown HCV status. Risk of bias was assessed with the use of the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2) tool; the quality of the body of evidence was assessed by means of GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology.

RESULTS

Of 1537 articles identified, 81 underwent full-text review, and 9 studies met the inclusion criteria. Compared with RNA detection, the sensitivity of the third-generation enzyme-linked immunosorbent assay was variable (61.0%-81.8%), and its specificity was high (97.5%-99.7%). As expected, there were more false-positive results when comparing antibody tests to RNA detection than to other immunoassays. Our GRADE assessment suggested that there was a high concern for risk of bias, particularly verification bias, and substantial inconsistency between studies in terms of their design.

INTERPRETATION

More research is needed to better characterize the accuracy of antibody tests used to screen for HCV infection in the general population. Jurisdictions that recently adopted birth cohort screening for HCV infection are encouraged to evaluate and report on the accuracy of HCV screening tests and screening benefits and harms. PROSPERO registration: no. CRD42016039710.

Lateral flow urine lipoarabinomannan assay for detecting active tuberculosis in HIV-positive adults

BACKGROUND

Rapid detection of tuberculosis (TB) among people living with human immunodeficiency virus (HIV) is a global health priority. HIV-associated TB may have different clinical presentations and is challenging to diagnose. Conventional sputum tests have reduced sensitivity in HIV-positive individuals, who have higher rates of extrapulmonary TB compared with HIV-negative individuals. The lateral flow urine lipoarabinomannan assay (LF-LAM) is a new, commercially available point-of-care test that detects lipoarabinomannan (LAM), a lipopolysaccharide present in mycobacterial cell walls, in people with active TB disease.

OBJECTIVES

To assess the accuracy of LF-LAM for the diagnosis of active TB disease in HIV-positive adults who have signs and symptoms suggestive of TB (TB diagnosis).To assess the accuracy of LF-LAM as a screening test for active TB disease in HIV-positive adults irrespective of signs and symptoms suggestive of TB (TB screening).

SEARCH METHODS

We searched the following databases without language restriction on 5 February 2015: the Cochrane Infectious Diseases Group Specialized Register; MEDLINE (PubMed,1966); EMBASE (OVID, from 1980); Science Citation Index Expanded (SCI-EXPANDED, from 1900), Conference Proceedings Citation Index-Science (CPCI-S, from 1900), and BIOSIS Previews (from 1926) (all three using the Web of Science platform; MEDION; LILACS (BIREME, from 1982); SCOPUS (from 1995); the metaRegister of Controlled Trials (mRCT); the search portal of the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP); and ProQuest Dissertations & Theses A&l (from 1861).

SELECTION CRITERIA

Eligible study types included randomized controlled trials, cross-sectional studies, and cohort studies that determined LF-LAM accuracy for TB against a microbiological reference standard (culture or nucleic acid amplification test from any body site). A higher quality reference standard was one in which two or more specimen types were evaluated for TB, and a lower quality reference standard was one in which only one specimen type was evaluated for TB. Participants were HIV-positive people aged 15 years and older.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data from each included study using a standardized form. We appraised the quality of studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. We evaluated the test at two different cut-offs: (grade 1 or 2, based on the reference card scale of five intensity bands). Most analyses used grade 2, the manufacturer's currently recommended cut-off for positivity. We carried out meta-analyses to estimate pooled sensitivity and specificity using a bivariate random-effects model and estimated the models using a Bayesian approach. We determined accuracy of LF-LAM combined with sputum microscopy or Xpert® MTB/RIF. In addition, we explored the influence of CD4 count on the accuracy estimates. We assessed the quality of the evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

MAIN RESULTS

We included 12 studies: six studies evaluated LF-LAM for TB diagnosis and six studies evaluated the test for TB screening. All studies were cross-sectional or cohort studies. Studies for TB diagnosis were largely conducted among inpatients (median CD4 range 71 to 210 cells per µL) and studies for TB screening were largely conducted among outpatients (median CD4 range 127 to 437 cells per µL). All studies were conducted in low- or middle-income countries. Only two studies for TB diagnosis (33%) and one study for TB screening (17%) used a higher quality reference standard.LF-LAM for TB diagnosis (grade 2 cut-off): meta-analyses showed median pooled sensitivity and specificity (95% credible interval (CrI)) of 45% (29% to 63%) and 92% (80% to 97%), (five studies, 2313 participants, 35% with TB, low quality evidence). The pooled sensitivity of a combination of LF-LAM and sputum microscopy (either test positive) was 59% (47% to 70%), which represented a 19% (4% to 36%) increase over sputum microscopy alone, while the pooled specificity was 92% (73% to 97%), which represented a 6% (1% to 24%) decrease from sputum microscopy alone (four studies, 1876 participants, 38% with TB). The pooled sensitivity of a combination of LF-LAM and sputum Xpert® MTB/RIF (either test positive) was 75% (61% to 87%) and represented a 13% (1% to 37%) increase over Xpert® MTB/RIF alone. The pooled specificity was 93% (81% to 97%) and represented a 4% (1% to 16%) decrease from Xpert® MTB/RIF alone (three studies, 909 participants, 36% with TB). Pooled sensitivity and specificity of LF-LAM were 56% (41% to 70%) and 90% (81% to 95%) in participants with a CD4 count of less than or equal to 100 cells per µL (five studies, 859 participants, 47% with TB) versus 26% (16% to 46%) and 92% (78% to 97%) in participants with a CD4 count greater than 100 cells per µL (five studies, 1410 participants, 30% with TB).LF-LAM for TB screening (grade 2 cut-off): for individual studies, sensitivity estimates (95% CrI) were 44% (30% to 58%), 28% (16% to 42%), and 0% (0% to 71%) and corresponding specificity estimates were 95% (92% to 97%), 94% (90% to 97%), and 95% (92% to 97%) (three studies, 1055 participants, 11% with TB, very low quality evidence). There were limited data for additional analyses.The main limitations of the review were the use of a lower quality reference standard in most included studies, and the small number of studies and participants included in the analyses. The results should, therefore, be interpreted with caution.

AUTHORS' CONCLUSIONS

We found that LF-LAM has low sensitivity to detect TB in adults living with HIV whether the test is used for diagnosis or screening. For TB diagnosis, the combination of LF-LAM with sputum microscopy suggests an increase in sensitivity for TB compared to either test alone, but with a decrease in specificity. In HIV-positive individuals with low CD4 counts who are seriously ill, LF-LAM may help with the diagnosis of TB.

Validation of Diagnostic Groups Based on Health Care Utilization Data Should Adjust for Sampling Strategy

Supplemental Digital Content is available in the text.

Objective:

Valid measurement of outcomes such as disease prevalence using health care utilization data is fundamental to the implementation of a “learning health system.” Definitions of such outcomes can be complex, based on multiple diagnostic codes. The literature on validating such data demonstrates a lack of awareness of the need for a stratified sampling design and corresponding statistical methods. We propose a method for validating the measurement of diagnostic groups that have: (1) different prevalences of diagnostic codes within the group; and (2) low prevalence.

Methods:

We describe an estimation method whereby: (1) low-prevalence diagnostic codes are oversampled, and the positive predictive value (PPV) of the diagnostic group is estimated as a weighted average of the PPV of each diagnostic code; and (2) claims that fall within a low-prevalence diagnostic group are oversampled relative to claims that are not, and bias-adjusted estimators of sensitivity and specificity are generated.

Application:

We illustrate our proposed method using an example from population health surveillance in which diagnostic groups are applied to physician claims to identify cases of acute respiratory illness.

Conclusions:

Failure to account for the prevalence of each diagnostic code within a diagnostic group leads to the underestimation of the PPV, because low-prevalence diagnostic codes are more likely to be false positives. Failure to adjust for oversampling of claims that fall within the low-prevalence diagnostic group relative to those that do not leads to the overestimation of sensitivity and underestimation of specificity.

Xpert MTB/RIF assay for the diagnosis of pulmonary tuberculosis in children: a systematic review and meta-analysis

BACKGROUND

Microbiological confirmation of childhood tuberculosis is rare because of the difficulty of collection of specimens, low sensitivity of smear microscopy, and poor access to culture. We aimed to establish summary estimates for sensitivity and specificity of of the Xpert MTB/RIF assay compared with microscopy in the diagnosis of pulmonary tuberculosis in children.

METHODS

We searched for studies published up to Jan 6, 2015, that used Xpert in any setting in children with and without HIV infection. We systematically reviewed studies that compared the diagnostic accuracy of Xpert MTB/RIF (Xpert) with microscopy for detection of pulmonary tuberculosis and rifampicin resistance in children younger than 16 years against two reference standards-culture results and culture-negative children who were started on anti-tuberculosis therapy. We did meta-analyses using a bivariate random-effects model.

FINDINGS

We identified 15 studies including 4768 respiratory specimens in 3640 children investigated for pulmonary tuberculosis. Culture tests were positive for tuberculosis in 12% (420 of 3640) of all children assessed and Xpert was positive in 11% (406 of 3640). Compared with culture, the pooled sensitivities and specificities of Xpert for tuberculosis detection were 62% (95% credible interval 51-73) and 98% (97-99), respectively, with use of expectorated or induced sputum samples and 66% (51-81) and 98% (96-99), respectively, with use of samples from gastric lavage. Xpert sensitivity was 36-44% higher than was sensitivity for microscopy. Xpert sensitivity in culture-negative children started on antituberculosis therapy was 2% (1-3) for expectorated or induced sputum. Xpert's pooled sensitivity and specificity to detect rifampicin resistance was 86% (95% credible interval 53-98) and 98% (94-100), respectively.

INTERPRETATION

Compared with microscopy, Xpert offers better sensitivity for the diagnosis of pulmonary tuberculosis in children and its scale-up will improve access to tuberculosis diagnostics for children. Although Xpert helps to provide rapid confirmation of disease, its sensitivity remains suboptimum compared with culture tests. A negative Xpert result does not rule out tuberculosis. Good clinical acumen is still needed to decide when to start antituberculosis therapy and continued research for better diagnostics is crucial.

FUNDING

WHO, Global TB Program of Texas Children's Hospital.

Predictors of asymptomatic Clostridium difficile colonization on hospital admission

BACKGROUND

Clostridium difficile (CD) is the leading cause of health care-associated diarrhea and can result in asymptomatic carriage. Rates of asymptomatic CD colonization on hospital admission range from 1.4%-21%. The objective of this study was to evaluate host and bacterial factors associated with colonization on admission.

METHODS

The Consortium de recherche québécois sur le Clostridium difficile study provided data for analysis, including demographic information, known risk factors, and potential confounding factors, prospectively collected for 5,232 patients from 6 hospitals in Quebec and Ontario over 15 months from 2006-2007. Stool or rectal swabs were obtained for culture on admission. Pulsed-field gel electrophoresis was performed on the isolates. The presence of antibody against CD toxins A and B was measured.

RESULTS

There were 212 (4.05%) patients colonized with CD on admission, and 5,020 patients were not colonized with CD. Multivariate logistic regression analysis showed that hospitalization within the last 12 months, use of corticosteroids, prior CD infection, and presence of antibody against toxin B were associated with colonization on admission. Of patients colonized on admission, 79.4% had non-NAP1, non-NAP2 strains.

CONCLUSION

There are identifiable risk factors among asymptomatic CD carriers that could serve in their detection and provide a basis for targeted screening.

Lifetime Prevalence of Congenital Heart Disease in the General Population From 2000 to 2010

Background—

Our objective was to obtain contemporary lifetime estimates of congenital heart disease (CHD) prevalence using population-based data sources up to year 2010.

Methods and Results—

The Quebec CHD database contains 28 years of longitudinal data on all individuals with CHD from 1983 to 2010. Severe CHD was defined as tetralogy of Fallot, truncus arteriosus, transposition complexes, endocardial cushion defects, and univentricular hearts. We used latent class bayesian models combining case definitions from physician claims, hospitalization, and surgical data to obtain point and interval prevalence estimates of CHD in the first year of life, in children (<18 years of age) and in adults. We identified 107 559 CHD patients from 1983 to 2010. Prevalence of CHD in the first year of life was 8.21 per 1000 live births (95% confidence interval, 7.47–9.02) from 1998 to 2005. In 2010, overall prevalence of CHD was 13.11 per 1000 (95% confidence interval, 12.43–13.81) in children and 6.12 per 1000 (95% confidence interval, 5.69–6.57) in adults. CHD prevalence increased by 11% in children and 57% in adults from 2000 to 2010. Prevalence in the severe CHD subgroup increased by 19% (95% confidence interval, 17%–21%) in children and 55% (51%–62%) in adults. By 2010, adults accounted for 66% of the entire CHD population.

Conclusions—

With an increase of >50% in CHD prevalence since 2000, by 2010 adults accounted for two thirds of patients with severe and other forms of CHD in the general population. Our findings should inform allocation of resources and the planning of workforce needs for the predominantly adult CHD population.

A Bayesian framework for estimating the incremental value of a diagnostic test in the absence of a gold standard

BACKGROUND

The absence of a gold standard, i.e., a diagnostic reference standard having perfect sensitivity and specificity, is a common problem in clinical practice and in diagnostic research studies. There is a need for methods to estimate the incremental value of a new, imperfect test in this context.

METHODS

We use a Bayesian approach to estimate the probability of the unknown disease status via a latent class model and extend two commonly-used measures of incremental value based on predictive values [difference in the area under the ROC curve (AUC) and integrated discrimination improvement (IDI)] to the context where no gold standard exists. The methods are illustrated using simulated data and applied to the problem of estimating the incremental value of a novel interferon-gamma release assay (IGRA) over the tuberculin skin test (TST) for latent tuberculosis (TB) screening. We also show how to estimate the incremental value of IGRAs when decisions are based on observed test results rather than predictive values.

RESULTS

We showed that the incremental value is greatest when both sensitivity and specificity of the new test are better and that conditional dependence between the tests reduces the incremental value. The incremental value of the IGRA depends on the sensitivity and specificity of the TST, as well as the prevalence of latent TB, and may thus vary in different populations.

CONCLUSIONS

Even in the absence of a gold standard, incremental value statistics may be estimated and can aid decisions about the practical value of a new diagnostic test.

Xpert MTB/RIF assay for the diagnosis of extrapulmonary tuberculosis: a systematic review and meta-analysis

Xpert MTB/RIF (Cepheid, Sunnyvale, CA, USA) is endorsed for the detection of pulmonary tuberculosis (TB). We performed a systematic review and meta-analysis to assess the accuracy of Xpert for the detection of extrapulmonary TB. We searched multiple databases to October 15, 2013. We determined the accuracy of Xpert compared with culture and a composite reference standard (CRS). We grouped data by sample type and performed meta-analyses using a bivariate random-effects model. We assessed sources of heterogeneity using meta-regression for predefined covariates. We identified 18 studies involving 4461 samples. Sample processing varied greatly among the studies. Xpert sensitivity differed substantially between sample types. In lymph node tissues or aspirates, Xpert pooled sensitivity was 83.1% (95% CI 71.4-90.7%) versus culture and 81.2% (95% CI 72.4-87.7%) versus CRS. In cerebrospinal fluid, Xpert pooled sensitivity was 80.5% (95% CI 59.0-92.2%) against culture and 62.8% (95% CI 47.7-75.8%) against CRS. In pleural fluid, pooled sensitivity was 46.4% (95% CI 26.3-67.8%) against culture and 21.4% (95% CI 8.8-33.9%) against CRS. Xpert pooled specificity was consistently >98.7% against CRS across different sample types. Based on this systematic review, the World Health Organization now recommends Xpert over conventional tests for diagnosis of TB in lymph nodes and other tissues, and as the preferred initial test for diagnosis of TB meningitis.

Statistics for quantifying heterogeneity in univariate and bivariate meta-analyses of binary data: the case of meta-analyses of diagnostic accuracy

Heterogeneity in diagnostic meta-analyses is common because of the observational nature of diagnostic studies and the lack of standardization in the positivity criterion (cut-off value) for some tests. So far the unexplained heterogeneity across studies has been quantified by either using the I(2) statistic for a single parameter (i.e. either the sensitivity or the specificity) or visually examining the data in a receiver-operating characteristic space. In this paper, we derive improved I(2) statistics measuring heterogeneity for dichotomous outcomes, with a focus on diagnostic tests. We show that the currently used estimate of the 'typical' within-study variance proposed by Higgins and Thompson is not able to properly account for the variability of the within-study variance across studies for dichotomous variables. Therefore, when the between-study variance is large, the 'typical' within-study variance underestimates the expected within-study variance, and the corresponding I(2) is overestimated. We propose to use the expected value of the within-study variation in the construction of I(2) in cases of univariate and bivariate diagnostic meta-analyses. For bivariate diagnostic meta-analyses, we derive a bivariate version of I(2) that is able to account for the correlation between sensitivity and specificity. We illustrate the performance of these new estimators using simulated data as well as two real data sets.