Analysis of the diagnostic accuracy of the gamma interferon assay for detection of bovine tuberculosis in U.S. herds

Evaluation of the performance of the IFN-γ release assay in bovine tuberculosis free herds from five European countries

The diagnostic methods for granting and maintenance of the official tuberculosis-free (OTF) status and for intra-Community movement of cattle are the tuberculin skin tests (single or comparative) and the interferon-γ (IFN-γ) release assay (IGRA). However, until now, IGRAs have been primarily applied in infected farms in parallel to the skin test to maximize the number of infected animals detected. Therefore, an evaluation of the performance of IGRAs in OTF herds to assess whether if their specificity is equal to or higher than that of the skin tests is needed. For this, a panel of 4365 plasma samples coming from 84 OTF herds in six European regions (five countries) was assembled and analysed using two IGRA kits, the ID Screen^® Ruminant IFN-g (IDvet) and the Bovigam™ TB Kit (Bovigam). Results were evaluated using different cut-offs, and the impact of herd and animal-level factors on the probability of positivity was assessed using hierarchical Bayesian multivariable logistic regression models. The percentage of reactors ranged from 1.7 to 21.0% (IDvet: S/P ≥ 35%), and 2.1-26.3% (Bovigam: OD_bovis-OD_PBS ≥ 0.1 and OD_bovis-OD_avium ≥ 0.1) depending on the region, with Bovigam disclosing more reactors in all regions. The results suggest that specificity of IGRAs can be influenced by the production type, age and region of origin of the animals. Changes in the cut-offs could lead to specificity values above 98-99% in certain OTF populations, but no single cut-off yielding a sufficiently high specificity (equal or higher than that of skin tests) in all populations was identified. Therefore, an exploratory analysis of the baseline IFN-γ reactivity in OTF populations could help to assess the usefulness of this technique when applied for the purpose of maintaining OTF status.

Accuracy of Tests for Diagnosis of Animal Tuberculosis: Moving Away from the Golden Calf (and towards Bayesian Models)

The last decades have seen major efforts to develop new and improved tools to maximize our ability to detect tuberculosis-infected animals and advance towards the objective of disease control and ultimately eradication. Nevertheless, there is still uncertainty regarding test performance due to the wide range of specificity and especially sensitivity estimates published in the scientific literature. Here, we performed a systematic review of the literature on studies that evaluated the performance of tuberculosis diagnostic tests used in animals through Bayesian Latent Class Models (BLCMs), which do not require the application of a (fallible) reference procedure to classify animals as infected with tuberculosis or not. BLCM-based sensitivity and specificity estimates deviated from those obtained using a reference procedure for certain antemortem tests: an overall lower sensitivity of skin tests and serology and a higher sensitivity of interferon-gamma (IFN-γ) assays was reported. In the case of postmortem diagnostic tests, sensitivity estimates from BLCMs were similar to estimates from studies based on other methodologies. For specificity, the range of BLCM-based estimates was narrower than those based on a reference test, reaching values close to 100% (but lower in the case of IFN-γ assays). In conclusion, Bayesian methods have been increasingly applied for the evaluation of tuberculosis diagnostic tests in animals, yielding results that differ (sometimes substantially) from previously reported test performance in the literature, particularly for in vivo tests and sensitivity estimates. Newly developed models that allow adjustment for relevant factors (e.g., age, breed, region, and herd size) can contribute to the generation of more unbiased estimates of test performance. Nevertheless, although BLCMs for tuberculosis do not require the use of an imperfect reference procedure and are therefore not influenced by its limited performance, they require careful implementation, and transparent systematic reporting should be the norm.

Diagnostic accuracy of the Enferplex Bovine Tuberculosis antibody test in cattle sera

Bovine tuberculosis is a contagious bacterial disease of worldwide economic, zoonotic and welfare importance caused mainly by Mycobacterium bovis infection. Current regulatory diagnostic methods lack sensitivity and require improvement. We have developed a multiplex serological test for bovine tuberculosis and here we provide an estimate of the diagnostic accuracy of the test in cattle. Positive and negative reference serum samples were obtained from animals from Europe and the United States of America. The diagnostic specificity estimate was 98.4% and 99.7% using high sensitivity and high specificity settings of the test respectively. Tuberculin boosting did not affect the overall specificity estimate. The diagnostic sensitivity in samples from Mycobacterium bovis culture positive animals following tuberculin boosting was 93.9%.The relative sensitivity following boosting in tuberculin test positive, lesion positive animals and interferon gamma test positive, lesion positive animals was 97.2% and 96.9% respectively. In tuberculin test negative, lesion positive animals and in interferon gamma test negative, lesion positive animals, the relative sensitivity following tuberculin boosting was 88.2% and 83.6% respectively. The results show that the test has high diagnostic sensitivity and specificity and can detect infected animals that are missed by tuberculin and interferon gamma testing.

Diagnostic accuracy of the interferon-gamma release assay (IGRA) for cases of feline mycobacteriosis

The aim of this study was to evaluate the sensitivity and specificity of the interferon-gamma release assay (IGRA) for diagnosing infections with members of the Mycobacterium (M.) tuberculosis-complex (MTBC) and non-tuberculous mycobacteria (NTM) in domestic cats, and to generate defined feline-specific cut-off values using receiver operating characteristic (ROC) curve analysis to improve test performance. Records of 594 cats that had been tested by IGRA were explored to identify individuals that had a culture and/or polymerase chain reaction (PCR)-confirmed case of mycobacterial disease, and those that had a final diagnosis of non-mycobacterial disease. A total of 117 cats - 80 with mycobacterial disease and 37 diagnosed with a condition other than mycobacteriosis - were identified for further detailed analysis. This population was used to estimate test sensitivity and specificity, as well as likelihood ratios for the IGRA to correctly identify a cat with or without mycobacterial disease. Agreement between IGRA results and culture/PCR using current and proposed new cut-off values was also determined. ROC analysis of defined confirmed infected and non-mycobacterial disease control cats allowed an adjustment of current test cut-offs that increased the overall test sensitivity for MTBC infections from 83.1 % (95 % confidence interval [CI]: 71.5-90.5 %) to 90.2 % (95 % CI: 80.2-95.4%), and M. bovis infection from 43 % (95 % CI: 28.2-60.7%) to 68 % (95 % CI: 51.4-82.1%) while maintaining high test specificity (100 % in both cases). Overall agreement between IGRA results and culture/PCR, while recognising that neither culture nor PCR tests have perfect sensitivity, improved from weak (κ = 0.57) to moderate (κ = 0.71) using new proposed IGRA test cut-off values. Application of these results, based upon the statistical analysis of accumulated test data, can improve the diagnostic performance of the feline IGRA, particularly for identifying infections with M. bovis, without compromising specificity.

Early Detection of Mycobacterium avium subsp. paratuberculosis Infected Cattle: Use of Experimental Johnins and Innovative Interferon-Gamma Test Interpretative Criteria

Paratuberculosis (PTB), also known as Johne's disease, is a chronic proliferative enteritis of ruminants caused by Mycobacterium avium subsp.paratuberculosis (MAP). To date, PTB diagnosis, based on serology, fecal culture, and real-time polymerase chain reaction, has identified animals in advanced stages of infection. To detect MAP infection in animals earlier, the interferon-gamma (IFN-γ) test may be applied. This assay detects cytokines produced by T-lymphocytes of infected subjects after stimulation with purified protein derivatives (PPDs), extracted from Mycobacterium bovis (MB) and from M. avium (MA). The study involved three bovine herds: one PTB-infected herd, one PTB-free herd, and one with an outbreak of bovine tuberculosis. The IFN-γ test was performed on 235 animals, using bovine PPD (PPDB), avian PPD (PPDA), and three experimental PPD Johnins (PPDJs) extracted from a synthetic liquid medium culture of MAP (PPDJ A, B, and C), to assess early MAP detection and avoid false reactions to MB. Furthermore, IFN-γ results were evaluated using 12 interpretative criteria (ICs), based on the differences and ratio between PPD optical density (OD) and IFN-γ basal OD values after lymphocytic stimulation. IC accuracy was expressed as area under the receiver operating characteristic curve. Through a longitudinal study, PPDJs proved to be specific and sensitive in the detection of MAP-infected animals. Among the evaluated ICs, six showed the best performance in terms of accuracy (p < 0.0001), highlighting PTB subclinical infections. In particular, the two best criteria reached sensitivity values of 100% [confidence interval (CI) 95%, 94.1–100%] with a specificity of 91.8% (CI 95%, 81.9–97.3%) and sensitivity levels of 80.6% (CI 95%, 69.1–89.2%) with a specificity of 100% (CI 95%, 94.1–100%). Thus, the IFN-γ assay proved to be a useful diagnostic tool to identify early subclinical MAP-infected animals, in order to manage infected cattle or those exposed to MAP and to monitor younger calves within a herd. Furthermore, the IFN-γ test can be considered an additional test to avoid the introduction of MAP-infected animals, especially in herds where disease has already been eradicated and preservation of the health status is required to maintain the PTB certification level.

Determination of the sensitivity and specificity of bovine tuberculosis screening tests in dairy herds in Thailand using a Bayesian approach

Background

The objective of this study was to determine the sensitivity (Se) and specificity (Sp) of bovine tuberculosis (bTB) screening tests including a single intradermal tuberculin (SIT) test, interferon gamma (IFN-γ) assay, and a commercial ELISA test (M. bovis Ab) in dairy cattle, under field conditions, using a Bayesian approach.

Results

The study population consisted of 128 dairy cows from 25 bTB-infected herds in Chiang Mai and Chiang Rai provinces, Thailand. A single-population Bayesian model was implemented assuming conditional dependence between the SIT test and IFN-γ assays. The 95% posterior probability interval (PPI) of the SIT test (severe interpretation) Se ranged from 75.3 to 95.2% (median = 87.6%), while the Sp was slightly lower (median = 83.6%, PPI = 74.2–92.8%). The IFN-γ assay Se was moderate and the 95% PPI ranged from 38.6 to 74.4% (median = 55.7%) with higher Sp (median = 93.5.4%, PPI = 87.0–98.1%). The M. bovis Ab ELISA Se was low, with 95% PPI ranging between 30.0 and 71.2% (median = 47.4%); however, the Sp was high (median = 90.9%, PPI = 84.5–95.5%).

Conclusion

The SIT test sensitivity was similar to that demonstrated in other regions and can, therefore, be used effectively as part of control programs in this area. The IFN-γ and M. bovis Ab ELISA assays can be applied as supplementary techniques. The test performance of these tests when used as single tests without confirmation, however, are expected to continue to challenge disease eradication efforts.

Electronic supplementary material

The online version of this article (10.1186/s12917-019-1905-x) contains supplementary material, which is available to authorized users.

Evaluating diagnostic tests for bovine tuberculosis in the southern part of Germany: A latent class analysis

Germany has been officially free of bovine tuberculosis since 1996. However, in the last years there has been an increase of bovine tuberculosis cases, particularly in the southern part of Germany, in the Allgäu region. As a consequence a one-time tuberculosis surveillance program was revisited with different premortal and postmortal tests. The aim of this paper was to estimate diagnostic sensitivities and specificities of the different tests used within this surveillance program. In the absence of a perfect test with 100% sensitivity and 100% specificity, thus in the absence of a gold standard, a Bayesian latent class approach with two different datasets was performed. The first dataset included 389 animals, tested with single intra-dermal comparative cervical tuberculin (SICCT) test, PCR and pathology; the second dataset contained 175 animals, tested with single intra-dermal cervical tuberculin (SICT) test, Bovigam® assay, pathology and culture. Two-way conditional dependencies were considered within the models. Additionally, inter-laboratory agreement (five officially approved laboratories) of the Bovigam® assay was assessed with Cohen's kappa test (21 blood samples). The results are given in posterior means and 95% credibility intervals. The specificities of the SICT test, SICCT test, PCR and pathology ranged between 75.8% [68.8-82.2%] and 99.0% [96.8-100%]. The Bovigam® assay stood out with a very low specificity (6.9% [3.6-11.1%]), though it had the highest sensitivity (95.7% [91.3-99.2%]). The sensitivities of the SICCT test, PCR, SICT test, pathology and culture varied from 57.8% [48.0-67.6%] to 88.9% [65.5-99.7%]. The prevalences were 19.8% [14.6-26.5%] (three-test dataset) and 7.7% [4.2-12.3%] (four-test dataset). Among all pairwise comparisons the highest agreement was 0.62 [0.15-1]). In conclusion, the specificity of the Bovigam® assay and the inter-laboratory agreement were lower than expected.

Meta-analyses of the sensitivity and specificity of ante-mortem and post-mortem diagnostic tests for bovine tuberculosis in the UK and Ireland

Bovine Tuberculosis (bTB) in cattle is a global health problem and eradication of the disease requires accurate estimates of diagnostic test performance to optimize their efficiency. The objective of this study was, through statistical meta-analyses, to obtain estimates of sensitivity (Se) and specificity (Sp), for 14 different ante-mortem and post-mortem diagnostic tests for bTB in cattle. Using data from a systematic review of the scientific literature (published 1934-2009) diagnostic Se and Sp were estimated using Bayesian logistic regression models adjusting for confounding factors. Random effect terms were used to account for unexplained heterogeneity. Parameters in the models were implemented using Markov Chain Monte Carlo (MCMC), and posterior distributions for the diagnostic parameters with adjustment for covariates (confounding factors) were obtained using the inverse logit function. Estimates for Se and/or Sp of the tuberculin skin tests and the IFN-γ blood test were compared with estimates published 2010-2015. Median Se for the single intradermal comparative cervical tuberculin skin (SICCT) test (standard interpretation) was 0.50 and Bayesian credible intervals (CrI) were wide (95% CrI 0.26, 0.78). Median Sp for the SICCT test was 1.00 (95% CrI 0.99, 1.00). Estimates for the IFN-γ blood test Bovine Purified Protein Derivative (PPD)-Avian PPD and Early Secreted Antigen target 6 and Culture Filtrate Protein 10 (ESAT-6/CFP10) ESAT6/CFP10 were 0.67 (95% CrI 0.49, 0.82) and 0.78 (95% CrI 0.60, 0.90) respectively for Se, and 0.98 (95% CrI 0.96, 0.99) and 0.99 (95% CrI 0.99, 1.00) for Sp. The study provides an overview of the accuracy of a range of contemporary diagnostic tests for bTB in cattle. Better understanding of diagnostic test performance is essential for the design of effective control strategies and their evaluation.

Bovine tuberculosis: within-herd transmission models to support and direct the decision-making process

Use of mathematical models to study the transmission dynamics of infectious diseases is becoming increasingly common in veterinary sciences. However, modeling chronic infectious diseases such as bovine tuberculosis (bTB) is particularly challenging due to the substantial uncertainty associated with the epidemiology of the disease. Here, the methodological approaches used to model bTB and published in the peer-reviewed literature in the last decades were reviewed with a focus on the impact that the models' assumptions may have had on their results, such as the assumption of density vs. frequency-dependent transmission, the existence of non-infectious and non-detectable stages, and the effect of extrinsic sources of infection (usually associated with wildlife reservoirs). Although all studies suggested a relatively low rate of within-herd transmission of bTB when test-and-cull programs are in place, differences in the estimated length of the infection stages, sensitivity and specificity of the tests used and probable type of transmission (density or frequency dependent) were observed. Additional improvements, such as exploring the usefulness of contact-networks instead of assuming homogeneous mixing of animals, may help to build better models that can help to design, evaluate and monitor control and eradication strategies against bTB.

Circulating Mycobacterium bovis peptides and host response proteins as biomarkers for unambiguous detection of subclinical infection

Bovine tuberculosis remains one of the most damaging diseases to agriculture, and there is also a concern for human spillover. A critical need exists for rapid, thorough, and inexpensive diagnostic methods capable of detecting and differentiating Mycobacterium bovis infection from other pathogenic and environmental mycobacteria at multiple surveillance levels. In a previous study, Seth et al. (PLoS One 4:e5478, 2009, doi:10.1371/journal.pone.0005478) identified 32 host peptides that specifically increased in the blood serum of M. bovis-infected animals). In the current study, 16 M. bovis proteins were discovered in the blood serum proteomics data sets. A large-scale validation analysis was undertaken for selected host and M. bovis proteins using a cattle serum repository containing M. bovis (n = 128), Mycobacterium kansasii (n = 10), and Mycobacterium avium subsp. paratuberculosis (n = 10), cases exposed to M. bovis (n = 424), and negative controls (n = 38). Of the host biomarkers, vitamin D binding protein (VDBP) showed the greatest sensitivity and specificity for M. bovis detection. Circulating M. bovis proteins, specifically polyketide synthetase 5, detected M. bovis-infected cattle with little to no seroreactivity against M. kansasii- and M. avium subsp. paratuberculosis-infected animals. These data indicate that host and pathogen serum proteins can serve as reliable biomarkers for tracking M. bovis infection in animal populations.

Evaluation of the interferon-γ assay on blood collected at exsanguination of cattle under field conditions for surveillance of bovine tuberculosis

Development of point of concentration (POC) surveillance strategies for bovine tuberculosis (bTB) would facilitate global efforts to eradicate bTB. The interferon-gamma (IFNγ) assay can detect IFNγ responses to Mycobacterium bovis in blood collected at commencement of exsanguination (COE) of experimentally challenged cattle but has not been evaluated under field conditions. The current study was aimed at determining (i) whether blood collected at COE of cattle at slaughter, under field conditions, is practical to obtain and useful for identifying cattle as IFNγ positive for bTB, (ii) whether the results of the IFNγ assay obtained at COE reliably compare with results obtained from live animals in the field, and (iii) whether the identified animal(s) originated from bTB-infected or bTB-exposed herds. Cattle from three risk groups were used: the highest risk group consisted of 49 cattle from 3 bTB-infected herds; the medium risk group consisted of 24 cattle from a potentially exposed herd; and the lowest risk group consisted of 60 cattle from herds with no known history of bTB exposure. The IFNγ assay was performed on blood collected both before stunning and at COE of cattle at slaughter. An enhanced slaughter inspection for gross lesions consistent with bTB was performed on all cattle. In addition, lymph nodes were cultured for M. bovis for cattle that tested positive for bTB via the IFNγ assay and for most cattle that tested negative for bTB. Cattle, both with and without lesions consistent with bTB, were identified as positive for bTB by the IFNγ assay using blood collected at COE, but none of the positive cattle originated from the lowest risk group. The current study demonstrates that blood collected at COE of cattle is both a practical and moderately reliable sample for accessing bTB infection using the IFNγ assay.

Accurate diagnostics for Bovine tuberculosis based on high-throughput sequencing

Background

Bovine tuberculosis (bTB) is an enduring contagious disease of cattle that has caused substantial losses to the global livestock industry. Despite large-scale eradication efforts, bTB continues to persist. Current bTB tests rely on the measurement of immune responses in vivo (skin tests), and in vitro (bovine interferon-γ release assay). Recent developments are characterized by interrogating the expression of an increasing number of genes that participate in the immune response. Currently used assays have the disadvantages of limited sensitivity and specificity, which may lead to incomplete eradication of bTB. Moreover, bTB that reemerges from wild disease reservoirs requires early and reliable diagnostics to prevent further spread. In this work, we use high-throughput sequencing of the peripheral blood mononuclear cells (PBMCs) transcriptome to identify an extensive panel of genes that participate in the immune response. We also investigate the possibility of developing a reliable bTB classification framework based on RNA-Seq reads.

Methodology/Principal Findings

Pooled PBMC mRNA samples from unaffected calves as well as from those with disease progression of 1 and 2 months were sequenced using the Illumina Genome Analyzer II. More than 90 million reads were splice-aligned against the reference genome, and deposited to the database for further expression analysis and visualization. Using this database, we identified 2,312 genes that were differentially expressed in response to bTB infection (p<10⁻⁸). We achieved a bTB infected status classification accuracy of more than 99% with split-sample validation on newly designed and learned mixtures of expression profiles.

Conclusions/Significance

We demonstrated that bTB can be accurately diagnosed at the early stages of disease progression based on RNA-Seq high-throughput sequencing. The inclusion of multiple genes in the diagnostic panel, combined with the superior sensitivity and broader dynamic range of RNA-Seq, has the potential to improve the accuracy of bTB diagnostics. The computational pipeline used for the project is available from http://code.google.com/p/bovine-tb-prediction.