Accuracy of computer-aided chest X-ray in community-based tuberculosis screening: Lessons from the 2016 Kenya National Tuberculosis Prevalence Survey

Evaluation of C-Reactive Protein and Computer-Aided Analysis of Chest X-rays as Tuberculosis Triage Tests at Health Facilities in Lesotho and South Africa

BACKGROUND

To improve tuberculosis case-finding, rapid, non-sputum triage tests need to be developed according to the World Health Organization target product profile (TPP) (>90% sensitivity, >70% specificity). We prospectively evaluated and compared artificial intelligence-based, computer-aided detection software, CAD4TBv7, and C-reactive protein assay (CRP) as triage tests at health facilities in Lesotho and South Africa.

METHODS

Adults (≥18 years) presenting with ≥1 of the 4 cardinal tuberculosis symptoms were consecutively recruited between February 2021 and April 2022. After informed consent, each participant underwent a digital chest X-ray for CAD4TBv7 and a CRP test. Participants provided 1 sputum sample for Xpert MTB/RIF Ultra and Xpert MTB/RIF and 1 for liquid culture. Additionally, an expert radiologist read the chest X-rays via teleradiology. For primary analysis, a composite microbiological reference standard (ie, positive culture or Xpert Ultra) was used.

RESULTS

We enrolled 1392 participants, 48% were people with HIV and 24% had previously tuberculosis. The receiver operating characteristic curve for CAD4TBv7 and CRP showed an area under the curve of .87 (95% CI: .84-.91) and .80 (95% CI: .76-.84), respectively. At thresholds corresponding to 90% sensitivity, specificity was 68.2% (95% CI: 65.4-71.0%) and 38.2% (95% CI: 35.3-41.1%) for CAD4TBv7 and CRP, respectively. CAD4TBv7 detected tuberculosis as well as an expert radiologist. CAD4TBv7 almost met the TPP criteria for tuberculosis triage.

CONCLUSIONS

CAD4TBv7 is accurate as a triage test for patients with tuberculosis symptoms from areas with a high tuberculosis and HIV burden. The role of CRP in tuberculosis triage requires further research.

CLINICAL TRIALS REGISTRATION

Clinicaltrials.gov identifier: NCT04666311.

Simultaneous alleviation of verification and reference standard biases in a community-based tuberculosis screening study using Bayesian latent class analysis

BACKGROUND

Estimation of prevalence and diagnostic test accuracy in tuberculosis (TB) prevalence surveys suffer from reference standard and verification biases. The former is attributed to the imperfect reference test used to bacteriologically confirm TB disease. The latter occurs when only the participants screening positive for any TB-compatible symptom or chest X-ray abnormality are selected for bacteriological testing (verification). Bayesian latent class analysis (LCA) alleviates the reference standard bias but suffers verification bias in TB prevalence surveys. This work aims to identify best-practice approaches to simultaneously alleviate the reference standard and verification biases in the estimates of pulmonary TB prevalence and diagnostic test performance in TB prevalence surveys.

METHODS

We performed a secondary analysis of 9869 participants aged ≥15 years from a community-based multimorbidity screening study in a rural district of KwaZulu-Natal, South Africa (Vukuzazi study). Participants were eligible for bacteriological testing using Xpert Ultra and culture if they reported any cardinal TB symptom or had an abnormal chest X-ray finding. We conducted Bayesian LCA in five ways to handle the unverified individuals: (i) complete-case analysis, (ii) analysis assuming the unverified individuals would be negative if bacteriologically tested, (iii) analysis of multiply-imputed datasets with imputation of the missing bacteriological test results for the unverified individuals using multivariate imputation via chained equations (MICE), and simultaneous imputation of the missing bacteriological test results in the analysis model assuming the missing bacteriological test results were (iv) missing at random (MAR), and (v) missing not at random (MNAR). We compared the results of (i)-(iii) to the analysis based on a composite reference standard (CRS) of Xpert Ultra and culture. Through simulation with an overall true prevalence of 2.0%, we evaluated the ability of the models to alleviate both biases simultaneously.

RESULTS

Based on simulation, Bayesian LCA with simultaneous imputation of the missing bacteriological test results under the assumption that the missing data are MAR and MNAR alleviate the reference standard and verification biases. CRS-based analysis and Bayesian LCA assuming the unverified are negative for TB alleviate the biases only when the true overall prevalence is <3.0%. Complete-case analysis produced biased estimates. In the Vukuzazi study, Bayesian LCA with simultaneous imputation of the missing bacteriological test results under the MAR and MNAR assumptions produced overall PTB prevalence of 0.9% (95% Credible Interval (CrI): 0.6-1.9) and 0.7% (95% CrI: 0.5-1.1) respectively alongside realistic estimates of overall diagnostic test sensitivity and specificity with substantially overlapping 95% CrI. The CRS-based analysis and Bayesian LCA assuming the unverified were negative for TB produced 0.7% (95% CrI: 0.5-0.9) and 0.7% (95% CrI: 0.5-1.2) overall PTB prevalence respectively with realistic estimates of overall diagnostic test sensitivity and specificity. Unlike CRS-based analysis, Bayesian LCA of multiply-imputed data using MICE mitigates both biases.

CONCLUSION

The findings demonstrate the efficacy of these advanced techniques in alleviating the reference standard and verification biases, enhancing the robustness of community-based screening programs. Imputing missing values as negative for bacteriological tests is plausible under realistic assumptions.

AI for Detection of Tuberculosis: Implications for Global Health

Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity. Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning Supplemental material is available for this article. © RSNA, 2024.

Community-based active-case finding for tuberculosis: navigating a complex minefield

Community-based active case finding (ACF) for tuberculosis (TB) involves an offer of screening to populations at risk of TB, oftentimes with additional health promotion, community engagement and health service strengthening. Recently updated World Health Organization TB screening guidelines conditionally recommend expanded offer of ACF for communities where the prevalence of undiagnosed pulmonary TB is greater than 0.5% among adults, or with other structural risk factors for TB. Subclinical TB is thought to be a major contributor to TB transmission, and ACF, particularly with chest X-ray screening, could lead to earlier diagnosis. However, the evidence base for the population-level impact of ACF is mixed, with effectiveness likely highly dependent on the screening approach used, the intensity with which ACF is delivered, and the success of community- and health-system participation. With recent changes in TB epidemiology due to the effective scale-up of treatment for HIV in Africa, the impacts of the COVID-19 pandemic, and the importance of subclinical TB, researchers and public health practitioners planning to implement ACF programmes must carefully and repeatedly consider the potential population and individual benefits and harms from these programmes. Here we synthesise evidence and experience from implementing ACF programmes to provide practical guidance, focusing on the selection of populations, screening algorithms, selecting outcomes, and monitoring and evaluation. With careful planning and substantial investment, community-based ACF for TB can be an impactful approach to accelerating progress towards elimination of TB in high-burden countries. However, ACF cannot and should not be a substitute for equitable access to responsive, affordable, accessible primary care services for all.

Accuracy of digital chest x-ray analysis with artificial intelligence software as a triage and screening tool in hospitalized patients being evaluated for tuberculosis in Lima, Peru

Tuberculosis (TB) transmission in healthcare facilities is common in high-incidence countries. Yet, the optimal approach for identifying inpatients who may have TB is unclear. We evaluated the diagnostic accuracy of qXR (Qure.ai, India) computer-aided detection (CAD) software versions 3.0 and 4.0 (v3 and v4) as a triage and screening tool within the FAST (Find cases Actively, Separate safely, and Treat effectively) transmission control strategy. We prospectively enrolled two cohorts of patients admitted to a tertiary hospital in Lima, Peru: one group had cough or TB risk factors (triage) and the other did not report cough or TB risk factors (screening). We evaluated the sensitivity and specificity of qXR for the diagnosis of pulmonary TB using culture and Xpert as primary and secondary reference standards, including stratified analyses based on risk factors. In the triage cohort (n = 387), qXR v4 sensitivity was 0.91 (59/65, 95% CI 0.81-0.97) and specificity was 0.32 (103/322, 95% CI 0.27-0.37) using culture as reference standard. There was no difference in the area under the receiver-operating-characteristic curve (AUC) between qXR v3 and qXR v4 with either a culture or Xpert reference standard. In the screening cohort (n = 191), only one patient had a positive Xpert result, but specificity in this cohort was high (>90%). A high prevalence of radiographic lung abnormalities, most notably opacities (81%), consolidation (62%), or nodules (58%), was detected by qXR on digital CXR images from the triage cohort. qXR had high sensitivity but low specificity as a triage in hospitalized patients with cough or TB risk factors. Screening patients without cough or risk factors in this setting had a low diagnostic yield. These findings further support the need for population and setting-specific thresholds for CAD programs.

Diagnostic Accuracy of Computer-Aided Detection During Active Case Finding for Pulmonary Tuberculosis in Africa: A Systematic Review and Meta-analysis

Background

Computer-aided detection (CAD) may be a useful screening tool for tuberculosis (TB). However, there are limited data about its utility in active case finding (ACF) in a community-based setting, and particularly in an HIV-endemic setting where performance may be compromised.

Methods

We performed a systematic review and evaluated articles published between January 2012 and February 2023 that included CAD as a screening tool to detect pulmonary TB against a microbiological reference standard (sputum culture and/or nucleic acid amplification test [NAAT]). We collected and summarized data on study characteristics and diagnostic accuracy measures. Two reviewers independently extracted data and assessed methodological quality against Quality Assessment of Diagnostic Accuracy Studies-2 criteria. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines were followed.

Results

Of 1748 articles reviewed, 5 met with the eligibility criteria and were included in this review. A meta-analysis revealed pooled sensitivity of 0.87 (95% CI, 0.78-0.96) and specificity of 0.74 (95% CI, 0.55-0.93), just below the World Health Organization (WHO)-recommended target product profile (TPP) for a screening test (sensitivity ≥0.90 and specificity ≥0.70). We found a high risk of bias and applicability concerns across all studies. Subgroup analyses, including the impact of HIV and previous TB, were not possible due to the nature of the reporting within the included studies.

Conclusions

This review provides evidence, specifically in the context of ACF, for CAD as a potentially useful and cost-effective screening tool for TB in a resource-poor HIV-endemic African setting. However, given methodological concerns, caution is required with regards to applicability and generalizability.

Computer-aided detection thresholds for digital chest radiography interpretation in tuberculosis diagnostic algorithms

Objectives

Use of computer-aided detection (CAD) software is recommended to improve tuberculosis screening and triage, but threshold determination is challenging if reference testing has not been performed in all individuals. We aimed to determine such thresholds through secondary analysis of the 2019 Lesotho national tuberculosis prevalence survey.

Methods

Symptom screening and chest radiographs were performed in participants aged ≥15 years; those symptomatic or with abnormal chest radiographs provided samples for Xpert MTB/RIF and culture testing. Chest radiographs were processed using CAD4TB version 7. We used six methodological approaches to deal with participants who did not have bacteriological test results to estimate pulmonary tuberculosis prevalence and assess diagnostic accuracy.

Results

Among 17 070 participants, 5214 (31%) had their tuberculosis status determined; 142 had tuberculosis. Prevalence estimates varied between methodological approaches (0.83-2.72%). Using multiple imputation to estimate tuberculosis status for those eligible but not tested, and assuming those not eligible for testing were negative, a CAD4TBv7 threshold of 13 had a sensitivity of 89.7% (95% CI 84.6-94.8) and a specificity of 74.2% (73.6-74.9), close to World Health Organization (WHO) target product profile criteria. Assuming all those not tested were negative produced similar results.

Conclusions

This is the first study to evaluate CAD4TB in a community screening context employing a range of approaches to account for unknown tuberculosis status. The assumption that those not tested are negative - regardless of testing eligibility status - was robust. As threshold determination must be context specific, our analytically straightforward approach should be adopted to leverage prevalence surveys for CAD threshold determination in other settings with a comparable proportion of eligible but not tested participants.

Accuracy of digital chest x-ray analysis with artificial intelligence software as a triage and screening tool in hospitalized patients being evaluated for tuberculosis in Lima, Peru

Introduction

Tuberculosis (TB) transmission in healthcare facilities is common in high-incidence countries. Yet, the optimal approach for identifying inpatients who may have TB is unclear. We evaluated the diagnostic accuracy of qXR (Qure.ai, India) computer-aided detection (CAD) software versions 3.0 and 4.0 (v3 and v4) as a triage and screening tool within the FAST (Find cases Actively, Separate safely, and Treat effectively) transmission control strategy.

Methods

We prospectively enrolled two cohorts of patients admitted to a tertiary hospital in Lima, Peru: one group had cough or TB risk factors (triage) and the other did not report cough or TB risk factors (screening). We evaluated the sensitivity and specificity of qXR for the diagnosis of pulmonary TB using culture and Xpert as primary and secondary reference standards, including stratified analyses based on risk factors.

Results

In the triage cohort (n=387), qXR v4 sensitivity was 0.91 (59/65, 95% CI 0.81-0.97) and specificity was 0.32 (103/322, 95% CI 0.27-0.37) using culture as reference standard. There was no difference in the area under the receiver-operating-characteristic curve (AUC) between qXR v3 and qXR v4 with either a culture or Xpert reference standard. In the screening cohort (n=191), only one patient had a positive Xpert result, but specificity in this cohort was high (>90%). A high prevalence of radiographic lung abnormalities, most notably opacities (81%), consolidation (62%), or nodules (58%), was detected by qXR on digital CXR images from the triage cohort.

Conclusions

qXR had high sensitivity but low specificity as a triage in hospitalized patients with cough or TB risk factors. Screening patients without cough or risk factors in this setting had a low diagnostic yield. These findings further support the need for population and setting-specific thresholds for CAD programs.