The Performance of Computer-Aided Detection Digital Chest X-ray Reading Technologies for Triage of Active Tuberculosis Among Persons With a History of Previous Tuberculosis

Breaking the threshold: Developing multivariable models using computer-aided chest X-ray analysis for tuberculosis triage

OBJECTIVES

Computer-aided detection (CAD) software packages quantify tuberculosis (TB)-compatible chest X-ray (CXR) abnormality as continuous scores. In practice, a threshold value is selected for binary CXR classification. We assessed the diagnostic accuracy of an alternative approach to applying CAD for TB triage: incorporating CAD scores in multivariable modeling.

METHODS

We pooled individual patient data from four studies. Separately, for two commercial CAD, we used logistic regression to model microbiologically confirmed TB. Models included CAD score, study site, age, sex, human immunodeficiency virus status, and prior TB. We compared specificity at target sensitivities ≥90% between the multivariable model and the current threshold-based approach for CAD use.

RESULTS

We included 4,733/5,640 (84%) participants with complete covariate data (median age 36 years; 45% female; 22% with prior TB; 22% people living with human immunodeficiency virus). A total of 805 (17%) had TB. Multivariable models demonstrated excellent performance (areas under the receiver operating characteristic curve [95% confidence interval]: software A, 0.91 [0.90-0.93]; software B, 0.92 [0.91-0.93]). Compared with threshold scores, multivariable models increased specificity (e.g., at 90% sensitivity, threshold vs model specificity [95% confidence interval]: software A, 71% [68-74%] vs 75% [74-77%]; software B, 69% [63-75%] vs 75% [74-77%]).

CONCLUSION

Using CAD scores in multivariable models outperformed the current practice of CAD-threshold-based CXR classification for TB diagnosis.

Development and validation of a deep learning model for detecting signs of tuberculosis on chest radiographs among US-bound immigrants and refugees

Immigrants and refugees seeking admission to the United States must first undergo an overseas medical exam, overseen by the US Centers for Disease Control and Prevention (CDC), during which all persons ≥15 years old receive a chest x-ray to look for signs of tuberculosis. Although individual screening sites often implement quality control (QC) programs to ensure radiographs are interpreted correctly, the CDC does not currently have a method for conducting similar QC reviews at scale. We obtained digitized chest radiographs collected as part of the overseas immigration medical exam. Using radiographs from applicants 15 years old and older, we trained deep learning models to perform three tasks: identifying abnormal radiographs; identifying abnormal radiographs suggestive of tuberculosis; and identifying the specific findings (e.g., cavities or infiltrates) in abnormal radiographs. We then evaluated the models on both internal and external testing datasets, focusing on two classes of performance metrics: individual-level metrics, like sensitivity and specificity, and sample-level metrics, like accuracy in predicting the prevalence of abnormal radiographs. A total of 152,012 images (one image per applicant; mean applicant age 39 years) were used for model training. On our internal test dataset, our models performed well both in identifying abnormalities suggestive of TB (area under the curve [AUC] of 0.97; 95% confidence interval [CI]: 0.95, 0.98) and in estimating sample-level counts of the same (-2% absolute percentage error; 95% CIC: -8%, 6%). On the external test datasets, our models performed similarly well in identifying both generic abnormalities (AUCs ranging from 0.89 to 0.92) and those suggestive of TB (AUCs from 0.94 to 0.99). This performance was consistent across metrics, including those based on thresholded class predictions, like sensitivity, specificity, and F1 score. Strong performance relative to high-quality radiological reference standards across a variety of datasets suggests our models may make reliable tools for supporting chest radiography QC activities at CDC.

Evaluation of C-Reactive Protein and Computer-Aided Analysis of Chest X-rays as Tuberculosis Triage Tests at Health Facilities in Lesotho and South Africa

BACKGROUND

To improve tuberculosis case-finding, rapid, non-sputum triage tests need to be developed according to the World Health Organization target product profile (TPP) (>90% sensitivity, >70% specificity). We prospectively evaluated and compared artificial intelligence-based, computer-aided detection software, CAD4TBv7, and C-reactive protein assay (CRP) as triage tests at health facilities in Lesotho and South Africa.

METHODS

Adults (≥18 years) presenting with ≥1 of the 4 cardinal tuberculosis symptoms were consecutively recruited between February 2021 and April 2022. After informed consent, each participant underwent a digital chest X-ray for CAD4TBv7 and a CRP test. Participants provided 1 sputum sample for Xpert MTB/RIF Ultra and Xpert MTB/RIF and 1 for liquid culture. Additionally, an expert radiologist read the chest X-rays via teleradiology. For primary analysis, a composite microbiological reference standard (ie, positive culture or Xpert Ultra) was used.

RESULTS

We enrolled 1392 participants, 48% were people with HIV and 24% had previously tuberculosis. The receiver operating characteristic curve for CAD4TBv7 and CRP showed an area under the curve of .87 (95% CI: .84-.91) and .80 (95% CI: .76-.84), respectively. At thresholds corresponding to 90% sensitivity, specificity was 68.2% (95% CI: 65.4-71.0%) and 38.2% (95% CI: 35.3-41.1%) for CAD4TBv7 and CRP, respectively. CAD4TBv7 detected tuberculosis as well as an expert radiologist. CAD4TBv7 almost met the TPP criteria for tuberculosis triage.

CONCLUSIONS

CAD4TBv7 is accurate as a triage test for patients with tuberculosis symptoms from areas with a high tuberculosis and HIV burden. The role of CRP in tuberculosis triage requires further research.

CLINICAL TRIALS REGISTRATION

Clinicaltrials.gov identifier: NCT04666311.

Prior tuberculosis, radiographic lung abnormalities and prevalent diabetes in rural South Africa

BACKGROUND

Growing evidence suggests that chronic inflammation caused by tuberculosis (TB) may increase the incidence of diabetes. However, the relationship between post-TB pulmonary abnormalities and diabetes has not been well characterized.

METHODS

We analyzed data from a cross-sectional study in KwaZulu-Natal, South Africa, of people 15 years and older who underwent chest X-ray and diabetes screening with hemoglobin A1c testing. The analytic sample was restricted to persons with prior TB, defined by either (1) a self-reported history of TB treatment, (2) radiologist-confirmed prior TB on chest radiography, and (3) a negative sputum culture and GeneXpert. Chest X-rays of all participants were evaluated by the study radiologist to determine the presence of TB lung abnormalities. To assess the relationships between our outcome of interest, prevalent diabetes (HBA1c ≥6.5%), and our exposure of interest, chest X-ray abnormalities, we fitted logistic regression models adjusted for potential clinical and demographic confounders. In secondary analyses, we used the computer-aided detection system CAD4TB, which scores X-rays from 10 to 100 for detection of TB disease, as our exposure interest, and repeated analyses with a comparator group that had no history of TB disease.

RESULTS

In the analytic cohort of people with prior TB (n = 3,276), approximately two-thirds (64.9%) were women, and the average age was 50.8 years (SD 17.4). The prevalence of diabetes was 10.9%, and 53.0% of people were living with HIV. In univariate analyses, there was no association between diabetes prevalence and radiologist chest X-ray abnormalities (OR 1.23, 95%CI 0.95-1.58). In multivariate analyses, the presence of pulmonary abnormalities was associated with an 29% reduction in the odds of prevalent diabetes (aOR 0.71, 95%CI 0.53-0.97, p = 0.030). A similar inverse relationship was observed for diabetes with each 10-unit increase in the CAD4TB chest X-ray scores among people with prior TB (aOR 0.92, 95%CI 0.87-0.97; p = 0.002), but this relationship was less pronounced in the no TB comparator group (aOR 0.96, 95%CI 0.94-0.99).

CONCLUSIONS

Among people with prior TB, pulmonary abnormalities on digital chest X-ray are inversely associated with prevalent diabetes. The severity of radiographic post-TB lung disease does not appear to be a determinant of diabetes in this South African population.

[Clinical Application of Artificial Intelligence-Based Detection Assistance Devices for Chest X-Ray Interpretation: Current Status and Practical Considerations]

Artificial intelligence (AI) technology is actively being applied for the interpretation of medical imaging, such as chest X-rays. AI-based software medical devices, which automatically detect various types of abnormal findings in chest X-ray images to assist physicians in their interpretation, are actively being commercialized and clinically implemented in Korea. Several important issues need to be considered for AI-based detection assistant tools to be applied in clinical practice: the evaluation of performance and efficacy prior to implementation; the determination of the target application, range, and method of delivering results; and monitoring after implementation and legal liability issues. Appropriate decision making regarding these devices based on the situation in each institution is necessary. Radiologists must be engaged as medical assessment experts using the software for these devices as well as in medical image interpretation to ensure the safe and efficient implementation and operation of AI-based detection assistant tools.

Head-to-head comparison of diagnostic accuracy of TB screening tests: Chest-X-ray, Xpert TB host response, and C-reactive protein

Background

Accessible, accurate screening tests are necessary to advance tuberculosis (TB) case finding and early detection in high-burden countries. We compared the diagnostic accuracy of available TB triage tests.

Methods

We prospectively screened consecutive adults with ≥2 weeks of cough presenting to primary health centers in the Philippines, Vietnam, South Africa, Uganda, and India. All participants received the index tests: chest-X-ray (CXR), venous or capillary Cepheid Xpert TB Host Response (HR) testing, and point-of-care C-reactive protein (CRP) testing (Boditech iChroma II). CXR images were processed using computer-aided detection (CAD) algorithms. We assessed diagnostic accuracy against a microbiologic reference standard (sputum Xpert Ultra, culture). Optimal cut-points were chosen to achieve sensitivity ≥90% and maximize specificity. Two-test screening algorithms were considered, using two approaches: 1) sequential negative serial screening in which the second screening test is conducted only if the first is negative and positive is defined as positive on either test and 2) sequential positive serial screening, in which the second screening test is conducted only if the first is positive and positive is defined as positive on both tests.

Results

Between July 2021 and August 2022, 1,392 participants with presumptive TB had valid results on index tests and the reference standard, and 303 (22%) had confirmed TB. In head-to-head comparisons, CAD4TB v7 showed the highest specificity when using a cut-point that achieves 90% sensitivity (70.3% vs. 65.1% for Xpert HR, difference 95% CI 1.6 to 8.9; 49.7% for CRP, difference 95% CI 17.0 to 24.3). Among the possible two-test screening algorithms, three met WHO target product profile (TPP) minimum accuracy thresholds and had higher accuracy than any test alone. At 90% sensitivity, the specificity was 79.6% for Xpert HR-CAD4TB [sequential negative], 75.9% for CRP-CAD4TB [sequential negative], and 73.7% for Xpert HR-CAD4TB [sequential positive].

Conclusions

CAD4TB achieves TPP targets and outperforms Xpert HR and CRP. Combining screening tests further increased accuracy. Cost and feasibility of two-test screening algorithms should be explored.

Registration

NCT04923958.

AI for Detection of Tuberculosis: Implications for Global Health

Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity. Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning Supplemental material is available for this article. © RSNA, 2024.

Post-treatment Radiographic Severity and Mortality in Mycobacterium avium Complex Pulmonary Disease

Rationale: Imaging studies are widely performed when treating Mycobacterium avium complex pulmonary disease (MAC-PD); however, the clinical significance of post-treatment radiographic change is unknown. Objectives: To determine whether a deep neural network trained with pulmonary tuberculosis could adequately score the radiographic severity of MAC-PD and then to examine relationships between post-treatment radiographic severity and its change from baseline and long-term prognosis. Methods: We retrospectively collected chest radiographs of adult patients with MAC-PD treated for ⩾6 months at baseline and at 3, 6, 9, and 12 months of treatment. We correlated the radiographic severity score generated by a deep neural network with visual and clinical severity as determined by radiologists and mycobacterial culture status, respectively. The associations between the score, improvement from baseline, and mortality were analyzed using Cox proportional hazards regression. Results: In total, 342 and 120 patients were included in the derivation and validation cohorts, respectively. The network's severity score correlated with radiologists' grading (Spearman coefficient, 0.40) and mycobacterial culture results (odds ratio, 1.02; 95% confidence interval [CI], 1.0-1.05). A significant decreasing trend in the severity score was observed over time (P < 0.001). A higher score at 12 months of treatment was independently associated with higher mortality (adjusted hazard ratio, 1.07; 95% CI, 1.03-1.10). Improvements in radiographic scores from baseline were associated with reduced mortality, regardless of culture conversion (adjusted hazard ratio, 0.42; 95% CI, 0.22-0.80). These findings were replicated in the validation cohort. Conclusions: Post-treatment radiographic severity and improvement from baseline in patients with MAC-PD were associated with long-term survival.

Diagnostic Accuracy of Computer-Aided Detection During Active Case Finding for Pulmonary Tuberculosis in Africa: A Systematic Review and Meta-analysis

Background

Computer-aided detection (CAD) may be a useful screening tool for tuberculosis (TB). However, there are limited data about its utility in active case finding (ACF) in a community-based setting, and particularly in an HIV-endemic setting where performance may be compromised.

Methods

We performed a systematic review and evaluated articles published between January 2012 and February 2023 that included CAD as a screening tool to detect pulmonary TB against a microbiological reference standard (sputum culture and/or nucleic acid amplification test [NAAT]). We collected and summarized data on study characteristics and diagnostic accuracy measures. Two reviewers independently extracted data and assessed methodological quality against Quality Assessment of Diagnostic Accuracy Studies-2 criteria. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines were followed.

Results

Of 1748 articles reviewed, 5 met with the eligibility criteria and were included in this review. A meta-analysis revealed pooled sensitivity of 0.87 (95% CI, 0.78-0.96) and specificity of 0.74 (95% CI, 0.55-0.93), just below the World Health Organization (WHO)-recommended target product profile (TPP) for a screening test (sensitivity ≥0.90 and specificity ≥0.70). We found a high risk of bias and applicability concerns across all studies. Subgroup analyses, including the impact of HIV and previous TB, were not possible due to the nature of the reporting within the included studies.

Conclusions

This review provides evidence, specifically in the context of ACF, for CAD as a potentially useful and cost-effective screening tool for TB in a resource-poor HIV-endemic African setting. However, given methodological concerns, caution is required with regards to applicability and generalizability.

Computer-aided diagnostic accuracy of pulmonary tuberculosis on chest radiography among lower respiratory tract symptoms patients

Even though the Gaza Strip is a low pulmonary tuberculosis (TB) burden region, it is well-known that TB is primarily a socioeconomic problem associated with overcrowding, poor hygiene, a lack of fresh water, and limited access to healthcare, which is the typical case in the Gaza Strip. Therefore, this study aimed at assessing the accuracy of the automatic software computer-aided detection for tuberculosis (CAD4TB) in diagnosing pulmonary TB on chest radiography and compare the CAD4TB software reading with the results of geneXpert. Using a census sampling method, the study was conducted in radiology departments in the Gaza Strip hospitals between 1 December 2022 and 31 March 2023. A digital X-ray, printer, and online X-ray system backed by CAD4TBv6 software were used to screen patients with lower respiratory tract symptoms. GeneXpert analysis was performed for all patients having a score > 40. A total of 1,237 patients presenting with lower respiratory tract symptoms participated in this current study. Chest X-ray readings showed that 7.8% (n = 96) were presumptive for TB. The CAD4TBv6 scores showed that 11.8% (n = 146) of recruited patients were presumptive for TB. GeneXpert testing on sputum samples showed that 6.2% (n = 77) of those with a score > 40 on CAD4TB were positive for pulmonary TB. Significant differences were found in chest X-ray readings, CAD4TBv6 scores, and GeneXpert results among sociodemographic and health status variables (P-value < 0.05). The study showed that the incidence rate of TB in the Gaza Strip is 3.5 per 100,000 population in the Gaza strip. The sensitivity of the CAD4TBv6 score and the symptomatic review for tuberculosis with a threshold score of >40 is 80.2%, and the specificity is 94.0%. The positive Likelihood Ratio is 13.3%, Negative Likelihood Ratio is 0.2 with 7.8% prevalence. Positive Predictive Value is 52.7%, Negative Predictive Value is 98.3%, and accuracy is 92.9%. In a resource-limited country with a high burden of neglected disease, combining chest X-ray readings by CAD4TB and symptomatology is extremely valuable for screening a population at risk. CAD4TB is noticeably more efficient than other methods for TB screening and early diagnosis in people who would otherwise go undetected.

Diagnostic Accuracy of the Artificial Intelligence Methods in Medical Imaging for Pulmonary Tuberculosis: A Systematic Review and Meta-Analysis

Tuberculosis (TB) remains one of the leading causes of death among infectious diseases worldwide. Early screening and diagnosis of pulmonary tuberculosis (PTB) is crucial in TB control, and tend to benefit from artificial intelligence. Here, we aimed to evaluate the diagnostic efficacy of a variety of artificial intelligence methods in medical imaging for PTB. We searched MEDLINE and Embase with the OVID platform to identify trials published update to November 2022 that evaluated the effectiveness of artificial-intelligence-based software in medical imaging of patients with PTB. After data extraction, the quality of studies was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using a bivariate random-effects model. In total, 3987 references were initially identified and 61 studies were finally included, covering a wide range of 124,959 individuals. The pooled sensitivity and the specificity were 91% (95% confidence interval (CI), 89-93%) and 65% (54-75%), respectively, in clinical trials, and 94% (89-96%) and 95% (91-97%), respectively, in model-development studies. These findings have demonstrated that artificial-intelligence-based software could serve as an accurate tool to diagnose PTB in medical imaging. However, standardized reporting guidance regarding AI-specific trials and multicenter clinical trials is urgently needed to truly transform this cutting-edge technology into clinical practice.