Evaluation of computer aided detection of tuberculosis on chest radiography among people with diabetes in Karachi Pakistan

Prior tuberculosis, radiographic lung abnormalities and prevalent diabetes in rural South Africa

BACKGROUND

Growing evidence suggests that chronic inflammation caused by tuberculosis (TB) may increase the incidence of diabetes. However, the relationship between post-TB pulmonary abnormalities and diabetes has not been well characterized.

METHODS

We analyzed data from a cross-sectional study in KwaZulu-Natal, South Africa, of people 15 years and older who underwent chest X-ray and diabetes screening with hemoglobin A1c testing. The analytic sample was restricted to persons with prior TB, defined by either (1) a self-reported history of TB treatment, (2) radiologist-confirmed prior TB on chest radiography, and (3) a negative sputum culture and GeneXpert. Chest X-rays of all participants were evaluated by the study radiologist to determine the presence of TB lung abnormalities. To assess the relationships between our outcome of interest, prevalent diabetes (HBA1c ≥6.5%), and our exposure of interest, chest X-ray abnormalities, we fitted logistic regression models adjusted for potential clinical and demographic confounders. In secondary analyses, we used the computer-aided detection system CAD4TB, which scores X-rays from 10 to 100 for detection of TB disease, as our exposure interest, and repeated analyses with a comparator group that had no history of TB disease.

RESULTS

In the analytic cohort of people with prior TB (n = 3,276), approximately two-thirds (64.9%) were women, and the average age was 50.8 years (SD 17.4). The prevalence of diabetes was 10.9%, and 53.0% of people were living with HIV. In univariate analyses, there was no association between diabetes prevalence and radiologist chest X-ray abnormalities (OR 1.23, 95%CI 0.95-1.58). In multivariate analyses, the presence of pulmonary abnormalities was associated with an 29% reduction in the odds of prevalent diabetes (aOR 0.71, 95%CI 0.53-0.97, p = 0.030). A similar inverse relationship was observed for diabetes with each 10-unit increase in the CAD4TB chest X-ray scores among people with prior TB (aOR 0.92, 95%CI 0.87-0.97; p = 0.002), but this relationship was less pronounced in the no TB comparator group (aOR 0.96, 95%CI 0.94-0.99).

CONCLUSIONS

Among people with prior TB, pulmonary abnormalities on digital chest X-ray are inversely associated with prevalent diabetes. The severity of radiographic post-TB lung disease does not appear to be a determinant of diabetes in this South African population.

An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis

Background

Computer-aided detection (CAD) algorithms for automated chest X-ray (CXR) reading have been endorsed by the World Health Organization for tuberculosis (TB) triage, but independent, multi-country assessment and comparison of current products are needed to guide implementation.

Methods

We conducted a head-to-head evaluation of five CAD algorithms for TB triage across seven countries. We included CXRs from adults who presented to outpatient facilities with at least two weeks of cough in India, Madagascar, the Philippines, South Africa, Tanzania, Uganda, and Vietnam. The participants completed a standard evaluation for pulmonary TB, including sputum collection for Xpert MTB/RIF Ultra and culture. Against a microbiological reference standard, we calculated and compared the accuracy overall, by country and key groups for five CAD algorithms: CAD4TB (Delft Imaging), INSIGHT CXR (Lunit), DrAid (Vinbrain), Genki (Deeptek), and qXR (qure.AI). We determined the area under the ROC curve (AUC) and if any CAD product could achieve the minimum target accuracy for a TB triage test (≥90% sensitivity and ≥70% specificity). We then applied country- and population-specific thresholds and recalculated accuracy to assess any improvement in performance.

Results

Of 3,927 individuals included, the median age was 41 years (IQR 29-54), 12.9% were people living with HIV (PLWH), 8.2% living with diabetes, and 21.2% had a prior history of TB. The overall AUC ranged from 0.774-0.819, and specificity ranged from 64.8-73.8% at 90% sensitivity. CAD4TB had the highest overall accuracy (73.8% specific, 95% CI 72.2-75.4, at 90% sensitivity), although qXR and INSIGHT CXR also achieved the target 70% specificity. There was heterogeneity in accuracy by country, and females and PLWH had lower sensitivity while males and people with a history of TB had lower specificity. The performance remained stable regardless of diabetes status. When country- and population-specific thresholds were applied, at least one CAD product could achieve or approach the target accuracy for each country and sub-group, except for PLWH and those with a history of TB.

Conclusions

Multiple CAD algorithms can achieve or exceed the minimum target accuracy for a TB triage test, with improvement when using setting- or population-specific thresholds. Further efforts are needed to integrate CAD into routine TB case detection programs in high-burden communities.

AI for Detection of Tuberculosis: Implications for Global Health

Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity. Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning Supplemental material is available for this article. © RSNA, 2024.

Benchmarking the diagnostic test accuracy of certified AI products for screening pulmonary tuberculosis in digital chest radiographs: Preliminary evidence from a rapid review and meta-analysis

BACKGROUND AND OBJECTIVE

The global market for AI systems used in lung tuberculosis (TB) detection has expanded significantly in recent years. Verifying their performance across diverse settings is crucial before medical organisations can invest in them and pursue safe, wide-scale deployment. The goal of this research was to synthesise the clinical evidence for the diagnostic accuracy of certified AI products designed for screening TB in chest X-rays (CXRs) compared to a microbiological reference standard.

METHODS

Four databases were searched between June to September 2022. Data concerning study methodology, system characteristics, and diagnostic accuracy metrics was extracted and summarised. Study bias was evaluated using QUADAS-2 and by examining sources of funding. Forest plots for diagnostic odds ratio (DOR) and summary receiver operating characteristic (SROC) curves were constructed for the AI products individually and collectively.

RESULTS

10 out of 3642 studies satisfied the review criteria however only 8 were subject to meta-analysis following bias assessment. Three AI products were evaluated with a 95 % confidence interval producing the following pooled estimates for accuracy rankings: qXR v2 (sensitivity of 0.944 [0.887-0.973], specificity of 0.692 [0.549-0.805], DOR of 3.63 [3.17-4.09], Lunit INSIGHT CXR v3.1 (sensitivity of 0.853 [0.787-0.901], specificity of 0.646 [0.627-0.665], DOR of 2.37 [1.96-2.78]), and CAD4TB v3.07 (sensitivity of 0.917 [0.848-0.956], specificity of 0.371 [0.336-0.408], DOR of 1.91 [1.4-2.47]). Overall, the products had a sensitivity of 0.903 (0.859-0.934), specificity of 0.526 (0.409-0.641), and DOR of 2.31 (1.78-2.84).

CONCLUSION

Current publicly available evidence indicates considerable variability in the diagnostic accuracy of available AI products although overall they have high sensitivity and modest specificity which is improving with time. These preliminary results are limited by the small number of studies and poor coverage for low TB burden settings. More research is needed to expand the clinical evidence base for the performance of AI products.

Enhancing the reliability and accuracy of AI-enabled diagnosis via complementarity-driven deferral to clinicians

Predictive artificial intelligence (AI) systems based on deep learning have been shown to achieve expert-level identification of diseases in multiple medical imaging settings, but can make errors in cases accurately diagnosed by clinicians and vice versa. We developed Complementarity-Driven Deferral to Clinical Workflow (CoDoC), a system that can learn to decide between the opinion of a predictive AI model and a clinical workflow. CoDoC enhances accuracy relative to clinician-only or AI-only baselines in clinical workflows that screen for breast cancer or tuberculosis (TB). For breast cancer screening, compared to double reading with arbitration in a screening program in the UK, CoDoC reduced false positives by 25% at the same false-negative rate, while achieving a 66% reduction in clinician workload. For TB triaging, compared to standalone AI and clinical workflows, CoDoC achieved a 5-15% reduction in false positives at the same false-negative rate for three of five commercially available predictive AI systems. To facilitate the deployment of CoDoC in novel futuristic clinical settings, we present results showing that CoDoC's performance gains are sustained across several axes of variation (imaging modality, clinical setting and predictive AI system) and discuss the limitations of our evaluation and where further validation would be needed. We provide an open-source implementation to encourage further research and application.

Diagnostic Accuracy of the Artificial Intelligence Methods in Medical Imaging for Pulmonary Tuberculosis: A Systematic Review and Meta-Analysis

Tuberculosis (TB) remains one of the leading causes of death among infectious diseases worldwide. Early screening and diagnosis of pulmonary tuberculosis (PTB) is crucial in TB control, and tend to benefit from artificial intelligence. Here, we aimed to evaluate the diagnostic efficacy of a variety of artificial intelligence methods in medical imaging for PTB. We searched MEDLINE and Embase with the OVID platform to identify trials published update to November 2022 that evaluated the effectiveness of artificial-intelligence-based software in medical imaging of patients with PTB. After data extraction, the quality of studies was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using a bivariate random-effects model. In total, 3987 references were initially identified and 61 studies were finally included, covering a wide range of 124,959 individuals. The pooled sensitivity and the specificity were 91% (95% confidence interval (CI), 89-93%) and 65% (54-75%), respectively, in clinical trials, and 94% (89-96%) and 95% (91-97%), respectively, in model-development studies. These findings have demonstrated that artificial-intelligence-based software could serve as an accurate tool to diagnose PTB in medical imaging. However, standardized reporting guidance regarding AI-specific trials and multicenter clinical trials is urgently needed to truly transform this cutting-edge technology into clinical practice.

eHealth in TB clinical management

BACKGROUND: The constant expansion of internet and mobile technologies has created new opportunities in the field of eHealth, or the digital delivery of healthcare services. This TB meta-analysis aims to examine eHealth and its impact on TB clinical management in order to formulate recommendations for further development.METHODS: A systematic search was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework in PubMed and Embase of articles published up to April 2021. Screening, extraction and quality assessment were performed by two independent researchers. Studies evaluating an internet and/or mobile-based eHealth intervention with an impact on TB clinical management were included. Outcomes were organised following the five domains described in the WHO "Recommendations on Digital Interventions for Health System Strengthening" guideline.RESULTS: Search strategy yielded 3,873 studies, and 89 full texts were finally included. eHealth tended to enhance screening, diagnosis and treatment indicators, while being cost-effective and acceptable to users. The main challenges concern hardware malfunction and software misuse.CONCLUSION: This study offers a broad overview of the innovative field of eHealth applications in TB. Different studies implementing eHealth solutions consistently reported on benefits, but also on specific challenges. eHealth is a promising field of research and could enhance clinical management of TB.

The yield of chest X-ray based versus symptom-based screening among patients with diabetes mellitus in public health facilities in Addis Ababa, Ethiopia

Background

Patients with diabetes mellitus (DM) are at increased risk of developing TB, but the best screening algorithm for early detection and treatment of TB remains unknown. Our objective was to determine if combining routine chest X-ray screening could have a better yield compared with symptom-based screening alone.

Methods

We conducted this cross-sectional study between September 2020 and September 2021 in 26 public health facilities in Addis Ababa, Ethiopia. All DM patients attending the clinics during the study period were offered chest X-ray and symptom screening simultaneously followed by confirmatory Xpert testing. We analyzed the number and proportion of patients with TB by the diagnostic algorithm category and performed binary logistic regression analysis to identify predictors of TB diagnosis.

Results

Of 7394 patients screened, 54.6 % were female, and their median age was 53 years. Type-2 diabetes accounted for 89.6 % of all participants of the patients. Of 172 symptomatic patients, chest X-ray suggested TB in 19, and 11 of these were confirmed to have TB (8 bacterilogicially confirmed and 3 clinically diagnosed). Only 2 of the 152 asymptomatic patients without X-ray findings had TB (both bacteriologically confirmed). X-ray was not done for one patient. On the other hand, 28 of 7222 symptom-negative patients had X-ray findings suggestive of TB, and 7 of these were subsequently confirmed with TB (6 clinically diagnosed). When combined with 8 patients who were on treatment for TB at the time of the screening, the overall point prevalence of TB was 380 per 100,000. The direct cost associated with the X-ray-based screening was 42-times higher.

Conclusion

Chest X-ray led to detection of about a third of TB patients which otherwise would have been missed but the algorithm is more expensive. Its full cost implication needs further economic evaluation.

Diagnostic accuracy of computer aided reading of chest x-ray in screening for pulmonary tuberculosis in comparison with Gene-Xpert

Background & Objectives:

Pakistan ranked fifth amongst 22 high-burden Tuberculosis countries, and it is an epidemic in Pakistan, hence screening is performed nationally, as part of the ambitious ZERO TB drive. Our objective was to assess the diagnostic accuracy of Computer Aided Detection (CAD4TB) software on chest Xray in screening for pulmonary tuberculosis in comparison with gene-Xpert.

Methods:

The study was conducted by Radiology Department Lady Reading Hospital Peshawar in affiliation with Indus Hospital network over a period of one year. Screening was done by using mobile Xray unit equipped with CAD4TB software with scoring system. All of those having score of more than 70 and few selected cases with strong clinical suspicion but score of less than 70 were referred to dedicated TB clinic for Gene-Xpert analysis.

Results:

Among 26,997 individuals screened, 2617 (9.7%) individuals were found presumptive for pulmonary TB. Sputum samples for Gene-Xpert were obtained in 2100 (80.24%) individuals, out of which 1825 (86.9%) were presumptive for pulmonary TB on CAD4TB only. Gene-Xpert was positive in 159 (8.7%) patients and negative in 1,666(91.3%). Sensitivity and specificity of CAD4TB and symptomatology with threshold score of ≥70 was 83.2% and 12.7% respectively keeping Gene-Xpert as gold standard.

Conclusion:

Combination of chest X-ray analysis by CAD4TB and symptomatology is of immense value to screen a large population at risk in a developing high burden country. It is significantly a more effective tool for screening and early diagnosis of TB in individuals, who would otherwise go undiagnosed.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.