Accuracy of an automated system for tuberculosis detection on chest radiographs in high-risk screening

Diagnostic Accuracy of Computer-Aided Detection During Active Case Finding for Pulmonary Tuberculosis in Africa: A Systematic Review and Meta-analysis

Background

Computer-aided detection (CAD) may be a useful screening tool for tuberculosis (TB). However, there are limited data about its utility in active case finding (ACF) in a community-based setting, and particularly in an HIV-endemic setting where performance may be compromised.

Methods

We performed a systematic review and evaluated articles published between January 2012 and February 2023 that included CAD as a screening tool to detect pulmonary TB against a microbiological reference standard (sputum culture and/or nucleic acid amplification test [NAAT]). We collected and summarized data on study characteristics and diagnostic accuracy measures. Two reviewers independently extracted data and assessed methodological quality against Quality Assessment of Diagnostic Accuracy Studies-2 criteria. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines were followed.

Results

Of 1748 articles reviewed, 5 met with the eligibility criteria and were included in this review. A meta-analysis revealed pooled sensitivity of 0.87 (95% CI, 0.78-0.96) and specificity of 0.74 (95% CI, 0.55-0.93), just below the World Health Organization (WHO)-recommended target product profile (TPP) for a screening test (sensitivity ≥0.90 and specificity ≥0.70). We found a high risk of bias and applicability concerns across all studies. Subgroup analyses, including the impact of HIV and previous TB, were not possible due to the nature of the reporting within the included studies.

Conclusions

This review provides evidence, specifically in the context of ACF, for CAD as a potentially useful and cost-effective screening tool for TB in a resource-poor HIV-endemic African setting. However, given methodological concerns, caution is required with regards to applicability and generalizability.

Tuberculosis prevention: current strategies and future directions

BACKGROUND

An estimated one fourth of the world's population is infected with Mycobacterium tuberculosis, and 5-10% of those infected develop tuberculosis in their lifetime. Preventing tuberculosis is one of the most underutilized but essential components of curtailing the tuberculosis epidemic. Moreover, current evidence illustrates that tuberculosis manifestations occur along a dynamic spectrum from infection to disease rather than a binary state as historically conceptualized. Elucidating determinants of transition between these states is crucial to decreasing the tuberculosis burden and reaching the END-TB Strategy goals as defined by the WHO. Vaccination, detection of infection, and provision of preventive treatment are key elements of tuberculosis prevention.

OBJECTIVES

This review provides a comprehensive summary of recent evidence and state-of-the-art updates on advancements to prevent tuberculosis in various settings and high-risk populations.

SOURCES

We identified relevant studies in the literature and synthesized the findings to provide an overview of the current state of tuberculosis prevention strategies and latest research developments.

CONTENT

We present the current knowledge and recommendations regarding tuberculosis prevention, with a focus on M. bovis Bacille-Calmette-Guérin vaccination and novel vaccine candidates, tests for latent infection with M. tuberculosis, regimens available for tuberculosis preventive treatment and recommendations in low- and high-burden settings.

IMPLICATIONS

Effective tuberculosis prevention worldwide requires a multipronged approach that addresses social determinants, and improves access to tuberculosis detection and to new short tuberculosis preventive treatment regimens. Robust collaboration and innovative research are needed to reduce the global burden of tuberculosis and develop new detection tools, vaccines, and preventive treatments that serve all populations and ages.

Artificial Intelligence-Based Software with CE Mark for Chest X-ray Interpretation: Opportunities and Challenges

Chest X-ray (CXR) is the most important technique for performing chest imaging, despite its well-known limitations in terms of scope and sensitivity. These intrinsic limitations of CXR have prompted the development of several artificial intelligence (AI)-based software packages dedicated to CXR interpretation. The online database "AI for radiology" was queried to identify CE-marked AI-based software available for CXR interpretation. The returned studies were divided according to the targeted disease. AI-powered computer-aided detection software is already widely adopted in screening and triage for pulmonary tuberculosis, especially in countries with few resources and suffering from high a burden of this disease. AI-based software has also been demonstrated to be valuable for the detection of lung nodules detection, automated flagging of positive cases, and post-processing through the development of digital bone suppression software able to produce digital bone suppressed images. Finally, the majority of available CE-marked software packages for CXR are designed to recognize several findings, with potential differences in sensitivity and specificity for each of the recognized findings.

Diagnostic Accuracy of the Artificial Intelligence Methods in Medical Imaging for Pulmonary Tuberculosis: A Systematic Review and Meta-Analysis

Tuberculosis (TB) remains one of the leading causes of death among infectious diseases worldwide. Early screening and diagnosis of pulmonary tuberculosis (PTB) is crucial in TB control, and tend to benefit from artificial intelligence. Here, we aimed to evaluate the diagnostic efficacy of a variety of artificial intelligence methods in medical imaging for PTB. We searched MEDLINE and Embase with the OVID platform to identify trials published update to November 2022 that evaluated the effectiveness of artificial-intelligence-based software in medical imaging of patients with PTB. After data extraction, the quality of studies was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using a bivariate random-effects model. In total, 3987 references were initially identified and 61 studies were finally included, covering a wide range of 124,959 individuals. The pooled sensitivity and the specificity were 91% (95% confidence interval (CI), 89-93%) and 65% (54-75%), respectively, in clinical trials, and 94% (89-96%) and 95% (91-97%), respectively, in model-development studies. These findings have demonstrated that artificial-intelligence-based software could serve as an accurate tool to diagnose PTB in medical imaging. However, standardized reporting guidance regarding AI-specific trials and multicenter clinical trials is urgently needed to truly transform this cutting-edge technology into clinical practice.

Advances in Deep Learning for Tuberculosis Screening using Chest X-rays: The Last 5 Years Review

There has been an explosive growth in research over the last decade exploring machine learning techniques for analyzing chest X-ray (CXR) images for screening cardiopulmonary abnormalities. In particular, we have observed a strong interest in screening for tuberculosis (TB). This interest has coincided with the spectacular advances in deep learning (DL) that is primarily based on convolutional neural networks (CNNs). These advances have resulted in significant research contributions in DL techniques for TB screening using CXR images. We review the research studies published over the last five years (2016-2021). We identify data collections, methodical contributions, and highlight promising methods and challenges. Further, we discuss and compare studies and identify those that offer extension beyond binary decisions for TB, such as region-of-interest localization. In total, we systematically review 54 peer-reviewed research articles and perform meta-analysis.

Triage of Persons With Tuberculosis Symptoms Using Artificial Intelligence-Based Chest Radiograph Interpretation: A Cost-Effectiveness Analysis

BACKGROUND

In settings without access to rapid expert radiographic interpretation, artificial intelligence (AI)-based chest radiograph (CXR) analysis can triage persons presenting with possible tuberculosis (TB) symptoms, to identify those who require additional microbiological testing. However, there is limited evidence of the cost-effectiveness of this technology as a triage tool.

METHODS

A decision analysis model was developed to evaluate the cost-effectiveness of triage strategies with AI-based CXR analysis for patients presenting with symptoms suggestive of pulmonary TB in Karachi, Pakistan. These strategies were compared to the current standard of care using microbiological testing with smear microscopy or GeneXpert, without prior triage. Positive triage CXRs were considered to improve referral success for microbiologic testing, from 91% to 100% for eligible persons. Software diagnostic accuracy was based on a prospective field study in Karachi. Other inputs were obtained from the Pakistan TB Program. The analysis was conducted from the healthcare provider perspective, and costs were expressed in 2020 US dollars.

RESULTS

Compared to upfront smear microscopy for all persons with presumptive TB, triage strategies with AI-based CXR analysis were projected to lower costs by 19%, from $23233 per 1000 persons, and avert 3%-4% disability-adjusted life-years (DALYs), from 372 DALYs. Compared to upfront GeneXpert, AI-based triage strategies lowered projected costs by 37%, from $34346 and averted 4% additional DALYs, from 369 DALYs. Reinforced follow-up for persons with positive triage CXRs but negative microbiologic tests was particularly cost-effective.

CONCLUSIONS

In lower-resource settings, the addition of AI-based CXR triage before microbiologic testing for persons with possible TB symptoms can reduce costs, avert additional DALYs, and improve TB detection.

Computer-Aided Diagnosis for Pneumoconiosis Staging Based on Multi-scale Feature Mapping

In this research, we explored a method of multi-scale feature mapping to pre-screen radiographs quickly and accurately in the aided diagnosis of pneumoconiosis staging. We utilized an open dataset and a self-collected dataset as research datasets. We proposed a multi-scale feature mapping model based on deep learning feature extraction technology for detecting pulmonary fibrosis and a discrimination method for pneumoconiosis staging. The diagnostic accuracy was evaluated using under the curve (AUC) of the receiver operating characteristic (ROC) curve. The AUC value of our model was 0.84, which showed the best performance compared with previous work on datasets. The diagnosis results indicated that our method was highly consistent with that of clinical experts on real patient. Furthermore, the AUC value obtained through categories I–IV on the testing dataset demonstrated that categories I (AUC = 0.86) and IV (AUC = 0.82) obtained the best performance and achieved the level of clinician categorization. Our research could be applied to the pre-screening and diagnosis of pneumoconiosis in the clinic.

Use of targeted mobile X-ray screening and computer-aided detection software to identify tuberculosis among high-risk groups in Romania: descriptive results of the E-DETECT TB active case-finding project

OBJECTIVE

To implement and assess the mobile X-ray unit (MXU) equipped with digital radiography, computer-aided detection (CAD) software and molecular point of care tests to improve early tuberculosis (TB) diagnosis in vulnerable populations in a TB outreach screening programme in Romania.

DESIGN

Descriptive study.

SETTINGS

Prisons in Bucharest and other cities in the southern part of Romania, homeless shelters and services for problem drug users in Bucharest, and Roma populations in Bucharest and Craiova.

PARTICIPANTS

5510 individuals attended the MXU service; 5003 persons were radiologically screened, 61% prisoners, 15% prison staff, 11% Roma population, 10% homeless persons and/or problem drug users and 3% other.

INTERVENTIONS

Radiological digital chest X-ray (CXR) screening of people at risk for TB, followed by CAD and human reading of the CXRs, and further TB diagnostics when the pulmonologist classified the CXR as suggestive for TB.

PRIMARY AND SECONDARY OUTCOME MEASURES

Ten bacteriologically confirmed TB cases were identified translating into an overall yield of 200 per 100 000 persons screened (95% CIs of 109 to 368 per 100 000). Prevalence rates among homeless persons and/or problem drug users (826/100 000; 95% CI 326 to 2105/100 000) and the Roma population (345/100 000; 95% CI 95 to 1251/100 000) were particularly high.

RESULTS

The human reader classified 6.4% (n=317) of the CXRs as suspect for TB (of which 32 were highly suggestive for TB); 16.3% of all CXRs had a CAD4TB version 6 score >50. All 10 diagnosed TB patients had a CAD4TB score >50; 9 had a CAD4TB score >60.

CONCLUSIONS

Given the high TB prevalence rates found among homeless persons and problem drug users and in the Roma population, targeted active case finding has the potential to deliver a major contribution to TB control in Romania.

Management and control of tuberculosis control in socially complex groups: a research programme including three RCTs

Background

Socially complex groups, including people experiencing homelessness, prisoners and drug users, have very high levels of tuberculosis, often complicated by late diagnosis and difficulty in adhering to treatment.

Objective

To assess a series of interventions to improve tuberculosis control in socially complex groups.

Design

A series of observational surveys, evaluations and trials of interventions.

Setting

The pan-London Find&Treat service, which supports tuberculosis screening and case management in socially complex groups across London.

Participants

Socially complex groups with tuberculosis or at risk of tuberculosis, including people experiencing homelessness, prisoners, drug users and those at high risk of poor adherence to tuberculosis treatment.

Interventions and main outcome measures

We screened 491 people in homeless hostels and 511 people in prison for latent tuberculosis infection, human immunodeficiency virus, hepatitis B and hepatitis C. We evaluated an NHS-led prison radiographic screening programme. We conducted a cluster randomised controlled trial (2348 eligible people experiencing homelessness in 46 hostels) of the effectiveness of peer educators (22 hostels) compared with NHS staff (24 hostels) at encouraging the uptake of mobile radiographic screening. We initiated a trial of the use of point-of-care polymerase chain reaction diagnostics to rapidly confirm tuberculosis alongside mobile radiographic screening. We undertook a randomised controlled trial to improve treatment adherence, comparing face-to-face, directly observed treatment with video-observed treatment using a smartphone application. The primary outcome was completion of ≥ 80% of scheduled treatment observations over the first 2 months following enrolment. We assessed the cost-effectiveness of latent tuberculosis screening alongside radiographic screening of people experiencing homelessness. The costs of video-observed treatment and directly observed treatment were compared.

Results

In the homeless hostels, 16.5% of people experiencing homelessness had latent tuberculosis infection, 1.4% had current hepatitis B infection, 10.4% had hepatitis C infection and 1.0% had human immunodeficiency virus infection. When a quality-adjusted life-year is valued at £30,000, the latent tuberculosis screening of people experiencing homelessness was cost-effective provided treatment uptake was ≥ 25% (for a £20,000 quality-adjusted life-year threshold, treatment uptake would need to be > 50%). In prison, 12.6% of prisoners had latent tuberculosis infection, 1.9% had current hepatitis B infection, 4.2% had hepatitis C infection and 0.0% had human immunodeficiency virus infection. In both settings, levels of latent tuberculosis infection and blood-borne viruses were higher among injecting drug users. A total of 1484 prisoners were screened using chest radiography over a total of 112 screening days (new prisoner screening coverage was 43%). Twenty-nine radiographs were reported as potentially indicating tuberculosis. One prisoner began, and completed, antituberculosis treatment in prison. In the cluster randomised controlled trial of peer educators to increase screening uptake, the median uptake was 45% in the control arm and 40% in the intervention arm (adjusted risk ratio 0.98, 95% confidence interval 0.80 to 1.20). A rapid diagnostic service was established on the mobile radiographic unit but the trial of rapid diagnostics was abandoned because of recruitment and follow-up difficulties. We randomly assigned 112 patients to video-observed treatment and 114 patients to directly observed treatment. Fifty-eight per cent of those recruited had a history of homelessness, addiction, imprisonment or severe mental health problems. Seventy-eight (70%) of 112 patients on video-observed treatment achieved the primary outcome, compared with 35 (31%) of 114 patients on directly observed treatment (adjusted odds ratio 5.48, 95% confidence interval 3.10 to 9.68; p < 0.0001). Video-observed treatment was superior to directly observed treatment in all demographic and social risk factor subgroups. The cost for 6 months of treatment observation was £1645 for daily video-observed treatment, £3420 for directly observed treatment three times per week and £5700 for directly observed treatment five times per week.

Limitations

Recruitment was lower than anticipated for most of the studies. The peer advocate study may have been contaminated by the fact that the service was already using peer educators to support its work.

Conclusions

There are very high levels of latent tuberculosis infection among prisoners, people experiencing homelessness and drug users. Screening for latent infection in people experiencing homelessness alongside mobile radiographic screening would be cost-effective, providing the uptake of treatment was 25–50%. Despite ring-fenced funding, the NHS was unable to establish static radiographic screening programmes. Although we found no evidence that peer educators were more effective than health-care workers in encouraging the uptake of mobile radiographic screening, there may be wider benefits of including peer educators as part of the Find&Treat team. Utilising polymerase chain reaction-based rapid diagnostic testing on a mobile radiographic unit is feasible. Smartphone-enabled video-observed treatment is more effective and cheaper than directly observed treatment for ensuring that treatment is observed.

Future work

Trials of video-observed treatment in high-incidence settings are needed.

Trial registration

Current Controlled Trials ISRCTN17270334 and ISRCTN26184967.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 8, No. 9. See the NIHR Journals Library website for further project information.

Computer aided detection of tuberculosis on chest radiographs: An evaluation of the CAD4TB v6 system

There is a growing interest in the automated analysis of chest X-Ray (CXR) as a sensitive and inexpensive means of screening susceptible populations for pulmonary tuberculosis. In this work we evaluate the latest version of CAD4TB, a commercial software platform designed for this purpose. Version 6 of CAD4TB was released in 2018 and is here tested on a fully independent dataset of 5565 CXR images with GeneXpert (Xpert) sputum test results available (854 Xpert positive subjects). A subset of 500 subjects (50% Xpert positive) was reviewed and annotated by 5 expert observers independently to obtain a radiological reference standard. The latest version of CAD4TB is found to outperform all previous versions in terms of area under receiver operating curve (ROC) with respect to both Xpert and radiological reference standards. Improvements with respect to Xpert are most apparent at high sensitivity levels with a specificity of 76% obtained at a fixed 90% sensitivity. When compared with the radiological reference standard, CAD4TB v6 also outperformed previous versions by a considerable margin and achieved 98% specificity at the 90% sensitivity setting. No substantial difference was found between the performance of CAD4TB v6 and any of the various expert observers against the Xpert reference standard. A cost and efficiency analysis on this dataset demonstrates that in a standard clinical situation, operating at 90% sensitivity, users of CAD4TB v6 can process 132 subjects per day at an average cost per screen of $5.95 per subject, while users of version 3 process only 85 subjects per day at a cost of $8.38 per subject. At all tested operating points version 6 is shown to be more efficient and cost effective than any other version.

Identifying gaps in the quality of latent tuberculosis infection care

Latent tuberculosis infection (LTBI) occurs after transmission and acquisition of infection, when the tuberculosis (TB) bacteria lie dormant in a person. Nearly one-quarter of the world's population is estimated to have LTBI, yet few studies have been published assessing the quality of LTBI services globally. This paper reviews issues to providing patient-centered LTBI services and offers an example framework to formally assess the quality of LTBI patient care. By applying the LTBI cascade of care model, TB programmes can evaluate the gaps and barriers to high-quality care and develop locally-driven solutions to improve LTBI services. Quality care for LTBI must address some of the key challenges to services including: (1) low prioritization of LTBI; (2) gaps in healthcare provider knowledge about testing and treatment; and (3) patient concerns about side effects of preventive treatment regimens. TB programmes need to ensure that these issues are addressed in a patient-centered manner, with clear communication and ongoing evaluation of the quality of LTBI services. Quality LTBI care must be a central focus, particularly identifying and engaging more household contacts in preventive treatment, in order to halt the progression to active disease thereby stopping TB transmission globally.

Deep learning: definition and perspectives for thoracic imaging

Relevance and penetration of machine learning in clinical practice is a recent phenomenon with multiple applications being currently under development. Deep learning-and especially convolutional neural networks (CNNs)-is a subset of machine learning, which has recently entered the field of thoracic imaging. The structure of neural networks, organized in multiple layers, allows them to address complex tasks. For several clinical situations, CNNs have demonstrated superior performance as compared with classical machine learning algorithms and in some cases achieved comparable or better performance than clinical experts. Chest radiography, a high-volume procedure, is a natural application domain because of the large amount of stored images and reports facilitating the training of deep learning algorithms. Several algorithms for automated reporting have been developed. The training of deep learning algorithm CT images is more complex due to the dimension, variability, and complexity of the 3D signal. The role of these methods is likely to increase in clinical practice as a complement of the radiologist's expertise. The objective of this review is to provide definitions for understanding the methods and their potential applications for thoracic imaging. KEY POINTS: • Deep learning outperforms other machine learning techniques for number of tasks in radiology. • Convolutional neural network is the most popular deep learning architecture in medical imaging. • Numerous deep learning algorithms are being currently developed; some of them may become part of clinical routine in the near future.

A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis

We undertook a systematic review of the diagnostic accuracy of artificial intelligence-based software for identification of radiologic abnormalities (computer-aided detection, or CAD) compatible with pulmonary tuberculosis on chest x-rays (CXRs). We searched four databases for articles published between January 2005-February 2019. We summarized data on CAD type, study design, and diagnostic accuracy. We assessed risk of bias with QUADAS-2. We included 53 of the 4712 articles reviewed: 40 focused on CAD design methods (“Development” studies) and 13 focused on evaluation of CAD (“Clinical” studies). Meta-analyses were not performed due to methodological differences. Development studies were more likely to use CXR databases with greater potential for bias as compared to Clinical studies. Areas under the receiver operating characteristic curve (median AUC [IQR]) were significantly higher: in Development studies AUC: 0.88 [0.82–0.90]) versus Clinical studies (0.75 [0.66–0.87]; p-value 0.004); and with deep-learning (0.91 [0.88–0.99]) versus machine-learning (0.82 [0.75–0.89]; p = 0.001). We conclude that CAD programs are promising, but the majority of work thus far has been on development rather than clinical evaluation. We provide concrete suggestions on what study design elements should be improved.

Automatic diagnostics of tuberculosis using convolutional neural networks analysis of MODS digital images

Tuberculosis is an infectious disease that causes ill health and death in millions of people each year worldwide. Timely diagnosis and treatment is key to full patient recovery. The Microscopic Observed Drug Susceptibility (MODS) is a test to diagnose TB infection and drug susceptibility directly from a sputum sample in 7-10 days with a low cost and high sensitivity and specificity, based on the visual recognition of specific growth cording patterns of M. Tuberculosis in a broth culture. Despite its advantages, MODS is still limited in remote, low resource settings, because it requires permanent and trained technical staff for the image-based diagnostics. Hence, it is important to develop alternative solutions, based on reliable automated analysis and interpretation of MODS cultures. In this study, we trained and evaluated a convolutional neural network (CNN) for automatic interpretation of MODS cultures digital images. The CNN was trained on a dataset of 12,510 MODS positive and negative images obtained from three different laboratories, where it achieved 96.63 +/- 0.35% accuracy, and a sensitivity and specificity ranging from 91% to 99%, when validated across held-out laboratory datasets. The model's learned features resemble visual cues used by expert diagnosticians to interpret MODS cultures, suggesting that our model may have the ability to generalize and scale. It performed robustly when validated across held-out laboratory datasets and can be improved upon with data from new laboratories. This CNN can assist laboratory personnel, in low resource settings, and is a step towards facilitating automated diagnostics access to critical areas in developing countries.

Segmentation of lung fields from chest radiographs-a radiomic feature-based approach

Precisely segmented lung fields restrict the region-of-interest from which radiological patterns are searched, and is thus an indispensable prerequisite step in any chest radiographic CADx system. Recently, a number of deep learning-based approaches have been proposed to implement this step. However, deep learning has its own limitations and cannot be used in resource-constrained settings. Medical systems generally have limited RAM, computational power, storage, and no GPUs. They are thus not always suited for running deep learning-based models. Shallow learning-based models with appropriately selected features give comparable performance but with modest resources. The present paper thus proposes a shallow learning-based method that makes use of 40 radiomic features to segment lung fields from chest radiographs. A distance regularized level set evolution (DRLSE) method along with other post-processing steps are used to refine its output. The proposed method is trained and tested using publicly available JSRT dataset. The testing results indicate that the performance of the proposed method is comparable to the state-of-the-art deep learning-based lung field segmentation (LFS) methods and better than other LFS methods.