1
|
Blake SR, Das N, Tadepalli M, Reddy B, Singh A, Agrawal R, Chattoraj S, Shah D, Putha P. Using Artificial Intelligence to Stratify Normal versus Abnormal Chest X-rays: External Validation of a Deep Learning Algorithm at East Kent Hospitals University NHS Foundation Trust. Diagnostics (Basel) 2023; 13:3408. [PMID: 37998543 PMCID: PMC10670411 DOI: 10.3390/diagnostics13223408] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Revised: 10/16/2023] [Accepted: 11/02/2023] [Indexed: 11/25/2023] Open
Abstract
Background: The chest radiograph (CXR) is the most frequently performed radiological examination worldwide. The increasing volume of CXRs performed in hospitals causes reporting backlogs and increased waiting times for patients, potentially compromising timely clinical intervention and patient safety. Implementing computer-aided detection (CAD) artificial intelligence (AI) algorithms capable of accurate and rapid CXR reporting could help address such limitations. A novel use for AI reporting is the classification of CXRs as 'abnormal' or 'normal'. This classification could help optimize resource allocation and aid radiologists in managing their time efficiently. Methods: qXR is a CE-marked computer-aided detection (CAD) software trained on over 4.4 million CXRs. In this retrospective cross-sectional pre-deployment study, we evaluated the performance of qXR in stratifying normal and abnormal CXRs. We analyzed 1040 CXRs from various referral sources, including general practices (GP), Accident and Emergency (A&E) departments, and inpatient (IP) and outpatient (OP) settings at East Kent Hospitals University NHS Foundation Trust. The ground truth for the CXRs was established by assessing the agreement between two senior radiologists. Results: The CAD software had a sensitivity of 99.7% and a specificity of 67.4%. The sub-group analysis showed no statistically significant difference in performance across healthcare settings, age, gender, and X-ray manufacturer. Conclusions: The study showed that qXR can accurately stratify CXRs as normal versus abnormal, potentially reducing reporting backlogs and resulting in early patient intervention, which may result in better patient outcomes.
Collapse
Affiliation(s)
- Sarah R. Blake
- East Kent Hospitals University NHS Foundation Trust, Ashford TN24 OLZ, UK;
| | - Neelanjan Das
- East Kent Hospitals University NHS Foundation Trust, Ashford TN24 OLZ, UK;
| | - Manoj Tadepalli
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Bhargava Reddy
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Anshul Singh
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Rohitashva Agrawal
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Subhankar Chattoraj
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Dhruv Shah
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| | - Preetham Putha
- Qure.ai, Mumbai 400063, Maharashtra, India; (M.T.); (B.R.); (A.S.); (R.A.); (D.S.); (P.P.)
| |
Collapse
|
2
|
Mandal S, Sathyamurthy S, Govindarajan A, Dwivedi E, Challa V, Vanapalli P, Prakash R, Modi A, Chouhan R, Putha P, Kothari A, Reddy B. PP01.58 Multi City Opportunistic Screening of Lung Nodules amidst COVID-19. J Thorac Oncol 2023. [DOI: 10.1016/j.jtho.2022.09.084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
|
3
|
Tadepalli M, Chhaparwal A, Chawla S, Srivastava S, Dao T, Chhaparwal A, Naren S, Sathyamurthy S, Mukkavilli S, Putha P, Reddy B, Vo L, Warrier P. PP01.59 Performance of a Deep Learning Algorithm for the Early Detection of Malignant Lung Nodules. J Thorac Oncol 2023. [DOI: 10.1016/j.jtho.2022.09.085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
|
4
|
Govindarajan A, Govindarajan A, Tanamala S, Chattoraj S, Reddy B, Agrawal R, Iyer D, Srivastava A, Kumar P, Putha P. Role of an Automated Deep Learning Algorithm for Reliable Screening of Abnormality in Chest Radiographs: A Prospective Multicenter Quality Improvement Study. Diagnostics (Basel) 2022; 12:2724. [PMID: 36359565 PMCID: PMC9689183 DOI: 10.3390/diagnostics12112724] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 11/01/2022] [Accepted: 11/03/2022] [Indexed: 11/10/2023] Open
Abstract
In medical practice, chest X-rays are the most ubiquitous diagnostic imaging tests. However, the current workload in extensive health care facilities and lack of well-trained radiologists is a significant challenge in the patient care pathway. Therefore, an accurate, reliable, and fast computer-aided diagnosis (CAD) system capable of detecting abnormalities in chest X-rays is crucial in improving the radiological workflow. In this prospective multicenter quality-improvement study, we have evaluated whether artificial intelligence (AI) can be used as a chest X-ray screening tool in real clinical settings. Methods: A team of radiologists used the AI-based chest X-ray screening tool (qXR) as a part of their daily reporting routine to report consecutive chest X-rays for this prospective multicentre study. This study took place in a large radiology network in India between June 2021 and March 2022. Results: A total of 65,604 chest X-rays were processed during the study period. The overall performance of AI achieved in detecting normal and abnormal chest X-rays was good. The high negatively predicted value (NPV) of 98.9% was achieved. The AI performance in terms of area under the curve (AUC), NPV for the corresponding subabnormalities obtained were blunted CP angle (0.97, 99.5%), hilar dysmorphism (0.86, 99.9%), cardiomegaly (0.96, 99.7%), reticulonodular pattern (0.91, 99.9%), rib fracture (0.98, 99.9%), scoliosis (0.98, 99.9%), atelectasis (0.96, 99.9%), calcification (0.96, 99.7%), consolidation (0.95, 99.6%), emphysema (0.96, 99.9%), fibrosis (0.95, 99.7%), nodule (0.91, 99.8%), opacity (0.92, 99.2%), pleural effusion (0.97, 99.7%), and pneumothorax (0.99, 99.9%). Additionally, the turnaround time (TAT) decreased by about 40.63% from pre-qXR period to post-qXR period. Conclusions: The AI-based chest X-ray solution (qXR) screened chest X-rays and assisted in ruling out normal patients with high confidence, thus allowing the radiologists to focus more on assessing pathology on abnormal chest X-rays and treatment pathways.
Collapse
|
5
|
Ebrahimian S, Homayounieh F, Rockenbach MABC, Putha P, Raj T, Dayan I, Bizzo BC, Buch V, Wu D, Kim K, Li Q, Digumarthy SR, Kalra MK. Artificial intelligence matches subjective severity assessment of pneumonia for prediction of patient outcome and need for mechanical ventilation: a cohort study. Sci Rep 2021; 11:858. [PMID: 33441578 PMCID: PMC7807029 DOI: 10.1038/s41598-020-79470-0] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 12/04/2020] [Indexed: 02/08/2023] Open
Abstract
To compare the performance of artificial intelligence (AI) and Radiographic Assessment of Lung Edema (RALE) scores from frontal chest radiographs (CXRs) for predicting patient outcomes and the need for mechanical ventilation in COVID-19 pneumonia. Our IRB-approved study included 1367 serial CXRs from 405 adult patients (mean age 65 ± 16 years) from two sites in the US (Site A) and South Korea (Site B). We recorded information pertaining to patient demographics (age, gender), smoking history, comorbid conditions (such as cancer, cardiovascular and other diseases), vital signs (temperature, oxygen saturation), and available laboratory data (such as WBC count and CRP). Two thoracic radiologists performed the qualitative assessment of all CXRs based on the RALE score for assessing the severity of lung involvement. All CXRs were processed with a commercial AI algorithm to obtain the percentage of the lung affected with findings related to COVID-19 (AI score). Independent t- and chi-square tests were used in addition to multiple logistic regression with Area Under the Curve (AUC) as output for predicting disease outcome and the need for mechanical ventilation. The RALE and AI scores had a strong positive correlation in CXRs from each site (r2 = 0.79-0.86; p < 0.0001). Patients who died or received mechanical ventilation had significantly higher RALE and AI scores than those with recovery or without the need for mechanical ventilation (p < 0.001). Patients with a more substantial difference in baseline and maximum RALE scores and AI scores had a higher prevalence of death and mechanical ventilation (p < 0.001). The addition of patients' age, gender, WBC count, and peripheral oxygen saturation increased the outcome prediction from 0.87 to 0.94 (95% CI 0.90-0.97) for RALE scores and from 0.82 to 0.91 (95% CI 0.87-0.95) for the AI scores. AI algorithm is as robust a predictor of adverse patient outcome (death or need for mechanical ventilation) as subjective RALE scores in patients with COVID-19 pneumonia.
Collapse
Affiliation(s)
- Shadi Ebrahimian
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA.
| | - Fatemeh Homayounieh
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
| | | | - Preetham Putha
- Employee of qure.ai, Level 6, Oberoi Commerz II, Goregaon East, Mumbai, 400063, India
| | - Tarun Raj
- Employee of qure.ai, Level 6, Oberoi Commerz II, Goregaon East, Mumbai, 400063, India
| | - Ittai Dayan
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
- MGH & BWH Center for Clinical Data Science, Boston, MA, USA
| | - Bernardo C Bizzo
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
- MGH & BWH Center for Clinical Data Science, Boston, MA, USA
| | - Varun Buch
- MGH & BWH Center for Clinical Data Science, Boston, MA, USA
| | - Dufan Wu
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
- Gordon Center for Medical Imaging, Bartlett 501, 55 Fruit Street, Boston, MA, 02114, USA
| | - Kyungsang Kim
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
- Gordon Center for Medical Imaging, Bartlett 501, 55 Fruit Street, Boston, MA, 02114, USA
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
- Gordon Center for Medical Imaging, Bartlett 501, 55 Fruit Street, Boston, MA, 02114, USA
| | - Subba R Digumarthy
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital and the Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
| |
Collapse
|
6
|
Singh R, Kalra MK, Nitiwarangkul C, Patti JA, Homayounieh F, Padole A, Rao P, Putha P, Muse VV, Sharma A, Digumarthy SR. Deep learning in chest radiography: Detection of findings and presence of change. PLoS One 2018; 13:e0204155. [PMID: 30286097 PMCID: PMC6171827 DOI: 10.1371/journal.pone.0204155] [Citation(s) in RCA: 101] [Impact Index Per Article: 16.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2018] [Accepted: 09/04/2018] [Indexed: 11/18/2022] Open
Abstract
Background Deep learning (DL) based solutions have been proposed for interpretation of several imaging modalities including radiography, CT, and MR. For chest radiographs, DL algorithms have found success in the evaluation of abnormalities such as lung nodules, pulmonary tuberculosis, cystic fibrosis, pneumoconiosis, and location of peripherally inserted central catheters. Chest radiography represents the most commonly performed radiological test for a multitude of non-emergent and emergent clinical indications. This study aims to assess accuracy of deep learning (DL) algorithm for detection of abnormalities on routine frontal chest radiographs (CXR), and assessment of stability or change in findings over serial radiographs. Methods and findings We processed 874 de-identified frontal CXR from 724 adult patients (> 18 years) with DL (Qure AI). Scores and prediction statistics from DL were generated and recorded for the presence of pulmonary opacities, pleural effusions, hilar prominence, and enlarged cardiac silhouette. To establish a standard of reference (SOR), two thoracic radiologists assessed all CXR for these abnormalities. Four other radiologists (test radiologists), unaware of SOR and DL findings, independently assessed the presence of radiographic abnormalities. A total 724 radiographs were assessed for detection of findings. A subset of 150 radiographs with follow up examinations was used to asses change over time. Data were analyzed with receiver operating characteristics analyses and post-hoc power analysis. Results About 42% (305/ 724) CXR had no findings according to SOR; single and multiple abnormalities were seen in 23% (168/724) and 35% (251/724) of CXR. There was no statistical difference between DL and SOR for all abnormalities (p = 0.2–0.8). The area under the curve (AUC) for DL and test radiologists ranged between 0.837–0.929 and 0.693–0.923, respectively. DL had lowest AUC (0.758) for assessing changes in pulmonary opacities over follow up CXR. Presence of chest wall implanted devices negatively affected the accuracy of DL algorithm for evaluation of pulmonary and hilar abnormalities. Conclusions DL algorithm can aid in interpretation of CXR findings and their stability over follow up CXR. However, in its present version, it is unlikely to replace radiologists due to its limited specificity for categorizing specific findings.
Collapse
Affiliation(s)
- Ramandeep Singh
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Mannudeep K. Kalra
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Chayanin Nitiwarangkul
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
- Division of Diagnostic Radiology, Department of Diagnostic and Therapeutic Radiology, Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | - John A. Patti
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Fatemeh Homayounieh
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Atul Padole
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Pooja Rao
- Qure.ai, 101 Raheja Titanium, Goregaon East, Mumbai, India
| | - Preetham Putha
- Qure.ai, 101 Raheja Titanium, Goregaon East, Mumbai, India
| | - Victorine V. Muse
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Amita Sharma
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
| | - Subba R. Digumarthy
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
- * E-mail:
| |
Collapse
|