5
|
Lee S, Yim JJ, Kwak N, Lee YJ, Lee JK, Lee JY, Kim JS, Kang YA, Jeon D, Jang MJ, Goo JM, Yoon SH. Deep Learning to Determine the Activity of Pulmonary Tuberculosis on Chest Radiographs. Radiology 2021; 301:435-442. [PMID: 34342505 DOI: 10.1148/radiol.2021210063] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Background Determining the activity of pulmonary tuberculosis on chest radiographs is difficult. Purpose To develop a deep learning model to identify active pulmonary tuberculosis on chest radiographs. Materials and Methods Chest radiographs were retrospectively gathered from a multicenter consecutive cohort with pulmonary tuberculosis who were successfully treated between 2011 and 2017, along with normal radiographs to enrich a negative class. The pretreatment and posttreatment radiographs were labeled as positive and negative classes, respectively. A neural network was trained with those radiographs to calculate the probability of active versus healed tuberculosis. A single-center consecutive cohort (test set 1; 89 patients, 148 radiographs) and data from one multicenter randomized controlled trial (test set 2; 366 patients, 3774 radiographs) were used to test the model. The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance of the model and of the four expert readers. Results In total, 6654 pre- and posttreatment radiographs from 3327 patients (mean age ± standard deviation, 55 years ± 19; 1884 men) with pulmonary tuberculosis and 3182 normal radiographs from as many patients (mean age, 53 years ± 14; 1629 men) were gathered. For test set 1, the model showed a higher AUC (0.83; 95% CI: 0.73, 0.89) than one pulmonologist (0.69; 95% CI: 0.61, 0.76; P < .001) and performed similarly to the other readers (AUC, 0.79-0.80; P = .14-.23). For 200 randomly selected radiographs from test set 2, the model had a higher AUC (0.84) than the pulmonologists (0.71 and 0.74; P < .001 and .01, respectively) and performed similarly to the radiologists (0.79 and 0.80; P = .08 and .06, respectively). The model output increased by 0.30 on average with a higher degree of smear positivity (95% CI: 0.20, 0.39; P < .001) and decreased during treatment (baseline, 3 months, and 6 months: 0.85, 0.51, and 0.26, respectively). Conclusion A deep learning model performed similarly to radiologists for accurately determining the activity of pulmonary tuberculosis on chest radiographs; it also was able to follow posttreatment changes. © RSNA, 2021 Online supplemental material is available for this article.
Collapse
Affiliation(s)
- Seowoo Lee
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Jae-Joon Yim
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Nakwon Kwak
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Yeon Joo Lee
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Jung-Kyu Lee
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Ji Yeon Lee
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Ju Sang Kim
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Young Ae Kang
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Doosoo Jeon
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Myoung-Jin Jang
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Jin Mo Goo
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| | - Soon Ho Yoon
- From the Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Chongno-gu, Seoul 03080, Korea (S.L., J.M.G., S.H.Y.); Division of Pulmonary and Critical Medicine, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea (J.J.Y., N.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Korea (Y.J.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea (J.K.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, National Medical Center, Seoul, Korea (J.Y.L.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Korea (J.S.K.); Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea (Y.A.K.); Department of Internal Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea (D.J.); Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Korea (M.J.J.); Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (J.M.G.); and Department of Radiology, UMass Memorial Medical Center, Worcester, Mass (S.H.Y.)
| |
Collapse
|