|
1
|
Komolafe TE, Cao Y, Nguchu BA, Monkam P, Olaniyi EO, Sun H, Zheng J, Yang X. Diagnostic Test Accuracy of Deep Learning Detection of COVID-19: A Systematic Review and Meta-Analysis. Acad Radiol 2021; 28:1507-1523. [PMID: 34649779 PMCID: PMC8445811 DOI: 10.1016/j.acra.2021.08.008] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 06/18/2021] [Accepted: 08/12/2021] [Indexed: 12/11/2022]
Abstract
RATIONALE AND OBJECTIVE To perform a meta-analysis to compare the diagnostic test accuracy (DTA) of deep learning (DL) in detecting coronavirus disease 2019 (COVID-19), and to investigate how network architecture and type of datasets affect DL performance. MATERIALS AND METHODS We searched PubMed, Web of Science and Inspec from January 1, 2020, to December 3, 2020, for retrospective and prospective studies on deep learning detection with at least reported sensitivity and specificity. Pooled DTA was obtained using random-effect models. Sub-group analysis between studies was also carried out for data source and network architectures. RESULTS The pooled sensitivity and specificity were 91% (95% confidence interval [CI]: 88%, 93%; I2 = 69%) and 92% (95% CI: 88%, 94%; I2 = 88%), respectively for 19 studies. The pooled AUC and diagnostic odds ratio (DOR) were 0.95 (95% CI: 0.88, 0.92) and 112.5 (95% CI: 57.7, 219.3; I2 = 90%) respectively. The overall accuracy, recall, F1-score, LR+ and LR- are 89.5%, 89.5%, 89.7%, 23.13 and 0.13. Sub-group analysis shows that the sensitivity and DOR significantly vary with the type of network architectures and sources of data with low heterogeneity are (I2 = 0%) and (I2 = 18%) for ResNet architecture and single-source datasets, respectively. CONCLUSION The diagnosis of COVID-19 via deep learning has achieved incredible performance, and the source of datasets, as well as network architectures, strongly affect DL performance.
Collapse
Affiliation(s)
- Temitope Emmanuel Komolafe
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Yuzhu Cao
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Benedictor Alexander Nguchu
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Patrice Monkam
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Ebenezer Obaloluwa Olaniyi
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Haotian Sun
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Jian Zheng
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China
| | - Xiaodong Yang
- School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China.
| |
Collapse
|
|
2
|
Hosse C, Büttner L, Fleckenstein FN, Hamper CM, Jonczyk M, Scholz O, Aigner A, Böning G. CT-Based Risk Stratification for Intensive Care Need and Survival in COVID-19 Patients-A Simple Solution. Diagnostics (Basel) 2021; 11:diagnostics11091616. [PMID: 34573957 PMCID: PMC8465083 DOI: 10.3390/diagnostics11091616] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 08/27/2021] [Accepted: 08/31/2021] [Indexed: 12/11/2022] Open
Abstract
We evaluated a simple semi-quantitative (SSQ) method for determining pulmonary involvement in computed tomography (CT) scans of COVID-19 patients. The extent of lung involvement in the first available CT was assessed with the SSQ method and subjectively. We identified risk factors for the need of invasive ventilation, intensive care unit (ICU) admission and for time to death after infection. Additionally, the diagnostic performance of both methods was evaluated. With the SSQ method, a 10% increase in the affected lung area was found to significantly increase the risk for need of ICU treatment with an odds ratio (OR) of 1.68 and for invasive ventilation with an OR of 1.35. Male sex, age, and pre-existing chronic lung disease were also associated with higher risks. A larger affected lung area was associated with a higher instantaneous risk of dying (hazard ratio (HR) of 1.11) independently of other risk factors. SSQ measurement was slightly superior to the subjective approach with an AUC of 73.5% for need of ICU treatment and 72.7% for invasive ventilation. SSQ assessment of the affected lung in the first available CT scans of COVID-19 patients may support early identification of those with higher risks for need of ICU treatment, invasive ventilation, or death.
Collapse
Affiliation(s)
- Clarissa Hosse
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
| | - Laura Büttner
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
- Correspondence:
| | - Florian Nima Fleckenstein
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany;
| | - Christina Maria Hamper
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
| | - Martin Jonczyk
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany;
| | - Oriane Scholz
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
| | - Annette Aigner
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany;
- Institute of Biometry and Clinical Epidemiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 10117 Berlin, Germany
| | - Georg Böning
- Institute of Radiology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, 10117 Berlin, Germany; (C.H.); (F.N.F.); (C.M.H.); (M.J.); (O.S.); (G.B.)
| |
Collapse
|