1
|
Arnaout R, Curran L, Zhao Y, Levine JC, Chinn E, Moon-Grady AJ. An ensemble of neural networks provides expert-level prenatal detection of complex congenital heart disease. Nat Med 2021; 27:882-891. [PMID: 33990806 DOI: 10.1038/s41591-021-01342-5] [Citation(s) in RCA: 75] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2020] [Accepted: 04/08/2021] [Indexed: 12/12/2022]
Abstract
Congenital heart disease (CHD) is the most common birth defect. Fetal screening ultrasound provides five views of the heart that together can detect 90% of complex CHD, but in practice, sensitivity is as low as 30%. Here, using 107,823 images from 1,326 retrospective echocardiograms and screening ultrasounds from 18- to 24-week fetuses, we trained an ensemble of neural networks to identify recommended cardiac views and distinguish between normal hearts and complex CHD. We also used segmentation models to calculate standard fetal cardiothoracic measurements. In an internal test set of 4,108 fetal surveys (0.9% CHD, >4.4 million images), the model achieved an area under the curve (AUC) of 0.99, 95% sensitivity (95% confidence interval (CI), 84-99%), 96% specificity (95% CI, 95-97%) and 100% negative predictive value in distinguishing normal from abnormal hearts. Model sensitivity was comparable to that of clinicians and remained robust on outside-hospital and lower-quality images. The model's decisions were based on clinically relevant features. Cardiac measurements correlated with reported measures for normal and abnormal hearts. Applied to guideline-recommended imaging, ensemble learning models could significantly improve detection of fetal CHD, a critical and global diagnostic challenge.
Collapse
Affiliation(s)
- Rima Arnaout
- Division of Cardiology, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA. .,Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, USA. .,Center for Intelligent Imaging, University of California, San Francisco, San Francisco, CA, USA. .,Biological and Medical Informatics, University of California, San Francisco, San Francisco, CA, USA. .,Chan Zuckerberg Biohub, University of California, San Francisco, San Francisco, CA, USA.
| | - Lara Curran
- Division of Cardiology, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA.,Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, USA
| | - Yili Zhao
- Division of Cardiology, Department of Pediatrics, University of California, San Francisco,, San Francisco, CA, USA
| | - Jami C Levine
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA.,Department of Pediatrics, Harvard School of Medicine, Boston, MA, USA
| | - Erin Chinn
- Division of Cardiology, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA.,Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, USA
| | - Anita J Moon-Grady
- Division of Cardiology, Department of Pediatrics, University of California, San Francisco,, San Francisco, CA, USA
| |
Collapse
|
2
|
Hu WY, Yu YC, Dai LY, Li SY, Zhao BW. Reliability of Sonography-based Volume Computer Aided Diagnosis in the Normal Fetal Heart. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2021; 40:953-962. [PMID: 32856729 DOI: 10.1002/jum.15469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/02/2020] [Revised: 06/25/2020] [Accepted: 07/13/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVES To explore the inter- and intra-observer reliability of Sonography-based Volume Computer Aided Diagnosis (SonoVCAD) in the display of 8 diagnostic planes of fetal echocardiography and to evaluate its efficiency. METHODS Three-dimensional volume data sets of the 56 normal singleton fetuses were acquired from a 4-chamber view by using a volume probe. After processing the data sets by using SonoVCAD, 8 cardiac diagnostic planes were displayed automatically. Three doctors with different experiences of performing fetal echocardiography evaluated each diagnostic plane and the success rates of 8 diagnostic planes were calculated. Inter-observer and intra-observer reliabilities were estimated by Cohen's kappa statistics. RESULTS A total of 276 volume data sets acquired from the 56 normal fetuses were used for SonoVCAD analysis and display. The success rate of each diagnostic section was more than 90%, ranging from 90.6% to 99.6%. Among 276 volumes, 81.5% (225/276) of volumes were able to generate all 8 diagnostic views successfully. Moderate to substantial agreement (kappa, 0.509-0.794) was found between 2 less experienced operators. Moderate to near-perfect agreement (kappa, 0.439-0.933) was found between an expert and 2 less experienced sonographers. Intra-observer reliability was substantial to near-perfect (kappa, 0.602-0.903). The efficiency of SonoVCAD was assessed. The expert spent less time than 2 less experienced examiners (P < 0.001) but no significant difference was found between 2 less experienced examiners (P = 0.176). Besides, SonoVCAD consumed significantly less time than 2-dimensional ultrasound (P < 0.001). CONCLUSIONS SonoVCAD can significantly improve the success rates of 8 diagnostic planes in fetal echocardiography with low operator dependency, good reproducibility and high efficiency.
Collapse
Affiliation(s)
- Wan Yu Hu
- Department of Diagnostic Ultrasound & Echocardiography, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, People's Republic of China
| | - Yan Cheng Yu
- Department of Ultrasonography, Affiliated Hospital of Hangzhou Normal University, Hangzhou, People's Republic of China
| | - Li Ya Dai
- Department of Ultrasonography, Lishui Central Hospital, Lishui, People's Republic of China
| | - Shi Yan Li
- Department of Diagnostic Ultrasound & Echocardiography, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, People's Republic of China
| | - Bo Wen Zhao
- Department of Diagnostic Ultrasound & Echocardiography, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, People's Republic of China
| |
Collapse
|
3
|
Pinto NM, Nelson R, Puchalski M, Metz TD, Smith KJ. Cost-effectiveness of prenatal screening strategies for congenital heart disease. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2014; 44:50-7. [PMID: 24357432 PMCID: PMC5278773 DOI: 10.1002/uog.13287] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/15/2013] [Revised: 11/22/2013] [Accepted: 12/05/2013] [Indexed: 05/13/2023]
Abstract
OBJECTIVE The economic implications of strategies to improve prenatal screening for congenital heart disease (CHD) in low-risk mothers have not been explored. The aim was to perform a cost-effectiveness analysis of different screening methods. METHODS We constructed a decision analytic model of CHD prenatal screening strategies (four-chamber screen (4C), 4C + outflow, nuchal translucency (NT) or fetal echocardiography) populated with probabilities from the literature. The model included whether initial screens were interpreted by a maternal-fetal medicine (MFM) specialist and different referral strategies if they were read by a non-MFM specialist. The primary outcome was the incremental cost per defect detected. Costs were obtained from Medicare National Fee estimates. A probabilistic sensitivity analysis was undertaken on model variables commensurate with their degree of uncertainty. RESULTS In base-case analysis, 4C + outflow referred to an MFM specialist was the least costly strategy per defect detected. The 4C screen and the NT screen were dominated by other strategies (i.e. were more costly and less effective). Fetal echocardiography was the most effective, but most costly. On simulation of 10 000 low-risk pregnancies, 4C + outflow screen referred to an MFM specialist remained the least costly per defect detected. For an additional $580 per defect detected, referral to cardiology after a 4C + outflow was the most cost-effective for the majority of iterations, increasing CHD detection by 13 percentage points. CONCLUSIONS The addition of examination of the outflow tracts to second-trimester ultrasound increases detection of CHD in the most cost-effective manner. Strategies to improve outflow-tract imaging and to refer with the most efficiency may be the best way to improve detection at a population level.
Collapse
Affiliation(s)
- N M Pinto
- Department of Pediatrics, University of Utah, Salt Lake City, UT, USA
| | | | | | | | | |
Collapse
|