1
|
Fang X, Huang S, Yin Y, Chen T, Liao Z, Zhong W. Advancing Underlying Cause of Death Inference Through Wide and Deep Model. China CDC Wkly 2024; 6:487-492. [PMID: 38854462 PMCID: PMC11154107 DOI: 10.46234/ccdcw2024.094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Accepted: 05/13/2024] [Indexed: 06/11/2024] Open
Abstract
Introduction Accurately filling out death certificates is essential for death surveillance. However, manually determining the underlying cause of death is often imprecise. In this study, we investigate the Wide and Deep framework as a method to improve the accuracy and reliability of inferring the underlying cause of death. Methods Death report data from national-level cause of death surveillance sites in Fujian Province from 2016 to 2022, involving 403,547 deaths, were analyzed. The Wide and Deep embedded with Convolutional Neural Networks (CNN) was developed. Model performance was assessed using weighted accuracy, weighted precision, weighted recall, and weighted area under the curve (AUC). A comparison was made with XGBoost, CNN, Gated Recurrent Unit (GRU), Transformer, and GRU with Attention. Results The Wide and Deep achieved strong performance metrics on the test set: precision of 95.75%, recall of 92.08%, F1 Score of 93.78%, and an AUC of 95.99%. The model also displayed specific F1 Scores for different cause-of-death chain lengths: 97.13% for single causes, 95.08% for double causes, 91.24% for triple causes, and 79.50% for quadruple causes. Conclusions The Wide and Deep significantly enhances the ability to determine the root causes of death, providing a valuable tool for improving cause-of-death surveillance quality. Integrating artificial intelligence (AI) in this field is anticipated to streamline death registration and reporting procedures, thereby boosting the precision of public health data.
Collapse
Affiliation(s)
- Xin Fang
- Department for Chronic and Noncommunicable Disease Control and Prevention, Fujian Provincial Center for Disease Control and Prevention, Fuzhou City, Fujian Province, China
| | - Shaofen Huang
- Department for Chronic and Noncommunicable Disease Control and Prevention, Fujian Provincial Center for Disease Control and Prevention, Fuzhou City, Fujian Province, China
| | - Yanrong Yin
- Department for Chronic and Noncommunicable Disease Control and Prevention, Fujian Provincial Center for Disease Control and Prevention, Fuzhou City, Fujian Province, China
| | - Tiehui Chen
- Department for Chronic and Noncommunicable Disease Control and Prevention, Fujian Provincial Center for Disease Control and Prevention, Fuzhou City, Fujian Province, China
| | - Zhijun Liao
- Department of Biochemistry and Molecular Biology, School of Basic Medical Sciences, Fujian Medical University, Fuzhou City, Fujian Province, China
| | - Wenling Zhong
- Department for Chronic and Noncommunicable Disease Control and Prevention, Fujian Provincial Center for Disease Control and Prevention, Fuzhou City, Fujian Province, China
| |
Collapse
|
2
|
Pita Ferreira P, Godinho Simões D, Pinto de Carvalho C, Duarte F, Fernandes E, Casaca Carvalho P, Loff JF, Soares AP, Albuquerque MJ, Pinto-Leite P, Peralta-Santos A. Real-Time Classification of Causes of Death Using AI: Sensitivity Analysis. JMIR AI 2023; 2:e40965. [PMID: 38875558 PMCID: PMC11041420 DOI: 10.2196/40965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 04/21/2023] [Accepted: 06/02/2023] [Indexed: 06/16/2024]
Abstract
BACKGROUND In 2021, the European Union reported >270,000 excess deaths, including >16,000 in Portugal. The Portuguese Directorate-General of Health developed a deep neural network, AUTOCOD, which determines the primary causes of death by analyzing the free text of physicians' death certificates (DCs). Although AUTOCOD's performance has been established, it remains unclear whether its performance remains consistent over time, particularly during periods of excess mortality. OBJECTIVE This study aims to assess the sensitivity and other performance metrics of AUTOCOD in classifying underlying causes of death compared with manual coding to identify specific causes of death during periods of excess mortality. METHODS We included all DCs between 2016 and 2019. AUTOCOD's performance was evaluated by calculating various performance metrics, such as sensitivity, specificity, positive predictive value (PPV), and F1-score, using a confusion matrix. This compared International Statistical Classification of Diseases and Health-Related Problems, 10th Revision (ICD-10), classifications of DCs by AUTOCOD with those by human coders at the Directorate-General of Health (gold standard). Subsequently, we compared periods without excess mortality with periods of excess, severe, and extreme excess mortality. We defined excess mortality as 2 consecutive days with a Z score above the 95% baseline limit, severe excess mortality as 2 consecutive days with a Z score >4 SDs, and extreme excess mortality as 2 consecutive days with a Z score >6 SDs. Finally, we repeated the analyses for the 3 most common ICD-10 chapters focusing on block-level classification. RESULTS We analyzed a large data set comprising 330,098 DCs classified by both human coders and AUTOCOD. AUTOCOD demonstrated high sensitivity (≥0.75) for 10 ICD-10 chapters examined, with values surpassing 0.90 for the more prevalent chapters (chapter II-"Neoplasms," chapter IX-"Diseases of the circulatory system," and chapter X-"Diseases of the respiratory system"), accounting for 67.69% (223,459/330,098) of all human-coded causes of death. No substantial differences were observed in these high-sensitivity values when comparing periods without excess mortality with periods of excess, severe, and extreme excess mortality. The same holds for specificity, which exceeded 0.96 for all chapters examined, and for PPV, which surpassed 0.75 in 9 chapters, including the more prevalent ones. When considering block classification within the 3 most common ICD-10 chapters, AUTOCOD maintained a high performance, demonstrating high sensitivity (≥0.75) for 13 ICD-10 blocks, high PPV for 9 blocks, and specificity of >0.98 in all blocks, with no significant differences between periods without excess mortality and those with excess mortality. CONCLUSIONS Our findings indicate that, during periods of excess and extreme excess mortality, AUTOCOD's performance remains unaffected by potential text quality degradation because of pressure on health services. Consequently, AUTOCOD can be dependably used for real-time cause-specific mortality surveillance even in extreme excess mortality situations.
Collapse
Affiliation(s)
- Patrícia Pita Ferreira
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
- Unidade de Saúde Pública Zé Povinho, Agrupamento de Centros de Saúde do Oeste Norte, Administração Regional de Saúde de Lisboa e Vale do Tejo, Caldas da Rainha, Portugal
- NOVA National School of Public Health, Universidade NOVA de Lisboa, Lisbon, Portugal
| | - Diogo Godinho Simões
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
- Unidade de Saúde Pública Higeia, Agrupamento de Centros de Saúde de Almada-Seixal, Administração Regional de Saúde de Lisboa e Vale do Tejo, Almada, Portugal
| | - Constança Pinto de Carvalho
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
- Unidade de Saúde Pública do Litoral Alentejano, Unidade Local de Saúde do Litoral Alentejano, Administração Regional de Saúde do Alentejo, Santiago do Cacém, Portugal
| | | | - Eugénia Fernandes
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
| | - Pedro Casaca Carvalho
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
| | | | - Ana Paula Soares
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
| | | | - Pedro Pinto-Leite
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
| | - André Peralta-Santos
- Direção de Serviços de Informação e Análise, Direção-Geral da Saúde, Lisbon, Portugal
- NOVA National School of Public Health, Public Health Research Centre, Universidade NOVA de Lisboa, Lisbon, Portugal
- Comprehensive Health Research Centre, Universidade NOVA de Lisboa, Lisbon, Portugal
| |
Collapse
|