Real-Time Classification of Causes of Death Using AI: Sensitivity Analysis

BACKGROUND

In 2021, the European Union reported >270,000 excess deaths, including >16,000 in Portugal. The Portuguese Directorate-General of Health developed a deep neural network, AUTOCOD, which determines the primary causes of death by analyzing the free text of physicians' death certificates (DCs). Although AUTOCOD's performance has been established, it remains unclear whether its performance remains consistent over time, particularly during periods of excess mortality.

OBJECTIVE

This study aims to assess the sensitivity and other performance metrics of AUTOCOD in classifying underlying causes of death compared with manual coding to identify specific causes of death during periods of excess mortality.

METHODS

We included all DCs between 2016 and 2019. AUTOCOD's performance was evaluated by calculating various performance metrics, such as sensitivity, specificity, positive predictive value (PPV), and F1-score, using a confusion matrix. This compared International Statistical Classification of Diseases and Health-Related Problems, 10th Revision (ICD-10), classifications of DCs by AUTOCOD with those by human coders at the Directorate-General of Health (gold standard). Subsequently, we compared periods without excess mortality with periods of excess, severe, and extreme excess mortality. We defined excess mortality as 2 consecutive days with a Z score above the 95% baseline limit, severe excess mortality as 2 consecutive days with a Z score >4 SDs, and extreme excess mortality as 2 consecutive days with a Z score >6 SDs. Finally, we repeated the analyses for the 3 most common ICD-10 chapters focusing on block-level classification.

RESULTS

We analyzed a large data set comprising 330,098 DCs classified by both human coders and AUTOCOD. AUTOCOD demonstrated high sensitivity (≥0.75) for 10 ICD-10 chapters examined, with values surpassing 0.90 for the more prevalent chapters (chapter II-"Neoplasms," chapter IX-"Diseases of the circulatory system," and chapter X-"Diseases of the respiratory system"), accounting for 67.69% (223,459/330,098) of all human-coded causes of death. No substantial differences were observed in these high-sensitivity values when comparing periods without excess mortality with periods of excess, severe, and extreme excess mortality. The same holds for specificity, which exceeded 0.96 for all chapters examined, and for PPV, which surpassed 0.75 in 9 chapters, including the more prevalent ones. When considering block classification within the 3 most common ICD-10 chapters, AUTOCOD maintained a high performance, demonstrating high sensitivity (≥0.75) for 13 ICD-10 blocks, high PPV for 9 blocks, and specificity of >0.98 in all blocks, with no significant differences between periods without excess mortality and those with excess mortality.

CONCLUSIONS

Our findings indicate that, during periods of excess and extreme excess mortality, AUTOCOD's performance remains unaffected by potential text quality degradation because of pressure on health services. Consequently, AUTOCOD can be dependably used for real-time cause-specific mortality surveillance even in extreme excess mortality situations.

[Monitoring heat-related mortality in Hesse]

BACKGROUND

Continuous monitoring of the mortality phenomenon is given high priority in the current recommendations for the preparation of heat action plans in Germany with respect to problem detection and evaluation of interventions. International monitoring systems are heterogeneous concerning the procedures used. In Germany, such monitoring systems are rarely established.

OBJECTIVES

Under what circumstances can a mortality monitoring system be operated on a regional basis using routine data?

MATERIALS AND METHODS

Summer mortality data from Hesse from 2000 to 2018 and their associations with climate variables were analyzed. Different approaches regarding spatial analyses, definition of excess criteria, and adjusting procedures were explored.

RESULTS

In Hesse, daily mean temperatures averaged over all operating weather stations proved appropriate as a climate parameter. The expected daily number of deaths was estimated by a moving average based on 25 daily mortality datasets from reference periods of five years adjusted for mortality peaks using data from three previous years. Mortality excess was defined as twice the value of the standard deviation of the expected values including an empirically determined temperature threshold. This threshold was derived from analyzing relative frequencies of observed excess number of deaths per 1 ℃ temperature interval. Based on this approach, 49 mortality excesses with a total of 889 excess deaths were estimated in Hesse during days with a daily mean temperature of more than 23.0 ℃ during summer from 2005 to 2018.

CONCLUSIONS

The system described in this article turned out to be practicable for systematically monitoring mortality during summer. Timely availability of mortality and climate data is crucial.

Implementation of the Montreal heat response plan during the 2010 heat wave

OBJECTIVES

The objective of this paper is to describe Montreal's heat response plan and its application during the July 2010 heat wave.

PARTICIPANTS AND SETTING

The Montreal heat response plan is designed to ensure the surveillance of weather and health indicators during the summer season and to coordinate actions to be undertaken during this period to reduce morbidity and mortality due to heat, particularly when weather thresholds are reached or an increase in health indicators is observed. It was developed to coordinate and apply intervention measures on the Island of Montreal and has been in effect since 2004.

INTERVENTION

In the beginning of July 2010, Montreal experienced a heat wave that lasted 5 days. During this period, health indicators such as total mortality, prehospital emergency transports, calls to the health information line and hospital admissions were monitored by the Montreal public health surveillance system. The decision to implement emergency interventions and actions performed by regional and local public health and municipal partners (intervention level) was made following attainment of a predetermined weather threshold and increases in health indicators. The significant increase in daily observed mortality from all causes and in particular people dying at home or in the community prompted the Director of public health to conduct a chart review of all people deceased from July 5 to July 11, 2010 to determine cause of death and underlying health conditions.

OUTCOME

During the heat wave, there were 304 reported deaths from all causes in Montreal residents, of which 106 were probably or possibly heat-related. Major underlying health conditions in heat-related deaths included cardiovascular problems and mental health illness. Furthermore, in the case of people with mental illness who died during the heat wave, the chart review revealed that many were contacted 24 hours prior to their death by health care professionals, family members, neighbours or friends.

CONCLUSION

Following the 2010 heat wave, the Montreal heat response plan and heat surveillance system were updated to include initiatives to better communicate preventive measures to the vulnerable populations and to intervene earlier during a heat wave.

The effects of temperature on heat-related illness according to the characteristics of patients during the summer of 2012 in the Republic of Korea

Objectives

This study was conducted to investigate the relationship between heat-related illnesses developed in the summer of 2012 and temperature.

Methods

The study analyzed data generated by a heat wave surveillance system operated by the Korea Centers for Disease Control and Prevention during the summer of 2012. The daily maximum temperature, average temperature, and maximum heat index were compared to identify the most suitable index for this study. A piecewise linear model was used to identify the threshold temperature and the relative risk (RR) above the threshold temperature according to patient characteristics and region.

Results

The total number of patients during the 3 months was 975. Of the three temperature indicators, the daily maximum temperature showed the best goodness of fit with the model. The RR of the total patient incidence was 1.691 (1.641 to 1.743) per 1℃ after 31.2℃. The RR above the threshold temperature of women (1.822, 1.716 to 1.934) was greater than that of men (1.643, 1.587 to 1.701). The threshold temperature was the lowest in the age group of 20 to 64 (30.4℃), and the RR was the highest in the ≥65 age group (1.863, 1.755 to 1.978). The threshold temperature of the provinces (30.5℃) was lower than that of the metropolitan cities (32.2℃). Metropolitan cities at higher latitudes had a greater RR than other cities at lower latitudes.

Conclusions

The influences of temperature on heat-related illnesses vary according to gender, age, and region. A surveillance system and public health program should reflect these factors in their implementation.

Emergency ambulance dispatches and apparent temperature: a time series analysis in Emilia-Romagna, Italy

INTRODUCTION

Increases in mortality associated with oppressive weather have been widely investigated in several epidemiological studies. However, to properly understand the full public health significance of heat-related health effects, as well as to develop an effective surveillance system, it is also important to investigate the impact of stressful meteorological conditions on non-fatal events. The objective of our study was to evaluate the exposure-response relationship of ambulance dispatch data in association with biometeorological conditions using time series techniques similar to those used in previous studies on mortality.

METHODS

Daily data of emergency ambulance dispatches for people aged 35 or older in the summer periods from 2002 to 2006 were collected for the major towns in the Emilia-Romagna region. In the first stage of the analysis, the city-specific relationship between daily ambulance dispatches and increasing apparent temperature was explored using Generalized Additive Models while controlling for air pollution, seasonality, long-term trend, holidays and weekends. The relationship between ambulance dispatches and apparent temperature was approximated by linear splines. The effects of high temperatures on health were evaluated for respiratory and cardiovascular diseases as well as for all non-traumatic conditions. In the second stage of the analysis, city-specific effects were combined in fixed or random effect meta-analyses.

RESULTS

The percent change in the ambulance dispatches associated with every 1 °C increase in the mean apparent temperature between 25 and 30 °C was 1.45% (95% confidence interval: 0.95, 1.95) for non-traumatic diseases and 2.74% (95% CI: 1.34, 4.14) for respiratory diseases. The percent increase in risk was greater on days in which the mean apparent temperature exceeded 30 °C (8.85%, 95% CI: 7.12, 10.58 for non-traumatic diseases). In this interval of biometeorological conditions, cardiovascular diseases became positively associated with the apparent temperature. The risks increased with age. The increase in risk for the non-traumatic diseases reached 13.34% for people aged 75 or older compared to 4.75% for those aged 35-64.

CONCLUSION

Time series analysis techniques were adopted for the first time to investigate emergency ambulance dispatches to evaluate the risks associated with biometeorological discomfort. Our findings show a strong relationship between biometeorological conditions and ambulance dispatches.

Excess mortality associated with influenza epidemics in Portugal, 1980 to 2004

Background

Influenza epidemics have a substantial impact on human health, by increasing the mortality from pneumonia and influenza, respiratory and circulatory diseases, and all causes. This paper provides estimates of excess mortality rates associated with influenza virus circulation for 7 causes of death and 8 age groups in Portugal during the period of 1980–2004.

Methodology/Principal Findings

We compiled monthly mortality time series data by age for all-cause mortality, cerebrovascular diseases, ischemic heart diseases, diseases of the respiratory system, chronic respiratory diseases, pneumonia and influenza. We also used a control outcome, deaths from injuries. Age- and cause-specific baseline mortality was modelled by the ARIMA approach; excess deaths attributable to influenza were calculated by subtracting expected deaths from observed deaths during influenza epidemic periods. Influenza was associated with a seasonal average of 24.7 all-cause excess deaths per 100,000 inhabitants, approximately 90% of which were among seniors over 65 yrs. Excess mortality was 3–6 fold higher during seasons dominated by the A(H3N2) subtype than seasons dominated by A(H1N1)/B. High excess mortality impact was also seen in children under the age of four years. Seasonal excess mortality rates from all the studied causes of death were highly correlated with each other (Pearson correlation range, 0.65 to 0.95, P<0.001) and with seasonal rates of influenza-like-illness (ILI) among seniors over 65 years (Pearson correlation rho>0.64, P<0.05). By contrast, there was no correlation with excess mortality from injuries.

Conclusions/Significance

Our excess mortality approach is specific to influenza virus activity and produces influenza-related mortality rates for Portugal that are similar to those published for other countries. Our results indicate that all-cause excess mortality is a robust indicator of influenza burden in Portugal, and could be used to monitor the impact of influenza epidemics in this country. Additional studies are warranted to confirm these findings in other settings.