Automated verbal autopsy classification: using one-against-all ensemble method and Naïve Bayes classifier

Temporal changes in cause of death among adolescents and adults in six countries in eastern and southern Africa in 1995-2019: a multi-country surveillance study of verbal autopsy data

BACKGROUND

The absence of high-quality comprehensive civil registration and vital statistics systems across many settings in Africa has led to little empirical data on causes of death in the region. We aimed to use verbal autopsy data to provide comparative, population-based estimates of cause-specific mortality among adolescents and adults in eastern and southern Africa.

METHODS

In this surveillance study, we harmonised verbal autopsy and residency data from nine health and demographic surveillance system (HDSS) sites in Kenya, Malawi, Tanzania, South Africa, Uganda, and Zimbabwe, each with variable coverage from Jan 1, 1995, to Dec 31, 2019. We included all deaths to adolescents and adults aged 12 or over that were residents of the study sites and had a verbal autopsy conducted. InSilicoVA, a probabilistic model, was used to assign cause of death on the basis of the signs and symptoms reported in the verbal autopsy. Levels and trends in all-cause and cause-specific mortality rates and cause-specific mortality fractions were calculated, stratified by HDSS site, sex, age, and calendar periods.

FINDINGS

52 484 deaths and 5 157 802 person-years were reported among 1 071 913 individuals across the nine sites during the study period. 47 961 (91·4%) deaths had a verbal autopsy, of which 46 570 (97·1%) were assigned a cause of death. All-cause mortality generally decreased across the HDSS sites during this period, particularly for adults aged 20-59 years. In many of the HDSS sites, these decreases were driven by reductions in HIV and tuberculosis-related deaths. In 2010-14, the top causes of death were: road traffic accidents, HIV or tuberculosis, and meningitis or sepsis in adolescents (12-19 years); HIV or tuberculosis in adults aged 20-59 years; and neoplasms and cardiovascular disease in adults aged 60 years and older. There was greater between-HDSS and between-sex variation in causes of death for adolescents compared with adults.

INTERPRETATION

This study shows progress in reducing mortality across eastern and southern Africa but also highlights age, sex, within-HDSS, and between-HDSS differences in causes of adolescent and adult deaths. These findings highlight the importance of detailed local data to inform health needs to ensure continued improvements in survival.

FUNDING

National Institute of Child Health and Human Development of the US National Institutes of Health.

Evaluation of methods for assigning causes of death from verbal autopsies in India

Background

Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (COD) in countries where medical certification of death is low. Computer-coded verbal autopsy (CCVA), an alternative method to PCVA for assigning the COD is considered to be efficient and cost-effective. However, the performance of CCVA as compared to PCVA is yet to be established in the Indian context.

Methods

We evaluated the performance of PCVA and three CCVA methods i.e., InterVA 5, InSilico, and Tariff 2.0 on verbal autopsies done using the WHO 2016 VA tool on 2,120 reference standard cases developed from five tertiary care hospitals of Delhi. PCVA methodology involved dual independent review with adjudication, where required. Metrics to assess performance were Cause Specific Mortality Fraction (CSMF), sensitivity, positive predictive value (PPV), CSMF Accuracy, and Kappa statistic.

Results

In terms of the measures of the overall performance of COD assignment methods, for CSMF Accuracy, the PCVA method achieved the highest score of 0.79, followed by 0.67 for Tariff_2.0, 0.66 for Inter-VA and 0.62 for InSilicoVA. The PCVA method also achieved the highest agreement (57%) and Kappa scores (0.54). The PCVA method showed the highest sensitivity for 15 out of 20 causes of death.

Conclusion

Our study found that the PCVA method had the best performance out of all the four COD assignment methods that were tested in our study sample. In order to improve the performance of CCVA methods, multicentric studies with larger sample sizes need to be conducted using the WHO VA tool.

Validation studies of verbal autopsy methods: a systematic review

BACKGROUND

Verbal autopsy (VA) has emerged as an increasingly popular technique to assign cause of death in parts of the world where the majority of deaths occur without proper medical certification. The purpose of this study was to examine the key characteristics of studies that have attempted to validate VA cause of death against an established cause of death.

METHODS

A systematic review was conducted by searching the MEDLINE, EMBASE, Cochrane-library, and Scopus electronic databases. Included studies contained 1) a VA component, 2) a validation component, and 3) original analysis or re-analysis. Characteristics of VA studies were extracted. A total of 527 studies were assessed, and 481 studies screened to give 66 studies selected for data extraction.

RESULTS

Sixty-six studies were included from multiple countries. Ten studies used an existing database. Sixteen studies used the World Health Organization VA questionnaire and 5 studies used the Population Health Metrics Research Consortium VA questionnaire. Physician certification was used in 36 studies and computer coded methods were used in 14 studies. Thirty-seven studies used high level comparator data with detailed laboratory investigations.

CONCLUSION

Most studies found VA to be an effective cause of death assignment method and compared VA cause of death to a high-quality established cause of death. Nonetheless, there were inconsistencies in the methodologies of the validation studies, and many used poor quality comparison cause of death data. Future VA validation studies should adhere to consistent methodological criteria so that policymakers can easily interpret the findings to select the most appropriate VA method.

PROSPERO REGISTRATION

CRD42020186886.

Performance evaluation of machine learning and Computer Coded Verbal Autopsy (CCVA) algorithms for cause of death determination: A comparative analysis of data from rural South Africa

Computer Coded Verbal Autopsy (CCVA) algorithms are commonly used to determine the cause of death (CoD) from questionnaire responses extracted from verbal autopsies (VAs). However, they can only operate on structured data and cannot effectively harness information from unstructured VA narratives. Machine Learning (ML) algorithms have also been applied successfully in determining the CoD from VA narratives, allowing the use of auxiliary information that CCVA algorithms cannot directly utilize. However, most ML-based studies only use responses from the structured questionnaire, and the results lack generalisability and comparability across studies. We present a comparative performance evaluation of ML methods and CCVA algorithms on South African VA narratives data, using data from Agincourt Health and Demographic Surveillance Site (HDSS) with physicians' classifications as the gold standard. The data were collected from 1993 to 2015 and have 16,338 cases. The random forest and extreme gradient boosting classifiers outperformed the other classifiers on the combined dataset, attaining accuracy of 96% respectively, with significant statistical differences in algorithmic performance (p < 0.0001). All our models attained Area Under Receiver Operating Characteristics (AUROC) of greater than 0.884. The InterVA CCVA attained 83% Cause Specific Mortality Fraction accuracy and an Overall Chance-Corrected Concordance of 0.36. We demonstrate that ML models could accurately determine the cause of death from VA narratives. Additionally, through mortality trends and pattern analysis, we discovered that in the first decade of the civil registration system in South Africa, the average life expectancy was approximately 50 years. However, in the second decade, life expectancy significantly dropped, and the population was dying at a much younger average age of 40 years, mostly from the leading HIV related causes. Interestingly, in the third decade, we see a gradual improvement in life expectancy, possibly attributed to effective health intervention programmes. Through a structure and semantic analysis of narratives where experts disagree, we also demonstrate the most frequent terms of traditional healer consultations and visits. The comparative approach also makes this study a baseline that can be used for future research enforcing generalization and comparability. Future study will entail exploring deep learning models for CoD classification.