Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets

Seasonal and temporal trends in all-cause and malaria mortality in rural Burkina Faso, 1998-2007

BACKGROUND

High mortality levels in sub-Saharan Africa are still a major public health problem. Children are the most affected group with malaria as one of the major causes of death in this region. To plan health interventions, reliable empirical information on cause-specific mortality patterns is essential, yet such data are often not available in developing countries. Health and Demographic Surveillance Systems (HDSS) implementing the verbal autopsy (VA) method provide such data on a longitudinal basis. Physician Coded VA is usually used to determine cause of death, but recently a computerized method, Interpreting VA (InterVA) was alternatively introduced. This study investigates the effect of season on all-cause and malaria mortality analysing cause of death data from 1998 to 2007 obtained by the Nouna HDSS in rural Burkina Faso and derived by InterVA.

METHODS

Monthly mortality rates were calculated for different age groups (infants, children, adolescents, adults, elderly). Seasonal and temporal trends were modelled with parametric Poisson regression adjusted for sex, area of residence and year of death.

RESULTS

Overall, 7,378 deaths occurred corresponding to a mortality rate of 11.9/1,000 with highest rates in infants (56.8/1,000) and children (22.0/1,000). Young children were most affected by malaria. Malaria mortality patterns in children showed significantly higher rates during the rainy season and a stagnant long-term trend. The seasonal trend is well described parametrically with a sinusoidal function. InterVA assigned about half as many deaths to malaria than physicians.

CONCLUSIONS

Malaria mortality remains highly seasonal in rural Burkina Faso. The InterVA method appears to determine reasonably well seasonal mortality patterns, which should be considered for the planning of health resources and activities.

Verbal Autopsy: Evaluation of Methods to Certify Causes of Death in Uganda

To assess different methods for determining cause of death from verbal autopsy (VA) questionnaire data, the intra-rater reliability of Physician-Certified Verbal Autopsy (PCVA) and the accuracy of PCVA, expert-derived (non-hierarchical) and data-driven (hierarchal) algorithms were assessed for determining common causes of death in Ugandan children. A verbal autopsy validation study was conducted from 2008-2009 in three different sites in Uganda. The dataset included 104 neonatal deaths (0-27 days) and 615 childhood deaths (1-59 months) with the cause(s) of death classified by PCVA and physician review of hospital medical records (the 'reference standard'). Of the original 719 questionnaires, 141 (20%) were selected for a second review by the same physicians; the repeat cause(s) of death were compared to the original,and agreement assessed using the Kappa statistic.Physician reviewers' refined non-hierarchical algorithms for common causes of death from existing expert algorithms, from which, hierarchal algorithms were developed. The accuracy of PCVA, non-hierarchical, and hierarchical algorithms for determining cause(s) of death from all 719 VA questionnaires was determined using the reference standard. Overall, intra-rater repeatability was high (83% agreement, Kappa 0.79 [95% CI 0.76-0.82]). PCVA performed well, with high specificity for determining cause of neonatal (>67%), and childhood (>83%) deaths, resulting in fairly accurate cause-specific mortality fraction (CSMF) estimates. For most causes of death in children, non-hierarchical algorithms had higher sensitivity, but correspondingly lower specificity, than PCVA and hierarchical algorithms, resulting in inaccurate CSMF estimates. Hierarchical algorithms were specific for most causes of death, and CSMF estimates were comparable to the reference standard and PCVA. Inter-rater reliability of PCVA was high, and overall PCVA performed well. Hierarchical algorithms performed better than non-hierarchical algorithms due to higher specificity and more accurate CSMF estimates. Use of PCVA to determine cause of death from VA questionnaire data is reasonable while automated data-driven algorithms are improved.

Two-year survival following discharge from hospital after spinal cord injury in Bangladesh

STUDY DESIGN

Mixed retrospective-prospective cohort study.

OBJECTIVES

To determine 2-year survival following discharge from hospital after spinal cord injury in Bangladesh.

SETTING

Bangladesh.

METHODS

Medical records were used to identify all patients admitted in 2011 with a recent spinal cord injury to the Centre for Rehabilitation of the Paralysed, a large Bangladeshi hospital that specialises in care of people with spinal cord injury. Patients or their families were subsequently visited or contacted by telephone in 2014. Vital status and, where relevant, date and cause of death were determined by verbal autopsy.

RESULTS

350 of 371 people admitted with a recent spinal cord injury in 2011 were discharged alive from hospital. All but eleven were accounted for two years after discharge (97% follow-up). Two-year survival was 87% (95% CI 83% to 90%). Two-year survival of those who were wheelchair-dependent was 81% (95% CI 76% to 86%). The most common cause of death was sepsis due to pressure ulcers.

CONCLUSION

In Bangladesh, approximately one in five people with spinal cord injury who are wheelchair-dependent die within two years of discharge from hospital. Most deaths are due to sepsis from potentially preventable pressure ulcers.

Direct estimates of national neonatal and child cause-specific mortality proportions in Niger by expert algorithm and physician-coded analysis of verbal autopsy interviews

BACKGROUND

This study was one of a set of verbal autopsy investigations undertaken by the WHO/UNCEF-supported Child Health Epidemiology Reference Group (CHERG) to derive direct estimates of the causes of neonatal and child deaths in high priority countries of sub-Saharan Africa. The objective of the study was to determine the cause distributions of neonatal (0-27 days) and child (1-59 months) mortality in Niger.

METHODS

Verbal autopsy interviews were conducted of random samples of 453 neonatal deaths and 620 child deaths from 2007 to 2010 identified by the 2011 Niger National Mortality Survey. The cause of each death was assigned using two methods: computerized expert algorithms arranged in a hierarchy and physician completion of a death certificate for each child. The findings of the two methods were compared to each other, and plausibility checks were conducted to assess which is the preferred method. Comparison of some direct measures from this study with CHERG modeled cause of death estimates are discussed.

FINDINGS

The cause distributions of neonatal deaths as determined by expert algorithms and the physician were similar, with the same top three causes by both methods and all but two other causes within one rank of each other. Although child causes of death differed more, the reasons often could be discerned by analyzing algorithmic criteria alongside the physician's application of required minimal diagnostic criteria. Including all algorithmic (primary and co-morbid) and physician (direct, underlying and contributing) diagnoses in the comparison minimized the differences, with kappa coefficients greater than 0.40 for five of 11 neonatal diagnoses and nine of 13 child diagnoses. By algorithmic diagnosis, early onset neonatal infection was significantly associated (χ(2) = 13.2, P < 0.001) with maternal infection, and the geographic distribution of child meningitis deaths closely corresponded with that for meningitis surveillance cases and deaths.

CONCLUSIONS

Verbal autopsy conducted in the context of a national mortality survey can provide useful estimates of the cause distributions of neonatal and child deaths. While the current study found reasonable agreement between the expert algorithm and physician analyses, it also demonstrated greater plausibility for two algorithmic diagnoses and validation work is needed to ascertain the findings. Direct, large-scale measurement of causes of death complement, can strengthen, and in some settings may be preferred over modeled estimates.

Global, regional, and national causes of child mortality in 2000-13, with projections to inform post-2015 priorities: an updated systematic analysis

BACKGROUND

Trend data for causes of child death are crucial to inform priorities for improving child survival by and beyond 2015. We report child mortality by cause estimates in 2000-13, and cause-specific mortality scenarios to 2030 and 2035.

METHODS

We estimated the distributions of causes of child mortality separately for neonates and children aged 1-59 months. To generate cause-specific mortality fractions, we included new vital registration and verbal autopsy data. We used vital registration data in countries with adequate registration systems. We applied vital registration-based multicause models for countries with low under-5 mortality but inadequate vital registration, and updated verbal autopsy-based multicause models for high mortality countries. We used updated numbers of child deaths to derive numbers of deaths by causes. We applied two scenarios to derive cause-specific mortality in 2030 and 2035.

FINDINGS

Of the 6·3 million children who died before age 5 years in 2013, 51·8% (3·257 million) died of infectious causes and 44% (2·761 million) died in the neonatal period. The three leading causes are preterm birth complications (0·965 million [15·4%, uncertainty range (UR) 9·8-24·5]; UR 0·615-1·537 million), pneumonia (0·935 million [14·9%, 13·0-16·8]; 0·817-1·057 million), and intrapartum-related complications (0·662 million [10·5%, 6·7-16·8]; 0·421-1·054 million). Reductions in pneumonia, diarrhoea, and measles collectively were responsible for half of the 3·6 million fewer deaths recorded in 2013 versus 2000. Causes with the slowest progress were congenital, preterm, neonatal sepsis, injury, and other causes. If present trends continue, 4·4 million children younger than 5 years will still die in 2030. Furthermore, sub-Saharan Africa will have 33% of the births and 60% of the deaths in 2030, compared with 25% and 50% in 2013, respectively.

INTERPRETATION

Our projection results provide concrete examples of how the distribution of child causes of deaths could look in 15-20 years to inform priority setting in the post-2015 era. More evidence is needed about shifts in timing, causes, and places of under-5 deaths to inform child survival agendas by and beyond 2015, to end preventable child deaths in a generation, and to count and account for every newborn and every child.

FUNDING

Bill & Melinda Gates Foundation.

Validating estimates of prevalence of non-communicable diseases based on household surveys: the symptomatic diagnosis study

BACKGROUND

Easy-to-collect epidemiological information is critical for the more accurate estimation of the prevalence and burden of different non-communicable diseases around the world. Current measurement is restricted by limitations in existing measurement systems in the developing world and the lack of biometry tests for non-communicable diseases. Diagnosis based on self-reported signs and symptoms ("Symptomatic Diagnosis," or SD) analyzed with computer-based algorithms may be a promising method for collecting timely and reliable information on non-communicable disease prevalence. The objective of this study was to develop and assess the performance of a symptom-based questionnaire to estimate prevalence of non-communicable diseases in low-resource areas.

METHODS

As part of the Population Health Metrics Research Consortium study, we collected 1,379 questionnaires in Mexico from individuals who suffered from a non-communicable disease that had been diagnosed with gold standard diagnostic criteria or individuals who did not suffer from any of the 10 target conditions. To make the diagnosis of non-communicable diseases, we selected the Tariff method, a technique developed for verbal autopsy cause of death calculation. We assessed the performance of this instrument and analytical techniques at the individual and population levels.

RESULTS

The questionnaire revealed that the information on health care experience retrieved achieved 66.1% (95% uncertainty interval [UI], 65.6-66.5%) chance corrected concordance with true diagnosis of non-communicable diseases using health care experience and 0.826 (95% UI, 0.818-0.834) accuracy in its ability to calculate fractions of different causes. SD is also capable of outperforming the current estimation techniques for conditions estimated by questionnaire-based methods.

CONCLUSIONS

SD is a viable method for producing estimates of the prevalence of non-communicable diseases in areas with low health information infrastructure. This technology can provide higher-resolution prevalence data, more flexible data collection, and potentially individual diagnoses for certain conditions.

Malaria mortality in Africa and Asia: evidence from INDEPTH health and demographic surveillance system sites

Background

Malaria continues to be a major cause of infectious disease mortality in tropical regions. However, deaths from malaria are most often not individually documented, and as a result overall understanding of malaria epidemiology is inadequate. INDEPTH Network members maintain population surveillance in Health and Demographic Surveillance System sites across Africa and Asia, in which individual deaths are followed up with verbal autopsies.

Objective

To present patterns of malaria mortality determined by verbal autopsy from INDEPTH sites across Africa and Asia, comparing these findings with other relevant information on malaria in the same regions.

Design

From a database covering 111,910 deaths over 12,204,043 person-years in 22 sites, in which verbal autopsy data were handled according to the WHO 2012 standard and processed using the InterVA-4 model, over 6,000 deaths were attributed to malaria. The overall period covered was 1992–2012, but two-thirds of the observations related to 2006–2012. These deaths were analysed by site, time period, age group and sex to investigate epidemiological differences in malaria mortality.

Results

Rates of malaria mortality varied by 1:10,000 across the sites, with generally low rates in Asia (one site recording no malaria deaths over 0.5 million person-years) and some of the highest rates in West Africa (Nouna, Burkina Faso: 2.47 per 1,000 person-years). Childhood malaria mortality rates were strongly correlated with Malaria Atlas Project estimates of Plasmodium falciparum parasite rates for the same locations. Adult malaria mortality rates, while lower than corresponding childhood rates, were strongly correlated with childhood rates at the site level.

Conclusions

The wide variations observed in malaria mortality, which were nevertheless consistent with various other estimates, suggest that population-based registration of deaths using verbal autopsy is a useful approach to understanding the details of malaria epidemiology.

Cause of death among infants in rural western China: a community-based study using verbal autopsy

OBJECTIVES

To determine the causes of death among infants in high-mortality areas of western China with the use of globally recognized methods.

STUDY DESIGN

A survey of all infant deaths identified over 1 year in 4 counties in Yunnan and Xinjiang in which combined verbal autopsy was combined with a physician's diagnosis of the cause to calculate the local infant mortality rate.

RESULTS

Among 470 completed investigations, a cause of death was assigned to 423 cases (90%). Overall, pneumonia (34.5%), preterm birth complications (16.5%), diarrhea (10.4%), birth asphyxia (10.3%), and congenital abnormalities (8.5%) were the main causes, with 56.6% of deaths occurring in the neonatal period. Deaths were attributable predominantly to prematurity or birth asphyxia in the early neonatal period, whereas infection accounted for more than 60% and 80% of deaths in the late and postneonatal periods, respectively. Calculated infant mortality was 21.9 in 1000 live births.

CONCLUSIONS

The pattern of infant mortality observed in the surveyed counties differs markedly from that reported previously at the national level, with a high proportion the result of causes that may be preventable with globally recommended interventions. Financial and political support is needed to promote improved cause of death surveillance and newborn and infant health care in China's western region.

Assessing the reliability of causes of death reported by the Vital Registration System in Sri Lanka: medical records review in Colombo

Information on causes of death is critical for informed decision making in the health sector. This paper reports findings from a study that measured the accuracy of registered causes of death and quality of medical records for a sample of deaths occurring in hospitals in Colombo, Sri Lanka. Five physicians, trained in medical certification of cause of death, reconstructed death certificates for hospital deaths from medical records and assessed the quality of medical records for this purpose. The majority of medical records were found to be of average quality. Concordance between the underlying cause of death in the vital registration data and that from the 'gold standard' (medical records review) diagnosis was 41.4% (n=249). The sensitivity of all leading causes of death and positive predictive value were below 67%. Major misclassification errors were found in identifying deaths due to vascular diseases and diabetes mellitus. Certified causes of death in Sri Lanka are frequently incorrect, thus limiting their value for health policy and for monitoring progress towards development goals. Sri Lanka, and other countries at a similar level of statistical development, should consider periodically conducting research to evaluate the quality of cause of death reporting at both local and national levels.

Verbal autopsy as a tool for identifying children dying of sickle cell disease: a validation study conducted in Kilifi district, Kenya

BACKGROUND

Sickle cell disease (SCD) is common in many parts of sub-Saharan Africa (SSA), where it is associated with high early mortality. In the absence of newborn screening, most deaths among children with SCD go unrecognized and unrecorded. As a result, SCD does not receive the attention it deserves as a leading cause of death among children in SSA. In the current study, we explored the potential utility of verbal autopsy (VA) as a tool for attributing underlying cause of death (COD) in children to SCD.

METHODS

We used the 2007 WHO Sample Vital Registration with Verbal Autopsy (SAVVY) VA tool to determine COD among child residents of the Kilifi Health and Demographic Surveillance System (KHDSS), Kenya, who died between January 2008 and April 2011. VAs were coded both by physician review (physician coded verbal autopsy, PCVA) using COD categories based on the WHO International Classification of Diseases 10th Edition (ICD-10) and by using the InterVA-4 probabilistic model after extracting data according to the 2012 WHO VA standard. Both of these methods were validated against one of two gold standards: hospital ICD-10 physician-assigned COD for children who died in Kilifi District Hospital (KDH) and, where available, laboratory confirmed SCD status for those who died in the community.

RESULTS

Overall, 6% and 5% of deaths were attributed to SCD on the basis of PCVA and the InterVA-4 model, respectively. Of the total deaths, 22% occurred in hospital, where the agreement coefficient (AC1) for SCD between PCVA and hospital physician diagnosis was 95.5%, and agreement between InterVA-4 and hospital physician diagnosis was 96.9%. Confirmatory laboratory evidence of SCD status was available for 15% of deaths, in which the AC1 against PCVA was 87.5%.

CONCLUSIONS

Other recent studies and provisional data from this study, outlining the importance of SCD as a cause of death in children in many parts of the developing world, contributed to the inclusion of specific SCD questions in the 2012 version of the WHO VA instruments, and a specific code for SCD has now been included in the WHO and InterVA-4 COD listings. With these modifications, VA may provide a useful approach to quantifying the contribution of SCD to childhood mortality in rural African communities. Further studies will be needed to evaluate the generalizability of our findings beyond our local context.

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods

BACKGROUND

Verbal Autopsy (VA) is widely viewed as the only immediate strategy for registering cause of death in much of Africa and Asia, where routine physician certification of deaths is not widely practiced. VA involves a lay interview with family or friends after a death, to record essential details of the circumstances. These data can then be processed automatically to arrive at standardized cause of death information.

METHODS

The Population Health Metrics Research Consortium (PHMRC) undertook a study at six tertiary hospitals in low- and middle-income countries which documented over 12,000 deaths clinically and subsequently undertook VA interviews. This dataset, now in the public domain, was compared with the WHO 2012 VA standard and the InterVA-4 interpretative model.

RESULTS

The PHMRC data covered 70% of the WHO 2012 VA input indicators, and categorized cause of death according to PHMRC definitions. After eliminating some problematic or incomplete records, 11,984 VAs were compared. Some of the PHMRC cause definitions, such as 'preterm delivery', differed substantially from the International Classification of Diseases, version 10 equivalent. There were some appreciable inconsistencies between the hospital and VA data, including 20% of the hospital maternal deaths being described as non-pregnant in the VA data. A high proportion of VA cases (66%) reported respiratory symptoms, but only 18% of assigned hospital causes were respiratory-related. Despite these issues, the concordance correlation coefficient between hospital and InterVA-4 cause of death categories was 0.61.

CONCLUSIONS

The PHMRC dataset is a valuable reference source for VA methods, but has to be interpreted with care. Inherently inconsistent cases should not be included when using these data to build other VA models. Conversely, models built from these data should be independently evaluated. It is important to distinguish between the internal and external validity of VA models. The effects of using tertiary hospital data, rather than the more usual application of VA to all-community deaths, are hard to evaluate. However, it would still be of value for VA method development to have further studies of population-based post-mortem examinations.

Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review

BACKGROUND

Computer-coded verbal autopsy (CCVA) methods to assign causes of death (CODs) for medically unattended deaths have been proposed as an alternative to physician-certified verbal autopsy (PCVA). We conducted a systematic review of 19 published comparison studies (from 684 evaluated), most of which used hospital-based deaths as the reference standard. We assessed the performance of PCVA and five CCVA methods: Random Forest, Tariff, InterVA, King-Lu, and Simplified Symptom Pattern.

METHODS

The reviewed studies assessed methods' performance through various metrics: sensitivity, specificity, and chance-corrected concordance for coding individual deaths, and cause-specific mortality fraction (CSMF) error and CSMF accuracy at the population level. These results were summarized into means, medians, and ranges.

RESULTS

The 19 studies ranged from 200 to 50,000 deaths per study (total over 116,000 deaths). Sensitivity of PCVA versus hospital-assigned COD varied widely by cause, but showed consistently high specificity. PCVA and CCVA methods had an overall chance-corrected concordance of about 50% or lower, across all ages and CODs. At the population level, the relative CSMF error between PCVA and hospital-based deaths indicated good performance for most CODs. Random Forest had the best CSMF accuracy performance, followed closely by PCVA and the other CCVA methods, but with lower values for InterVA-3.

CONCLUSIONS

There is no single best-performing coding method for verbal autopsies across various studies and metrics. There is little current justification for CCVA to replace PCVA, particularly as physician diagnosis remains the worldwide standard for clinical diagnosis on live patients. Further assessments and large accessible datasets on which to train and test combinations of methods are required, particularly for rural deaths without medical attention.

Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries

BACKGROUND

Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.

METHODS

We compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.

RESULTS

The positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).

CONCLUSIONS

On an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs.

Using verbal autopsy to measure causes of death: the comparative performance of existing methods

BACKGROUND

Monitoring progress with disease and injury reduction in many populations will require widespread use of verbal autopsy (VA). Multiple methods have been developed for assigning cause of death from a VA but their application is restricted by uncertainty about their reliability.

METHODS

We investigated the validity of five automated VA methods for assigning cause of death: InterVA-4, Random Forest (RF), Simplified Symptom Pattern (SSP), Tariff method (Tariff), and King-Lu (KL), in addition to physician review of VA forms (PCVA), based on 12,535 cases from diverse populations for which the true cause of death had been reliably established. For adults, children, neonates and stillbirths, performance was assessed separately for individuals using sensitivity, specificity, Kappa, and chance-corrected concordance (CCC) and for populations using cause specific mortality fraction (CSMF) accuracy, with and without additional diagnostic information from prior contact with health services. A total of 500 train-test splits were used to ensure that results are robust to variation in the underlying cause of death distribution.

RESULTS

Three automated diagnostic methods, Tariff, SSP, and RF, but not InterVA-4, performed better than physician review in all age groups, study sites, and for the majority of causes of death studied. For adults, CSMF accuracy ranged from 0.764 to 0.770, compared with 0.680 for PCVA and 0.625 for InterVA; CCC varied from 49.2% to 54.1%, compared with 42.2% for PCVA, and 23.8% for InterVA. For children, CSMF accuracy was 0.783 for Tariff, 0.678 for PCVA, and 0.520 for InterVA; CCC was 52.5% for Tariff, 44.5% for PCVA, and 30.3% for InterVA. For neonates, CSMF accuracy was 0.817 for Tariff, 0.719 for PCVA, and 0.629 for InterVA; CCC varied from 47.3% to 50.3% for the three automated methods, 29.3% for PCVA, and 19.4% for InterVA. The method with the highest sensitivity for a specific cause varied by cause.

CONCLUSIONS

Physician review of verbal autopsy questionnaires is less accurate than automated methods in determining both individual and population causes of death. Overall, Tariff performs as well or better than other methods and should be widely applied in routine mortality surveillance systems with poor cause of death certification practices.

Revising the WHO verbal autopsy instrument to facilitate routine cause-of-death monitoring

Objective

Verbal autopsy (VA) is a systematic approach for determining causes of death (CoD) in populations without routine medical certification. It has mainly been used in research contexts and involved relatively lengthy interviews. Our objective here is to describe the process used to shorten, simplify, and standardise the VA process to make it feasible for application on a larger scale such as in routine civil registration and vital statistics (CRVS) systems.

Methods

A literature review of existing VA instruments was undertaken. The World Health Organization (WHO) then facilitated an international consultation process to review experiences with existing VA instruments, including those from WHO, the Demographic Evaluation of Populations and their Health in Developing Countries (INDEPTH) Network, InterVA, and the Population Health Metrics Research Consortium (PHMRC). In an expert meeting, consideration was given to formulating a workable VA CoD list [with mapping to the International Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) CoD] and to the viability and utility of existing VA interview questions, with a view to undertaking systematic simplification.

Findings

A revised VA CoD list was compiled enabling mapping of all ICD-10 CoD onto 62 VA cause categories, chosen on the grounds of public health significance as well as potential for ascertainment from VA. A set of 221 indicators for inclusion in the revised VA instrument was developed on the basis of accumulated experience, with appropriate skip patterns for various population sub-groups. The duration of a VA interview was reduced by about 40% with this new approach.

Conclusions

The revised VA instrument resulting from this consultation process is presented here as a means of making it available for widespread use and evaluation. It is envisaged that this will be used in conjunction with automated models for assigning CoD from VA data, rather than involving physicians.

Global burden of alcoholic liver diseases

Liver diseases contribute markedly to the global burden of mortality and disease. This paper provides an overview from a global perspective of the contribution of alcohol to liver diseases. The Global Burden of Disease study methodology was used to estimate the burden of alcohol-attributable liver cirrhosis and alcohol-attributable liver cancer in 2010 as measured by deaths and disability adjusted life years (DALYs). This methodology estimates attributable fractions based on alcohol exposure distribution and relative risks associated with different levels of drinking. Globally, in 2010, alcohol-attributable liver cirrhosis was responsible for 493,300 deaths (156,900 female deaths and 336,400 male deaths) and 14,544,000 DALYs (4,112,000 DALYs for women and 10,432,000 DALYs for men), representing 0.9% (0.7% for women and 1.2% for men) of all global deaths and 0.6% (0.4% for women and 0.8% for men) of all global DALYs, and 47.9% of all liver cirrhosis deaths (46.5% for women and 48.5% for men) and 46.9% of all liver cirrhosis DALYs (44.5% for women and 47.9% for men). Alcohol-attributable liver cancer was responsible for 80,600 deaths (14,800 female deaths and 65,900 male deaths) and 2,142,000 DALYs (335,000 DALYs for women and 1,807,000 DALYs for men). The burden of alcohol-attributable liver cirrhosis and liver cancer is high and entirely preventable. Interventions to reduce alcohol consumption are recommended as a population health priority and may range from taxation increases for alcoholic beverages to increases in screening and treatment rates for alcohol use disorders.

Infant illness spanning the antenatal to early neonatal continuum in rural northern Ghana: local perceptions, beliefs and practices

OBJECTIVE

To explore community understanding of perinatal illness in northern Ghana.

STUDY DESIGN

A cross-sectional descriptive study design.

RESULT

253 community members participated in in-depth interviews and focus group discussions, including women with newborn infants, grandmothers and health care providers. Four overarching themes emerged: (1) Local understanding of illness affects treatment practices. Respondents recognized danger signs of illness spanning antenatal to early neonatal periods. Understanding of causation often had a distinctly local flavor, and thus treatment sometimes differed from mainstream recommendations; (2) Mothers are frequently blamed for their infant's illness; (3) Healthcare decisions regarding infant care are often influenced by community members aside from the infant's mother and (4) Confidence in healthcare providers is issue-specific, and many households use a blended approach to meet their health needs.

CONCLUSION

Despite widespread recognition of danger signs and reported intentions to treat ill infants through the formal health care system, traditional approaches to perinatal illness remain common. Interventions need to be aligned with community perceptions if they are to succeed.

Measuring unsafe abortion-related mortality: a systematic review of the existing methods

BACKGROUND

The WHO estimates that 13% of maternal mortality is due to unsafe abortion, but challenges with measurement and data quality persist. To our knowledge, no systematic assessment of the validity of studies reporting estimates of abortion-related mortality exists.

STUDY DESIGN

To be included in this study, articles had to meet the following criteria: (1) published between September 1(st), 2000-December 1(st), 2011; (2) utilized data from a country where abortion is "considered unsafe"; (3) specified and enumerated causes of maternal death including "abortion"; (4) enumerated ≥100 maternal deaths; (5) a quantitative research study; (6) published in a peer-reviewed journal.

RESULTS

7,438 articles were initially identified. Thirty-six studies were ultimately included. Overall, studies rated "Very Good" found the highest estimates of abortion related mortality (median 16%, range 1-27.4%). Studies rated "Very Poor" found the lowest overall proportion of abortion related deaths (median: 2%, range 1.3-9.4%).

CONCLUSIONS

Improvements in the quality of data collection would facilitate better understanding global abortion-related mortality. Until improved data exist, better reporting of study procedures and standardization of the definition of abortion and abortion-related mortality should be encouraged.

Reflections on the global burden of disease 2010 estimates

Peter Byass and colleagues raise questions about the recent, high-profile Global Burden of Disease estimates. Please see later in the article for the Editors' Summary

Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010

BACKGROUND

Reliable and timely information on the leading causes of death in populations, and how these are changing, is a crucial input into health policy debates. In the Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010), we aimed to estimate annual deaths for the world and 21 regions between 1980 and 2010 for 235 causes, with uncertainty intervals (UIs), separately by age and sex.

METHODS

We attempted to identify all available data on causes of death for 187 countries from 1980 to 2010 from vital registration, verbal autopsy, mortality surveillance, censuses, surveys, hospitals, police records, and mortuaries. We assessed data quality for completeness, diagnostic accuracy, missing data, stochastic variations, and probable causes of death. We applied six different modelling strategies to estimate cause-specific mortality trends depending on the strength of the data. For 133 causes and three special aggregates we used the Cause of Death Ensemble model (CODEm) approach, which uses four families of statistical models testing a large set of different models using different permutations of covariates. Model ensembles were developed from these component models. We assessed model performance with rigorous out-of-sample testing of prediction error and the validity of 95% UIs. For 13 causes with low observed numbers of deaths, we developed negative binomial models with plausible covariates. For 27 causes for which death is rare, we modelled the higher level cause in the cause hierarchy of the GBD 2010 and then allocated deaths across component causes proportionately, estimated from all available data in the database. For selected causes (African trypanosomiasis, congenital syphilis, whooping cough, measles, typhoid and parathyroid, leishmaniasis, acute hepatitis E, and HIV/AIDS), we used natural history models based on information on incidence, prevalence, and case-fatality. We separately estimated cause fractions by aetiology for diarrhoea, lower respiratory infections, and meningitis, as well as disaggregations by subcause for chronic kidney disease, maternal disorders, cirrhosis, and liver cancer. For deaths due to collective violence and natural disasters, we used mortality shock regressions. For every cause, we estimated 95% UIs that captured both parameter estimation uncertainty and uncertainty due to model specification where CODEm was used. We constrained cause-specific fractions within every age-sex group to sum to total mortality based on draws from the uncertainty distributions.

FINDINGS

In 2010, there were 52·8 million deaths globally. At the most aggregate level, communicable, maternal, neonatal, and nutritional causes were 24·9% of deaths worldwide in 2010, down from 15·9 million (34·1%) of 46·5 million in 1990. This decrease was largely due to decreases in mortality from diarrhoeal disease (from 2·5 to 1·4 million), lower respiratory infections (from 3·4 to 2·8 million), neonatal disorders (from 3·1 to 2·2 million), measles (from 0·63 to 0·13 million), and tetanus (from 0·27 to 0·06 million). Deaths from HIV/AIDS increased from 0·30 million in 1990 to 1·5 million in 2010, reaching a peak of 1·7 million in 2006. Malaria mortality also rose by an estimated 19·9% since 1990 to 1·17 million deaths in 2010. Tuberculosis killed 1·2 million people in 2010. Deaths from non-communicable diseases rose by just under 8 million between 1990 and 2010, accounting for two of every three deaths (34·5 million) worldwide by 2010. 8 million people died from cancer in 2010, 38% more than two decades ago; of these, 1·5 million (19%) were from trachea, bronchus, and lung cancer. Ischaemic heart disease and stroke collectively killed 12·9 million people in 2010, or one in four deaths worldwide, compared with one in five in 1990; 1·3 million deaths were due to diabetes, twice as many as in 1990. The fraction of global deaths due to injuries (5·1 million deaths) was marginally higher in 2010 (9·6%) compared with two decades earlier (8·8%). This was driven by a 46% rise in deaths worldwide due to road traffic accidents (1·3 million in 2010) and a rise in deaths from falls. Ischaemic heart disease, stroke, chronic obstructive pulmonary disease (COPD), lower respiratory infections, lung cancer, and HIV/AIDS were the leading causes of death in 2010. Ischaemic heart disease, lower respiratory infections, stroke, diarrhoeal disease, malaria, and HIV/AIDS were the leading causes of years of life lost due to premature mortality (YLLs) in 2010, similar to what was estimated for 1990, except for HIV/AIDS and preterm birth complications. YLLs from lower respiratory infections and diarrhoea decreased by 45-54% since 1990; ischaemic heart disease and stroke YLLs increased by 17-28%. Regional variations in leading causes of death were substantial. Communicable, maternal, neonatal, and nutritional causes still accounted for 76% of premature mortality in sub-Saharan Africa in 2010. Age standardised death rates from some key disorders rose (HIV/AIDS, Alzheimer's disease, diabetes mellitus, and chronic kidney disease in particular), but for most diseases, death rates fell in the past two decades; including major vascular diseases, COPD, most forms of cancer, liver cirrhosis, and maternal disorders. For other conditions, notably malaria, prostate cancer, and injuries, little change was noted.

INTERPRETATION

Population growth, increased average age of the world's population, and largely decreasing age-specific, sex-specific, and cause-specific death rates combine to drive a broad shift from communicable, maternal, neonatal, and nutritional causes towards non-communicable diseases. Nevertheless, communicable, maternal, neonatal, and nutritional causes remain the dominant causes of YLLs in sub-Saharan Africa. Overlaid on this general pattern of the epidemiological transition, marked regional variation exists in many causes, such as interpersonal violence, suicide, liver cancer, diabetes, cirrhosis, Chagas disease, African trypanosomiasis, melanoma, and others. Regional heterogeneity highlights the importance of sound epidemiological assessments of the causes of death on a regular basis.

FUNDING

Bill & Melinda Gates Foundation.

Assessing the global burden of ischemic heart disease, part 2: analytic methods and estimates of the global epidemiology of ischemic heart disease in 2010

BACKGROUND

Ischemic Heart Disease (IHD) is the leading cause of death worldwide. The Global Burden of Diseases, Injuries and Risk Factors (GBD) 2010 Study estimated IHD mortality and disability burden for 21 world regions for the years 1990 to 2010.

METHODS

Data sources for GBD IHD epidemiology estimates were mortality surveillance, verbal autopsy, and vital registration data (for IHD mortality) and systematic review of IHD epidemiology literature published 1980-2008 (for non-fatal IHD outcomes). An estimation and validation process led to an ensemble model of IHD mortality by country for all 21 world regions, adjusted for country-level covariates. Disease models were developed for the nonfatal sequelae of IHD: myocardial infarction, stable angina pectoris, and ischemic heart failure.

RESULTS

Country level covariates including metabolic and nutritional risk factors, education, war, and annual income per capita contributed to the ensemble model for the analysis of IHD death. In the acute myocardial infarction model, inclusion of troponin in the diagnostic criteria of studies published after the year 2000 was associated with a 50% higher incidence. Self-reported diagnosis of angina significantly overestimated stable angina prevalence compared with "definite" angina elicited by the Rose angina questionnaire. For 2010, Eastern Europe and Central Asia had the highest rates of IHD death and the Asia Pacific High-Income, East Asia, Latin American Andean, and sub-Saharan Africa regions had the lowest.

CONCLUSIONS

Global and regional IHD epidemiology estimates are needed for estimating the worldwide burden of IHD. Using descriptive meta-analysis tools, the GBD 2010 standardized and pooled international data by adjusting for region-level mortality and risk factor data, and study level diagnostic method. Analyses maximized internal consistency, generalizability, and adjustment for known sources of bias. The GBD IHD analysis nonetheless highlights the need for improved IHD epidemiology surveillance in many regions and the need for uniform diagnostic standards.

Causes of death in Tonga: quality of certification and implications for statistics

Background

Detailed cause of death data by age group and sex are critical to identify key public health issues and target interventions appropriately. In this study the quality of local routinely collected cause of death data from medical certification is reviewed, and a cause of death profile for Tonga based on amended data is presented.

Methods

Medical certificates of death for all deaths in Tonga for 2001 to 2008 and medical records for all deaths in the main island Tongatapu for 2008 were sought from the national hospital. Cause of death data for 2008 were reviewed for quality through (a) a review of current tabulation procedures and (b) a medical record review. Data from each medical record were extracted and provided to an independent medical doctor to assign cause of death, with underlying cause from the medical record tabulated against underlying cause from the medical certificate. Significant associations in reporting patterns were evaluated and final cause of death for each case in 2008 was assigned based on the best quality information from the medical certificate or medical record. Cause of death data from 2001 to 2007 were revised based on findings from the evaluation of certification of the 2008 data and added to the dataset. Proportional mortality was calculated and applied to age- and sex-specific mortality for all causes from 2001 to 2008. Cause of death was tabulated by age group and sex, and age-standardized (all ages) mortality rates for each sex by cause were calculated.

Results

Reported tabulations of cause of death in Tonga are of immediate cause, with ischemic heart disease and diabetes underrepresented. In the majority of cases the reported (immediate) cause fell within the same broad category as the underlying cause of death from the medical certificate. Underlying cause of death from the medical certificate, attributed to neoplasms, diabetes, and cardiovascular disease were assigned to other underlying causes by the medical record review in 70% to 77% of deaths. Of the 28 (6.5%) deaths attributed to nonspecific or unknown causes on the medical certificate, 17 were able to be attributed elsewhere following review of the medical record. Final cause of death tabulations for 2001 to 2008 demonstrate that noncommunicable diseases are leading adult mortality, and age-standardized rates for cardiovascular diseases, neoplasms, and diabetes increased significantly between 2001 to 2004 and 2005 to 2008. Cause of death data for 2001 to 2008 show increasing cause-specific mortality (deaths per 100,000) from 2001-2004 to 2005-2008 from cardiovascular (194-382 to 423-644 in 2005-2008 for males and 108-227 to 194-321 for females) and other noncommunicable diseases that cannot be accounted for by changes in the age structure of the population. Mortality from diabetes for 2005 to 2008 is estimated at 94 to 222 deaths per 100,000 population for males and 98 to 190 for females (based on the range of plausible all-cause mortality estimates) compared with 2008 estimates from the global burden of disease study of 40 (males) and 53 (females) deaths per 100,000 population.

Discussion

Certification of death was generally found to be the most reliable data on cause of death in Tonga available for Tonga, with 93% of the final assigned causes following review of the 2008 data matching those listed on the medical certificate of death. Cause of death data available in Tonga can be improved by routinely tabulating data by underlying cause and ensuring contributory causes are not recorded in Part I of the certificate during data entry to the database. There is significantly more data on cause of death available in Tonga than are routinely reported or known to international agencies.

Exploring the role narrative free-text plays in discrepancies between physician coding and the InterVA regarding determination of malaria as cause of death, in a malaria holo-endemic region

BACKGROUND

In countries where tracking mortality and clinical cause of death are not routinely undertaken, gathering verbal autopsies (VA) is the principal method of estimating cause of death. The most common method for determining probable cause of death from the VA interview is Physician-Certified Verbal Autopsy (PCVA). A recent alternative method to interpret Verbal Autopsy (InterVA) is a computer model using a Bayesian approach to derive posterior probabilities for causes of death, given an a priori distribution at population level and a set of interview-based indicators. The model uses the same input information as PCVA, with the exception of narrative text information, which physicians can consult but which were not inputted into the model. Comparing the results of physician coding with the model, large differences could be due to difficulties in diagnosing malaria, especially in holo-endemic regions. Thus, the aim of the study was to explore whether physicians' access to electronically unavailable narrative text helps to explain the large discrepancy in malaria cause-specific mortality fractions (CSMFs) in physician coding versus the model.

METHODS

Free-texts of electronically available records (N = 5,649) were summarised and incorporated into the InterVA version 3 (InterVA-3) for three sub-groups: (i) a 10%-representative subsample (N = 493) (ii) records diagnosed as malaria by physicians and not by the model (N = 1035), and (iii) records diagnosed by the model as malaria, but not by physicians (N = 332). CSMF results before and after free-text incorporation were compared.

RESULTS

There were changes of between 5.5-10.2% between models before and after free-text incorporation. No impact on malaria CSMFs was seen in the representative sub-sample, but the proportion of malaria as cause of death increased in the physician sub-sample (2.7%) and saw a large decrease in the InterVA subsample (9.9%). Information on 13/106 indicators appeared at least once in the free-texts that had not been matched to any item in the structured, electronically available portion of the Nouna questionnaire.

DISCUSSION

Free-texts are helpful in gathering information not adequately captured in VA questionnaires, though access to free-text does not explain differences in physician and model determination of malaria as cause of death.

Modeling causes of death: an integrated approach using CODEm

BACKGROUND

Data on causes of death by age and sex are a critical input into health decision-making. Priority setting in public health should be informed not only by the current magnitude of health problems but by trends in them. However, cause of death data are often not available or are subject to substantial problems of comparability. We propose five general principles for cause of death model development, validation, and reporting.

METHODS

We detail a specific implementation of these principles that is embodied in an analytical tool - the Cause of Death Ensemble model (CODEm) - which explores a large variety of possible models to estimate trends in causes of death. Possible models are identified using a covariate selection algorithm that yields many plausible combinations of covariates, which are then run through four model classes. The model classes include mixed effects linear models and spatial-temporal Gaussian Process Regression models for cause fractions and death rates. All models for each cause of death are then assessed using out-of-sample predictive validity and combined into an ensemble with optimal out-of-sample predictive performance.

RESULTS

Ensemble models for cause of death estimation outperform any single component model in tests of root mean square error, frequency of predicting correct temporal trends, and achieving 95% coverage of the prediction interval. We present detailed results for CODEm applied to maternal mortality and summary results for several other causes of death, including cardiovascular disease and several cancers.

CONCLUSIONS

CODEm produces better estimates of cause of death trends than previous methods and is less susceptible to bias in model specification. We demonstrate the utility of CODEm for the estimation of several major causes of death.

Breast and cervical cancer in 187 countries between 1980 and 2010: a systematic analysis

BACKGROUND

Breast and cervical cancer are important causes of mortality in women aged ≥15 years. We undertook annual age-specific assessments of breast and cervical cancer in 187 countries.

METHODS

We systematically collected cancer registry data on mortality and incidence, vital registration, and verbal autopsy data for the period 1980-2010. We modelled the mortality-to-incidence (MI) ratio using a hierarchical model. Vital registration and verbal autopsy were supplemented with incidence multiplied by the MI ratio to yield a comprehensive database of mortality rates. We used Gaussian process regression to develop estimates of mortality with uncertainty by age, sex, country, and year. We used out-of-sample predictive validity to select the final model. Estimates of incidence with uncertainty were also generated with mortality and MI ratios.

FINDINGS

Global breast cancer incidence increased from 641,000 (95% uncertainty intervals 610,000-750,000) cases in 1980 to 1,643,000 (1,421,000-1,782,000) cases in 2010, an annual rate of increase of 3·1%. Global cervical cancer incidence increased from 378,000 (256,000-489,000) cases per year in 1980 to 454,000 (318,000-620,000) cases per year in 2010-a 0·6% annual rate of increase. Breast cancer killed 425,000 (359,000-453,000) women in 2010, of whom 68,000 (62,000-74,000) were aged 15-49 years in developing countries. Cervical cancer death rates have been decreasing but the disease still killed 200,000 (139,000-276,000) women in 2010, of whom 46,000 (33,000-64,000) were aged 15-49 years in developing countries. We recorded pronounced variation in the trend in breast cancer mortality across regions and countries.

INTERPRETATION

More policy attention is needed to strengthen established health-system responses to reduce breast and cervical cancer, especially in developing countries.

FUNDING

Susan G Komen for the Cure and the Bill & Melinda Gates Foundation.

Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards

Background

InterVA is a widely disseminated tool for cause of death attribution using information from verbal autopsies. Several studies have attempted to validate the concordance and accuracy of the tool, but the main limitation of these studies is that they compare cause of death as ascertained through hospital record review or hospital discharge diagnosis with the results of InterVA. This study provides a unique opportunity to assess the performance of InterVA compared to physician-certified verbal autopsies (PCVA) and alternative automated methods for analysis.

Methods

Using clinical diagnostic gold standards to select 12,542 verbal autopsy cases, we assessed the performance of InterVA on both an individual and population level and compared the results to PCVA, conducting analyses separately for adults, children, and neonates. Following the recommendation of Murray et al., we randomly varied the cause composition over 500 test datasets to understand the performance of the tool in different settings. We also contrasted InterVA with an alternative Bayesian method, Simplified Symptom Pattern (SSP), to understand the strengths and weaknesses of the tool.

Results

Across all age groups, InterVA performs worse than PCVA, both on an individual and population level. On an individual level, InterVA achieved a chance-corrected concordance of 24.2% for adults, 24.9% for children, and 6.3% for neonates (excluding free text, considering one cause selection). On a population level, InterVA achieved a cause-specific mortality fraction accuracy of 0.546 for adults, 0.504 for children, and 0.404 for neonates. The comparison to SSP revealed four specific characteristics that lead to superior performance of SSP. Increases in chance-corrected concordance are attained by developing cause-by-cause models (2%), using all items as opposed to only the ones that mapped to InterVA items (7%), assigning probabilities to clusters of symptoms (6%), and using empirical as opposed to expert probabilities (up to 8%).

Conclusions

Given the widespread use of verbal autopsy for understanding the burden of disease and for setting health intervention priorities in areas that lack reliable vital registrations systems, accurate analysis of verbal autopsies is essential. While InterVA is an affordable and available mechanism for assigning causes of death using verbal autopsies, users should be aware of its suboptimal performance relative to other methods.

Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards

Background

Verbal autopsy (VA) is used to estimate the causes of death in areas with incomplete vital registration systems. The King and Lu method (KL) for direct estimation of cause-specific mortality fractions (CSMFs) from VA studies is an analysis technique that estimates CSMFs in a population without predicting individual-level cause of death as an intermediate step. In previous studies, KL has shown promise as an alternative to physician-certified verbal autopsy (PCVA). However, it has previously been impossible to validate KL with a large dataset of VAs for which the underlying cause of death is known to meet rigorous clinical diagnostic criteria.

Methods

We applied the KL method to adult, child, and neonatal VA datasets from the Population Health Metrics Research Consortium gold standard verbal autopsy validation study, a multisite sample of 12,542 VAs where gold standard cause of death was established using strict clinical diagnostic criteria. To emulate real-world populations with varying CSMFs, we evaluated the KL estimations for 500 different test datasets of varying cause distribution. We assessed the quality of these estimates in terms of CSMF accuracy as well as linear regression and compared this with the results of PCVA.

Results

KL performance is similar to PCVA in terms of CSMF accuracy, attaining values of 0.669, 0.698, and 0.795 for adult, child, and neonatal age groups, respectively, when health care experience (HCE) items were included. We found that the length of the cause list has a dramatic effect on KL estimation quality, with CSMF accuracy decreasing substantially as the length of the cause list increases. We found that KL is not reliant on HCE the way PCVA is, and without HCE, KL outperforms PCVA for all age groups.

Conclusions

Like all computer methods for VA analysis, KL is faster and cheaper than PCVA. Since it is a direct estimation technique, though, it does not produce individual-level predictions. KL estimates are of similar quality to PCVA and slightly better in most cases. Compared to other recently developed methods, however, KL would only be the preferred technique when the cause list is short and individual-level predictions are not needed.

Simplified Symptom Pattern Method for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards

Background

Verbal autopsy can be a useful tool for generating cause of death data in data-sparse regions around the world. The Symptom Pattern (SP) Method is one promising approach to analyzing verbal autopsy data, but it has not been tested rigorously with gold standard diagnostic criteria. We propose a simplified version of SP and evaluate its performance using verbal autopsy data with accompanying true cause of death.

Methods

We investigated specific parameters in SP's Bayesian framework that allow for its optimal performance in both assigning individual cause of death and in determining cause-specific mortality fractions. We evaluated these outcomes of the method separately for adult, child, and neonatal verbal autopsies in 500 different population constructs of verbal autopsy data to analyze its ability in various settings.

Results

We determined that a modified, simpler version of Symptom Pattern (termed Simplified Symptom Pattern, or SSP) performs better than the previously-developed approach. Across 500 samples of verbal autopsy testing data, SSP achieves a median cause-specific mortality fraction accuracy of 0.710 for adults, 0.739 for children, and 0.751 for neonates. In individual cause of death assignment in the same testing environment, SSP achieves 45.8% chance-corrected concordance for adults, 51.5% for children, and 32.5% for neonates.

Conclusions

The Simplified Symptom Pattern Method for verbal autopsy can yield reliable and reasonably accurate results for both individual cause of death assignment and for determining cause-specific mortality fractions. The method demonstrates that verbal autopsies coupled with SSP can be a useful tool for analyzing mortality patterns and determining individual cause of death from verbal autopsy data.

Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals

BACKGROUND

In Mexico, the vital registration system relies on information collected from death certificates to generate official mortality figures. Although the death certificate has high coverage across the country, there is little information regarding its validity. The objective of this study was to assess the concordance between the underlying cause of death in official statistics obtained from death certificates and a gold standard diagnosis of the same deaths derived from medical records of hospitals.

METHODS

The study sample consisted of 1,589 deaths that occurred in 34 public hospitals in the Federal District and the state of Morelos, Mexico in 2009. Neonatal, child, and adult cases were selected for causes of death that included infectious diseases, noncommunicable diseases, and injuries. We compared the underlying cause of death, obtained from medical death certificates, against a gold standard diagnosis derived from a review of medical records developed by the Population Health Metrics Research Consortium. We used chance-corrected concordance and accuracy as metrics to evaluate the quality of performance of the death certificate.

RESULTS

Analysis considering only the underlying cause of death resulted in a median chance-corrected concordance between the cause of death in medical death certificates versus the gold standard of 54.3% (95% uncertainty interval [UI]: 52.2, 55.6) for neonates, 38.5% (37.0, 40.0) for children, and 66.5% (65.9, 66.9) for adults. The accuracy resulting from the same analysis was 0.756 (0.747, 0.769) for neonates, 0.683 (0.663, 0.701) for children, and 0.780 (0.774, 0.785) for adults. Median chance-corrected concordance and accuracy increased when considering the mention of any cause of death in the death certificate, not just the underlying cause. Concordance varied substantially depending on cause of death, and accuracy varied depending on the true cause-specific mortality fraction composition.

CONCLUSIONS

Although we cannot generalize our conclusions to Mexico as a whole, the results demonstrate important problems with the quality of the main source of information for causes of death used by decision-makers in settings with highly technological vital registration systems. It is necessary to improve death certification procedures, especially in the case of child and neonatal deaths. This requires an important commitment from the health system and health institutions.

Performance of physician-certified verbal autopsies: multisite validation study using clinical diagnostic gold standards

BACKGROUND

Physician review of a verbal autopsy (VA) and completion of a death certificate remains the most widely used approach for VA analysis. This study provides new evidence about the performance of physician-certified verbal autopsy (PCVA) using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 VAs. The study was also designed to analyze issues related to PCVA, such as the impact of a second physician reader on the cause of death assigned, the variation in performance with and without household recall of health care experience (HCE), and the importance of local information for physicians reading VAs.

METHODS

The certification was performed by 24 physicians. The assignment of VA was random and blinded. Each VA was certified by one physician. Half of the VAs were reviewed by a different physician with household recall of health care experience included. The completed death certificate was processed for automated ICD-10 coding of the underlying cause of death. PCVA was compared to gold standard cause of death assignment based on strictly defined clinical diagnostic criteria that are part of the Population Health Metrics Research Consortium (PHMRC) gold standard verbal autopsy study.

RESULTS

For individual cause assignment, the overall chance-corrected concordance for PCVA against the gold standard cause of death is less than 50%, with substantial variability by cause and physician. Physicians assign the correct cause around 30% of the time without HCE, and addition of HCE improves performance in adults to 45% and slightly higher in children to 48%. Physicians estimate cause-specific mortality fractions (CSMFs) with considerable error for adults, children, and neonates. Only for neonates for a cause list of six causes with HCE is accuracy above 0.7. In all three age groups, CSMF accuracy improves when household recall of health care experience is available.

CONCLUSIONS

Results show that physician coding for cause of death assignment may not be as robust as previously thought. The time and cost required to initially collect the verbal autopsies must be considered in addition to the analysis, as well as the impact of diverting physicians from servicing immediate health needs in a population to review VAs. All of these considerations highlight the importance and urgency of developing better methods to more reliably analyze past and future verbal autopsies to obtain the highest quality mortality data from populations without reliable death certification.

Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards

Background

Computer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs).

Methods

The Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE).

Results

For all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates.

Conclusions

We found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

Background

Verbal autopsy (VA) is an important method for obtaining cause of death information in settings without vital registration and medical certification of causes of death. An array of methods, including physician review and computer-automated methods, have been proposed and used. Choosing the best method for VA requires the appropriate metrics for assessing performance. Currently used metrics such as sensitivity, specificity, and cause-specific mortality fraction (CSMF) errors do not provide a robust basis for comparison.

Methods

We use simple simulations of populations with three causes of death to demonstrate that most metrics used in VA validation studies are extremely sensitive to the CSMF composition of the test dataset. Simulations also demonstrate that an inferior method can appear to have better performance than an alternative due strictly to the CSMF composition of the test set.

Results

VA methods need to be evaluated across a set of test datasets with widely varying CSMF compositions. We propose two metrics for assessing the performance of a proposed VA method. For assessing how well a method does at individual cause of death assignment, we recommend the average chance-corrected concordance across causes. This metric is insensitive to the CSMF composition of the test sets and corrects for the degree to which a method will get the cause correct due strictly to chance. For the evaluation of CSMF estimation, we propose CSMF accuracy. CSMF accuracy is defined as one minus the sum of all absolute CSMF errors across causes divided by the maximum total error. It is scaled from zero to one and can generalize a method's CSMF estimation capability regardless of the number of causes. Performance of a VA method for CSMF estimation by cause can be assessed by examining the relationship across test datasets between the estimated CSMF and the true CSMF.

Conclusions

With an increasing range of VA methods available, it will be critical to objectively assess their performance in assigning cause of death. Chance-corrected concordance and CSMF accuracy assessed across a large number of test datasets with widely varying CSMF composition provide a robust strategy for this assessment.

Performance of the Tariff Method: validation of a simple additive algorithm for analysis of verbal autopsies

BACKGROUND

Verbal autopsies provide valuable information for studying mortality patterns in populations that lack reliable vital registration data. Methods for transforming verbal autopsy results into meaningful information for health workers and policymakers, however, are often costly or complicated to use. We present a simple additive algorithm, the Tariff Method (termed Tariff), which can be used for assigning individual cause of death and for determining cause-specific mortality fractions (CSMFs) from verbal autopsy data.

METHODS

Tariff calculates a score, or "tariff," for each cause, for each sign/symptom, across a pool of validated verbal autopsy data. The tariffs are summed for a given response pattern in a verbal autopsy, and this sum (score) provides the basis for predicting the cause of death in a dataset. We implemented this algorithm and evaluated the method's predictive ability, both in terms of chance-corrected concordance at the individual cause assignment level and in terms of CSMF accuracy at the population level. The analysis was conducted separately for adult, child, and neonatal verbal autopsies across 500 pairs of train-test validation verbal autopsy data.

RESULTS

Tariff is capable of outperforming physician-certified verbal autopsy in most cases. In terms of chance-corrected concordance, the method achieves 44.5% in adults, 39% in children, and 23.9% in neonates. CSMF accuracy was 0.745 in adults, 0.709 in children, and 0.679 in neonates.

CONCLUSIONS

Verbal autopsies can be an efficient means of obtaining cause of death data, and Tariff provides an intuitive, reliable method for generating individual cause assignment and CSMFs. The method is transparent and flexible and can be readily implemented by users without training in statistics or computer science.