Evaluating malaria programmes in moderate- and low-transmission settings: practical ways to generate robust evidence

Development of WHO Recommendations for the Final Phase of Elimination and Prevention of Re-Establishment of Malaria

The WHO recommends that all affected countries work toward the elimination of malaria, even those still experiencing a high burden of disease. However, malaria programs in the final phase of elimination or those working to prevent re-establishment of transmission after elimination could benefit from specific evidence-based recommendations for these settings as part of comprehensive and quality-controlled malaria guidelines. The WHO convened an external guideline development group to formulate recommendations for interventions to reduce or prevent malaria transmission in areas with very low- to low-transmission levels and those that have eliminated malaria. In addition, several interventions that could be deployed in higher burden areas to accelerate elimination, such as mass drug administration, were reviewed. Systematic reviews were conducted that synthesized and evaluated evidence for the benefits and harms of public health interventions and summarized critical contextual factors from a health systems perspective. A total of 12 recommendations were developed, with five related to mass interventions that could be deployed at higher transmission levels and seven that would be most appropriate for programs in areas close to elimination or those working to prevent re-establishment of transmission. Four chemoprevention, two active case detection, and one vector control interventions were positively recommended, whereas two chemoprevention and three active case detection interventions were not recommended by the WHO. None of the recommendations were classified as strong given the limited and low-quality evidence base. Approaches to conducting higher quality research in very low- to low-transmission settings to improve the strength of WHO recommendations are discussed.

Evaluating the performance of Plasmodium falciparum genetic metrics for inferring National Malaria Control Programme reported incidence in Senegal

BACKGROUND

Genetic surveillance of the Plasmodium falciparum parasite shows great promise for helping National Malaria Control Programmes (NMCPs) assess parasite transmission. Genetic metrics such as the frequency of polygenomic (multiple strain) infections, genetic clones, and the complexity of infection (COI, number of strains per infection) are correlated with transmission intensity. However, despite these correlations, it is unclear whether genetic metrics alone are sufficient to estimate clinical incidence.

METHODS

This study examined parasites from 3147 clinical infections sampled between the years 2012-2020 through passive case detection (PCD) across 16 clinic sites spread throughout Senegal. Samples were genotyped with a 24 single nucleotide polymorphism (SNP) molecular barcode that detects parasite strains, distinguishes polygenomic (multiple strain) from monogenomic (single strain) infections, and identifies clonal infections. To determine whether genetic signals can predict incidence, a series of Poisson generalized linear mixed-effects models were constructed to predict the incidence level at each clinical site from a set of genetic metrics designed to measure parasite clonality, superinfection, and co-transmission rates.

RESULTS

Model-predicted incidence was compared with the reported standard incidence data determined by the NMCP for each clinic and found that parasite genetic metrics generally correlated with reported incidence, with departures from expected values at very low annual incidence (< 10/1000/annual [‰]).

CONCLUSIONS

When transmission is greater than 10 cases per 1000 annual parasite incidence (annual incidence > 10‰), parasite genetics can be used to accurately infer incidence and is consistent with superinfection-based hypotheses of malaria transmission. When transmission was < 10‰, many of the correlations between parasite genetics and incidence were reversed, which may reflect the disproportionate impact of importation and focal transmission on parasite genetics when local transmission levels are low.

Higher outdoor mosquito density and Plasmodium infection rates in and around malaria index case households in low transmission settings of Ethiopia: Implications for vector control

BACKGROUND

Understanding the clustering of infections for persistent malaria transmission is critical to determining how and where to target specific interventions. This study aimed to determine the density, blood meal sources and malaria transmission risk of anopheline vectors by targeting malaria index cases, their neighboring households and control villages in Arjo-Didessa, southwestern Ethiopia.

METHODS

An entomological study was conducted concurrently with a reactive case detection (RCD) study from November 2019 to October 2021 in Arjo Didessa and the surrounding vicinity, southwestern Ethiopia. Anopheline mosquitoes were collected indoors and outdoors in index case households and their surrounding households (neighboring households), as well as in control households, using pyrethrum spray cache (PSC) and U.S. Centers for Disease Control and Prevention (CDC) light traps. Adult mosquitoes were morphologically identified, and speciation in the Anopheles gambiae complex was done by PCR. Mosquito Plasmodium infections and host blood meal sources were detected by circumsporozoite protein enzyme-linked immunosorbent assay (CSP-ELISA) and cytochrome b-based blood meal PCR, respectively.

RESULTS

Among the 770 anopheline mosquitoes collected, An. gambiae sensu lato (A. gambiae s.l.) was the predominant species, accounting for 87.1% (n = 671/770) of the catch, followed by the Anopheles coustani complex and Anopheles pharoensis, which accounted for 12.6% (n = 97/770) and 0.26% (n = 2/770) of the catch, respectively. From the sub-samples of An. gambiae s.l.analyzed with PCR, An. arabiensis and Anopheles amharicus were identified. The overall mean density of mosquitoes was 1.26 mosquitoes per trap per night using the CDC light traps. Outdoor mosquito density was significantly higher than indoor mosquito density in the index and neighboring households (P = 0.0001). The human blood index (HBI) and bovine blood index (BBI) of An. arabiensis were 20.8% (n = 34/168) and 24.0% (n = 41/168), respectively. The overall Plasmodium sporozoite infection rate of anophelines (An. arabiensis and An. coustani complex) was 4.4% (n = 34/770). Sporozoites were detected indoors and outdoors in captured anopheline mosquitoes. Of these CSP-positive species for Pv-210, Pv-247 and Pf, 41.1% (n = 14/34) were captured outdoors. A significantly higher proportion of sporozoite-infected mosquitoes were caught in index case households (5.6%, n = 8/141) compared to control households (1.1%, n = 2/181) (P = 0.02), and in neighboring households (5.3%, n = 24/448) compared to control households (P = 0.01).

CONCLUSIONS

The findings of this study indicated that malaria index cases and their neighboring households had higher outdoor mosquito densities and Plasmodium infection rates. The study also highlighted a relatively higher outdoor mosquito density, which could increase the potential risk of outdoor malaria transmission and may play a role in residual malaria transmission. Thus, it is important to strengthen the implementation of vector control interventions, such as targeted indoor residual spraying, long-lasting insecticidal nets and other supplementary vector control measures such as larval source management and community engagement approaches. Furthermore, in low transmission settings, such as the Arjo Didessa Sugarcane Plantation, providing health education to local communities, enhanced environmental management and entomological surveillance, along with case detection and management by targeting of malaria index cases and their immediate neighboring households, could be important measures to control residual malaria transmission and achieve the targeted elimination goals.

From efficacy to effectiveness: a comprehensive framework for monitoring, evaluating and optimizing seasonal malaria chemoprevention programmes

BACKGROUND

Seasonal Malaria Chemoprevention (SMC) is a highly effective intervention for preventing malaria, particularly in areas with highly seasonal transmission. Monitoring and evaluating (M&E) SMC programmes are complex due to the scale, time-sensitive delivery of the programme, and influence of external factors. This paper describes the process followed to develop a comprehensive M&E framework tailored specifically for the SMC context.

METHODS

The Framework was developed through a literature and programme review, and stakeholder dialogues across three implementing countries-Burkina Faso, Chad, and Nigeria. Expert consultation further refined the Framework through an iterative approach drawing upon data collected through the three sources. The Framework was designed using the Logical Framework Approach incorporating external factors and intentionally aligned with global malaria M&E standards.

RESULTS

An overall aim and seven programme objectives were developed measured by 70 indicators. The indicators also capture the causal links between the implementation and results of the programme. The Framework leverages the use of current data sources and existing mechanisms, ensuring efficient data use without requiring a significant increase in resources for overall programme optimization. It also promotes the use of data triangulation, and stratification for a more nuanced understanding of factors affecting programme performance and timely data informed decision-making.

CONCLUSIONS

The SMC M&E Framework presented here provides a standardized approach for programme implementers to enhance decision-making for optimal programme performance. This is an essential tool as the scope of SMC programmes expands to new geographies and target age groups.

Evaluating the performance of Plasmodium falciparum genetics for inferring National Malaria Control Program reported incidence in Senegal

Genetic surveillance of the Plasmodium falciparum parasite shows great promise for helping National Malaria Control Programs (NMCPs) assess parasite transmission. Genetic metrics such as the frequency of polygenomic (multiple strain) infections, genetic clones, and the complexity of infection (COI, number of strains per infection) are correlated with transmission intensity. However, despite these correlations, it is unclear whether genetic metrics alone are sufficient to estimate clinical incidence. Here, we examined parasites from 3,147 clinical infections sampled between the years 2012-2020 through passive case detection (PCD) across 16 clinic sites spread throughout Senegal. Samples were genotyped with a 24 single nucleotide polymorphism (SNP) molecular barcode that detects parasite strains, distinguishes polygenomic (multiple strain) from monogenomic (single strain) infections, and identifies clonal infections. To determine whether genetic signals can predict incidence, we constructed a series of Poisson generalized linear mixed-effects models to predict the incidence level at each clinical site from a set of genetic metrics designed to measure parasite clonality, superinfection, and co-transmission rates. We compared the model-predicted incidence with the reported standard incidence data determined by the NMCP for each clinic and found that parasite genetic metrics generally correlated with reported incidence, with departures from expected values at very low annual incidence (<10/1000/annual [‰]). When transmission is greater than 10 cases per 1000 annual parasite incidence (annual incidence >10 ‰), parasite genetics can be used to accurately infer incidence and is consistent with superinfection-based hypotheses of malaria transmission. When transmission was <10 ‰, we found that many of the correlations between parasite genetics and incidence were reversed, which we hypothesize reflects the disproportionate impact of importation and focal transmission on parasite genetics when local transmission levels are low.

Projecting malaria elimination in Thailand using Bayesian hierarchical spatiotemporal models

Thailand has set a goal of eliminating malaria by 2024 in its national strategic plan. In this study, we used the Thailand malaria surveillance database to develop hierarchical spatiotemporal models to analyze retrospective patterns and predict Plasmodium falciparum and Plasmodium vivax malaria incidences at the provincial level. We first describe the available data, explain the hierarchical spatiotemporal framework underlying the analysis, and then display the results of fitting various space-time formulations to the malaria data with the different model selection metrics. The Bayesian model selection process assessed the sensitivity of different specifications to obtain the optimal models. To assess whether malaria could be eliminated by 2024 per Thailand's National Malaria Elimination Strategy, 2017-2026, we used the best-fitted model to project the estimated cases for 2022-2028. The study results based on the models revealed different predicted estimates between both species. The model for P. falciparum suggested that zero P. falciparum cases might be possible by 2024, in contrast to the model for P. vivax, wherein zero P. vivax cases might not be reached. Innovative approaches in the P. vivax-specific control and elimination plans must be implemented to reach zero P. vivax and consequently declare Thailand as a malaria-free country.

Surveillance as a Core Intervention to Strengthen Malaria Control Programs in Moderate to High Transmission Settings

New tools are needed for malaria control, and recent improvements in malaria surveillance have opened the possibility of transforming surveillance into a core intervention. Implementing this strategy can be challenging in moderate to high transmission settings. However, there is a wealth of practical experience among national malaria control programs and partners working to improve and use malaria surveillance data to guide programming. Granular and timely data are critical to understanding geographic heterogeneity, appropriately defining and targeting interventions packages, and enabling timely decision-making at the operational level. Resources to be targeted based on surveillance data include vector control, case management commodities, outbreak responses, quality improvement interventions, and human resources, including community health workers, as they contribute to a more refined granularity of the surveillance system. Effectively transforming malaria surveillance into a core intervention will require strong global and national leadership, empowerment of subnational and local leaders, collaboration among development partners, and global coordination. Ensuring that national health systems include community health work can contribute to a successful transformation. It will require a strong supply chain to ensure that all suspected cases can be diagnosed and data reporting tools including appropriate electronic devices to provide timely data. Regular data quality audits, decentralized implementation, supportive supervision, data-informed decision-making processes, and harnessing technology for data analysis and visualization are needed to improve the capacity for data-driven decision-making at all levels. Finally, resources must be available to respond programmatically to these decisions.

Assessing Thailand's 1-3-7 surveillance strategy in accelerating malaria elimination

Background

Thailand’s strong malaria elimination programme relies on effective implementation of its 1-3-7 surveillance strategy, which was endorsed and implemented nationwide in 2016. For each confirmed malaria patient, the Ministry of Public Health’s Division of Vector Borne Diseases (DVBD) ensures completion of case notification within 1 day, case investigation within 3 days, and foci investigation within 7 days. To date, there has not been a comprehensive assessment of the performance and achievements of the 1-3-7 surveillance strategy although such results could help Thailand’s future malaria elimination strategic planning.

Methods

This study examined adherence to the 1-3-7 protocols, tracked progress against set targets, and examined geographic variations in implementation of the 1-3-7 strategy in the programme’s initial 5 years. An auto-regressive integrated moving average (ARIMA) time series analysis with seasonal decomposition assessed the plausible implementation effect of the 1-3-7 strategy on malaria incidence in the programme’s initial 5 years. The quantitative analysis included all confirmed malaria cases from public health and non-governmental community facilities from October 2014 to September 2021 (fiscal year [FY] 2015 to FY 2021) (n = 77,405). The spatial analysis included active foci with known geocoordinates that reported more than five cases from FY 2018 to FY 2021.

Results

From FY 2017 to FY 2021, on-time case notification improved from 24.4% to 89.3%, case investigations from 58.0% to 96.5%, and foci investigations from 37.9% to 87.2%. Adherence to timeliness protocols did not show statistically significant variation by area risk classification. However, adherence to 1-3-7 protocols showed a marked spatial heterogeneity among active foci, and the ARIMA model showed a statistically significant acceleration in the reduction of malaria incidence. The 1-3-7 strategy national indicators and targets in Thailand have shown progressive success, and most targets were achieved for FY 2021.

Conclusion

The results of Thailand’s 1-3-7 surveillance strategy are associated with a decreased incidence in the period following the adoption of the strategy although there is notable geographic variation. The DVBD will continue to implement and adapt the 1-3-7 strategy to accelerate progress toward malaria elimination. This assessment may be useful for domestic strategic planning and to other countries considering more intensive case and foci investigation and response strategies.

The relative impact of interventions on sympatric Plasmodium vivax and Plasmodium falciparum malaria: A systematic review

Background

In areas with both Plasmodium vivax and Plasmodium falciparum malaria, interventions can reduce the burden of both species but the impact may vary due to their different biology. Knowing the expected relative impact on the two species over time for vector- and drug-based interventions, and the factors affecting this, could help plan and evaluate intervention strategies.

Methods

For three interventions (treated bed nets (ITN), mass drug administration (MDA) and indoor residual spraying (IRS)), we identified studies providing information on the proportion of clinical illness and patent infections attributed to P. vivax over time using a literature search. The change in the proportion of malaria attributed to P. vivax up to two years since implementation was estimated using logistic regression accounting for clustering with random effects. Potential factors (intervention type, coverage, relapse pattern, transmission intensity, seasonality, initial proportion of P. vivax and round of intervention) were assessed.

Results

In total there were 55 studies found that led to 72 series of time-points for clinical case data and 69 series for patent infection data. The main reason of study exclusion was insufficient information on interventions. There was considerable variation in the proportion of malaria attributed to P. vivax over time by study and location for all of the interventions. Overall, there was an increase apart from MDA in the short-term. The potential factors could not be ruled in or out. Although not consistently significant, coverage, transmission intensity and relapse pattern are possible factors that explain some of the variation found.

Conclusion

While there are reports of an increase in the proportion of malaria due to P. vivax following interventions in the long-term, there was substantial variation for the shorter time-scales considered in this study (up to 24 months for IRS and ITN, and up to six months for MDA). The large variability points to the need for the monitoring of both species after an intervention. Studies should report intervention timing and characteristics to allow inclusion in systematic reviews.

Dynamic changes in genetic diversity, drug resistance mutations, and treatment outcomes of falciparum malaria from the low-transmission to the pre-elimination phase on the islands of São Tomé and Príncipe

Background

With effective vector control and case management, substantial progress has been made towards eliminating malaria on the islands of São Tomé and Príncipe (STP). This study assessed the dynamic changes in the genetic diversity of Plasmodium falciparum, the anti-malarial drug resistance mutations, and malaria treatment outcomes between 2010 and 2016 to provide insights for the prevention of malaria rebounding.

Methods

Polymorphic regions of merozoite surface proteins 1 and 2 (msp1 and msp2) were sequenced in 118 dried blood spots (DBSs) collected from malaria patients who had visited the Central Hospital in 2010–2016. Mutations in the multi-drug resistance I (pfmdr1), chloroquine resistance transporter (pfcrt), and kelch 13 (pfk13) genes were analysed by polymerase chain reaction-restriction fragment length polymorphism (PCR–RFLP) and sequencing in 111 DBSs. A total of 7482 cases that completed a 28-day follow-up were evaluated for treatment outcomes based on the microscopic results. Regression models were used to characterize factors associated with levels of parasite density and treatment failures.

Results

Parasite strains in STP showed significant changes during and after the peak incidence in 2012. The prevalent allelic type in msp1 changed from K1 to MAD20, and that in msp2 changed from 3D7/IC to FC27. The dominant alleles of drug-resistance markers were pfmdr1 86Y, 184F, D1246, and pfcrt 76 T (Y-F-D-T, 51.4%). The average parasite density in malaria cases declined threefold from low-transmission (2010–2013) to pre-elimination period (2014–2016). Logistic regression models showed that patients with younger age (OR for age = 0.97–0.98, p < 0.001), higher initial parasite density (log₁₀-transformed, OR = 1.44, p < 0.001), and receiving quinine treatment (compared to artemisinin-based combination therapy, OR = 1.91–1.96, p < 0.001) were more likely to experience treatment failures during follow-up.

Conclusions

Plasmodium falciparum in STP had experienced changes in prevalent strains, and increased mutation frequencies in drug-resistance genes from the low-transmission to the pre-elimination settings. Notably, patients with younger age and receiving quinine treatment were more likely to show parasitological treatment failure during follow-up. Therapeutic efficacy should be carefully monitored to inform future treatment policy in STP.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12936-021-04007-3.

Estimating malaria chemoprevention and vector control coverage using program and campaign data: A scoping review of current practices and opportunities

Background

Accurate estimation of intervention coverage is a vital component of malaria program monitoring and evaluation, both for process evaluation (how well program targets are achieved), and impact evaluation (whether intervention coverage had an impact on malaria burden). There is growing interest in maximizing the utility of program data to generate interim estimates of intervention coverage in the periods between large-scale cross-sectional surveys (the gold standard). As such, this study aimed to identify relevant concepts and themes that may guide future optimization of intervention coverage estimation using routinely collected data, or data collected during and following intervention campaigns, with a particular focus on strategies to define the denominator.

Methods

We conducted a scoping review of current practices to estimate malaria intervention coverage for insecticide-treated nets (ITNs); indoor residual spray (IRS); intermittent preventive treatment in pregnancy (IPTp); mass drug administration (MDA); and seasonal malaria chemoprevention (SMC) interventions; case management was excluded. Multiple databases were searched for relevant articles published from January 1, 2015 to June 1, 2018. Additionally, we identified and included other guidance relevant to estimating population denominators, with a focus on innovative techniques.

Results

While program data have the potential to provide intervention coverage data, there are still substantial challenges in selecting appropriate denominators. The review identified a lack of consistency in how coverage was defined and reported for each intervention type, with denominator estimation methods not clearly or consistently reported, and denominator estimates rarely triangulated with other data sources to present the feasible range of denominator values and consequently the range of likely coverage estimates.

Conclusions

Though household survey-based estimates of intervention coverage remain the gold standard, efforts should be made to further standardize practices for generating interim measurements of intervention coverage from program data, and for estimating and reporting population denominators. This includes fully describing any projections or adjustments made to existing census or population data, exploring opportunities to validate available data by comparing with other sources, and explaining how the denominator has been restricted (or not) to reflect exclusion criteria.

Prevalence and seroprevalence of Plasmodium infection in Myanmar reveals highly heterogeneous transmission and a large hidden reservoir of infection

Malaria incidence in Myanmar has significantly reduced over recent years, however, completeness and timeliness of incidence data remain a challenge. The first ever nationwide malaria infection and seroprevalence survey was conducted in Myanmar in 2015 to better understand malaria epidemiology and highlight gaps in Annual Parasite Index (API) data. The survey was a cross-sectional two-stage stratified cluster-randomised household survey conducted from July-October 2015. Blood samples were collected from household members for ultra-sensitive PCR and serology testing for P. falciparum and P. vivax. Data was gathered on demography and a priori risk factors of participants. Data was analysed nationally and within each of four domains defined by API data. Prevalence and seroprevalence of malaria were 0.74% and 16.01% nationwide, respectively. Prevalent infection was primarily asymptomatic P. vivax, while P. falciparum was predominant in serology. There was large heterogeneity between villages and by domain. At the township level, API showed moderate correlation with P. falciparum seroprevalence. Risk factors for infection included socioeconomic status, domain, and household ownership of nets. Three K13 P. falciparum mutants were found in highly prevalent villages. There results highlight high heterogeneity of both P. falciparum and P. vivax transmission between villages, accentuated by a large hidden reservoir of asymptomatic P. vivax infection not captured by incidence data, and representing challenges for malaria elimination. Village-level surveillance and stratification to guide interventions to suit local context and targeting of transmission foci with evidence of drug resistance would aid elimination efforts.

World Malaria Day 2021: Commemorating 15 Years of Contribution by the United States President's Malaria Initiative

World Malaria Day 2021 coincides with the 15th anniversary of the United States President’s Malaria Initiative (PMI) and follows the first anniversary of the declaration of the coronavirus disease (COVID-19) pandemic. From 2006 to the present, the PMI has led to considerable country-managed progress in malaria prevention, care, and treatment in 24 of the highest-burden countries in sub-Saharan Africa and three countries in the Southeast Asia Greater Mekong subregion. Furthermore, it has contributed to a 29% reduction in malaria cases and a 60% reduction in the death rates in sub-Saharan Africa. In this context of progress, substantial heterogeneity persists within and between countries, such that malaria control programs can seek subnational elimination in some populations but others still experience substantial malaria disease and death. During the COVID-19 pandemic, most malaria programs have shown resilience in delivering prevention campaigns, but many experienced important disruptions in their care and treatment of malaria illness. Confronting the COVID-19 pandemic and building on the progress against malaria will require fortitude, including strengthening the quality and ensuring the safety and resiliency of the existing programs, extending services to those currently not reached, and supporting the people and partners closest to those in need.

Near-term climate change impacts on sub-national malaria transmission

The role of climate change on global malaria is often highlighted in World Health Organisation reports. We modelled a Zambian socio-environmental dataset from 2000 to 2016, against malaria trends and investigated the relationship of near-term environmental change with malaria incidence using Bayesian spatio-temporal, and negative binomial mixed regression models. We introduced the diurnal temperature range (DTR) as an alternative environmental measure to the widely used mean temperature. We found substantial sub-national near-term variations and significant associations with malaria incidence-trends. Significant spatio-temporal shifts in DTR/environmental predictors influenced malaria incidence-rates, even in areas with declining trends. We highlight the impact of seasonally sensitive DTR, especially in the first two quarters of the year and demonstrate how substantial investment in intervention programmes is negatively impacted by near-term climate change, most notably since 2010. We argue for targeted seasonally-sensitive malaria chemoprevention programmes.

A Narrative Review of Methods for Causal Inference and Associated Educational Resources

BACKGROUND AND OBJECTIVES

Root cause analysis involves evaluation of causal relationships between exposures (or interventions) and adverse outcomes, such as identification of direct (eg, medication orders missed) and root causes (eg, clinician's fatigue and workload) of adverse rare events. To assess causality requires either randomization or sophisticated methods applied to carefully designed observational studies. In most cases, randomized trials are not feasible in the context of root cause analysis. Using observational data for causal inference, however, presents many challenges in both the design and analysis stages. Methods for observational causal inference often fall outside the toolbox of even well-trained statisticians, thus necessitating workforce training.

METHODS

This article synthesizes the key concepts and statistical perspectives for causal inference, and describes available educational resources, with a focus on observational clinical data. The target audience for this review is clinical researchers with training in fundamental statistics or epidemiology, and statisticians collaborating with those researchers.

RESULTS

The available literature includes a number of textbooks and thousands of review articles. However, using this literature for independent study or clinical training programs is extremely challenging for numerous reasons. First, the published articles often assume an advanced technical background with different notations and terminology. Second, they may be written from any number of perspectives across statistics, epidemiology, computer science, or philosophy. Third, the methods are rapidly expanding and thus difficult to capture within traditional publications. Fourth, even the most fundamental aspects of causal inference (eg, framing the causal question as a target trial) often receive little or no coverage. This review presents an overview of (1) key concepts and frameworks for causal inference and (2) online documents that are publicly available for better assisting researchers to gain the necessary perspectives for functioning effectively within a multidisciplinary team.

CONCLUSION

A familiarity with causal inference methods can help risk managers empirically verify, from observed events, the true causes of adverse sentinel events.