Reflection on modern methods: when worlds collide-prediction, machine learning and causal inference

Machine Learning Methods to Estimate Individualized Treatment Effects for Use in Health Technology Assessment

BACKGROUND

Recent developments in causal inference and machine learning (ML) allow for the estimation of individualized treatment effects (ITEs), which reveal whether treatment effectiveness varies according to patients' observed covariates. ITEs can be used to stratify health policy decisions according to individual characteristics and potentially achieve greater population health. Little is known about the appropriateness of available ML methods for use in health technology assessment.

METHODS

In this scoping review, we evaluate ML methods available for estimating ITEs, aiming to help practitioners assess their suitability in health technology assessment. We present a taxonomy of ML approaches, categorized by key challenges in health technology assessment using observational data, including handling time-varying confounding and time-to event data and quantifying uncertainty.

RESULTS

We found a wide range of algorithms for simpler settings with baseline confounding and continuous or binary outcomes. Not many ML algorithms can handle time-varying or unobserved confounding, and at the time of writing, no ML algorithm was capable of estimating ITEs for time-to-event outcomes while accounting for time-varying confounding. Many of the ML algorithms that estimate ITEs in longitudinal settings do not formally quantify uncertainty around the point estimates.

LIMITATIONS

This scoping review may not cover all relevant ML methods and algorithms as they are continuously evolving.

CONCLUSIONS

Existing ML methods available for ITE estimation are limited in handling important challenges posed by observational data when used for cost-effectiveness analysis, such as time-to-event outcomes, time-varying and hidden confounding, or the need to estimate sampling uncertainty around the estimates.

IMPLICATIONS

ML methods are promising but need further development before they can be used to estimate ITEs for health technology assessments.

HIGHLIGHTS

Estimating individualized treatment effects (ITEs) using observational data and machine learning (ML) can support personalized treatment advice and help deliver more customized information on the effectiveness and cost-effectiveness of health technologies.ML methods for ITE estimation are mostly designed for handling confounding at baseline but not time-varying or unobserved confounding. The few models that account for time-varying confounding are designed for continuous or binary outcomes, not time-to-event outcomes.Not all ML methods for estimating ITEs can quantify the uncertainty of their predictions.Future work on developing ML that addresses the concerns summarized in this review is needed before these methods can be widely used in clinical and health technology assessment-like decision making.

Designing target trials using electronic health records: A case study of second-line disease-modifying anti-rheumatic drugs and cardiovascular disease outcomes in patients with rheumatoid arthritis

BACKGROUND

Emulation of the "target trial" (TT), a hypothetical pragmatic randomized controlled trial (RCT), using observational data can be used to mitigate issues commonly encountered in comparative effectiveness research (CER) when randomized trials are not logistically, ethically, or financially feasible. However, cardiovascular (CV) health research has been slow to adopt TT emulation. Here, we demonstrate the design and analysis of a TT emulation using electronic health records to study the comparative effectiveness of the addition of a disease-modifying anti-rheumatic drug (DMARD) to a regimen of methotrexate on CV events among rheumatoid arthritis (RA) patients.

METHODS

We used data from an electronic medical records-based cohort of RA patients from Northwestern Medicine to emulate the TT. Follow-up began 3 months after initial prescription of MTX (2000-2020) and included all available follow-up through June 30, 2020. Weighted pooled logistic regression was used to estimate differences in CVD risk and survival. Cloning was used to handle immortal time bias and weights to improve baseline and time-varying covariate imbalance.

RESULTS

We identified 659 eligible people with RA with average follow-up of 46 months and 31 MACE events. The month 24 adjusted risk difference for MACE comparing initiation vs non-initiation of a DMARD was -1.47% (95% confidence interval [CI]: -4.74, 1.95%), and the marginal hazard ratio (HR) was 0.72 (95% CI: 0.71, 1.23). In analyses subject to immortal time bias, the HR was 0.62 (95% CI: 0.29-1.44).

CONCLUSION

In this sample, we did not observe evidence of differences in risk of MACE, a finding that is compatible with previously published meta-analyses of RCTs. Thoughtful application of the TT framework provides opportunities to conduct CER in observational data. Benchmarking results of observational analyses to previously published RCTs can lend credibility to interpretation.

Inverse probability weighting for self-selection bias correction in the investigation of social inequality in mortality

BACKGROUND

Empirical evaluation of inverse probability weighting (IPW) for self-selection bias correction is inaccessible without the full source population. We aimed to: (i) investigate how self-selection biases frequency and association measures and (ii) assess self-selection bias correction using IPW in a cohort with register linkage.

METHODS

The source population included 17 936 individuals invited to the Copenhagen Aging and Midlife Biobank during 2009-11 (ages 49-63 years). Participants counted 7185 (40.1%). Register data were obtained for every invited person from 7 years before invitation to the end of 2020. The association between education and mortality was estimated using Cox regression models among participants, IPW participants and the source population.

RESULTS

Participants had higher socioeconomic position and fewer hospital contacts before baseline than the source population. Frequency measures of participants approached those of the source population after IPW. Compared with primary/lower secondary education, upper secondary, short tertiary, bachelor and master/doctoral were associated with reduced risk of death among participants (adjusted hazard ratio [95% CI]: 0.60 [0.46; 0.77], 0.68 [0.42; 1.11], 0.37 [0.25; 0.54], 0.28 [0.18; 0.46], respectively). IPW changed the estimates marginally (0.59 [0.45; 0.77], 0.57 [0.34; 0.93], 0.34 [0.23; 0.50], 0.24 [0.15; 0.39]) but not only towards those of the source population (0.57 [0.51; 0.64], 0.43 [0.32; 0.60], 0.38 [0.32; 0.47], 0.22 [0.16; 0.29]).

CONCLUSIONS

Frequency measures of study participants may not reflect the source population in the presence of self-selection, but the impact on association measures can be limited. IPW may be useful for (self-)selection bias correction, but the returned results can still reflect residual or other biases and random errors.

Does cardiorespiratory fitness mediate or moderate the association between mid-life physical activity frequency and cognitive function? findings from the 1958 British birth cohort study

BACKGROUND

Physical activity (PA) is associated with a lower risk of cognitive decline and all-cause dementia in later life. Pathways underpinning this association are unclear but may involve either mediation and/or moderation by cardiorespiratory fitness (CRF).

METHODS

Data on PA frequency (exposure) at 42y, non-exercise testing CRF (NETCRF, mediator/moderator) at 45y and overall cognitive function (outcome) at 50y were obtained from 9,385 participants (50.8% female) in the 1958 British birth cohort study. We used a four-way decomposition approach to examine the relative contributions of mediation and moderation by NETCRF on the association between PA frequency at 42y and overall cognitive function at 50y.

RESULTS

In males, the estimated overall effect of 42y PA ≥once per week (vs.

CONCLUSION

We present the first evidence from a four-way decomposition analysis of the potential contribution that CRF plays in the relationship between mid-life PA frequency and subsequent cognitive function. Our lack of evidence in support of CRF mediating or moderating the PA frequency-cognitive function association suggests that other pathways underpin this association.

Collapse

Key Words Collapse

MESH Headings

• Humans
• Cardiorespiratory Fitness/physiology
• Male
• Female
• Cognition/physiology
• Exercise/physiology
• United Kingdom/epidemiology
• Adult
• Middle Aged
• Birth Cohort
• Cohort Studies

Collapse

Grants

• Medical Research Council
• British Heart Foundation

Collapse

Affiliation(s)

Tom Norris Faculty of Medical Sciences, Institute of Sport, Division of Surgery and Interventional Science, Exercise and Health, UCL, London, United Kingdom
John J. Mitchell Faculty of Medical Sciences, Institute of Sport, Division of Surgery and Interventional Science, Exercise and Health, UCL, London, United Kingdom
Joanna M. Blodgett Faculty of Medical Sciences, Institute of Sport, Division of Surgery and Interventional Science, Exercise and Health, UCL, London, United Kingdom
Mark Hamer Faculty of Medical Sciences, Institute of Sport, Division of Surgery and Interventional Science, Exercise and Health, UCL, London, United Kingdom
Snehal M. Pinto Pereira Faculty of Medical Sciences, Institute of Sport, Division of Surgery and Interventional Science, Exercise and Health, UCL, London, United Kingdom

Collapse

A computable biomedical knowledge system: Toward rapidly building candidate-directed acyclic graphs

AIM

It is essential for health researchers to have a systematic understanding of third-party variables that influence both the exposure and outcome under investigation, as shown by a directed acyclic graph (DAG). The traditional construction of DAGs through literature review and expert knowledge often needs to be more systematic and consistent, leading to potential biases. We try to introduce an automatic approach to building network linking variables of interest.

METHODS

Large-scale text mining from medical literature was utilized to construct a conceptual network based on the Semantic MEDLINE Database (SemMedDB). SemMedDB is a PubMed-scale repository of the "concept-relation-concept" triple format. Relations between concepts are categorized as Excitatory, Inhibitory, or General.

RESULTS

To facilitate the use of large-scale triple sets in SemMedDB, we have developed a computable biomedical knowledge (CBK) system (https://cbk.bjmu.edu.cn/), a website that enables direct retrieval of related publications and their corresponding triples without the necessity of writing SQL statements. Three case studies were elaborated to demonstrate the applications of the CBK system.

CONCLUSIONS

The CBK system is openly available and user-friendly for rapidly capturing a set of influencing factors for a phenotype and building candidate DAGs between exposure-outcome variables. It could be a valuable tool to reduce the exploration time in considering relationships between variables, and constructing a DAG. A reliable and standardized DAG could significantly improve the design and interpretation of observational health research.

Change in employment status and its causal effect on suicidal ideation and depressive symptoms: A marginal structural model with machine learning algorithms

OBJECTIVE

This study aimed to assess the causal effect of a change in employment status on suicidal ideation and depressive symptoms by applying marginal structural models (MSM) with machine-learning (ML) algorithms.

METHODS

We analyzed data from the 8-15th waves (2013-2020) of the Korean Welfare Panel Study, a nationally representative longitudinal dataset. Our analysis included 13 294 observations from 3621 participants who had standard employment at baseline (2013-2019). Based on employment status at follow-up year (2014-2020), respondents were classified into two groups: (i) maintained standard employment (reference group), (ii) changed to non-standard employment. Suicidal ideation during the past year and depressive symptoms during the past week were assessed through self-report questionnaire. To apply the ML algorithms to the MSM, we conducted eight ML algorithms to build the propensity score indicating a change in employment status. Then, we applied the MSM to examine the causal effect by using inverse probability weights calculated based on the propensity score from ML algorithms.

RESULTS

The random forest algorithm performed best among all algorithms, showing the highest area under the curve 0.702, 95% confidence interval (CI) 0.686-0.718. In the MSM with the random forest algorithm, workers who changed from standard to non-standard employment were 2.07 times more likely to report suicidal ideation compared to those who maintained standard employment (95% CI 1.16-3.70). A similar trend was observed in the analysis of depressive symptoms.

CONCLUSIONS

This study found that a change in employment status could lead to a higher risk of suicidal ideation and depressive symptoms.

An economic evaluation of chronic obstructive pulmonary disease clinical pathway in Saskatchewan, Canada: Data-driven techniques to identify cost-effectiveness among patient subgroups

BACKGROUND

Saskatchewan has implemented care pathways for several common health conditions. To date, there has not been any cost-effectiveness evaluation of care pathways in the province. The objective of this study was to evaluate the real-world cost-effectiveness of a chronic obstructive pulmonary disease (COPD) care pathway program in Saskatchewan.

METHODS

Using patient-level administrative health data, we identified adults (35+ years) with COPD diagnosis recruited into the care pathway program in Regina between April 1, 2018, and March 31, 2019 (N = 759). The control group comprised adults (35+ years) with COPD who lived in Saskatoon during the same period (N = 759). The control group was matched to the intervention group using propensity scores. Costs were calculated at the patient level. The outcome measure was the number of days patients remained without experiencing COPD exacerbation within 1-year follow-up. Both manual and data-driven policy learning approaches were used to assess heterogeneity in the cost-effectiveness by patient demographic and disease characteristics. Bootstrapping was used to quantify uncertainty in the results.

RESULTS

In the overall sample, the estimates indicate that the COPD care pathway was not cost-effective using the willingness to pay (WTP) threshold values in the range of $1,000 and $5,000/exacerbation day averted. The manual subgroup analyses show the COPD care pathway was dominant among patients with comorbidities and among patients aged 65 years or younger at the WTP threshold of $2000/exacerbation day averted. Although similar profiles as those identified in the manual subgroup analyses were confirmed, the data-driven policy learning approach suggests more nuanced demographic and disease profiles that the care pathway would be most appropriate for.

CONCLUSIONS

Both manual subgroup analysis and data-driven policy learning approach showed that the COPD care pathway consistently produced cost savings and better health outcomes among patients with comorbidities or among those relatively younger. The care pathway was not cost-effective in the entire sample.

Flexible Machine Learning Estimation of Conditional Average Treatment Effects: A Blessing and a Curse

Causal inference from observational data requires untestable identification assumptions. If these assumptions apply, machine learning methods can be used to study complex forms of causal effect heterogeneity. Recently, several machine learning methods were developed to estimate the conditional average treatment effect (ATE). If the features at hand cannot explain all heterogeneity, the individual treatment effects can seriously deviate from the conditional ATE. In this work, we demonstrate how the distributions of the individual treatment effect and the conditional ATE can differ when a causal random forest is applied. We extend the causal random forest to estimate the difference in conditional variance between treated and controls. If the distribution of the individual treatment effect equals that of the conditional ATE, this estimated difference in variance should be small. If they differ, an additional causal assumption is necessary to quantify the heterogeneity not captured by the distribution of the conditional ATE. The conditional variance of the individual treatment effect can be identified when the individual effect is independent of the outcome under no treatment given the measured features. Then, in the cases where the individual treatment effect and conditional ATE distributions differ, the extended causal random forest can appropriately estimate the variance of the individual treatment effect distribution, whereas the causal random forest fails to do so.

Real-world causal evidence for planned predictive enrichment in critical care trials: A scoping review

BACKGROUND

Randomised clinical trials in critical care are prone to inconclusiveness due, in part, to undue optimism about effect sizes and suboptimal accounting for heterogeneous treatment effects. Although causal evidence from rich real-world critical care can help overcome these challenges by informing predictive enrichment, no overview exists.

METHODS

We conducted a scoping review, systematically searching 10 general and speciality journals for reports published on or after 1 January 2018, of randomised clinical trials enrolling adult critically ill patients. We collected trial metadata on 22 variables including recruitment period, intervention type and early stopping (including reasons) as well as data on the use of causal evidence from secondary data for planned predictive enrichment.

RESULTS

We screened 9020 records and included 316 unique RCTs with a total of 268,563 randomised participants. One hundred seventy-three (55%) trials tested drug interventions, 101 (32%) management strategies and 42 (13%) devices. The median duration of enrolment was 2.2 (IQR: 1.3-3.4) years, and 83% of trials randomised less than 1000 participants. Thirty-six trials (11%) were restricted to COVID-19 patients. Of the 55 (17%) trials that stopped early, 23 (42%) used predefined rules; futility, slow enrolment and safety concerns were the commonest stopping reasons. None of the included RCTs had used causal evidence from secondary data for planned predictive enrichment.

CONCLUSION

Work is needed to harness the rich multiverse of critical care data and establish its utility in critical care RCTs. Such work will likely need to leverage methodology from interventional and analytical epidemiology as well as data science.

Artificial intelligence for dementia prevention

INTRODUCTION

A wide range of modifiable risk factors for dementia have been identified. Considerable debate remains about these risk factors, possible interactions between them or with genetic risk, and causality, and how they can help in clinical trial recruitment and drug development. Artificial intelligence (AI) and machine learning (ML) may refine understanding.

METHODS

ML approaches are being developed in dementia prevention. We discuss exemplar uses and evaluate the current applications and limitations in the dementia prevention field.

RESULTS

Risk-profiling tools may help identify high-risk populations for clinical trials; however, their performance needs improvement. New risk-profiling and trial-recruitment tools underpinned by ML models may be effective in reducing costs and improving future trials. ML can inform drug-repurposing efforts and prioritization of disease-modifying therapeutics.

DISCUSSION

ML is not yet widely used but has considerable potential to enhance precision in dementia prevention.

HIGHLIGHTS

Artificial intelligence (AI) is not widely used in the dementia prevention field. Risk-profiling tools are not used in clinical practice. Causal insights are needed to understand risk factors over the lifespan. AI will help personalize risk-management tools for dementia prevention. AI could target specific patient groups that will benefit most for clinical trials.

What is circulating factor disease and how is it currently explained?

Nephrotic syndrome (NS) consists of the clinical triad of hypoalbuminaemia, high levels of proteinuria and oedema, and describes a heterogeneous group of disease processes with different underlying drivers. The existence of circulating factor disease (CFD) as a driver of NS has been epitomised by a subset of patients who exhibit disease recurrence after transplantation, alongside laboratory work. Several circulating factors have been proposed and studied, broadly grouped into protease components such as soluble urokinase-type plasminogen activator (suPAR), hemopexin (Hx) and calcium/calmodulin-serine protease kinase (CASK), and other circulating proteases, and immune components such as TNF-α, CD40 and cardiotrophin-like cytokine-1 (CLC-1). While currently there is no definitive way of assessing risk of CFD pre-transplantation, promising work is emerging through the study of 'multi-omic' bioinformatic data from large national cohorts and biobanks.

Propensity-based standardization to enhance the validation and interpretation of prediction model discrimination for a target population

External validation of the discriminative ability of prediction models is of key importance. However, the interpretation of such evaluations is challenging, as the ability to discriminate depends on both the sample characteristics (ie, case-mix) and the generalizability of predictor coefficients, but most discrimination indices do not provide any insight into their respective contributions. To disentangle differences in discriminative ability across external validation samples due to a lack of model generalizability from differences in sample characteristics, we propose propensity-weighted measures of discrimination. These weighted metrics, which are derived from propensity scores for sample membership, are standardized for case-mix differences between the model development and validation samples, allowing for a fair comparison of discriminative ability in terms of model characteristics in a target population of interest. We illustrate our methods with the validation of eight prediction models for deep vein thrombosis in 12 external validation data sets and assess our methods in a simulation study. In the illustrative example, propensity score standardization reduced between-study heterogeneity of discrimination, indicating that between-study variability was partially attributable to case-mix. The simulation study showed that only flexible propensity-score methods (allowing for non-linear effects) produced unbiased estimates of model discrimination in the target population, and only when the positivity assumption was met. Propensity score-based standardization may facilitate the interpretation of (heterogeneity in) discriminative ability of a prediction model as observed across multiple studies, and may guide model updating strategies for a particular target population. Careful propensity score modeling with attention for non-linear relations is recommended.

The influence of the co-exposure to polycyclic aromatic hydrocarbons and toxic metals on DNA damage in brazilian lactating women and their infants: A cross-sectional study using machine learning approaches

Polycyclic aromatic hydrocarbons (PAHs) and toxic metals are widely spread pollutants of public health concern. The co-contamination of these chemicals in the environment is frequent, but relatively little is known about their combined toxicities. In this context, this study aimed to evaluate the influence of the co-exposure to PAHs and toxic metals on DNA damage in Brazilian lactating women and their infants using machine learning approaches. Data were collected from an observational, cross-sectional study with 96 lactating women and 96 infants living in two cities. The exposure to these pollutants was estimated by determining urinary levels of seven mono-hydroxylated PAH metabolites and the free form of three toxic metals. 8-Hydroxydeoxyguanosine (8-OHdG) levels in the urine were used as the oxidative stress biomarker and set as the outcome. Individual sociodemographic factors were also collected using questionnaires. Sixteen machine learning algorithms were trained using 10-fold cross-validation to investigate the associations of urinary OH-PAHs and metals with 8-OHdG levels. This approach was also compared with models attained by multiple linear regression. The results showed that the urinary concentration of OH-PAHs was highly correlated between the mothers and their infants. Multiple linear regression did not show a statistically significant association between the contaminants and urinary 8OHdG levels. Machine learning models indicated that all investigated variables did not present predictive performance on 8-OHdG concentrations. In conclusion, PAHs and toxic metals were not associated with 8-OHdG levels in Brazilian lactating women and their infants. These novelty and originality results were achieved even after applying sophisticated statistical models to capture non-linear relationships. However, these findings should be interpreted cautiously because the exposure to the studied contaminants was considerably low, which may not reflect other populations at risk.

Estimating the impact of the COVID-19 pandemic on dengue in Brazil

Atypical dengue prevalence was observed in 2020 in many dengue-endemic countries, including Brazil. Evidence suggests that the pandemic disrupted not only dengue dynamics due to changes in mobility patterns, but also several aspects of dengue surveillance, such as care seeking behavior, care availability, and monitoring systems. However, we lack a clear understanding of the overall impact on dengue in different parts of the country as well as the role of individual causal drivers. In this study, we estimated the gap between expected and observed dengue cases in 2020 using an interrupted time series design with forecasts from a neural network and a structural Bayesian time series model. We also decomposed the gap into the impacts of climate conditions, pandemic-induced changes in reporting, human susceptibility, and human mobility. We find that there is considerable variation across the country in both overall pandemic impact on dengue and the relative importance of individual drivers. Increased understanding of the causal mechanisms driving the 2020 dengue season helps mitigate some of the data gaps caused by the COVID-19 pandemic and is critical to developing effective public health interventions to control dengue in the future.

Predicting high health-cost users among people with cardiovascular disease using machine learning and nationwide linked social administrative datasets

OBJECTIVES

To optimise planning of public health services, the impact of high-cost users needs to be considered. However, most of the existing statistical models for costs do not include many clinical and social variables from administrative data that are associated with elevated health care resource use, and are increasingly available. This study aimed to use machine learning approaches and big data to predict high-cost users among people with cardiovascular disease (CVD).

METHODS

We used nationally representative linked datasets in New Zealand to predict CVD prevalent cases with the most expensive cost belonging to the top quintiles by cost. We compared the performance of four popular machine learning models (L1-regularised logistic regression, classification trees, k-nearest neighbourhood (KNN) and random forest) with the traditional regression models.

RESULTS

The machine learning models had far better accuracy in predicting high health-cost users compared with the logistic models. The harmony score F1 (combining sensitivity and positive predictive value) of the machine learning models ranged from 30.6% to 41.2% (compared with 8.6-9.1% for the logistic models). Previous health costs, income, age, chronic health conditions, deprivation, and receiving a social security benefit were among the most important predictors of the CVD high-cost users.

CONCLUSIONS

This study provides additional evidence that machine learning can be used as a tool together with big data in health economics for identification of new risk factors and prediction of high-cost users with CVD. As such, machine learning may potentially assist with health services planning and preventive measures to improve population health while potentially saving healthcare costs.

Implementing machine learning methods with complex survey data: Lessons learned on the impacts of accounting sampling weights in gradient boosting

Despite the prominent use of complex survey data and the growing popularity of machine learning methods in epidemiologic research, few machine learning software implementations offer options for handling complex samples. A major challenge impeding the broader incorporation of machine learning into epidemiologic research is incomplete guidance for analyzing complex survey data, including the importance of sampling weights for valid prediction in target populations. Using data from 15, 820 participants in the 1988-1994 National Health and Nutrition Examination Survey cohort, we determined whether ignoring weights in gradient boosting models of all-cause mortality affected prediction, as measured by the F1 score and corresponding 95% confidence intervals. In simulations, we additionally assessed the impact of sample size, weight variability, predictor strength, and model dimensionality. In the National Health and Nutrition Examination Survey data, unweighted model performance was inflated compared to the weighted model (F1 score 81.9% [95% confidence interval: 81.2%, 82.7%] vs 77.4% [95% confidence interval: 76.1%, 78.6%]). However, the error was mitigated if the F1 score was subsequently recalculated with observed outcomes from the weighted dataset (F1: 77.0%; 95% confidence interval: 75.7%, 78.4%). In simulations, this finding held in the largest sample size (N = 10,000) under all analytic conditions assessed. For sample sizes <5,000, sampling weights had little impact in simulations that more closely resembled a simple random sample (low weight variability) or in models with strong predictors, but findings were inconsistent under other analytic scenarios. Failing to account for sampling weights in gradient boosting models may limit generalizability for data from complex surveys, dependent on sample size and other analytic properties. In the absence of software for configuring weighted algorithms, post-hoc re-calculations of unweighted model performance using weighted observed outcomes may more accurately reflect model prediction in target populations than ignoring weights entirely.

Big data, machine learning, and population health: predicting cognitive outcomes in childhood

The application of machine learning (ML) to address population health challenges has received much less attention than its application in the clinical setting. One such challenge is addressing disparities in early childhood cognitive development-a complex public health issue rooted in the social determinants of health, exacerbated by inequity, characterised by intergenerational transmission, and which will continue unabated without novel approaches to address it. Early life, the period of optimal neuroplasticity, presents a window of opportunity for early intervention to improve cognitive development. Unfortunately for many, this window will be missed, and intervention may never occur or occur only when overt signs of cognitive delay manifest. In this review, we explore the potential value of ML and big data analysis in the early identification of children at risk for poor cognitive outcome, an area where there is an apparent dearth of research. We compare and contrast traditional statistical methods with ML approaches, provide examples of how ML has been used to date in the field of neurodevelopmental disorders, and present a discussion of the opportunities and risks associated with its use at a population level. The review concludes by highlighting potential directions for future research in this area. IMPACT: To date, the application of machine learning to address population health challenges in paediatrics lags behind other clinical applications. This review provides an overview of the public health challenge we face in addressing disparities in childhood cognitive development and focuses on the cornerstone of early intervention. Recent advances in our ability to collect large volumes of data, and in analytic capabilities, provide a potential opportunity to improve current practices in this field. This review explores the potential role of machine learning and big data analysis in the early identification of children at risk for poor cognitive outcomes.

Childcare attendance and risk of infections in childhood and adolescence

Background

It has been suggested that the transiently increased infection risk following childcare enrolment is compensated by decreased infection risk later in childhood and adolescence. We investigated how childcare enrolment affected rates of antimicrobial-treated infections during childhood and adolescence.

Methods

In a register-based cohort study of all children born in Denmark 1997–2014 with available exposure information (n = 1 007 448), we assessed the association between childcare enrolment before age 6 years and infection risks up to age 20 years, using antimicrobial exposure as proxy for infections. Nationwide childcare and prescription data were used. We estimated infection rates and the cumulative number of infections using adjusted Poisson regression models.

Results

We observed 4 599 993 independent episodes of infection (antimicrobial exposure) during follow-up. Childcare enrolment transiently increased infection rates; the younger the child, the greater the increase. The resulting increased cumulative number of infections associated with earlier age at childcare enrolment was not compensated by lower infection risk later in childhood or adolescence. Accordingly, children enrolled in childcare before age 12 months had experienced 0.5–0.7 more infections at age 6 years (in total 4.5–5.1 infections) than peers enrolled at age 3 years, differences that persisted throughout adolescence. The type of childcare had little impact on infection risks.

Conclusions

Early age at childcare enrolment is associated with a modest increase in the cumulative number of antimicrobial-treated infections at all ages through adolescence. Emphasis should be given to disrupting infectious disease transmission in childcare facilities through prevention strategies with particular focus on the youngest children.

Causal inference for planning randomised critical care trials: Protocol for a scoping review

BACKGROUND

Randomised clinical trials in critical care are prone to inconclusiveness owing, in part, to undue optimism about effect sizes and suboptimal accounting for heterogeneous treatment effects. Planned predictive enrichment based on secondary critical care data (often very rich with respect to both data types and temporal granularity) and causal inference methods may help overcome these challenges, but no overview exists about their use to this end.

METHODS

We will conduct a scoping review to assess the extent and nature of the use of causal inference from secondary data for planned predictive enrichment of randomised clinical trials in critical care. We will systematically search 10 general and specialty journals for reports published on or after 1 January 2018, of randomised clinical trials enrolling adult critically ill patients. We will collect trial metadata (e.g., recruitment period and phase) and, when available, information pertaining to the focus of the review (predictive enrichment based on causal inference estimates from secondary data): causal inference methods, estimation techniques and software used; types of patient populations; data provenance, types and models; and the availability of the data (public or not). The results will be reported in a descriptive manner.

DISCUSSION

The outlined scoping review aims to assess the use of causal inference methods and secondary data for planned predictive enrichment in randomised critical care trials. This will help guide methodological improvements to increase the utility, and facilitate the use, of causal inference estimates when planning such trials in the future.

Mapping of machine learning approaches for description, prediction, and causal inference in the social and health sciences

Machine learning (ML) methodology used in the social and health sciences needs to fit the intended research purposes of description, prediction, or causal inference. This paper provides a comprehensive, systematic meta-mapping of research questions in the social and health sciences to appropriate ML approaches by incorporating the necessary requirements to statistical analysis in these disciplines. We map the established classification into description, prediction, counterfactual prediction, and causal structural learning to common research goals, such as estimating prevalence of adverse social or health outcomes, predicting the risk of an event, and identifying risk factors or causes of adverse outcomes, and explain common ML performance metrics. Such mapping may help to fully exploit the benefits of ML while considering domain-specific aspects relevant to the social and health sciences and hopefully contribute to the acceleration of the uptake of ML applications to advance both basic and applied social and health sciences research.

Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease

The medical field has seen a rapid increase in the development of artificial intelligence (AI)-based prediction models. With the introduction of such AI-based prediction model tools and software in cardiovascular patient care, the cardiovascular researcher and healthcare professional are challenged to understand the opportunities as well as the limitations of the AI-based predictions. In this article, we present 12 critical questions for cardiovascular health professionals to ask when confronted with an AI-based prediction model. We aim to support medical professionals to distinguish the AI-based prediction models that can add value to patient care from the AI that does not.

Recommendations for the use of propensity score methods in multiple sclerosis research

BACKGROUND

With many disease-modifying therapies currently approved for the management of multiple sclerosis, there is a growing need to evaluate the comparative effectiveness and safety of those therapies from real-world data sources. Propensity score methods have recently gained popularity in multiple sclerosis research to generate real-world evidence. Recent evidence suggests, however, that the conduct and reporting of propensity score analyses are often suboptimal in multiple sclerosis studies.

OBJECTIVES

To provide practical guidance to clinicians and researchers on the use of propensity score methods within the context of multiple sclerosis research.

METHODS

We summarize recommendations on the use of propensity score matching and weighting based on the current methodological literature, and provide examples of good practice.

RESULTS

Step-by-step recommendations are presented, starting with covariate selection and propensity score estimation, followed by guidance on the assessment of covariate balance and implementation of propensity score matching and weighting. Finally, we focus on treatment effect estimation and sensitivity analyses.

CONCLUSION

This comprehensive set of recommendations highlights key elements that require careful attention when using propensity score methods.

Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force

Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.

Marginal structural models using calibrated weights with SuperLearner: application to type II diabetes cohort

As different scientific disciplines begin to converge on machine learning for causal inference, we demonstrate the application of machine learning algorithms in the context of longitudinal causal estimation using electronic health records. Our aim is to formulate a marginal structural model for estimating diabetes care provisions in which we envisioned hypothetical (i.e. counterfactual) dynamic treatment regimes using a combination of drug therapies to manage diabetes: metformin, sulfonylurea and SGLT-2i. The binary outcome of diabetes care provisions was defined using a composite measure of chronic disease prevention and screening elements [27] including (i) primary care visit, (ii) blood pressure, (iii) weight, (iv) hemoglobin A1c, (v) lipid, (vi) ACR, (vii) eGFR and (viii) statin medication. We used several statistical learning algorithms to describe causal relationships between the prescription of three common classes of diabetes medications and quality of diabetes care using the electronic health records contained in National Diabetes Repository. In particular, we generated an ensemble of statistical learning algorithms using the SuperLearner framework based on the following base learners: (i) least absolute shrinkage and selection operator, (ii) ridge regression, (iii) elastic net, (iv) random forest, (v) gradient boosting machines, and (vi) neural network. Each statistical learning algorithm was fitted using the pseudo-population generated from the marginalization of the time-dependent confounding process. Covariate balance was assessed using the longitudinal (i.e. cumulative-time product) stabilized weights with calibrated restrictions. Our results indicated that the treatment drop-in cohorts (with respect to metformin, sulfonylurea and SGLT-2i) may have improved diabetes care provisions in relation to treatment naive (i.e. no treatment) cohort. As a clinical utility, we hope that this article will facilitate discussions around the prevention of adverse chronic outcomes associated with type II diabetes through the improvement of diabetes care provisions in primary care.

Can Robots Do Epidemiology? Machine Learning, Causal Inference, and Predicting the Outcomes of Public Health Interventions

This paper argues that machine learning (ML) and epidemiology are on collision course over causation. The discipline of epidemiology lays great emphasis on causation, while ML research does not. Some epidemiologists have proposed imposing what amounts to a causal constraint on ML in epidemiology, requiring it either to engage in causal inference or restrict itself to mere projection. We whittle down the issues to the question of whether causal knowledge is necessary for underwriting predictions about the outcomes of public health interventions. While there is great plausibility to the idea that it is, conviction that something is impossible does not by itself motivate a constraint to forbid trying. We disambiguate the possible motivations for such a constraint into definitional, metaphysical, epistemological, and pragmatic considerations and argue that “Proceed with caution” (rather than “Stop!”) is the outcome of each. We then argue that there are positive reasons to proceed, albeit cautiously. Causal inference enforces existing classification schema prior to the testing of associational claims (causal or otherwise), but associations and classification schema are more plausibly discovered (rather than tested or justified) in a back-and-forth process of gaining reflective equilibrium. ML instantiates this kind of process, we argue, and thus offers the welcome prospect of uncovering meaningful new concepts in epidemiology and public health—provided it is not causally constrained.

Developing a prediction model to estimate the true burden of respiratory syncytial virus (RSV) in hospitalised children in Western Australia

Respiratory syncytial virus (RSV) is a leading cause of childhood morbidity, however there is no systematic testing in children hospitalised with respiratory symptoms. Therefore, current RSV incidence likely underestimates the true burden. We used probabilistically linked perinatal, hospital, and laboratory records of 321,825 children born in Western Australia (WA), 2000-2012. We generated a predictive model for RSV positivity in hospitalised children aged < 5 years. We applied the model to all hospitalisations in our population-based cohort to determine the true RSV incidence, and under-ascertainment fraction. The model's predictive performance was determined using cross-validated area under the receiver operating characteristic (AUROC) curve. From 321,825 hospitalisations, 37,784 were tested for RSV (22.8% positive). Predictors of RSV positivity included younger admission age, male sex, non-Aboriginal ethnicity, a diagnosis of bronchiolitis and longer hospital stay. Our model showed good predictive accuracy (AUROC: 0.87). The respective sensitivity, specificity, positive predictive value and negative predictive values were 58.4%, 92.2%, 68.6% and 88.3%. The predicted incidence rates of hospitalised RSV for children aged < 3 months was 43.7/1000 child-years (95% CI 42.1-45.4) compared with 31.7/1000 child-years (95% CI 30.3-33.1) from laboratory-confirmed RSV admissions. Findings from our study suggest that the true burden of RSV may be 30-57% higher than current estimates.

G-computation and doubly robust standardisation for continuous-time data: A comparison with inverse probability weighting

In time-to-event settings, g-computation and doubly robust estimators are based on discrete-time data. However, many biological processes are evolving continuously over time. In this paper, we extend the g-computation and the doubly robust standardisation procedures to a continuous-time context. We compare their performance to the well-known inverse-probability-weighting estimator for the estimation of the hazard ratio and restricted mean survival times difference, using a simulation study. Under a correct model specification, all methods are unbiased, but g-computation and the doubly robust standardisation are more efficient than inverse-probability-weighting. We also analyse two real-world datasets to illustrate the practical implementation of these approaches. We have updated the R package RISCA to facilitate the use of these methods and their dissemination.

AIPW: An R Package for Augmented Inverse Probability-Weighted Estimation of Average Causal Effects

An increasing number of recent studies have suggested that doubly robust estimators with cross-fitting should be used when estimating causal effects with machine learning methods. However, not all existing programs that implement doubly robust estimators support machine learning methods and cross-fitting, or provide estimates on multiplicative scales. To address these needs, we developed AIPW, a software package implementing augmented inverse probability weighting (AIPW) estimation of average causal effects in R (R Foundation for Statistical Computing, Vienna, Austria). Key features of the AIPW package include cross-fitting and flexible covariate adjustment for observational studies and randomized controlled trials (RCTs). In this paper, we use a simulated RCT to illustrate implementation of the AIPW estimator. We also perform a simulation study to evaluate the performance of the AIPW package compared with other doubly robust implementations, including CausalGAM, npcausal, tmle, and tmle3. Our simulation showed that the AIPW package yields performance comparable to that of other programs. Furthermore, we also found that cross-fitting substantively decreases the bias and improves the confidence interval coverage for doubly robust estimators fitted with machine learning algorithms. Our findings suggest that the AIPW package can be a useful tool for estimating average causal effects with machine learning methods in RCTs and observational studies.

High-dimensional generalized propensity score with application to omics data

Propensity score (PS) methods are popular when estimating causal effects in non-randomized studies. Drawing causal conclusion relies on the unconfoundedness assumption. This assumption is untestable and is considered more plausible if a large number of pre-treatment covariates are included in the analysis. However, previous studies have shown that including unnecessary covariates into PS models can lead to bias and efficiency loss. With the ever-increasing amounts of available data, such as the omics data, there is often little prior knowledge of the exact set of important covariates. Therefore, variable selection for causal inference in high-dimensional settings has received considerable attention in recent years. However, recent studies have focused mainly on binary treatments. In this study, we considered continuous treatments and proposed the generalized outcome-adaptive LASSO (GOAL) to select covariates that can provide an unbiased and statistically efficient estimation. Simulation studies showed that when the outcome model was linear, the GOAL selected almost all true confounders and predictors of outcome and excluded other covariates. The accuracy and precision of the estimates were close to ideal. Furthermore, the GOAL is robust to model misspecification. We applied the GOAL to seven DNA methylation datasets from the Gene Expression Omnibus database, which covered four brain regions, to estimate the causal effects of epigenetic aging acceleration on the incidence of Alzheimer's disease.

Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations

Background

Directed acyclic graphs (DAGs) are an increasingly popular approach for identifying confounding variables that require conditioning when estimating causal effects. This review examined the use of DAGs in applied health research to inform recommendations for improving their transparency and utility in future research.

Methods

Original health research articles published during 1999–2017 mentioning ‘directed acyclic graphs’ (or similar) or citing DAGitty were identified from Scopus, Web of Science, Medline and Embase. Data were extracted on the reporting of: estimands, DAGs and adjustment sets, alongside the characteristics of each article’s largest DAG.

Results

A total of 234 articles were identified that reported using DAGs. A fifth (n = 48, 21%) reported their target estimand(s) and half (n = 115, 48%) reported the adjustment set(s) implied by their DAG(s). Two-thirds of the articles (n = 144, 62%) made at least one DAG available. DAGs varied in size but averaged 12 nodes [interquartile range (IQR): 9–16, range: 3–28] and 29 arcs (IQR: 19–42, range: 3–99). The median saturation (i.e. percentage of total possible arcs) was 46% (IQR: 31–67, range: 12–100). 37% (n = 53) of the DAGs included unobserved variables, 17% (n = 25) included ‘super-nodes’ (i.e. nodes containing more than one variable) and 34% (n = 49) were visually arranged so that the constituent arcs flowed in the same direction (e.g. top-to-bottom).

Conclusion

There is substantial variation in the use and reporting of DAGs in applied health research. Although this partly reflects their flexibility, it also highlights some potential areas for improvement. This review hence offers several recommendations to improve the reporting and use of DAGs in future research.

Pharmacoepidemiology for nephrologists (part 2): potential biases and how to overcome them

Observational pharmacoepidemiological studies using routinely collected healthcare data are increasingly being used in the field of nephrology to answer questions on the effectiveness and safety of medications. This review discusses a number of biases that may arise in such studies and proposes solutions to minimize them during the design or statistical analysis phase. We first describe designs to handle confounding by indication (e.g. active comparator design) and methods to investigate the influence of unmeasured confounding, such as the E-value, the use of negative control outcomes and control cohorts. We next discuss prevalent user and immortal time biases in pharmacoepidemiology research and how these can be prevented by focussing on incident users and applying either landmarking, using a time-varying exposure, or the cloning, censoring and weighting method. Lastly, we briefly discuss the common issues with missing data and misclassification bias. When these biases are properly accounted for, pharmacoepidemiological observational studies can provide valuable information for clinical practice.

A scoping review of causal methods enabling predictions under hypothetical interventions

BACKGROUND

The methods with which prediction models are usually developed mean that neither the parameters nor the predictions should be interpreted causally. For many applications, this is perfectly acceptable. However, when prediction models are used to support decision making, there is often a need for predicting outcomes under hypothetical interventions.

AIMS

We aimed to identify published methods for developing and validating prediction models that enable risk estimation of outcomes under hypothetical interventions, utilizing causal inference. We aimed to identify the main methodological approaches, their underlying assumptions, targeted estimands, and potential pitfalls and challenges with using the method. Finally, we aimed to highlight unresolved methodological challenges.

METHODS

We systematically reviewed literature published by December 2019, considering papers in the health domain that used causal considerations to enable prediction models to be used for predictions under hypothetical interventions. We included both methodologies proposed in statistical/machine learning literature and methodologies used in applied studies.

RESULTS

We identified 4919 papers through database searches and a further 115 papers through manual searches. Of these, 87 papers were retained for full-text screening, of which 13 were selected for inclusion. We found papers from both the statistical and the machine learning literature. Most of the identified methods for causal inference from observational data were based on marginal structural models and g-estimation.

CONCLUSIONS

There exist two broad methodological approaches for allowing prediction under hypothetical intervention into clinical prediction models: (1) enriching prediction models derived from observational studies with estimated causal effects from clinical trials and meta-analyses and (2) estimating prediction models and causal effects directly from observational data. These methods require extending to dynamic treatment regimes, and consideration of multiple interventions to operationalise a clinical decision support system. Techniques for validating 'causal prediction models' are still in their infancy.

G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

In clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Machine Learning Within Studies of Early-Life Environmental Exposures and Child Health: Review of the Current Literature and Discussion of Next Steps

PURPOSE OF REVIEW

The goal of this article is to review the use of machine learning (ML) within studies of environmental exposures and children's health, identify common themes across studies, and provide recommendations to advance their use in research and practice.

RECENT FINDINGS

We identified 42 articles reporting upon the use of ML within studies of environmental exposures and children's health between 2017 and 2019. The common themes among the articles were analysis of mixture data, exposure prediction, disease prediction and forecasting, analysis of complex data, and causal inference. With the increasing complexity of environmental health data, we anticipate greater use of ML to address the challenges that cannot be handled by traditional analytics. In order for these methods to beneficially impact public health, the ML techniques we use need to be appropriate for our study questions, rigorously evaluated and reported in a way that can be critically assessed by the scientific community.

Enabling personalized decision support with patient-generated data and attributable components

Decision-making related to health is complex. Machine learning (ML) and patient generated data can identify patterns and insights at the individual level, where human cognition falls short, but not all ML-generated information is of equal utility for making health-related decisions. We develop and apply attributable components analysis (ACA), a method inspired by optimal transport theory, to type 2 diabetes self-monitoring data to identify patterns of association between nutrition and blood glucose control. In comparison with linear regression, we found that ACA offers a number of characteristics that make it promising for use in decision support applications. For example, ACA was able to identify non-linear relationships, was more robust to outliers, and offered broader and more expressive uncertainty estimates. In addition, our results highlight a tradeoff between model accuracy and interpretability, and we discuss implications for ML-driven decision support systems.

Proportional multistate lifetable modelling of preventive interventions: concepts, code and worked examples

Burden of Disease studies-such as the Global Burden of Disease (GBD) Study-quantify health loss in disability-adjusted life-years. However, these studies stop short of quantifying the future impact of interventions that shift risk factor distributions, allowing for trends and time lags. This methodology paper explains how proportional multistate lifetable (PMSLT) modelling quantifies intervention impacts, using comparisons between three tobacco control case studies [eradication of tobacco, tobacco-free generation i.e. the age at which tobacco can be legally purchased is lifted by 1 year of age for each calendar year) and tobacco tax]. We also illustrate the importance of epidemiological specification of business-as-usual in the comparator arm that the intervention acts on, by demonstrating variations in simulated health gains when incorrectly: (i) assuming no decreasing trend in tobacco prevalence; and (ii) not including time lags from quitting tobacco to changing disease incidence. In conjunction with increasing availability of baseline and forecast demographic and epidemiological data, PMSLT modelling is well suited to future multiple country comparisons to better inform national, regional and global prioritization of preventive interventions. To facilitate use of PMSLT, we introduce a Python-based modelling framework and associated tools that facilitate the construction, calibration and analysis of PMSLT models.

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council

Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.

Real-world evidence: the devil is in the detail

Much has been written about real-world evidence (RWE), a concept that offers an understanding of the effects of healthcare interventions using routine clinical data. The reflection of diverse real-world practices is a double-edged sword that makes RWE attractive but also opens doors to several biases that need to be minimised both in the design and analytical phases of non-experimental studies. Additionally, it is critical to ensure that researchers who conduct these studies possess adequate methodological expertise and ability to accurately implement these methods. Critical design elements to be considered should include a clearly defined research question using a causal inference framework, choice of a fit-for-purpose data source, inclusion of new users of a treatment with comparators that are as similar as possible to that group, accurately classifying person-time and deciding censoring approaches. Having taken measures to minimise bias 'by design', the next step is to implement appropriate analytical techniques (for example propensity scores) to minimise the remnant potential biases. A clear protocol should be provided at the beginning of the study and a report of the results after, including caveats to consider. We also point the readers to readings on some novel analytical methods as well as newer areas of application of RWE. While there is no one-size-fits-all solution to evaluating RWE studies, we have focused our discussion on key methods and issues commonly encountered in comparative observational cohort studies with the hope that readers are better equipped to evaluate non-experimental studies that they encounter in the future. Graphical abstract.

On the Convergence of Epidemiology, Biostatistics, and Data Science

Epidemiology, biostatistics, and data science are broad disciplines that incorporate a variety of substantive areas. Common among them is a focus on quantitative approaches for solving intricate problems. When the substantive area is health and health care, the overlap is further cemented. Researchers in these disciplines are fluent in statistics, data management and analysis, and health and medicine, to name but a few competencies. Yet there are important and perhaps mutually exclusive attributes of these fields that warrant a tighter integration. For example, epidemiologists receive substantial training in the science of study design, measurement, and the art of causal inference. Biostatisticians are well versed in the theory and application of methodological techniques, as well as the design and conduct of public health research. Data scientists receive equivalently rigorous training in computational and visualization approaches for high-dimensional data. Compared to data scientists, epidemiologists and biostatisticians may have less expertise in computer science and informatics, while data scientists may benefit from a working knowledge of study design and causal inference. Collaboration and cross-training offer the opportunity to share and learn of the constructs, frameworks, theories, and methods of these fields with the goal of offering fresh and innovate perspectives for tackling challenging problems in health and health care. In this article, we first describe the evolution of these fields focusing on their convergence in the era of electronic health data, notably electronic medical records (EMRs). Next we present how a collaborative team may design, analyze, and implement an EMR-based study. Finally, we review the curricula at leading epidemiology, biostatistics, and data science training programs, identifying gaps and offering suggestions for the fields moving forward.