Directed acyclic graphs and causal thinking in clinical risk prediction modeling

English Cocker Spaniels under primary veterinary care in the UK: disorder predispositions and protections

BACKGROUND

The English Cocker Spaniel (ECS) is one of the most popular dog breeds in the UK but information on disorder predisposition and protection is limited. Using anonymised veterinary clinical data from the VetCompass™ Programme, this study aimed to compare disorder predisposition and protection between the ECS and the remaining dogs under primary veterinary care in the UK during 2016. Electronic patient records for random samples of ECS and non-ECS were reviewed. The most common disorders diagnosed during 2016 were extracted and compared using multivariable logistic regression, controlling for confounders.

RESULTS

The analysis included random samples of 2510/10,313 (24.3%) ECS and 7813/326,552 (2.39%) non-ECS. After accounting for confounding by age, sex, bodyweight within breed-sex, insurance status and veterinary practice group, the ECS had increased odds of 21/43 (48.85%) disorders at fine-level precision, with highest odds for aural discharge (odds ratio (OR) 14.66, 95% confidence interval (CI): 7.73-30.90, P <  0.001) and keratoconjunctivitis sicca (OR 7.64, 95% CI: 4.33-14.14, P <  0.001) and lowest odds for atopic dermatitis (OR 0.14, 95% CI: 0.05-0.31, P <  0.001) and allergy (OR 0.14, 95% CI: 0.06-0.28, P <  0.001).

CONCLUSIONS

This study provides evidence for strong predisposition to aural and ocular disorders and protection from hypersensitivity disorders in the ECS. These results can aid dog owners, breeders, and veterinarians to better monitor health in ECS, and promote earlier diagnosis with improved prognosis. Further, the results can help breeding organisations establish key priorities the health-based reforms of the ECS.

Directed acyclic graph for epidemiological studies in childhood food allergy: Construction, user's guide, and application

Understanding modifiable prenatal and early life causal determinants of food allergy is important for the prevention of the disease. Randomized clinical trials studying environmental and dietary determinants of food allergy may not always be feasible. Identifying risk/protective factors for early-life food allergy often relies on observational studies, which may be affected by confounding bias. The directed acyclic graph (DAG) is a causal diagram useful to guide causal inference from observational epidemiological research. To date, research on food allergy has made little use of this promising method. We performed a literature review of existing evidence with a systematic search, synthesized 32 known risk/protective factors, and constructed a comprehensive DAG for early-life food allergy development. We present an easy-to-use online tool for researchers to re-construct, amend, and modify the DAG along with a user's guide to minimize confounding bias. We estimated that adjustment strategies in 57% of previous observational studies on modifiable factors of childhood food allergy could be improved if the researchers determined their adjustment sets by DAG. Future researchers who are interested in the causal inference of food allergy development in early life can apply the DAG to identify covariates that should and should not be controlled in observational studies.

Race as a Component of Cardiovascular Disease Risk Prediction Algorithms

PURPOSE OF REVIEW

Several prediction algorithms include race as a component to account for race-associated variations in disease frequencies. This practice has been questioned recently because of the risk of perpetuating race as a biological construct and diverting attention away from the social determinants of health (SDoH) for which race might be a proxy. We evaluated the appropriateness of including race in cardiovascular disease (CVD) prediction algorithms, notably the pooled cohort equations (PCE).

RECENT FINDINGS

In a recent investigation, we reported substantial and biologically implausible differences in absolute CVD risk estimates upon using PCE for predicting CVD risk in Black and White persons with identical risk factor profiles, which might result in differential treatment decisions based solely on their race. We recommend the development of raceless CVD risk prediction algorithms that obviate race-associated risk misestimation and racializing treatment practices, and instead incorporate measures of SDoH that mediate race-associated risk differences.

Does modeling causal relationships improve the accuracy of predicting lactation milk yields?

This study compared 3 correlational (best prediction, linear regression, and feed-forward neural networks) and 2 causal models (recursive structural equation model and recurrent neural networks) for estimating lactation milk yields. The correlational models assumed associations between test-day milk yields (health conditions), while the casual models postulated unidirectional recursive effects between these test-day variables. Wood lactation curves were used to simulate the data and served as a benchmark model. Individual Wood lactation curves provided an excellent parametric interpretation of lactation dynamics, with their prediction accuracies depending on the coverage of the lactation curve dynamics. Best prediction outperformed other models in the absence of mastitis but was suboptimal when mastitis was present and unaccounted for. Recurrent neural networks yielded the highest accuracy when mastitis was present. Although causal models facilitated the inference about the causality underlying lactation, precisely capturing the causal relationships was challenging because the underlying biology was complex. Misspecification of recursive effects in the recursive structural equation model resulted in a loss of accuracy. Hence, modeling causal relationships does not necessarily guarantee improved accuracies. In practice, a parsimonious model is preferred, balancing model complexity and accuracy. In addition to the choice of statistical models, the proper accounting for factors and covariates affecting milk yields is equally crucial.

Assessing the transportability of clinical prediction models for cognitive impairment using causal models

BACKGROUND

Machine learning models promise to support diagnostic predictions, but may not perform well in new settings. Selecting the best model for a new setting without available data is challenging. We aimed to investigate the transportability by calibration and discrimination of prediction models for cognitive impairment in simulated external settings with different distributions of demographic and clinical characteristics.

METHODS

We mapped and quantified relationships between variables associated with cognitive impairment using causal graphs, structural equation models, and data from the ADNI study. These estimates were then used to generate datasets and evaluate prediction models with different sets of predictors. We measured transportability to external settings under guided interventions on age, APOE ε4, and tau-protein, using performance differences between internal and external settings measured by calibration metrics and area under the receiver operating curve (AUC).

RESULTS

Calibration differences indicated that models predicting with causes of the outcome were more transportable than those predicting with consequences. AUC differences indicated inconsistent trends of transportability between the different external settings. Models predicting with consequences tended to show higher AUC in the external settings compared to internal settings, while models predicting with parents or all variables showed similar AUC.

CONCLUSIONS

We demonstrated with a practical prediction task example that predicting with causes of the outcome results in better transportability compared to anti-causal predictions when considering calibration differences. We conclude that calibration performance is crucial when assessing model transportability to external settings.

FASDetect as a machine learning-based screening app for FASD in youth with ADHD

Fetal alcohol-spectrum disorder (FASD) is underdiagnosed and often misdiagnosed as attention-deficit/hyperactivity disorder (ADHD). Here, we develop a screening tool for FASD in youth with ADHD symptoms. To develop the prediction model, medical record data from a German University outpatient unit are assessed including 275 patients aged 0-19 years old with FASD with or without ADHD and 170 patients with ADHD without FASD aged 0-19 years old. We train 6 machine learning models based on 13 selected variables and evaluate their performance. Random forest models yield the best prediction models with a cross-validated AUC of 0.92 (95% confidence interval [0.84, 0.99]). Follow-up analyses indicate that a random forest model with 6 variables - body length and head circumference at birth, IQ, socially intrusive behaviour, poor memory and sleep disturbance - yields equivalent predictive accuracy. We implement the prediction model in a web-based app called FASDetect - a user-friendly, clinically scalable FASD risk calculator that is freely available at https://fasdetect.dhc-lab.hpi.de .

Using LASSO Regression to Estimate the Population-Level Impact of Pneumococcal Conjugate Vaccines

Pneumococcal conjugate vaccines (PCVs) protect against diseases caused by Streptococcus pneumoniae, such as meningitis, bacteremia, and pneumonia. It is challenging to estimate their population-level impact due to the lack of a perfect control population and the subtleness of signals when the endpoint-such as all-cause pneumonia-is nonspecific. Here we present a new approach for estimating the impact of PCVs: using least absolute shrinkage and selection operator (LASSO) regression to select variables in a synthetic control model to predict the counterfactual outcome for vaccine impact inference. We first used a simulation study based on hospitalization data from Mexico (2000-2013) to test the performance of LASSO and established methods, including the synthetic control model with Bayesian variable selection (SC). We found that LASSO achieved accurate and precise estimation, even in complex simulation scenarios where the association between the outcome and all control variables was noncausal. We then applied LASSO to real-world hospitalization data from Chile (2001-2012), Ecuador (2001-2012), Mexico (2000-2013), and the United States (1996-2005), and found that it yielded estimates of vaccine impact similar to SC. The LASSO method is accurate and easily implementable and can be applied to study the impact of PCVs and other vaccines.

The dawn of directed acyclic graphs in primary care research and education

Dramatical increase in articles mentioning "directed acyclic graph."

Single Nucleotide Polymorphisms' Causal Structure Robustness within Coronary Artery Disease Patients

An ever-growing amount of accumulated data has materialized in several scientific fields, due to recent technological progress. New challenges emerge in exploiting these data and utilizing the valuable available information. Causal models are a powerful tool that can be employed towards this aim, by unveiling the structure of causal relationships between different variables. The causal structure may avail experts to better understand relationships, or even uncover new knowledge. Based on 963 patients with coronary artery disease, the robustness of the causal structure of single nucleotide polymorphisms was assessed, taking into account the value of the Syntax Score, an index that evaluates the complexity of the disease. The causal structure was investigated, both locally and globally, under different levels of intervention, reflected in the number of patients that were randomly excluded from the original datasets corresponding to two categories of the Syntax Score, zero and positive. It is shown that the causal structure of single nucleotide polymorphisms was more robust under milder interventions, whereas in the case of stronger interventions, the impact increased. The local causal structure around the Syntax Score was studied in the case of a positive Syntax Score, and it was found to be resilient, even when the intervention was strong. Consequently, employing causal models in this context may increase the understanding of the biological aspects of coronary artery disease.

Cardiovascular disease and type 2 diabetes in older adults: a combined protocol for an individual participant data analysis for risk prediction and a network meta-analysis of novel anti-diabetic drugs

Introduction

Older and multimorbid adults with type 2 diabetes (T2D) are at high risk of cardiovascular disease (CVD) and chronic kidney disease (CKD). Estimating risk and preventing CVD is a challenge in this population notably because it is underrepresented in clinical trials. Our study aims to (1) assess if T2D and haemoglobin A1c (HbA1c) are associated with the risk of CVD events and mortality in older adults, (2) develop a risk score for CVD events and mortality for older adults with T2D, (3) evaluate the comparative efficacy and safety of novel antidiabetics.

Methods and analysis

For Aim 1, we will analyse individual participant data on individuals aged ≥65 years from five cohort studies: the Optimising Therapy to Prevent Avoidable Hospital Admissions in Multimorbid Older People study; the Cohorte Lausannoise study; the Health, Aging and Body Composition study; the Health and Retirement Study; and the Survey of Health, Ageing and Retirement in Europe. We will fit flexible parametric survival models (FPSM) to assess the association of T2D and HbA1c with CVD events and mortality. For Aim 2, we will use data on individuals aged ≥65 years with T2D from the same cohorts to develop risk prediction models for CVD events and mortality using FPSM. We will assess model performance, perform internal-external cross validation, and derive a point-based risk score. For Aim 3, we will systematically search randomized controlled trials of novel antidiabetics. Network meta-analysis will be used to determine comparative efficacy in terms of CVD, CKD, and retinopathy outcomes, and safety of these drugs. Confidence in results will be judged using the CINeMA tool.

Ethics and dissemination

Aims 1 and 2 were approved by the local ethics committee (Kantonale Ethikkommission Bern); no approval is required for Aim 3. Results will be published in peer-reviewed journals and presented in scientific conferences.

Should Cognitive Screening Tests Be Corrected for Age and Education? Insights From a Causal Perspective

Cognitive screening tests such as the Mini-Mental State Examination are widely used in clinical routine to predict cognitive impairment. The raw test scores are often corrected for age and education, although documented poorer discrimination performance of corrected scores has challenged this practice. Nonetheless, test correction persists, perhaps due to the seemingly counterintuitive nature of the underlying problem. We used a causal framework to inform the long-standing debate from a more intuitive angle. We illustrate and quantify the consequences of applying the age-education correction of cognitive tests on discrimination performance. In an effort to bridge theory and practical implementation, we computed differences in discrimination performance under plausible causal scenarios using Open Access Series of Imaging Studies (OASIS)-1 data. We show that when age and education are causal risk factors for cognitive impairment and independently also affect the test score, correcting test scores for age and education removes meaningful information, thereby diminishing discrimination performance.

Risk Model Development and Validation in Clinical Oncology: Lessons Learned

Reliable risk models can greatly facilitate patient-centered inferences and decisions. Herein we summarize key considerations related to risk modeling in clinical oncology. Often overlooked challenges include data quality, missing data, effective sample size estimation, and selecting the variables to be included in the risk model. The stability and quality of the model should be carefully interrogated with particular emphasis on rigorous internal validation.

Seroprevalence, correlates and kinetics of SARS-CoV-2 nucleocapsid IgG antibody in healthcare workers and nonclinical staff at a tertiary hospital: A prevaccine census study

BACKGROUND

Healthcare workers and nonclinical staff in medical facilities are perceived to be a high-risk group for acquiring SAR-CoV-2 infection, and more so in countries where COVID-19 vaccination uptake is low. Serosurveillance may best determine the true extent of SARS-CoV-2 infection since most infected HCWs and other staff may be asymptomatic or present with only mild symptoms. Over time, determining the true extent of SARS-CoV-2 infection could inform hospital management and staff whether the preventive measures instituted are effective and valuable in developing targeted solutions.

METHODS

This was a census survey study conducted at the Aga Khan University Hospital, Nairobi, between November 2020 and February 2021 before the implementation of the COVID-19 vaccination. The SARS-CoV-2 nucleocapsid IgG test was performed using a chemiluminescent assay.

RESULTS

One thousand six hundred thirty-one (1631) staff enrolled, totalling 60% of the workforce. The overall crude seroprevalence was 18.4% and the adjusted value (for assay sensitivity of 86%) was 21.4% (95% CI; 19.2-23.7). The staff categories with higher prevalence included pharmacy (25.6%), outreach (24%), hospital- based nursing (22.2%) and catering staff (22.6%). Independent predictors of a positive IgG result after adjusting for age, sex and comorbidities included prior COVID-19 like symptoms, odds ratio (OR) 2.0 [95% confidence interval (CI) 1.3-3.0, p = 0.001], a prior positive SARS-CoV-2 PCR result OR 12.0 (CI: 7.7-18.7, p<0.001) and working in a clinical COVID-19 designated area, OR 1.9 (CI 1.1-3.3, p = 0.021). The odds of testing positive for IgG after a positive PCR test were lowest if the antibody test was performed more than 2 months later; OR 0.7 (CI: 0.48-0.95, p = 0.025).

CONCLUSIONS

The prevalence of anti- SARS-CoV-2 nucleocapsid IgG among HCWs and nonclinical staff was lower than in the general population. Staff working in clinical areas were not at increased risk when compared to staff working in non-clinical areas.

A Causal Analysis of the Effect of Age and Sex Differences on Brain Atrophy in the Elderly Brain

We studied how brain volume loss in old age is affected by age, the APOE gene, sex, and the level of education completed. The quantitative characterization of brain volume loss at an old age relative to a young age requires-at least in principle-two MRI scans, one performed at a young age and one at an old age. There is, however, a way to address this problem when having only one MRI scan obtained at an old age. We computed the total brain losses of elderly subjects as a ratio between the estimated brain volume and the estimated total intracranial volume. Magnetic resonance imaging (MRI) scans of 890 healthy subjects aged 70 to 85 years were assessed. A causal analysis of factors affecting brain atrophy was performed using probabilistic Bayesian modelling and the mathematics of causal inference. We found that both age and sex were causally related to brain atrophy, with women reaching an elderly age with a 1% larger brain volume relative to their intracranial volume than men. How the brain ages and the rationale for sex differences in brain volume losses during the adult lifespan are questions that need to be addressed with causal inference and empirical data. The graphical causal modelling presented here can be instrumental in understanding a puzzling scientific area of study-the biological aging of the brain.

Association between executive function and excess weight in pre-school children

The association between executive function and excess weight is becoming increasingly evident. However, the results of previous studies are still inconclusive, and there is a lack of evidence in early childhood. This study aims to examine the association between executive function, in terms of overall and subscales of executive function (e.g., inhibition, working memory, and shifting), and weight excess in preschoolers. A population-based cross-sectional study was conducted on children aged 2–5 years of age from public and private schools in Chiang Mai, Thailand. Participants’ weights and heights were measured and classified into three weight status groups (i.e., children with normal weight, overweight, and obesity groups). Executive function was assessed using the parent-report Behavior Rating Inventory of Executive Function-Preschool (BRIEF-P). Multivariable polynomial regression was performed to analyze the association between executive function and weight status. A total of 1,181 children were included in the study. After adjusting for confounders, impaired overall executive function significantly increased the probability of being overweight (odds ratio [OR] = 2.47; 95% confidence interval [CI] 1.33 to 4.56). A similar trend of association was also found between impaired inhibition and overweight status (OR = 2.33; 95%CI 1.11 to 4.90). Furthermore, poor working memory was associated with both overweight and obesity (OR = 1.87; 95%CI 1.09 to 3.20 and OR = 1.74; 95%CI 1.09 to 2.78, respectively). Our data suggest that deficits in executive function, particularly inhibition and working memory, are associated with weight excess in preschoolers. Early promotion of executive function may be needed at this developmental age to prevent unhealthy weight status.

A Causal Framework for Making Individualized Treatment Decisions in Oncology

Simple Summary

Physicians routinely make individualized treatment decisions by accounting for the joint effects of patient prognostic covariates and treatments on clinical outcomes. Ideally, this is performed using historical randomized clinical trial (RCT) data. Randomization ensures that unbiased estimates of causal treatment effect parameters can be obtained from the historical RCT data and used to predict each new patient’s outcome based on the joint effect of their baseline covariates and each treatment being considered. However, this process becomes problematic if a patient seen in the clinic is very different from the patients who were enrolled in the RCT. That is, if a new patient does not satisfy the entry criteria of the RCT, then the patient does not belong to the population represented by the patients who were studied in the RCT. In such settings, it still may be possible to utilize the RCT data to help choose a new patient’s treatment. This may be achieved by combining the RCT data with data from other clinical trials, or possibly preclinical experiments, and using the combined dataset to predict the patient’s expected outcome for each treatment being considered. In such settings, combining data from multiple sources in a way that is statistically reliable is not entirely straightforward, and correctly identifying and estimating the effects of treatments and patient covariates on clinical outcomes can be complex. Causal diagrams provide a rational basis to guide this process. The first step is to construct a causal diagram that reflects the plausible relationships between treatment variables, patient covariates, and clinical outcomes. If the diagram is correct, it can be used to determine what additional data may be needed, how to combine data from multiple sources, how to formulate a statistical model for clinical outcomes as a function of treatment and covariates, and how to compute an unbiased treatment effect estimate for each new patient. We use adjuvant therapy of renal cell carcinoma to illustrate how causal diagrams may be used to guide these steps.

Abstract

We discuss how causal diagrams can be used by clinicians to make better individualized treatment decisions. Causal diagrams can distinguish between settings where clinical decisions can rely on a conventional additive regression model fit to data from a historical randomized clinical trial (RCT) to estimate treatment effects and settings where a different approach is needed. This may be because a new patient does not meet the RCT’s entry criteria, or a treatment’s effect is modified by biomarkers or other variables that act as mediators between treatment and outcome. In some settings, the problem can be addressed simply by including treatment–covariate interaction terms in the statistical regression model used to analyze the RCT dataset. However, if the RCT entry criteria exclude a new patient seen in the clinic, it may be necessary to combine the RCT data with external data from other RCTs, single-arm trials, or preclinical experiments evaluating biological treatment effects. For example, external data may show that treatment effects differ between histological subgroups not recorded in an RCT. A causal diagram may be used to decide whether external observational or experimental data should be obtained and combined with RCT data to compute statistical estimates for making individualized treatment decisions. We use adjuvant treatment of renal cell carcinoma as our motivating example to illustrate how to construct causal diagrams and apply them to guide clinical decisions.

Predictive models for clinical decision making: Deep dives in practical machine learning

The deployment of machine learning for tasks relevant to complementing standard of care and advancing tools for precision health has gained much attention in the clinical community, thus meriting further investigations into its broader use. In an introduction to predictive modelling using machine learning, we conducted a review of the recent literature that explains standard taxonomies, terminology and central concepts to a broad clinical readership. Articles aimed at readers with little or no prior experience of commonly used methods or typical workflows were summarised and key references are highlighted. Continual interdisciplinary developments in data science, biostatistics and epidemiology also motivated us to further discuss emerging topics in predictive and data-driven (hypothesis-less) analytics with machine learning. Through two methodological deep dives using examples from precision psychiatry and outcome prediction after lymphoma, we highlight how the use of, for example, natural language processing can outperform established clinical risk scores and aid dynamic prediction and adaptive care strategies. Such realistic and detailed examples allow for critical analysis of the importance of new technological advances in artificial intelligence for clinical decision-making. New clinical decision support systems can assist in prevention and care by leveraging precision medicine.

Systematic review and meta-analysis to examine intrapartum interventions, and maternal and neonatal outcomes following immersion in water during labour and waterbirth

OBJECTIVES

Water immersion during labour using a birth pool to achieve relaxation and pain relief during the first and possibly part of the second stage of labour is an increasingly popular care option in several countries. It is used particularly by healthy women who experience a straightforward pregnancy, labour spontaneously at term gestation and plan to give birth in a midwifery led care setting. More women are also choosing to give birth in water. There is debate about the safety of intrapartum water immersion, particularly waterbirth. We synthesised the evidence that compared the effect of water immersion during labour or waterbirth on intrapartum interventions and outcomes to standard care with no water immersion. A secondary objective was to synthesise data relating to clinical care practices and birth settings that women experience who immerse in water and women who do not.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

A search was conducted using CINAHL, Medline, Embase, BioMed Central and PsycINFO during March 2020 and was replicated in May 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Primary quantitative studies published in 2000 or later, examining maternal or neonatal interventions and outcomes using the birthing pool for labour and/or birth.

DATA EXTRACTION AND SYNTHESIS

Full-text screening was undertaken independently against inclusion/exclusion criteria in two pairs. Risk of bias assessment included review of seven domains based on the Robbins-I Risk of Bias Tool. All outcomes were summarised using an OR and 95% CI. All calculations were conducted in Comprehensive Meta-Analysis V.3, using the inverse variance method. Results of individual studies were converted to log OR and SE for synthesis. Fixed effects models were used when I² was less than 50%, otherwise random effects models were used. The fail-safe N estimates were calculated to determine the number of studies necessary to change the estimates. Begg's test and Egger's regression risk assessed risk of bias across studies. Trim-and-fill analysis was used to estimate the magnitude of effect of the bias. Meta-regression was completed when at least 10 studies provided data for an outcome.

RESULTS

We included 36 studies in the review, (N=157 546 participants). Thirty-one studies were conducted in an obstetric unit setting (n=70 393), four studies were conducted in midwife led settings (n=61 385) and one study was a mixed setting (OU and homebirth) (n=25 768). Midwife led settings included planned home and freestanding midwifery unit (k=1), alongside midwifery units (k=1), planned homebirth (k=1), a freestanding midwifery unit and an alongside midwifery unit (k=1) and an alongside midwifery unit (k=1). For water immersion, 25 studies involved women who planned to have/had a waterbirth (n=151 742), seven involved water immersion for labour only (1901), three studies reported on water immersion during labour and waterbirth (n=3688) and one study was unclear about the timing of water immersion (n=215).Water immersion significantly reduced use of epidural (k=7, n=10 993; OR 0.17 95% CI 0.05 to 0.56), injected opioids (k=8, n=27 391; OR 0.22 95% CI 0.13 to 0.38), episiotomy (k=15, n=36 558; OR 0.16; 95% CI 0.10 to 0.27), maternal pain (k=8, n=1200; OR 0.24 95% CI 0.12 to 0.51) and postpartum haemorrhage (k=15, n=63 891; OR 0.69 95% CI 0.51 to 0.95). There was an increase in maternal satisfaction (k=6, n=4144; OR 1.95 95% CI 1.28 to 2.96) and odds of an intact perineum (k=17, n=59 070; OR 1.48; 95% CI 1.21 to 1.79) with water immersion. Waterbirth was associated with increased odds of cord avulsion (OR 1.94 95% CI 1.30 to 2.88), although the absolute risk remained low (4.3 per 1000 vs 1.3 per 1000). There were no differences in any other identified neonatal outcomes.

CONCLUSIONS

This review endorses previous reviews showing clear benefits resulting from intrapartum water immersion for healthy women and their newborns. While most included studies were conducted in obstetric units, to enable the identification of best practice regarding water immersion, future birthing pool research should integrate factors that are known to influence intrapartum interventions and outcomes. These include maternal parity, the care model, care practices and birth setting.

PROSPERO REGISTRATION NUMBER

CRD42019147001.

A novel machine learning model to predict respiratory failure and invasive mechanical ventilation in critically ill patients suffering from COVID-19

In hypoxemic patients at risk for developing respiratory failure, the decision to initiate invasive mechanical ventilation (IMV) may be extremely difficult, even more so among patients suffering from COVID-19. Delayed recognition of respiratory failure may translate into poor outcomes, emphasizing the need for stronger predictive models for IMV necessity. We developed a two-step model; the first step was to train a machine learning predictive model on a large dataset of non-COVID-19 critically ill hypoxemic patients from the United States (MIMIC-III). The second step was to apply transfer learning and adapt the model to a smaller COVID-19 cohort. An XGBoost algorithm was trained on data from the MIMIC-III database to predict if a patient would require IMV within the next 6, 12, 18 or 24 h. Patients’ datasets were used to construct the model as time series of dynamic measurements and laboratory results obtained during the previous 6 h with additional static variables, applying a sliding time-window once every hour. We validated the adaptation algorithm on a cohort of 1061 COVID-19 patients from a single center in Israel, of whom 160 later deteriorated and required IMV. The new XGBoost model for the prediction of the IMV onset was trained and tested on MIMIC-III data and proved to be predictive, with an AUC of 0.83 on a shortened set of features, excluding the clinician’s settings, and an AUC of 0.91 when the clinician settings were included. Applying these models “as is” (no adaptation applied) on the dataset of COVID-19 patients degraded the prediction results to AUCs of 0.78 and 0.80, without and with the clinician’s settings, respectively. Applying the adaptation on the COVID-19 dataset increased the prediction power to an AUC of 0.94 and 0.97, respectively. Good AUC results get worse with low overall precision. We show that precision of the prediction increased as prediction probability was higher. Our model was successfully trained on a specific dataset, and after adaptation it showed promise in predicting outcome on a completely different dataset. This two-step model successfully predicted the need for invasive mechanical ventilation 6, 12, 18 or 24 h in advance in both general ICU population and COVID-19 patients. Using the prediction probability as an indicator of the precision carries the potential to aid the decision-making process in patients with hypoxemic respiratory failure despite the low overall precision.

DNA methylation-based predictors of health: applications and statistical considerations

DNA methylation data have become a valuable source of information for biomarker development, because, unlike static genetic risk estimates, DNA methylation varies dynamically in relation to diverse exogenous and endogenous factors, including environmental risk factors and complex disease pathology. Reliable methods for genome-wide measurement at scale have led to the proliferation of epigenome-wide association studies and subsequently to the development of DNA methylation-based predictors across a wide range of health-related applications, from the identification of risk factors or exposures, such as age and smoking, to early detection of disease or progression in cancer, cardiovascular and neurological disease. This Review evaluates the progress of existing DNA methylation-based predictors, including the contribution of machine learning techniques, and assesses the uptake of key statistical best practices needed to ensure their reliable performance, such as data-driven feature selection, elimination of data leakage in performance estimates and use of generalizable, adequately powered training samples.

Circulating Palmitoyl Sphingomyelin Is Associated With Cardiovascular Disease in Individuals With Type 2 Diabetes: Findings From the China Da Qing Diabetes Study

OBJECTIVE

To investigate the association of potential cardiovascular disease (CVD) biomarkers in patients with type 2 diabetes.

RESEARCH DESIGN AND METHODS

We enrolled 120 participants (aged 61.5-69.5 years) with type 2 diabetes and 60 (aged 62.5-73.5 years) with normal glucose tolerance in the discovery group from the original Da Qing Diabetes Study. Their diabetes status was confirmed in 1986; then, the participants were followed over 23 years to collect CVD outcome data. Untargeted and targeted metabolomics analyses based on ultra-high-performance liquid chromatography-tandem mass spectrometry were used to identify potential markers. Multivariable regression analysis was used to evaluate the association between metabolites and CVD outcomes. An independent group of 335 patients (aged 67.0-77.0 years) with diabetes was used for biomarker validation.

RESULTS

In the discovery group, untargeted metabolomics analysis found 16 lipids and fatty acids metabolites associated with CVD risk in patients with diabetes, with palmitoyl sphingomyelin (PSM) having the strongest association. Plasma PSM concentrations were significantly higher in cases of diabetes with CVD than without (41.68 ± 10.47 vs. 9.69 ± 1.47 μg/mL; P < 0.0001). The odds ratio (OR) of CVD for 1 µg/mL PSM change was 1.19 (95% CI 1.13-1.25) after adjustment of clinical confounders. The validation study confirmed that PSM was significantly associated with increased CVD risk in diabetes (OR 1.22 [95% CI 1.16-1.30]).

CONCLUSIONS

Changes in lipid and fatty acid content were significantly associated with CVD risk in the Chinese population with diabetes. PSM is a potential biomarker of increased CVD risk in diabetes.

Developing a prediction model to estimate the true burden of respiratory syncytial virus (RSV) in hospitalised children in Western Australia

Respiratory syncytial virus (RSV) is a leading cause of childhood morbidity, however there is no systematic testing in children hospitalised with respiratory symptoms. Therefore, current RSV incidence likely underestimates the true burden. We used probabilistically linked perinatal, hospital, and laboratory records of 321,825 children born in Western Australia (WA), 2000-2012. We generated a predictive model for RSV positivity in hospitalised children aged < 5 years. We applied the model to all hospitalisations in our population-based cohort to determine the true RSV incidence, and under-ascertainment fraction. The model's predictive performance was determined using cross-validated area under the receiver operating characteristic (AUROC) curve. From 321,825 hospitalisations, 37,784 were tested for RSV (22.8% positive). Predictors of RSV positivity included younger admission age, male sex, non-Aboriginal ethnicity, a diagnosis of bronchiolitis and longer hospital stay. Our model showed good predictive accuracy (AUROC: 0.87). The respective sensitivity, specificity, positive predictive value and negative predictive values were 58.4%, 92.2%, 68.6% and 88.3%. The predicted incidence rates of hospitalised RSV for children aged < 3 months was 43.7/1000 child-years (95% CI 42.1-45.4) compared with 31.7/1000 child-years (95% CI 30.3-33.1) from laboratory-confirmed RSV admissions. Findings from our study suggest that the true burden of RSV may be 30-57% higher than current estimates.

Differences in estimates for 10-year risk of cardiovascular disease in Black versus White individuals with identical risk factor profiles using pooled cohort equations: an in silico cohort study

BACKGROUND

Sex-specific and race-specific pooled cohort equations (PCEs) are recommended for estimating the 10-year risk of cardiovascular disease, with an absolute risk of more than 7·5% indicating a clinical decision threshold. We compared differences between Black and White individuals in PCE-estimated absolute cardiovascular disease risk across various plausible risk factor combinations with the aim of evaluating if using the PCE might result in different clinical decisions in Black versus White individuals with identical risk profiles.

METHODS

We generated in silico patient risk profiles by combining numerical risk factors (age [5-year intervals], total cholesterol [20-mg/dl intervals], HDL cholesterol [5-mg/dl intervals], systolic blood pressure [10-mm Hg intervals]) and binary risk factors (smoking, diabetes, and antihypertensive treatment). We compared PCE-estimated 10-year cardiovascular disease risk in Black versus White individuals with identical risk profiles. We did similar comparisons using eligible participants in the Framingham Heart Study (FHS) third generation cohort and the National Health and Nutrition Examination Survey (NHANES) 2017-18.

FINDINGS

For our in silico analysis, we evaluated 29 515 risk factor combinations for women and 30 565 for men, after excluding profiles that generated 10-year cardiovascular disease risk estimates below 1% or above 30%. There were 6357 risk profiles associated with 10-year cardiovascular disease risk above 7·5% for Black men but not for White men (median risk difference [RD] 6·25%, range 0·15-22·8; median relative risk [RR] 2·40, range 1·02-12·6). There were 391 profiles with 10-year cardiovascular disease risk above 7·5% for White men but not Black men (median RD 2·68%, range 0·07-16·9%; median RR 1·42, range 1·01-3·57). There were 6543 risk profiles associated with 10-year estimated cardiovascular disease risk above 7·5% for Black women but not for White women (median RD 6·14%, range 0·35-26·8%; median RR 2·29, range 1·05-12·6). There were 318 profiles with 10-year cardiovascular disease risk above 7·5% for White women but not Black women (median RD 3·71%, range 0·22-20·1%; median RR 1·66, range 1·03-5·46). For the population-based samples, we calculated the PCE-based 10-year cardiovascular disease risk for 1272 eligible participants (378 women; median age 48 years [IQR 44-52]; 100% White) in the FHS third generation cohort and 550 participants (223 women [36·8% Black] and 327 men [40·4% Black]; median age 61 years [IQR 52-67]) in the NHANES cohort. The population-based samples showed similar risk differences to that of the in silico analyses.

INTERPRETATION

The PCE might generate substantially divergent cardiovascular disease risk estimates for Black versus White individuals with identical risk profiles, which could introduce race-related variations in clinical recommendations for cardiovascular disease prevention.

FUNDING

US National Institutes of Health.

French Bulldogs differ to other dogs in the UK in propensity for many common disorders: a VetCompass study

Background

The French Bulldog is a highly popular dog breed but is linked with many serious health issues. A holistic view of breed health in French Bulldogs would assist efforts to appreciate the overall health strengths and weaknesses in the French Bulldog and to take appropriate steps to mitigate these.

Based on random sampling of French Bulldogs and non-French Bulldogs under primary veterinary care during 2016 within the VetCompass Programme, a cohort study design was used to estimate the one-year (2016) period prevalence of the most commonly diagnosed disorders in each group. Risk factor analysis used multivariable logistic regression modelling methods.

Results

The analysis included 2,781 French Bulldogs and 21,850 non-French Bulldogs. French Bulldogs were younger (1.51 years, IQR 0.86 – 2.77 vs. 4.48 years, IQR 1.94 – 8.14) (p < 0.001) and lighter (12.45 kg, IQR 11.00 – 14.03 versus 13.80 kg, IQR 8.10 – 25.12) (p < 0.001) than non-French Bulldogs. Of 43 common specific-level disorders across both groups, French Bulldogs had significantly increased adjusted odds of 20/43 (46.5 %) disorders and significantly reduced adjusted odds of 11/43 (25.6 %) disorders compared to non-French Bulldogs. Highly predisposed disorders in French Bulldogs included stenotic nares (OR 42.14; 95 % CI 18.50 to 95.99; p < 0.001), Brachycephalic Obstructive Airway Syndrome (OR 30.89; 95 % CI 20.91 to 45.64; p < 0.001), aural discharge (OR 14.40; 95 % CI 9.08 to 22.86; p < 0.001), skin fold dermatitis (OR 11.18; 95 % CI 7.19 to 17.40; p < 0.001) and dystocia (OR 9.13; 95 % CI 5.17 to 16.13; p < 0.001). At a grouped-level of diagnostic precision, French Bulldogs had increased adjusted odds of 12/32 (37.5 %) disorders and reduced adjusted odds of 6/32 (18.8 %) disorders compared to non-French Bulldogs.

Conclusions

These results identified ultra-predispositions with worryingly higher odds in French Bulldogs for several disorders, suggesting that the health of French Bulldogs has diverged substantially from, and may be lower than, the health of the wider non-French Bulldog population. Many of these predispositions are closely associated with the conformational extremes that define the French Bulldog breed. Shifting the typical conformation of the French Bulldog population towards a more moderate phenotype is proposed as a logical opportunity to reduce the serious health issues endemic in the French Bulldog breed.

The French Bulldog is currently a hugely popular dog breed in the UK. However, the breed is linked with a range of serious health issues. Using veterinary clinical data from the VetCompass Programme at the Royal Veterinary College, this study aimed to compare the frequency of common disorders in French Bulldogs against that of all remaining dogs to identify health strengths and weaknesses in French Bulldogs. This overall view of breed health can assist owners, breeders and veterinarians to take appropriate actions to improve the health of French Bulldogs.

From an overall population of 905,544 dogs, random samples of 2,781 French Bulldogs and 21,850 non-French Bulldogs were included in the analysis. French Bulldogs were younger (1.51 years versus 4.48 years) and lighter (12.45 kg versus 13.80 kg) than non-French Bulldogs. French Bulldogs had increased risk of 20/43 (46.5 %) specific disorders and decreased risk of 11/43 (25.6 %) specific disorders compared to non-French Bulldogs. The disorders with greatest relative risk in French Bulldogs compared to non-French Bulldogs were narrowed nostrils (x 42.14), Brachycephalic Obstructive Airway Syndrome (x 30.89), ear discharge (x 14.40), skin fold dermatitis (x 11.18) and difficulty giving birth [dystocia] (x 9.13). When the disorders were grouped into broad disease categories, French Bulldogs had increased risk of 12/32 (37.5 %) disorder groups and reduced risk of 6/32 (18.8 %) disorder groups compared to non-French Bulldogs.

This study suggests that the health of French Bulldogs is very different, and largely much poorer, that the health of the wider non-French Bulldog population. Many of these differences are closely associated with the extreme body shape that defines the French Bulldog breed. Shifting the body shape of French Bulldogs to become more moderate, and hence less extreme, is proposed as a logical opportunity to reduce the current serious and common health issues in the French Bulldog breed.

Disorder predispositions and protections of Labrador Retrievers in the UK

The Labrador Retriever is one of the most popular dog breeds worldwide, therefore it is important to have reliable evidence on the general health issues of the breed. Using anonymised veterinary clinical data from the VetCompass Programme, this study aimed to explore the relative risk to common disorders in the Labrador Retriever. The clinical records of a random sample of dogs were reviewed to extract the most definitive diagnoses for all disorders recorded during 2016. A list of disorders was generated, including the 30 most common disorders in Labrador Retrievers and the 30 most common disorders in non-Labrador Retrievers. Multivariable logistic regression was used to report the odds of each of these disorders in 1462 (6.6%) Labrador Retrievers compared with 20,786 (93.4%) non-Labrador Retrievers. At a specific-level of diagnostic precision, after accounting for confounding, Labrador Retrievers had significantly increased odds of 12/35 (34.3%) disorders compared to non-Labrador Retrievers; osteoarthritis (OR 2.83) had the highest odds. Conversely, Labrador Retrievers had reduced odds of 7/35 (20.0%) disorders; patellar luxation (OR 0.18) had the lowest odds. This study provides useful information about breed-specific disorder predispositions and protections, which future research could evaluate further to produce definitive guidance for Labrador Retriever breeders and owners.

A scoping review of causal methods enabling predictions under hypothetical interventions

BACKGROUND

The methods with which prediction models are usually developed mean that neither the parameters nor the predictions should be interpreted causally. For many applications, this is perfectly acceptable. However, when prediction models are used to support decision making, there is often a need for predicting outcomes under hypothetical interventions.

AIMS

We aimed to identify published methods for developing and validating prediction models that enable risk estimation of outcomes under hypothetical interventions, utilizing causal inference. We aimed to identify the main methodological approaches, their underlying assumptions, targeted estimands, and potential pitfalls and challenges with using the method. Finally, we aimed to highlight unresolved methodological challenges.

METHODS

We systematically reviewed literature published by December 2019, considering papers in the health domain that used causal considerations to enable prediction models to be used for predictions under hypothetical interventions. We included both methodologies proposed in statistical/machine learning literature and methodologies used in applied studies.

RESULTS

We identified 4919 papers through database searches and a further 115 papers through manual searches. Of these, 87 papers were retained for full-text screening, of which 13 were selected for inclusion. We found papers from both the statistical and the machine learning literature. Most of the identified methods for causal inference from observational data were based on marginal structural models and g-estimation.

CONCLUSIONS

There exist two broad methodological approaches for allowing prediction under hypothetical intervention into clinical prediction models: (1) enriching prediction models derived from observational studies with estimated causal effects from clinical trials and meta-analyses and (2) estimating prediction models and causal effects directly from observational data. These methods require extending to dynamic treatment regimes, and consideration of multiple interventions to operationalise a clinical decision support system. Techniques for validating 'causal prediction models' are still in their infancy.

Staffordshire Bull Terriers in the UK: their disorder predispositions and protections

Background

The Staffordshire Bull Terrier is a popular dog breed in the UK but there is limited reliable evidence on disorder predispositions and protections within the breed. Using anonymised veterinary clinical data from the VetCompass™ Programme, this study aimed to identify common disorders with predisposition and protection in the Staffordshire Bull Terrier. The study hypothesised that Staffordshire Bull Terriers would have higher odds of aggression compared with non-Staffordshire Bull Terriers.

Results

The clinical records of a random sample of dogs of all types were reviewed to extract the most definitive diagnoses for all disorders existing during 2016. A combined list from the 30 most common disorders in Staffordshire Bull Terriers and the 30 most common disorders in non-Staffordshire Bull Terriers was generated. Multivariable logistic regression was used to report the odds of each of these disorders in 1304 (5.8%) Staffordshire Bull Terriers compared with 21,029 (94.2%) non-Staffordshire Bull Terriers. After accounting for confounding, Staffordshire Bull Terriers had significantly increased odds of 4/36 (11.1%) disorders compared to non-Staffordshire Bull Terriers with highest odds for seizure disorder (OR 2.06; 95% CI 1.24 to 3.40; p = 0.005). Conversely, Staffordshire Bull Terriers had reduced odds of 5/36 (13.9%) disorders, with lowest odds for patellar luxation (OR 0.15; 95% CI 0.04 to 0.61; p = 0.008). There was no significant difference in the odds of aggression between Staffordshire Bull Terriers compared with non-Staffordshire Bull Terriers (OR 1.09; 95% CI 0.75 to 1.58; p = 0.644).

Conclusions

This study provides a reliable evidence base of breed-specific disorder predispositions and protections that can be used by breeders to optimise breeding decisions. The findings can assist prospective owners of Staffordshire Bull Terriers to make informed decisions when acquiring a dog. From the relative number of predispositions to protections identified, there is no evidence that Staffordshire Bull Terriers have higher overall health problems than non-Staffordshire Bull Terriers.

Directed acyclic graphs and causal thinking in clinical risk prediction modeling

BACKGROUND

In epidemiology, causal inference and prediction modeling methodologies have been historically distinct. Directed Acyclic Graphs (DAGs) are used to model a priori causal assumptions and inform variable selection strategies for causal questions. Although tools originally designed for prediction are finding applications in causal inference, the counterpart has remained largely unexplored. The aim of this theoretical and simulation-based study is to assess the potential benefit of using DAGs in clinical risk prediction modeling.

METHODS

We explore how incorporating knowledge about the underlying causal structure can provide insights about the transportability of diagnostic clinical risk prediction models to different settings. We further probe whether causal knowledge can be used to improve predictor selection in clinical risk prediction models.

RESULTS

A single-predictor model in the causal direction is likely to have better transportability than one in the anticausal direction in some scenarios. We empirically show that the Markov Blanket, the set of variables including the parents, children, and parents of the children of the outcome node in a DAG, is the optimal set of predictors for that outcome.

CONCLUSIONS

Our findings provide a theoretical basis for the intuition that a diagnostic clinical risk prediction model including causes as predictors is likely to be more transportable. Furthermore, using DAGs to identify Markov Blanket variables may be a useful, efficient strategy to select predictors in clinical risk prediction models if strong knowledge of the underlying causal structure exists or can be learned.