Can machine learning augment clinician adjudication of events in cardiovascular trials? A case study of major adverse cardiovascular events (MACE) across CVRM trials

Background and introduction

Accurate identification of clinical outcome events is critical to obtaining reliable results in cardiovascular outcomes trials (CVOTs). Current processes for event adjudication are expensive and hampered by delays. As part of a larger project to more reliably identify outcomes, we evaluated the use of machine learning to automate event adjudication using data from the SOCRATES trial (NCT01994720), a large randomized trial comparing ticagrelor and aspirin in reducing risk of major cardiovascular events after acute ischemic stroke or transient ischemic attack (TIA).

Purpose

We studied whether machine learning algorithms could replicate the outcome of the expert adjudication process for clinical events of ischemic stroke and TIA. Could classification models be trained on historical CVOT data and demonstrate performance comparable to human adjudicators?

Methods

Using data from the SOCRATES trial, multiple machine learning algorithms were tested using grid search and cross validation. Models tested included Support Vector Machines, Random Forest and XGBoost. Performance was assessed on a validation subset of the adjudication data not used for training or testing in model development. Metrics used to evaluate model performance were Receiver Operating Characteristic (ROC), Matthews Correlation Coefficient, Precision and Recall. The contribution of features, attributes of data used by the algorithm as it is trained to classify an event, that contributed to a classification were examined using both Mutual Information and Recursive Feature Elimination.

Results

Classification models were trained on historical CVOT data using adjudicator consensus decision as the ground truth. Best performance was observed on models trained to classify ischemic stroke (ROC 0.95) and TIA (ROC 0.97). Top ranked features that contributed to classification of Ischemic Stroke or TIA corresponded to site investigator decision or variables used to define the event in the trial charter, such as duration of symptoms. Model performance was comparable across the different machine learning algorithms tested with XGBoost demonstrating the best ROC on the validation set for correctly classifying both stroke and TIA.

Conclusions

Our results indicate that machine learning may augment or even replace clinician adjudication in clinical trials, with potential to gain efficiencies, speed up clinical development, and retain reliability. Our current models demonstrate good performance at binary classification of ischemic stroke and TIA within a single CVOT with high consistency and accuracy between automated and clinician adjudication. Further work will focus on harmonizing features between multiple historical clinical trials and training models to classify several different endpoint events across trials. Our aim is to utilize these clinical trial datasets to optimize the delivery of CVOTs in further cardiovascular drug development.

Funding Acknowledgement

Type of funding sources: Private company. Main funding source(s): AstraZenca Plc

Patients with prostate cancer and androgen deprivation therapy have increased risk of fractures-a study from the fractures and fall injuries in the elderly cohort (FRAILCO)

Osteoporosis is a common complication of androgen deprivation therapy (ADT). In this large Swedish cohort study consisting of a total of nearly 180,000 older men, we found that those with prostate cancer and ADT have a significantly increased risk of future osteoporotic fractures.

INTRODUCTION

Androgen deprivation therapy (ADT) in patients with prostate cancer is associated to increased risk of fractures. In this study, we investigated the relationship between ADT in patients with prostate cancer and the risk of incident fractures and non-skeletal fall injuries both compared to those without ADT and compared to patients without prostate cancer.

METHODS

We included 179,744 men (79.1 ± 7.9 years (mean ± SD)) from the Swedish registry to which national directories were linked in order to study associations regarding fractures, fall injuries, morbidity, mortality and medications. We identified 159,662 men without prostate cancer, 6954 with prostate cancer and current ADT and 13,128 men with prostate cancer without ADT. During a follow-up of approximately 270,300 patient-years, we identified 10,916 incident fractures including 4860 hip fractures.

RESULTS

In multivariable Cox regression analyses and compared to men without prostate cancer, those with prostate cancer and ADT had increased risk of any fracture (HR 95% CI 1.40 (1.28-1.53)), hip fracture (1.38 (1.20-1.58)) and MOF (1.44 (1.28-1.61)) but not of non-skeletal fall injury (1.01 (0.90-1.13)). Patients with prostate cancer without ADT did not have increased risk of any fracture (0.97 (0.90-1.05)), hip fracture (0.95 (0.84-1.07)), MOF (1.01 (0.92-1.12)) and had decreased risk of non-skeletal fall injury (0.84 (0.77-0.92)).

CONCLUSIONS

Patients with prostate cancer and ADT is a fragile patient group with substantially increased risk of osteoporotic fractures both compared to patients without prostate cancer and compared to those with prostate cancer without ADT. We believe that this must be taken in consideration in all patients with prostate cancer already at the initiation of ADT.

Hip fracture risk and safety with alendronate treatment in the oldest-old

BACKGROUND

There is high evidence for secondary prevention of fractures, including hip fracture, with alendronate treatment, but alendronate's efficacy to prevent hip fractures in the oldest-old (≥80 years old), the population with the highest fracture risk, has not been studied.

OBJECTIVE

To investigate whether alendronate treatment amongst the oldest-old with prior fracture was related to decreased hip fracture rate and sustained safety.

METHODS

Using a national database of men and women undergoing a fall risk assessment at a Swedish healthcare facility, we identified 90 795 patients who were 80 years or older and had a prior fracture. Propensity score matching (four to one) was then used to identify 7844 controls to 1961 alendronate-treated patients. The risk of incident hip fracture was investigated with Cox models and the interaction between age and treatment was investigated using an interaction term.

RESULTS

The case and control groups were well balanced in regard to age, sex, anthropometrics and comorbidity. Alendronate treatment was associated with a decreased risk of hip fracture in crude (hazard ratio (HR) 0.62 (0.49-0.79), P < 0.001) and multivariable models (HR 0.66 (0.51-0.86), P < 0.01). Alendronate was related to reduced mortality risk (HR 0.88 (0.82-0.95) but increased risk of mild upper gastrointestinal symptoms (UGI) (HR 1.58 (1.12-2.24). The alendronate association did not change with age for hip fractures or mild UGI.

CONCLUSION

In old patients with prior fracture, alendronate treatment reduces the risk of hip fracture with sustained safety, indicating that this treatment should be considered in these high-risk patients.

Leptin: a predictor of abnormal glucose tolerance and prognosis in patients with myocardial infarction and without previously known Type 2 diabetes

AIMS

High levels of leptin and low adiponectin are associated with Type 2 diabetes mellitus (T2DM) and cardiovascular (CV) disease. We studied the prognostic implications of leptin and adiponectin in patients with acute myocardial infarction (AMI) without previously known Type 2 DM.

METHODS

One hundred and eighty-one patients were included. Based on an oral glucose tolerance test at hospital discharge (day 4-5), 168 (67% men) had normal or abnormal glucose tolerance (AGT), defined as impaired glucose tolerance or T2DM. Sex- and age-matched healthy persons served as control subjects (n = 185). The associations between fasting serum leptin and adiponectin (day 2) and newly discovered AGT and CV events (CV mortality, non-fatal stroke, reinfarction or severe heart failure) during a median follow-up of 34 months were investigated.

RESULTS

Compared with control subjects, patients of both genders had significantly higher levels of leptin 2 days after an AMI. These levels were higher than those obtained at hospital discharge and 3 months later. Circulating levels of (ln) leptin 2 days after the AMI predicted AGT at discharge (odds ratio 2.03, P = 0.042). Ln leptin at day 2 was the only biochemical variable that significantly predicted CV events both on univariate [hazard ratio (HR) 1.60, P = 0.018] and on multivariate analysis (HR 1.75, P = 0.045). Adiponectin levels did not differ between patients and control subjects and did not relate to AGT or CV events.

CONCLUSIONS

Elevated circulating levels of leptin on the first morning after an AMI are associated with the presence of AGT at discharge and with a poorer long-term prognosis.

Insulin-like growth factor I: a predictor of long-term glucose abnormalities in patients with acute myocardial infarction

AIMS/HYPOTHESIS

Low levels of IGF-I are associated with increased risk of cardiovascular disease and type 2 diabetes. The aim of this study was to investigate the IGF-I system in patients with acute myocardial infarction (AMI) without previously known diabetes.

MATERIALS AND METHODS

One hundred and sixty-eight AMI patients were classified before hospital discharge by means of an OGTT as having NGT, IGT or newly detected type 2 diabetes. Age- and sex-matched subjects from the background population (n=185) served as the control group. The associations between fasting levels of IGF-I and IGF binding proteins 1 and 3 (IGFBP-1, IGFBP-3) and glucose metabolism during a follow-up period of 12 months were studied.

RESULTS

At hospital discharge, age-adjusted IGF-I (IGF-I SD) was significantly lower in patients with abnormal glucose tolerance (AGT=IGT or type 2 diabetes) compared with patients with NGT (p=0.014) and control subjects (p<0.001). IGF-I was strongly correlated with IGFBP-3 (r=0.730, p<0.001), which was significantly lower in patients with AGT compared with patients with NGT (p=0.009) and control subjects (p<0.001). Fasting levels of IGFBP-1 did not differ significantly between patients with NGT and AGT or between patients and control subjects. In a multiple logistic regression analysis in patients, IGF-I at hospital discharge was a significant predictor of AGT at discharge and after 12 months (adjusted odds ratio 0.29, p=0.022, and adjusted odds ratio 0.29, p=0.034, respectively).

CONCLUSIONS/INTERPRETATION

Low levels of IGF-I may be a useful predictor of abnormal glucose metabolism in patients with AMI.

Beta cell dysfunction in patients with acute myocardial infarction but without previously known type 2 diabetes: a report from the GAMI study

AIMS/HYPOTHESIS

Patients with acute myocardial infarction (AMI) but without previously known type 2 diabetes have a high prevalence of undiagnosed IGT and type 2 diabetes. Such perturbations have dismal prognostic implications. The aim of this study was to characterise AMI patients in terms of insulin resistance and beta cell function.

METHODS

A total of 168 consecutive AMI patients were classified by means of an OGTT before hospital discharge as having NGT, IGT or type 2 diabetes. The homeostasis model assessment (HOMA-IR) was used to estimate insulin resistance. Beta cell responsiveness was quantified as insulinogenic index (IGI) at 30 min (DeltaI(30)/DeltaG(30)).

RESULTS

According to the HOMA-IR, patients with type 2 diabetes were more insulin resistant than those with IGT or NGT (p=0.003). Beta cell responsiveness deteriorated with decreasing glucose tolerance as measured by the IGI (median [quartile 1, quartile 3] in pmol/mmol: NGT, 70.1 [42.7, 101.4]; IGT, 48.7 [34.7, 86.8], type 2 diabetes, 38.1 [25.7, 61.6]; p<0.001). The IGI was significantly related to admission capillary blood glucose (r=-0.218, p=0.010) and to the area under the curve for glucose (r=-0.475, p<0.001).

CONCLUSIONS/INTERPRETATION

Glucose abnormalities are very common in patients with AMI but without previously known type 2 diabetes. To a significant extent, this seems to be related to impaired beta cell function and implies that dysglycaemia immediately after an infarction is not a stress epiphenomenon but reflects stable disturbances of glucose regulation preceding the AMI. Early beta cell dysfunction may have important pathophysiological implications and may serve as a future target for treatment strategies.