Can machine learning augment clinician adjudication of events in cardiovascular trials? A case study of major adverse cardiovascular events (MACE) across CVRM trials

Background and introduction

Accurate identification of clinical outcome events is critical to obtaining reliable results in cardiovascular outcomes trials (CVOTs). Current processes for event adjudication are expensive and hampered by delays. As part of a larger project to more reliably identify outcomes, we evaluated the use of machine learning to automate event adjudication using data from the SOCRATES trial (NCT01994720), a large randomized trial comparing ticagrelor and aspirin in reducing risk of major cardiovascular events after acute ischemic stroke or transient ischemic attack (TIA).

Purpose

We studied whether machine learning algorithms could replicate the outcome of the expert adjudication process for clinical events of ischemic stroke and TIA. Could classification models be trained on historical CVOT data and demonstrate performance comparable to human adjudicators?

Methods

Using data from the SOCRATES trial, multiple machine learning algorithms were tested using grid search and cross validation. Models tested included Support Vector Machines, Random Forest and XGBoost. Performance was assessed on a validation subset of the adjudication data not used for training or testing in model development. Metrics used to evaluate model performance were Receiver Operating Characteristic (ROC), Matthews Correlation Coefficient, Precision and Recall. The contribution of features, attributes of data used by the algorithm as it is trained to classify an event, that contributed to a classification were examined using both Mutual Information and Recursive Feature Elimination.

Results

Classification models were trained on historical CVOT data using adjudicator consensus decision as the ground truth. Best performance was observed on models trained to classify ischemic stroke (ROC 0.95) and TIA (ROC 0.97). Top ranked features that contributed to classification of Ischemic Stroke or TIA corresponded to site investigator decision or variables used to define the event in the trial charter, such as duration of symptoms. Model performance was comparable across the different machine learning algorithms tested with XGBoost demonstrating the best ROC on the validation set for correctly classifying both stroke and TIA.

Conclusions

Our results indicate that machine learning may augment or even replace clinician adjudication in clinical trials, with potential to gain efficiencies, speed up clinical development, and retain reliability. Our current models demonstrate good performance at binary classification of ischemic stroke and TIA within a single CVOT with high consistency and accuracy between automated and clinician adjudication. Further work will focus on harmonizing features between multiple historical clinical trials and training models to classify several different endpoint events across trials. Our aim is to utilize these clinical trial datasets to optimize the delivery of CVOTs in further cardiovascular drug development.

Funding Acknowledgement

Type of funding sources: Private company. Main funding source(s): AstraZenca Plc

Tissue factor overexpression in rat arterial neointima models thrombosis and progression of advanced atherosclerosis

BACKGROUND

Tissue factor located in the atherosclerotic plaque might cause the clinically significant thrombotic events associated with end-stage disease. It might also affect intimal area by increasing matrix accumulation and stimulating smooth muscle cell (SMC) migration and proliferation. To test this hypothesis, we overexpressed tissue factor in a rat model of the human fibrous plaque.

METHODS AND RESULTS

A neointima was generated by seeding tissue factor-overexpressing rat SMCs onto the luminal surface of a balloon-injured syngeneic rat carotid artery. The cells attached and expressed tissue factor over the long term. Mural thrombus accumulation was present at 4 and 7 days and increased neointimal SMC numbers and area by 2-fold at 2 and 4 weeks. Tissue factor overexpression accelerated reendothelialization compared with controls at 2 weeks and 1 month. Tissue factor-overexpressing SMCs exhibited increased migration both in vitro and in vivo. The increased migration by tissue factor-overexpressing SMCs in vitro was not dependent on activation of the coagulation cascade and could be blocked by an inhibitor of tissue factor.

CONCLUSIONS

These results suggest that tissue factor plays a direct role in neointimal development by coagulation-dependent and -independent pathways.

Effect of platelet-derived growth factor receptor-alpha and -beta blockade on flow-induced neointimal formation in endothelialized baboon vascular grafts

The growth of neointima and neointimal smooth muscle cells in baboon polytetrafluoroethylene grafts is regulated by blood flow. Because neointimal smooth muscle cells express both platelet-derived growth factor receptor-alpha and -beta (PDGFR-alpha and -beta), we designed this study to test the hypothesis that inhibiting either PDGFR-alpha or PDGFR-beta with a specific mouse/human chimeric antibody will modulate flow-induced neointimal formation. Bilateral aortoiliac grafts and distal femoral arteriovenous fistulae were placed in 17 baboons. After 8 weeks, 1 arteriovenous fistulae was ligated, normalizing flow through the ipsilateral graft while maintaining high flow in the contralateral graft. The experimental groups received a blocking antibody to PDGFR-alpha (Ab-PDGFR-alpha; 10 mg/kg; n=5) or PDGFR-beta (Ab-PDGFR-beta; 10 mg/kg; n=6) by pulsed intravenous administration 30 minutes before ligation and at 4, 8, 15, and 22 days after ligation. Controls received carrier medium alone (n=8). Serum antibody concentrations were followed. Grafts were harvested after 28 days and analyzed by videomorphometry. Serum Ab-PDGFR-alpha concentrations fell rapidly after day 7 to 0, whereas serum Ab-PDGFR-beta concentrations were maintained at the target levels (>50 microg/mL). Compared with controls (3.7+/-0.3), the ratio of the intimal areas (normalized flow/high flow) was significantly reduced in Ab-PDGFR-beta (1.2+/-0.2, P<0.01) but not in Ab-PDGFR-alpha (2.2+/-0.4). Ab-PDGFR-alpha decreased significantly the overall smooth muscle cell nuclear density of the neointima (P<0.01) compared with either the control or Ab-PDGFR-beta treated groups. PDGFR-beta is necessary for flow-induced neointimal formation in prosthetic grafts. Targeting PDGFR-beta may be an effective pharmacological strategy for suppressing graft neointimal development.

Local plasminogen activator inhibitor type 1 overexpression in rat carotid artery enhances thrombosis and endothelial regeneration while inhibiting intimal thickening

Elevated levels of plasminogen activator inhibitor type 1 (PAI-1) are found in advanced atherosclerotic plaque compared with normal vessel and may contribute to plaque progression and complications associated with plaque rupture. Increased expression of PAI-1 probably contributes to the thrombotic properties of advanced atherosclerotic plaque by impeding plasmin generation and degradation of fibrin. To test this hypothesis, we have deliberately created synthetic neointimas by seeding onto the denuded luminal surface of rat carotid arteries smooth muscle cells transduced with replication-defective retrovirus encoding rat PAI-1. This cell-based gene transfer method results in stable, long-term, and localized gene expression. PAI-1 overexpression increases mural thrombus accumulation at 4 days but decreases neointimal area by 30% and 25% at 1 week and 2 weeks, respectively. PAI-1 overexpression accelerates reendothelialization of injured arteries compared with control arteries at 1 week, 2 weeks, and 1 month. PAI-1 overexpression does not alter matrix accumulation at 1 week. Increased PAI-1 expression in the rat carotid artery enhances thrombosis and endothelial regeneration while inhibiting intimal thickening. These results suggest that PAI-1 could play a direct role in the development of advanced atherosclerotic plaque and in the repair of the diseased vessel after fibrous cap disruption.

Metalloproteinase blockade by local overexpression of TIMP-1 increases elastin accumulation in rat carotid artery intima

We have recently demonstrated that the blockade of matrix metalloproteinases by local overexpression of the intrinsic inhibitor tissue inhibitor of matrix metalloproteinase-1 (TIMP-1) reduces intimal hyperplasia. We now report a major change in the elastin content of the intima of rat carotid arteries seeded with TIMP-1-overexpressing smooth muscle cells. To understand the mechanism responsible for elastin accumulation, synthesis and degradation of elastin in TIMP-1 and control cell-seeded rats were measured. There were no differences in elastin mRNA or elastin synthesis, as documented by 14[C]proline incorporation between TIMP-1 and control cell-seeded arteries. In contrast, there was an increase in cross-linked elastin in the TIMP-1 group. In addition, in TIMP-1 and control rats, an elastase activity of approximately 28 kD was detected by elastin zymography and was decreased in TIMP-1 cell-seeded vessels. The 28 kD elastolytic activity was inhibited by exogenously added TIMP-1 and EDTA but not by PMSF, suggesting that it was a metalloelastase. Therefore, we have demonstrated that a shift of the proteolytic balance toward protease inhibition by TIMP-1 overexpression does not change elastin synthesis but rather changes posttranslational processing, resulting in increased elastin accumulation.

Overexpression of tissue inhibitor of matrix metalloproteinase-1 inhibits vascular smooth muscle cell functions in vitro and in vivo

Arterial smooth muscle cells (SMCs) are in a quiescent growth state under normal physiological conditions, but they can be stimulated to proliferate and migrate from one tissue compartment to another if the vessel is injured. This response might require a selective and focal increase in tissue degradation, which might be mediated through the increased production of matrix metalloproteinases (MMPs). Blockade of MMP activity might therefore inhibit the SMC response to injury. To test this hypothesis, we developed clones of rat SMCs that overexpress baboon tissue inhibitor of matrix metalloproteinase-I (TIMP-1), using retrovirally mediated gene transfer, and characterized the functional capacity of these cells in vitro and in vivo. SMCs transduced with the TIMP-1 vector (LTSN) grew more slowly and also migrated through a gel matrix in a Boyden chamber assay more slowly than the vector alone (LXSN) cells. The conditioned medium from LTSN cells completely inhibited the platelet-derived growth factor-BB-induced migration of normal SMCs across a matrix-coated filter, while the LXSN cell conditioned medium had no effect. The inhibitor activity in the LTSN conditioned medium could be neutralized with an antibody to TIMP-1. In vivo, local overexpression of TIMP-1 using LTSN cells implanted onto balloon-injured rat carotid artery inhibited intimal hyperplasia. Neutralizing antibodies against TIMP-1 suppressed the effect of LTSN cell seeding on intimal thickening. These data support the conclusion that the process of SMC activation leading to a thickened intima is dependent on MMP activity and that TIMP-1 could be utilized to inhibit this process.

Generating antibodies against secreted proteins using vascular smooth muscle cells transduced with replication-defective retrovirus

The traditional method of antibody (Ab) generation requires repeated injections of antigen (Ag). We have developed an alternative method that allows an investigator to generate a polyclonal antiserum with only a cDNA in hand. We cloned a cDNA encoding the coding frame for baboon tissue inhibitor of matrix metalloproteinase-1 (TIMP-1). Fischer rat arterial smooth muscle cells (SMC) transduced with the baboon TIMP-1 using a replication-defective retrovirus were propagated in culture. TIMP-1 overexpressing rat SMC were seeded into de-endothelialized rat carotid arteries. Three weeks after cell seeding in the rat, the presence of Ab to the baboon TIMP-1 was detected by dot blot and enzyme-linked immunosorbent assay in 5 of 6 of the animals. The major portion of the Ab generated against baboon TIMP-1 during the 12-month monitoring period after the cell seeding was identified as belonging to the IgG1 subtype. More interestingly, the titer of the Ab kept rising throughout an 8-month monitoring period. Among the salient features of this Ab are its capacity to block TIMP-1 activity and its utility for detecting TIMP-1 by immunohistochemistry. These results demonstrate that Ab against a secreted protein can be obtained in response to continuous expression of the cDNA by vascular SMC. Purified Ag is not required.

Cloning and characterization of a cDNA encoding the baboon tissue inhibitor of matrix metalloproteinase-1 (TIMP-1)

A baboon aortic smooth muscle cell (SMC) cDNA library was screened for the presence of tissue inhibitor of matrix metalloproteinase-1 (TIMP-1) by polymerase chain reaction (PCR); oligodeoxyribonucleotide primers corresponding to the coding frame of the known human TIMP-1 gene were used as primers. Sequencing of the PCR-amplified baboon cDNA demonstrated only eight single-nucleotide (nt) mismatches, when compared with the coding frame of human TIMP-1. The authenticity of the PCR-amplified TIMP-1 cDNA was further confirmed by clonal screening of the library with the PCR probe and sequencing of positive clones. On Northern blots from cultured baboon SMC, the baboon cDNA hybridized to a TIMP-1-specific mRNA of 800 bp. Phorbol ester (PMA) treatment of cultured baboon SMC produced a 2.5-fold increase in TIMP-1 transcript. TIMP-1 transcripts were also demonstrated in cultures of endothelial cells and fibroblasts obtained from baboon arteries. Immunohistochemical analysis demonstrated that TIMP-1 protein is localized to the adventitial layer of baboon artery. We conclude that TIMP-1 is a conserved molecule across species and localized to the tunica adventitia of baboon vessels.

Home care team in accident and emergency

We describe experience with a community paediatric service designed to provide more convenient follow up for children and their families and to reduce the number of children returning unnecessarily to the accident and emergency department.