Can machine learning augment clinician adjudication of events in cardiovascular trials? A case study of major adverse cardiovascular events (MACE) across CVRM trials

Background and introduction

Accurate identification of clinical outcome events is critical to obtaining reliable results in cardiovascular outcomes trials (CVOTs). Current processes for event adjudication are expensive and hampered by delays. As part of a larger project to more reliably identify outcomes, we evaluated the use of machine learning to automate event adjudication using data from the SOCRATES trial (NCT01994720), a large randomized trial comparing ticagrelor and aspirin in reducing risk of major cardiovascular events after acute ischemic stroke or transient ischemic attack (TIA).

Purpose

We studied whether machine learning algorithms could replicate the outcome of the expert adjudication process for clinical events of ischemic stroke and TIA. Could classification models be trained on historical CVOT data and demonstrate performance comparable to human adjudicators?

Methods

Using data from the SOCRATES trial, multiple machine learning algorithms were tested using grid search and cross validation. Models tested included Support Vector Machines, Random Forest and XGBoost. Performance was assessed on a validation subset of the adjudication data not used for training or testing in model development. Metrics used to evaluate model performance were Receiver Operating Characteristic (ROC), Matthews Correlation Coefficient, Precision and Recall. The contribution of features, attributes of data used by the algorithm as it is trained to classify an event, that contributed to a classification were examined using both Mutual Information and Recursive Feature Elimination.

Results

Classification models were trained on historical CVOT data using adjudicator consensus decision as the ground truth. Best performance was observed on models trained to classify ischemic stroke (ROC 0.95) and TIA (ROC 0.97). Top ranked features that contributed to classification of Ischemic Stroke or TIA corresponded to site investigator decision or variables used to define the event in the trial charter, such as duration of symptoms. Model performance was comparable across the different machine learning algorithms tested with XGBoost demonstrating the best ROC on the validation set for correctly classifying both stroke and TIA.

Conclusions

Our results indicate that machine learning may augment or even replace clinician adjudication in clinical trials, with potential to gain efficiencies, speed up clinical development, and retain reliability. Our current models demonstrate good performance at binary classification of ischemic stroke and TIA within a single CVOT with high consistency and accuracy between automated and clinician adjudication. Further work will focus on harmonizing features between multiple historical clinical trials and training models to classify several different endpoint events across trials. Our aim is to utilize these clinical trial datasets to optimize the delivery of CVOTs in further cardiovascular drug development.

Funding Acknowledgement

Type of funding sources: Private company. Main funding source(s): AstraZenca Plc

Targeted anti-corruption in LMICs: developmental governance and health systems research

Traditional approaches to anti-corruption have relied upon broad-based legal change and the introduction transparency and accountability measures. The evidence to date shows that these have been unsuccessful in reducing corruption in health and other sectors in low and middle income countries. Traditional approaches often assume that corruption is driven by individual greed, immorality or opportunism caused by a lack of accountability measures and that once corruption is rendered visible that there will be a channel through which it can be acted upon. In many LMICs, however, corruption and rule breaking is widespread and much better understood as a systemic problem. In these settings, health workers often break rules to solve the problems of working in overstretched, underfunded health systems. In these settings, policy often does not match the realities of an underfunded health system, and so sticking to the rules can have harm career progression or the ability to care for ones family. New approaches to anti-corruption based on Mushtaq Khan's idea of developmental governance take these context specific factors into account and look for targeted, feasible and high impact action that can create improvements of rule abiding behaviour that benefit the health system and the delivery of care. This presentation examines how it can be applied to the health system and the adaptations that it makes in the ways that we work on anti-corruption in health. It examines the ways in which policy can be changed so that groups of actors in the system are be incentivised to engage in abiding behaviour as they recognize that it is in their interests to do so.

Cancer Therapeutic Resistance: Progress and Perspectives (April 7-8, 2016 - Barcelona, Spain)

At the Cancer Therapeutic Resistance: Progress and Perspectives conference, in Barcelona, Spain, April 7-8, 2016, researchers, clinicians and students gathered to discuss our current understanding of intrinsic and acquired resistance of tumors to cancer therapies and to explore how to translate strategies to predict risk or overcome resistance to the clinic. The sessions covered a wide range of topics, including cancer omics, molecular classification, clinically relevant tumor models, biomarkers and novel therapeutic targets, and personalized medicine, with talks from many international experts in the field. This report highlights the main presentations that demonstrate the progress being made in predicting and identifying drug resistance in patients with cancer, personalized approaches to direct treatment and understanding the mechanisms involved. With better models of human cancer and powerful high-throughput screening techniques, translation to the clinic leading to tangible benefits for patients is attainable.

Age-dependent long-term structural and functional effects of early-life seizures: evidence for a hippocampal critical period influencing plasticity in adulthood

Neural activity promotes circuit formation in developing systems and during critical periods permanently modifies circuit organization and functional properties. These observations suggest that excessive neural activity, as occurs during seizures, might influence developing neural circuitry with long-term outcomes that depend on age at the time of seizures. We systematically examined long-term structural and functional consequences of seizures induced in rats by kainic acid, pentylenetetrazol, and hyperthermia across postnatal ages from birth through postnatal day 90 in adulthood (P90). Magnetic resonance imaging (MRI), diffusion tensor imaging (DTI), and electrophysiological methods at ⩾P95 following seizures induced from P1 to P90 demonstrated consistent patterns of gross atrophy, microstructural abnormalities in the corpus callosum (CC) and hippocampus, and functional alterations in hippocampal circuitry at ⩾P95 that were independent of the method of seizure induction and varied systematically as a function of age at the time of seizures. Three distinct epochs were observed in which seizures resulted in distinct long-term structural and functional outcomes at ⩾P95. Seizures prior to P20 resulted in DTI abnormalities in CC and hippocampus in the absence of gross cerebral atrophy, and increased paired-pulse inhibition (PPI) in the dentate gyrus (DG) at ⩾P95. Seizures after P30 induced a different pattern of DTI abnormalities in the fimbria and hippocampus accompanied by gross cerebral atrophy with increases in lateral ventricular volume, as well as increased PPI in the DG at ⩾P95. In contrast, seizures between P20 and P30 did not result in cerebral atrophy or significant imaging abnormalities in the hippocampus or white matter, but irreversibly decreased PPI in the DG compared to normal adult controls. These age-specific long-term structural and functional outcomes identify P20-30 as a potential critical period in hippocampal development defined by distinctive long-term structural and functional properties in adult hippocampal circuitry, including loss of capacity for seizure-induced plasticity in adulthood that could influence epileptogenesis and other hippocampal-dependent behaviors and functional properties.

Sport-related differences in biomarkers of bone resorption and cartilage degradation in endurance athletes

OBJECTIVE

By measuring urinary cross-linked N-telopeptide (NTx) as a bone resorption marker and urinary C-telopeptide of type II collagen (CTx-II) as a cartilage degradation marker, we asked whether differences in skeletal stresses in college athletes undergoing high-intensity training for diverse types of aerobic sports affect their skeletal metabolism and, if so, differentially or in unison.

METHODS

The study was cross-sectional at a Division 1 college campus with 60 student athletes representing crew, cross-country running and swimming. Controls were 16 non-athlete undergraduates. Urine samples were collected for NTx and CTx-II analysis by enzyme-linked immunosorbent assay, normalizing results to creatinine. Two-way analysis of variance models and pair-wise comparisons were used to test whether biomarker levels differed by sport and the significance when adjusted for body mass index (BMI).

RESULTS

NTx and CTx-II showed significant differences between groups before and after adjusting for BMI. NTx was highest in the rowers, and higher in rowers and runners than in swimmers or controls. CTx-II was significantly higher in runners than in crew, swimmers or controls, when unadjusted for BMI. After adjusting for BMI, these group differences remained significant except for runners over crew.

CONCLUSION

Athletes in-training in the three sports show significant differences in these markers of bone resorption and cartilage collagen degradation. The results suggest that crew undergo the highest bone remodeling and runners the highest cartilage degradation. The results also show how these markers can vary physiologically between individuals, at extremes of skeletal exercise.

The effects of a double blind, placebo controlled, artificial food colourings and benzoate preservative challenge on hyperactivity in a general population sample of preschool children

AIMS

To determine whether artificial food colourings and a preservative in the diet of 3 year old children in the general population influence hyperactive behaviour.

METHODS

A sample of 1873 children were screened in their fourth year for the presence of hyperactivity at baseline (HA), of whom 1246 had skin prick tests to identify atopy (AT). Children were selected to form the following groups: HA/AT, not-HA/AT, HA/not-AT, and not-HA/not-AT (n = 277). After baseline assessment, children were subjected to a diet eliminating artificial colourings and benzoate preservatives for one week; in the subsequent three week within subject double blind crossover study they received, in random order, periods of dietary challenge with a drink containing artificial colourings (20 mg daily) and sodium benzoate (45 mg daily) (active period), or a placebo mixture, supplementary to their diet. Behaviour was assessed by a tester blind to dietary status and by parents' ratings.

RESULTS

There were significant reductions in hyperactive behaviour during the withdrawal phase. Furthermore, there were significantly greater increases in hyperactive behaviour during the active than the placebo period based on parental reports. These effects were not influenced by the presence or absence of hyperactivity, nor by the presence or absence of atopy. There were no significant differences detected based on objective testing in the clinic.

CONCLUSIONS

There is a general adverse effect of artificial food colouring and benzoate preservatives on the behaviour of 3 year old children which is detectable by parents but not by a simple clinic assessment. Subgroups are not made more vulnerable to this effect by their prior levels of hyperactivity or by atopy.

Oxygen in nitrogen versus nitrous oxide during pediatric general anesthesia

BACKGROUND

Induction of anesthesia decreases lung volumes, giving areas of non-ventilated lung. Nitrogen is a slowly-absorbed gas that could prevent resorption of gases from these closed spaces, preventing atelectasis and improving oxygen saturations during recovery.

METHODS

We evaluated oxygen saturations during emergence after intra-operative administration of 33% oxygen in nitrogen versus 33% oxygen in nitrous oxide in 62 children having elective urologic surgery. Patients were randomly assigned to either group A (2.5 L/min air with 0.5 L/min oxygen) or group N (2 L/min nitrous oxide and 1 L/min oxygen). Flowmeters were covered to maintain blinding. Anesthetic technique was standardized (laryngeal mask airway, caudal, halothane, and deep extubation). Patients breathed room air during emergence, while a blinded observer recorded duration of desaturations by stopwatch for 15 min.

RESULTS

We found similar desaturations in both groups. The difference in desaturations < 94% at an interim 2-min total probably reflects diffusion hypoxia and was not significant at 15 min. The frequency of desaturations < 87% during emergence was significantly greater in children who were crying during induction.

CONCLUSIONS

The intra-operative use of air versus nitrous oxide has no substantial effect on oxygen saturations during emergence from anesthesia in children. These results are consistent with a recently published mathematical model.

Can parents accurately perceive hyperactivity in their child?

In all, 1872 children were recruited as part of a larger study concerning food additives and behaviours in preschool children. This figure represented 70% of the whole population of 3 1/4 -year-old children resident on the Isle of Wight, UK. Parents completed an assessment concerning their perceptions of their child's behaviour. The results of this assessment were compared with scores on two validated parental questionnaires, the Weiss Werry Peters (WWP) hyperactivity scale and the Emotionality, Activity and Sociability Temperament Questionnaire (EAS), which were used to assess hyperactivity. The accuracy of parents in perceiving hyperactivity in their children was found to be around 50% if the child was hyperactive, and 89% if the child was not hyperactive. The implications of these findings for services are discussed. Frequencies of potential risk groups for future Attention Deficit Hyperactivity Disorder (ADHD) and Conduct Disorder were also suggested.

Assessing outcome in lumbar disc surgery using patient completed measures

Measuring outcome after spinal surgery is difficult. The objective of this study was to assess the use of four measures in establishing outcome in patients undergoing lumbar discectomy. Forty-six consecutive patients who had undergone two operations for lumbar disc prolapse and 54 patients who had undergone one operation for the same condition over the same period were identified. The SF-36 questionnaire was used to assess general health. The Roland-Morris questionnaire and a simple modification of the Roland-Morris questionnaire were used to assess back and leg related disability, respectively. Analogue pain scales were used to measure back pain and sciatica. The SF-36 scores revealed significantly worse health status in the two operation compared with the one operation patients and in all patients compared with the normal population. Using the Roland-Morris and the leg disability questionnaires, patients who had undergone two operations reported significantly worse disability (Roland-Morris, 53%, poor outcome) than those who had undergone one operation (Roland-Morris, 19%, poor outcome). There was significantly greater back disability than leg disability in both groups of patients and this was confirmed by the analogue pain scales. In patients who had undergone two operations, 25% classified their back pain as very bad or unbearable, and 22.5% described very bad or unbearable leg pain. For the one operation patients these figures were 9.5 and 2.4%, respectively. The results demonstrate that both generic and condition specific patient completed measures have the potential to detect differences in outcome between patients who have undergone either one or two lumbar disc operations. The study provides support for the use of these patient completed measures in assessing outcome in lumbar disc surgery.

A novel human GnRH receptor homolog gene: abundant and wide tissue distribution of the antisense transcript

Gonadotropin releasing hormone (GnRH) regulates the reproductive system through a specific G-protein-coupled receptor (GPCR) in pituitary gonadotropes. The existence of two (or more) forms of GnRH in most vertebrates suggested the existence of GnRH receptor subtypes (I and II). Using sequence information for extracellular loop 3 of a putative Type II GnRH receptor from a reptile species, we have looked for a Type II GnRH receptor gene in the human genome EST (expressed sequence tag) database. A homolog was identified which has 45% and 41% amino acid identity with exons 2 and 3 of the known human GnRH pituitary receptor (designated Type I) and much lower homology with all other GPCRs. A total of 27 contiguous ESTs was found and comprised a continuous sequence of 1642 nucleotides. The EST sequences were confirmed in the cloned human gene and in PCR products of cDNA from several tissues. All EST transcripts detected were in the antisense orientation with respect to the novel GnRH receptor sequence and were highly expressed in a wide range of human brain and peripheral tissues. PCR of cDNA from a wide range of tissues revealed that intronic sequence equivalent to intron 2 of the Type I GnRH receptor was retained. The failure to splice out putative intron sequences in transcripts which spanned exon-intron boundaries is expected in antisense transcripts, as candidate donor and acceptor sites were only present in the gene when transcribed in the orientation encoding the GnRH receptor homolog. No transcripts extended 5' to the sequence corresponding to intron 2 of the Type I GnRH as the antisense transcripts terminated in poly A due to the presence of a polyadenylation signal sequence in the putative intron 2 when transcribed in the antisense orientation. These findings suggest that a Type II GnRH receptor gene has arisen during vertebrate evolution and is also present in the human. However, the receptor may have become vestigial in the human, possibly due to the abundant and universal tissue transcription of the opposite DNA strand to produce antisense RNA.

American Society of Gene Therapy--First Annual Meeting. Education session: the ABCs of non-viral vectors for gene therapy. 28-31 May 1998, Seattle, Washington, USA

In July 1996, the American Society of Gene Therapy (ASGT) was established in San Francisco by George Stamatoyannopoulos (President of ASGT, Professor of Genetics and Medical Genetics at the University of Washington, USA), James M Wilson (President Elect of ASGT, Director of the Institute for Human Gene Therapy, University of Pennsylvania, USA) and other gene therapy experts. In May 1998, this society held its first annual meeting, in Seattle, WA. The aim of the annual meeting is to provide a forum for all investigators, including students, young scientists, academic researchers and those working in industry. To avoid the risk of bias through election of one program committee, symposia, workshops and poster sessions are organized by different committees. There are nine committees in total, so each discipline within the diverse field of gene therapy is properly represented. An education program was organized by Savio LC Woo (Mount Sinai of Medicine, New York, USA) and there was a high school student symposium to encourage interest from the younger generation. Professor Stamatoyannopoulos said that he would like to see the NIH establishing centers of excellence of gene therapy, as they did for cancer 25 years ago. James Wilson then thanked George for all his work founding the society, as he took over as president. There are already 14,000 members of the ASGT and more than 1500 scientists attended this conference. There were 18 sessions of oral presentations and 750 abstracts.

The feasibility of MRI feedback control for intracavitary phased array hyperthermia treatments

Temperature feedback control has the potential to enhance hyperthermia treatments by providing more uniform heating of the target volume and improving the transient temperature response. A multivariable least squares batch algorithm was used to estimate system parameters for simulated prostate hyperthermia treatments. A multi-input, multi-output (MIMO) linear quadratic regulator (LQR) controller was designed for prostate hyperthermia treatments with an intracavitary phased array. A parametric study was performed for the one-dimensional control case, investigating factors relevant to magnetic resonance imaging (MRI) feedback control such as spatial resolution of temperature measurements (size of the averaging volume), sampling rate (image acquisition time), thermometry noise, control width, control depth, physiological parameter changes and reference input structure. Simulations utilizing the two dimensional (2-D) thermometry of MRI and the 2-D focusing capabilities of phased arrays demonstrated that near field heating can be controlled such that the size and shape of the heated volume can be tailored in 2-D. The control algorithms developed in this study show promising potential for incorporation into a non-invasive prostate hyperthermia system utilizing MRI feedback.

Survey of human immunodeficiency virus infection among pregnant women in England and Wales: 1990-93

We report on the first four years (1990-93) of a survey within the national HIV prevalence monitoring programme. The survey's objective is to monitor the prevalence of infection with the human immunodeficiency virus (HIV) in pregnant women in London and elsewhere in England. The survey--based in forty centres that offer antenatal care in London, Greater Manchester, West Yorkshire, and adjacent non-metropolitan areas--uses repeated cross sectional serosurveillance for anti-HIV-1 and 2 and the unlinked anonymous test method on blood left over from specimens collected for antenatal screening for immunity to rubella. The seroprevalence of HIV-1 ranged from 0.007% (1 in 14,530) in non-metropolitan areas, to 0.011% (1 in 8790) in metropolitan areas outside London, and 0.23% (1 in 440) in London. Evidence of HIV-2 infection was found in only four specimens, in London (1 in 50,300). The seroprevalence of HIV-1 in London varied more than tenfold between centres, from 0.03% (1 in 3190) to 0.51% (1 in 200). The highest prevalence of infection was in London in women aged between 20 and 30 (0.30%; 1 in 335). The seroprevalence in London centres rose from 0.18% in 1990 (1 in 560) to 0.26% in 1993 (1 in 390) and the rise was significant in all age groups. If voluntary confidential HIV testing (with counselling) among pregnant women in England were to be promoted, its cost effectiveness would be greater if focused on particular centres that provide antenatal care in London.

Lamellar ichthyosis with episodic psoriasiform reaction pattern

The skin of a girl born with the typical appearance of "collodion baby," evolved into an exfoliative erythroderma that clinically was lamellar ichthyosis. However, biopsy specimens done in early infancy showed psoriasis. Over the ensuing sixteen years she has continued to have clinical lamellar ichthyosis with rare occasions of febrile episodes and superficial pustules. Some biopsy specimens have been diagnosed as showing lamellar ichthyosis, while others have again shown psoriasis.

Methicillin-resistant Staphylococcus aureus (MRSA): risk and outcome of colonized vs. infected patients

A retrospective study of 204 patients culture positive for methicillin-resistant Staphylococcus aureus compared infected and colonized patients. Seventy-eight patients were colonized and never developed infection (C), 24 were colonized and subsequently infected (C----I), and 102 patients had 1 or more nosocomial infections with MRSA at time of first culture (I). The most prevalent sites of infection were wound (26.5%) and blood-stream (20.7%), whereas the respiratory tract and surgical wounds were both frequent sites of colonization. Stepwise discriminant analysis found the most important factors in differentiating likelihood of colonization vs. infection were recent prior hospitalization, history of wound debridement, and number of invasive procedures. Ten percent of (C) died and 25.5% of (I) died. MRSA contributed to death in 57.6% of the (I) deaths (p less than .05). These results underscore the importance of differentiating (C) vs. (I) in hospitals where MRSA is endemic so that early specific treatment may be initiated. Risk factors for infection should be discriminated from those for acquisition of the organism.