An Overview and Evaluation of Recent Machine Learning Imputation Methods Using Cardiac Imaging Data

Cardiometabolic risk estimation using exposome data and machine learning

BACKGROUND

The human exposome encompasses all exposures that individuals encounter throughout their lifetime. It is now widely acknowledged that health outcomes are influenced not only by genetic factors but also by the interactions between these factors and various exposures. Consequently, the exposome has emerged as a significant contributor to the overall risk of developing major diseases, such as cardiovascular disease (CVD) and diabetes. Therefore, personalized early risk assessment based on exposome attributes might be a promising tool for identifying high-risk individuals and improving disease prevention.

OBJECTIVE

Develop and evaluate a novel and fair machine learning (ML) model for CVD and type 2 diabetes (T2D) risk prediction based on a set of readily available exposome factors. We evaluated our model using internal and external validation groups from a multi-center cohort. To be considered fair, the model was required to demonstrate consistent performance across different sub-groups of the cohort.

METHODS

From the UK Biobank, we identified 5,348 and 1,534 participants who within 13 years from the baseline visit were diagnosed with CVD and T2D, respectively. An equal number of participants who did not develop these pathologies were randomly selected as the control group. 109 readily available exposure variables from six different categories (physical measures, environmental, lifestyle, mental health events, sociodemographics, and early-life factors) from the participant's baseline visit were considered. We adopted the XGBoost ensemble model to predict individuals at risk of developing the diseases. The model's performance was compared to that of an integrative ML model which is based on a set of biological, clinical, physical, and sociodemographic variables, and, additionally for CVD, to the Framingham risk score. Moreover, we assessed the proposed model for potential bias related to sex, ethnicity, and age. Lastly, we interpreted the model's results using SHAP, a state-of-the-art explainability method.

RESULTS

The proposed ML model presents a comparable performance to the integrative ML model despite using solely exposome information, achieving a ROC-AUC of 0.78±0.01 and 0.77±0.01 for CVD and T2D, respectively. Additionally, for CVD risk prediction, the exposome-based model presents an improved performance over the traditional Framingham risk score. No bias in terms of key sensitive variables was identified.

CONCLUSIONS

We identified exposome factors that play an important role in identifying patients at risk of CVD and T2D, such as naps during the day, age completed full-time education, past tobacco smoking, frequency of tiredness/unenthusiasm, and current work status. Overall, this work demonstrates the potential of exposome-based machine learning as a fair CVD and T2D risk assessment tool.

Railway accident causation analysis: Current approaches, challenges and potential solutions

Railway accident causation analysis is fundamental to understanding the nature of railway safety. Although a considerable number of prior studies have investigated this context, many of them suffer from the need to deal with a large amount of textual data given that most railway safety-related information is recorded and stored in the form of text. To gain a better understanding of the limitations imposed by overreliance on textual analysis, a scoping review of the academic literature on how railway accident causation analysis is addressed has been conducted. The results confirm the high frequency of using textual data, a single case study, and in-depth analysis frameworks. While the value of exploring causational factors is clear, the high level of human intervention and the labour-intensive analysis processes based on a large volume of textual data hinder researchers from understanding the complex nature of the rail safety system. Recently, growing attention has been given to the application of Natural Language Processing (NLP) to aid the practice of analysing a large corpus of textual data, but only limited studies to date in railway safety use such techniques and none address railway accident causation analysis. To fill this gap, a supplementary review is conducted to identify opportunities, challenges, boundaries and limitations in the application of NLP approaches to railway accident causation analysis. Findings indicate that novel techniques using off-the-shelf tools have strong potential to overcome the limitations of overreliance on manual analysis in practice and theory, but the absence of shared railway safety-related benchmark corpora restricts implementation. This study sheds light on a new approach to railway accident causation analysis and clarifies future applicable utilisations for further research.

Big Data in Stroke: How to Use Big Data to Make the Next Management Decision

The last decade has seen significant advances in the accumulation of medical data, the computational techniques to analyze that data, and corresponding improvements in management. Interventions such as thrombolytics and mechanical thrombectomy improve patient outcomes after stroke in selected patients; however, significant gaps remain in our ability to select patients, predict complications, and understand outcomes. Big data and the computational methods needed to analyze it can address these gaps. For example, automated analysis of neuroimaging to estimate the volume of brain tissue that is ischemic and salvageable can help triage patients for acute interventions. Data-intensive computational techniques can perform complex risk calculations that are too cumbersome to be completed by humans, resulting in more accurate and timely prediction of which patients require increased vigilance for adverse events such as treatment complications. To handle the accumulation of complex medical data, a variety of advanced computational techniques referred to as machine learning and artificial intelligence now routinely complement traditional statistical inference. In this narrative review, we explore data-intensive techniques in stroke research, how it has informed the management of stroke patients, and how current work could shape clinical practice in the future.

Application of Machine Learning Models to Predict Recurrence After Surgical Resection of Nonmetastatic Renal Cell Carcinoma

BACKGROUND

Predictive tools can be useful for adapting surveillance or including patients in adjuvant trials after surgical resection of nonmetastatic renal cell carcinoma (RCC). Current models have been built using traditional statistical modelling and prespecified variables, which limits their performance.

OBJECTIVE

To investigate the performance of machine learning (ML) framework to predict recurrence after RCC surgery and compare them with current validated models.

DESIGN, SETTING, AND PARTICIPANTS

In this observational study, we derived and tested several ML-based models (Random Survival Forests [RSF], Survival Support Vector Machines [S-SVM], and Extreme Gradient Boosting [XG boost]) to predict recurrence of patients who underwent radical or partial nephrectomy for a nonmetastatic RCC, between 2013 and 2020, at 21 French medical centres.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

The primary end point was disease-free survival. Model discrimination was assessed using the concordance index (c-index), and calibration was assessed using the Brier score. ML models were compared with four conventional prognostic models, using decision curve analysis (DCA).

RESULTS AND LIMITATIONS

A total of 4067 patients were included in this study (3253 in the development cohort and 814 in the validation cohort). Most tumours (69%) were clear cell RCC, 40% were of high grade (nuclear International Society of Urological Pathology grade 3 or 4), and 24% had necrosis. Of the patients, 4% had nodal involvement. After a median follow-up of 57 mo (interquartile range 29-76), 523 (13%) patients recurred. ML models obtained higher c-index values than conventional models. The RSF yielded the highest c-index values (0.794), followed by S-SVM (c-index 0.784) and XG boost (c-index 0.782). In addition, all models showed good calibration with low integrated Brier scores (all integrated brier scores <0.1). However, we found calibration drift over time for all models, albeit with a smaller magnitude for ML models. Finally, DCA showed an incremental net benefit from all ML models compared with conventional models currently used in practice.

CONCLUSIONS

Applying ML approaches to predict recurrence following surgical resection of RCC resulted in better prediction than that of current validated models available in clinical practice. However, there is still room for improvement, which may come from the integration of novel biological and/or imaging biomarkers.

PATIENT SUMMARY

We found that artificial intelligence algorithms could better predict the risk of recurrence after surgery for a localised kidney cancer. These algorithms may help better select patients who will benefit from medical treatment after surgery.

Non-linear missing data imputation for healthcare data via index-aware autoencoders

The availability of data in the healthcare domain provides great opportunities for the discovery of new or hidden patterns in medical data, which can eventually lead to improved clinical decision making. Predictive models play a crucial role in extracting this unknown information from data. However, medical data often contain missing values that can degrade the performance of predictive models. Autoencoder models have been widely used as non-linear functions for the imputation of missing data in fields such as computer vision, transportation, and finance. In this study, we assess the shortcomings of autoencoder models for data imputation and propose modified models to improve imputation performance. To evaluate, we compare the performance of the proposed model with five well-known imputation techniques on six medical datasets and five classification methods. Through extensive experiments, we demonstrate that the proposed non-linear imputation model outperforms the other models for all degrees of missing ratios and leads to the highest disease classification accuracy for all datasets.

Toward cardiac tissue characterization using machine learning and light-scattering spectroscopy

SIGNIFICANCE

The non-destructive characterization of cardiac tissue composition provides essential information for both planning and evaluating the effectiveness of surgical interventions such as ablative procedures. Although several methods of tissue characterization, such as optical coherence tomography and fiber-optic confocal microscopy, show promise, many barriers exist that reduce effectiveness or prevent adoption, such as time delays in analysis, prohibitive costs, and limited scope of application. Developing a rapid, low-cost non-destructive means of characterizing cardiac tissue could improve planning, implementation, and evaluation of cardiac surgical procedures.

AIM

To determine whether a new light-scattering spectroscopy (LSS) system that analyzes spectra via neural networks is capable of predicting the nuclear densities (NDs) of ventricular tissues.

APPROACH

We developed an LSS system with a fiber-optics probe and applied it for measurements on cardiac tissues from an ovine model. We quantified the ND in the cardiac tissues using fluorescent labeling, confocal microscopy, and image processing. Spectra acquired from the same cardiac tissues were analyzed with spectral clustering and convolutional neural networks (CNNs) to assess the feasibility of characterizing the ND of tissue via LSS.

RESULTS

Spectral clustering revealed distinct groups of spectra correlated to ranges of ND. CNNs classified three groups of spectra with low, medium, or high ND with an accuracy of 95.00  ±  11.77  %   (mean and standard deviation). Our analyses revealed the sensitivity of the classification accuracy to wavelength range and subsampling of spectra.

CONCLUSIONS

LSS and machine learning are capable of assessing ND in cardiac tissues. We suggest that the approach is useful for the diagnosis of cardiac diseases associated with changes of ND, such as hypertrophy and fibrosis.

The role of machine learning in clinical research: transforming the future of evidence generation

Background

Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum.

Results

Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas.

Conclusions

ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence.

A machine learning-based risk stratification model for ventricular tachycardia and heart failure in hypertrophic cardiomyopathy

BACKGROUND

Machine learning (ML) and artificial intelligence are emerging as important components of precision medicine that enhance diagnosis and risk stratification. Risk stratification tools for hypertrophic cardiomyopathy (HCM) exist, but they are based on traditional statistical methods. The aim was to develop a novel machine learning risk stratification tool for the prediction of 5-year risk in HCM. The goal was to determine if its predictive accuracy is higher than the accuracy of the state-of-the-art tools.

METHOD

Data from a total of 2302 patients were used. The data were comprised of demographic characteristics, genetic data, clinical investigations, medications, and disease-related events. Four classification models were applied to model the risk level, and their decisions were explained using the SHAP (SHapley Additive exPlanations) method. Unwanted cardiac events were defined as sustained ventricular tachycardia occurrence (VT), heart failure (HF), ICD activation, sudden cardiac death (SCD), cardiac death, and all-cause death.

RESULTS

The proposed machine learning approach outperformed the similar existing risk-stratification models for SCD, cardiac death, and all-cause death risk-stratification: it achieved higher AUC by 17%, 9%, and 1%, respectively. The boosted trees achieved the best performing AUC of 0.82. The resulting model most accurately predicts VT, HF, and ICD with AUCs of 0.90, 0.88, and 0.87, respectively.

CONCLUSIONS

The proposed risk-stratification model demonstrates high accuracy in predicting events in patients with hypertrophic cardiomyopathy. The use of a machine-learning risk stratification model may improve patient management, clinical practice, and outcomes in general.

Hybrid modelling for stroke care: Review and suggestions of new approaches for risk assessment and simulation of scenarios

Stroke is an example of a complex and multi-factorial disease involving multiple organs, timescales, and disease mechanisms. To deal with this complexity, and to realize Precision Medicine of stroke, mathematical models are needed. Such approaches include: 1) machine learning, 2) bioinformatic network models, and 3) mechanistic models. Since these three approaches have complementary strengths and weaknesses, a hybrid modelling approach combining them would be the most beneficial. However, no concrete approach ready to be implemented for a specific disease has been presented to date. In this paper, we both review the strengths and weaknesses of the three approaches, and propose a roadmap for hybrid modelling in the case of stroke care. We focus on two main tasks needed for the clinical setting: a) For stroke risk calculation, we propose a new two-step approach, where non-linear mixed effects models and bioinformatic network models yield biomarkers which are used as input to a machine learning model and b) For simulation of care scenarios, we propose a new four-step approach, which revolves around iterations between simulations of the mechanistic models and imputations of non-modelled or non-measured variables. We illustrate and discuss the different approaches in the context of Precision Medicine for stroke.

A systematic review of machine learning-based missing value imputation techniques

Purpose

The primary aim of this study is to review the studies from different dimensions including type of methods, experimentation setup and evaluation metrics used in the novel approaches proposed for data imputation, particularly in the machine learning (ML) area. This ultimately provides an understanding about how well the proposed framework is evaluated and what type and ratio of missingness are addressed in the proposals. The review questions in this study are (1) what are the ML-based imputation methods studied and proposed during 2010–2020? (2) How the experimentation setup, characteristics of data sets and missingness are employed in these studies? (3) What metrics were used for the evaluation of imputation method?

Design/methodology/approach

The review process went through the standard identification, screening and selection process. The initial search on electronic databases for missing value imputation (MVI) based on ML algorithms returned a large number of papers totaling at 2,883. Most of the papers at this stage were not exactly an MVI technique relevant to this study. The literature reviews are first scanned in the title for relevancy, and 306 literature reviews were identified as appropriate. Upon reviewing the abstract text, 151 literature reviews that are not eligible for this study are dropped. This resulted in 155 research papers suitable for full-text review. From this, 117 papers are used in assessment of the review questions.

Findings

This study shows that clustering- and instance-based algorithms are the most proposed MVI methods. Percentage of correct prediction (PCP) and root mean square error (RMSE) are most used evaluation metrics in these studies. For experimentation, majority of the studies sourced the data sets from publicly available data set repositories. A common approach is that the complete data set is set as baseline to evaluate the effectiveness of imputation on the test data sets with artificially induced missingness. The data set size and missingness ratio varied across the experimentations, while missing datatype and mechanism are pertaining to the capability of imputation. Computational expense is a concern, and experimentation using large data sets appears to be a challenge.

Originality/value

It is understood from the review that there is no single universal solution to missing data problem. Variants of ML approaches work well with the missingness based on the characteristics of the data set. Most of the methods reviewed lack generalization with regard to applicability. Another concern related to applicability is the complexity of the formulation and implementation of the algorithm. Imputations based on k-nearest neighbors (kNN) and clustering algorithms which are simple and easy to implement make it popular across various domains.

Defining natural baselines for rates of change in New Zealand's groundwater quality: Dealing with incomplete or disparate datasets, accounting for impacted sites, and merging into state of the-environment reporting

To effectively manage sustainably groundwater bodies, it is essential to establish what the naturally occurring ranges of chemical concentrations in groundwaters are and how they change over time. We defined baseline trends for New Zealand groundwaters using: 1) pattern recognition techniques to deal with inconsistent monitoring suites between the national (110 sites) and the denser regional network (>1000 sites), and 2) multivariate statistics to identify and remove impacted sites from the enhanced dataset. Rates of changes were calculated for 13 parameters between January 2005 and December 2014 at more than 1000 groundwater quality monitoring sites. The resulting dataset included 262 complete cases (CC), which was enhanced using Machine-Learning (ML) techniques to a total of 607 sites. Hierarchical cluster analysis was used to identify trend clusters that were consistent between the CC, ML-enhanced datasets and a 2006 study based on solely on the national network. The largest cluster (WR) consisted of low magnitude changes across all parameters and was attributed to water-rock interaction processes. The second largest cluster (I) exhibited fast changes particularly for parameters linked to human-induced impact. The third largest cluster (D) comprised decreases of all parameters and was associated with dilution processes. Trend clusters were further refined using groundwater quality state information, enabling the identification of impacted sites outside of Cluster I in the ML-enhanced and CC datasets. Corresponding trend baselines were subsequently derived at unimpacted sites using univariate quantile distribution (5th and 95th percentile thresholds). Finally, we developed classifications combining baselines (state and trend) and natural variability to enhance state of the environment reporting. This allowed the new identification of deteriorating trends at sites where groundwater quality state is not yet affected in addition to trend reversals. These classifications can be adapted to incorporate new knowledge or align with surface water quality reporting.

Increasing the Density of Laboratory Measures for Machine Learning Applications

Background. The imputation of missingness is a key step in Electronic Health Records (EHR) mining, as it can significantly affect the conclusions derived from the downstream analysis in translational medicine. The missingness of laboratory values in EHR is not at random, yet imputation techniques tend to disregard this key distinction. Consequently, the development of an adaptive imputation strategy designed specifically for EHR is an important step in improving the data imbalance and enhancing the predictive power of modeling tools for healthcare applications. Method. We analyzed the laboratory measures derived from Geisinger’s EHR on patients in three distinct cohorts—patients tested for Clostridioides difficile (Cdiff) infection, patients with a diagnosis of inflammatory bowel disease (IBD), and patients with a diagnosis of hip or knee osteoarthritis (OA). We extracted Logical Observation Identifiers Names and Codes (LOINC) from which we excluded those with 75% or more missingness. The comorbidities, primary or secondary diagnosis, as well as active problem lists, were also extracted. The adaptive imputation strategy was designed based on a hybrid approach. The comorbidity patterns of patients were transformed into latent patterns and then clustered. Imputation was performed on a cluster of patients for each cohort independently to show the generalizability of the method. The results were compared with imputation applied to the complete dataset without incorporating the information from comorbidity patterns. Results. We analyzed a total of 67,445 patients (11,230 IBD patients, 10,000 OA patients, and 46,215 patients tested for C. difficile infection). We extracted 495 LOINC and 11,230 diagnosis codes for the IBD cohort, 8160 diagnosis codes for the Cdiff cohort, and 2042 diagnosis codes for the OA cohort based on the primary/secondary diagnosis and active problem list in the EHR. Overall, the most improvement from this strategy was observed when the laboratory measures had a higher level of missingness. The best root mean square error (RMSE) difference for each dataset was recorded as −35.5 for the Cdiff, −8.3 for the IBD, and −11.3 for the OA dataset. Conclusions. An adaptive imputation strategy designed specifically for EHR that uses complementary information from the clinical profile of the patient can be used to improve the imputation of missing laboratory values, especially when laboratory codes with high levels of missingness are included in the analysis.

Prediction of Mechanical Properties by Artificial Neural Networks to Characterize the Plastic Behavior of Aluminum Alloys

In metal forming, the plastic behavior of metallic alloys is directly related to their formability, and it has been traditionally characterized by simplified models of the flow curves, especially in the analysis by finite element simulation and analytical methods. Tools based on artificial neural networks have shown high potential for predicting the behavior and properties of industrial components. Aluminum alloys are among the most broadly used materials in challenging industries such as aerospace, automotive, or food packaging. In this study, a computer-aided tool is developed to predict two of the most useful mechanical properties of metallic materials to characterize the plastic behavior, yield strength and ultimate tensile strength. These prognostics are based on the alloy chemical composition, tempers, and Brinell hardness. In this study, a material database is employed to train an artificial neural network that is able to make predictions with a confidence greater than 95%. It is also shown that this methodology achieves a performance similar to that of empirical equations developed expressly for a specific material, but it provides greater generality since it can approximate the properties of any aluminum alloy. The methodology is based on the usage of artificial neural networks supported by a big data collection about the properties of thousands of commercial materials. Thus, the input data go above 2000 entries. When the relevant information has been collected and organized, an artificial neural network is defined, and after the training, the artificial intelligence is able to make predictions about the material properties with an average confidence greater than 95%.

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council

Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.

Intervertebral Disc Degeneration Is Associated With Aberrant Endplate Remodeling and Reduced Small Molecule Transport

The intervertebral disc is the largest avascular structure in the body, and cells within the disc rely on diffusive transport via vasculature located within the vertebral endplate to receive nutrients, eliminate waste products, and maintain disc health. However, the mechanisms by which small molecule transport into the disc occurs in vivo and how these parameters change with disc degeneration remain understudied. Here, we utilize an in vivo rabbit puncture disc degeneration model to study these interactions and provide evidence that remodeling of the endplate adjacent to the disc occurs concomitant with degeneration. Our results identify significant increases in endplate bone volume fraction, increases in microscale stiffness of the soft tissue interfaces between the disc and vertebral bone, and reductions in endplate vascularity and small molecule transport into the disc as a function of degenerative state. A neural network model identified changes in diffusion into the disc as the most significant predictor of disc degeneration. These findings support the critical role of trans-endplate transport in disease progression and will improve patient selection to direct appropriate surgical intervention and inform new therapeutic approaches to improve disc health. © 2020 American Society for Bone and Mineral Research. Published 2020. This article is a U.S. Government work and is in the public domain in the USA.

Data Imputation in Wireless Sensor Networks Using a Machine Learning-Based Virtual Sensor

Data integrity in wireless sensor networks (WSN) is very important because incorrect or missing values could result in the system making suboptimal or catastrophic decisions. Data imputation allows for a system to counteract the effect of data loss by substituting faulty or missing sensor values with system-defined virtual values. This paper proposes a virtual sensor system that uses multi-layer perceptrons (MLP) to impute sensor values in a WSN. The MLP was trained using a genetic algorithm which efficiently reached an optimal solution for each sensor node. The system was able to successfully identify and replace physical sensor nodes that were disconnected from the network with corresponding virtual sensors. The virtual sensors imputed values with very high accuracies when compared to the physical sensor values.

eResearch in acute kidney injury: a primer for electronic health record research

Acute kidney injury (AKI) has a significant impact on patient morbidity and mortality as well as overall health care costs. eResearch, which integrates information technology and information management to optimize research strategies, provides a perfect platform for necessary ongoing AKI research. With the recent adoption of a widely accepted definition of AKI and near-universal use of electronic health records, eResearch is becoming an important tool in AKI research. Conducting eResearch in AKI should ideally be based on a relatively uniform methodology. This article is the first of its kind to describe a methodology for pursuing eResearch specific to AKI and includes an illustrative database example for critically ill patients. We discuss strategies for using serum creatinine and urine output in large databases to identify and stage AKI and ways to interpolate missing values and validate data. Issues specific to the pediatric population include variation in serum creatinine with growth, varied severity of illness scoring systems and medication dosage based on weight. Many of these same strategies used to optimize AKI eResearch can be applicable to real-time AKI alerts with potential integration of additional clinical variables.

Development and validation of multivariable prediction models of remission, recovery, and quality of life outcomes in people with first episode psychosis: a machine learning approach

BACKGROUND

Outcomes for people with first-episode psychosis are highly heterogeneous. Few reliable validated methods are available to predict the outcome for individual patients in the first clinical contact. In this study, we aimed to build multivariable prediction models of 1-year remission and recovery outcomes using baseline clinical variables in people with first-episode psychosis.

METHODS

In this machine learning approach, we applied supervised machine learning, using regularised regression and nested leave-one-site-out cross-validation, to baseline clinical data from the English Evaluating the Development and Impact of Early Intervention Services (EDEN) study (n=1027), to develop and internally validate prediction models at 1-year follow-up. We assessed four binary outcomes that were recorded at 1 year: symptom remission, social recovery, vocational recovery, and quality of life (QoL). We externally validated the prediction models by selecting from the top predictor variables identified in the internal validation models the variables shared with the external validation datasets comprised of two Scottish longitudinal cohort studies (n=162) and the OPUS trial, a randomised controlled trial of specialised assertive intervention versus standard treatment (n=578).

FINDINGS

The performance of prediction models was robust for the four 1-year outcomes of symptom remission (area under the receiver operating characteristic curve [AUC] 0·703, 95% CI 0·664-0·742), social recovery (0·731, 0·697-0·765), vocational recovery (0·736, 0·702-0·771), and QoL (0·704, 0·667-0·742; p<0·0001 for all outcomes), on internal validation. We externally validated the outcomes of symptom remission (AUC 0·680, 95% CI 0·587-0·773), vocational recovery (0·867, 0·805-0·930), and QoL (0·679, 0·522-0·836) in the Scottish datasets, and symptom remission (0·616, 0·553-0·679), social recovery (0·573, 0·504-0·643), vocational recovery (0·660, 0·610-0·710), and QoL (0·556, 0·481-0·631) in the OPUS dataset.

INTERPRETATION

In our machine learning analysis, we showed that prediction models can reliably and prospectively identify poor remission and recovery outcomes at 1 year for patients with first-episode psychosis using baseline clinical variables at first clinical contact.

FUNDING

Lundbeck Foundation.

Automated Echocardiographic Quantification of Left Ventricular Ejection Fraction Without Volume Measurements Using a Machine Learning Algorithm Mimicking a Human Expert

BACKGROUND

Echocardiographic quantification of left ventricular (LV) ejection fraction (EF) relies on either manual or automated identification of endocardial boundaries followed by model-based calculation of end-systolic and end-diastolic LV volumes. Recent developments in artificial intelligence resulted in computer algorithms that allow near automated detection of endocardial boundaries and measurement of LV volumes and function. However, boundary identification is still prone to errors limiting accuracy in certain patients. We hypothesized that a fully automated machine learning algorithm could circumvent border detection and instead would estimate the degree of ventricular contraction, similar to a human expert trained on tens of thousands of images.

METHODS

Machine learning algorithm was developed and trained to automatically estimate LVEF on a database of >50 000 echocardiographic studies, including multiple apical 2- and 4-chamber views (AutoEF, BayLabs). Testing was performed on an independent group of 99 patients, whose automated EF values were compared with reference values obtained by averaging measurements by 3 experts using conventional volume-based technique. Inter-technique agreement was assessed using linear regression and Bland-Altman analysis. Consistency was assessed by mean absolute deviation among automated estimates from different combinations of apical views. Finally, sensitivity and specificity of detecting of EF ≤35% were calculated. These metrics were compared side-by-side against the same reference standard to those obtained from conventional EF measurements by clinical readers.

RESULTS

Automated estimation of LVEF was feasible in all 99 patients. AutoEF values showed high consistency (mean absolute deviation =2.9%) and excellent agreement with the reference values: r=0.95, bias=1.0%, limits of agreement =±11.8%, with sensitivity 0.90 and specificity 0.92 for detection of EF ≤35%. This was similar to clinicians' measurements: r=0.94, bias=1.4%, limits of agreement =±13.4%, sensitivity 0.93, specificity 0.87.

CONCLUSIONS

Machine learning algorithm for volume-independent LVEF estimation is highly feasible and similar in accuracy to conventional volume-based measurements, when compared with reference values provided by an expert panel.

Predicting one-year outcome in first episode psychosis using machine learning

BACKGROUND

Early illness course correlates with long-term outcome in psychosis. Accurate prediction could allow more focused intervention. Earlier intervention corresponds to significantly better symptomatic and functional outcomes. Our study objective is to use routinely collected baseline demographic and clinical characteristics to predict employment, education or training (EET) status, and symptom remission in patients with first episode psychosis (FEP) at one-year.

METHODS AND FINDINGS

83 FEP patients were recruited from National Health Service (NHS) Glasgow between 2011 and 2014 to a 24-month prospective cohort study with regular assessment of demographic and psychometric measures. An external independent cohort of 79 FEP patients were recruited from NHS Glasgow and Edinburgh during a 12-month study between 2006 and 2009. Elastic net regularised logistic regression models were built to predict binary EET status, period and point remission outcomes at one-year on 83 Glasgow patients (training dataset). Models were externally validated on an independent dataset of 79 patients from Glasgow and Edinburgh (validation dataset). Only baseline predictors shared across both cohorts were made available for model training and validation. After excluding participants with missing outcomes, models were built on the training dataset for EET status, period and point remission outcomes and externally validated on the validation dataset. Models predicted EET status, period and point remission with receiver operating curve (ROC) area under the curve (AUC) performances of 0.876 (95%CI: 0.864, 0.887), 0.630 (95%CI: 0.612, 0.647) and 0.652 (95%CI: 0.635, 0.670) respectively. Positive predictors of EET included baseline EET and living with spouse/children. Negative predictors included higher PANSS suspiciousness, hostility and delusions scores. Positive predictors for symptom remission included living with spouse/children, and affective symptoms on the Positive and Negative Syndrome Scale (PANSS). Negative predictors of remission included passive social withdrawal symptoms on PANSS. A key limitation of this study is the small sample size (n) relative to the number of predictors (p), whereby p approaches n. The use of elastic net regularised regression rather than ordinary least squares regression helped circumvent this difficulty. Further, we did not have information for biological and additional social variables, such as nicotine dependence, which observational studies have linked to outcomes in psychosis.

CONCLUSIONS AND RELEVANCE

Using advanced statistical machine learning techniques, we provide the first externally validated evidence, in a temporally and geographically independent cohort, for the ability to predict one-year EET status and symptom remission in individual FEP patients.

A dynamic model for predicting graft function in kidney recipients' upcoming follow up visits: A clinical application of artificial neural network

BACKGROUND

Predicting the function of transplanted kidneys would help clinicians in individualized medical interventions. We aimed to develop and validate a predictive tool for a future value of estimated glomerular filtration rate (eGFR) at upcoming visits.

METHODS

We used static and time-dependent covariates as inputs of artificial neural network based prediction models for predicting an eGFR value for an upcoming visit. We included 675 kidney recipients, who received transplant in the Urmia kidney transplant center in 2001-2013 and were longitudinally cared for in 2001-2017. The first 75% of records of longitudinal data of each patient were used to develop the prediction models and the remaining last 25% for evaluating its performance. Models' performances were evaluated by Mean Square Error (MSE) and Mean Absolute Error (MAE).

RESULTS

The development and validation datasets included 18,773 and 7038 records of historical data, respectively. The most accurate model included 3 static covariates of recipients' gender and donors' age and gender as well as 11 dynamic covariates of recipients including current age, time since transplant, serum creatinine, fasting blood sugar, weight and blood pressures available at each visit time. The performance of prediction models in the validation cohort was improved when history window of time dependent variables' recent values was increased from 1 to 10 (an MSE decline from 161 to 99).

CONCLUSIONS

Our best performed model is able to dynamically predict a future eGFR value for kidney recipients' upcoming visits. Integrating such a clinical tool into daily workflow of outpatient clinics can potentially support clinicians in optimal and individualized decision makings.

Sex Differences in Outcomes After STEMI: Effect Modification by Treatment Strategy and Age

IMPORTANCE

Previous works have shown that women hospitalized with ST-segment elevation myocardial infarction (STEMI) have higher short-term mortality rates than men. However, it is unclear if these differences persist among patients undergoing contemporary primary percutaneous coronary intervention (PCI).

OBJECTIVE

To investigate whether the risk of 30-day mortality after STEMI is higher in women than men and, if so, to assess the role of age, medications, and primary PCI in this excess of risk.

DESIGN, SETTING, AND PARTICIPANTS

From January 2010 to January 2016, a total of 8834 patients were hospitalized and received medical treatment for STEMI in 41 hospitals referring data to the International Survey of Acute Coronary Syndromes in Transitional Countries (ISACS-TC) registry (NCT01218776).

EXPOSURES

Demographics, baseline characteristics, clinical profile, and pharmacological treatment within 24 hours and primary PCI.

MAIN OUTCOMES AND MEASURES

Adjusted 30-day mortality rates estimated using inverse probability of treatment weighted (IPTW) logistic regression models.

RESULTS

There were 2657 women with a mean (SD) age of 66.1 (11.6) years and 6177 men with a mean (SD) age of 59.9 (11.7) years included in the study. Thirty-day mortality was significantly higher for women than for men (11.6% vs 6.0%, P < .001). The gap in sex-specific mortality narrowed if restricting the analysis to men and women undergoing primary PCI (7.1% vs 3.3%, P < .001). After multivariable adjustment for comorbidities and treatment covariates, women under 60 had higher early mortality risk than men of the same age category (OR, 1.88; 95% CI, 1.04-3.26; P = .02). The risk in the subgroups aged 60 to 74 years and over 75 years was not significantly different between sexes (OR, 1.28; 95% CI, 0.88-1.88; P = .19 and OR, 1.17; 95% CI, 0.80-1.73; P = .40; respectively). After IPTW adjustment for baseline clinical covariates, the relationship among sex, age category, and 30-day mortality was similar (OR, 1.56 [95% CI, 1.05-2.3]; OR, 1.49 [95% CI, 1.15-1.92]; and OR, 1.21 [95% CI, 0.93-1.57]; respectively).

CONCLUSIONS AND RELEVANCE

Younger age was associated with higher 30-day mortality rates in women with STEMI even after adjustment for medications, primary PCI, and other coexisting comorbidities. This difference declines after age 60 and is no longer observed in oldest women.