Machine learning based decision support systems (DSS) for heart disease diagnosis: a review

Performance discrepancy mitigation in heart disease prediction for multisensory inter-datasets

Heart disease is one of the primary causes of morbidity and death worldwide. Millions of people have had heart attacks every year, and only early-stage predictions can help to reduce the number. Researchers are working on designing and developing early-stage prediction systems using different advanced technologies, and machine learning (ML) is one of them. Almost all existing ML-based works consider the same dataset (intra-dataset) for the training and validation of their method. In particular, they do not consider inter-dataset performance checks, where different datasets are used in the training and testing phases. In inter-dataset setup, existing ML models show a poor performance named the inter-dataset discrepancy problem. This work focuses on mitigating the inter-dataset discrepancy problem by considering five available heart disease datasets and their combined form. All potential training and testing mode combinations are systematically executed to assess discrepancies before and after applying the proposed methods. Imbalance data handling using SMOTE-Tomek, feature selection using random forest (RF), and feature extraction using principle component analysis (PCA) with a long preprocessing pipeline are used to mitigate the inter-dataset discrepancy problem. The preprocessing pipeline builds on missing value handling using RF regression, log transformation, outlier removal, normalization, and data balancing that convert the datasets to more ML-centric. Support vector machine, K-nearest neighbors, decision tree, RF, eXtreme Gradient Boosting, Gaussian naive Bayes, logistic regression, and multilayer perceptron are used as classifiers. Experimental results show that feature selection and classification using RF produce better results than other combination strategies in both single- and inter-dataset setups. In certain configurations of individual datasets, RF demonstrates 100% accuracy and 96% accuracy during the feature selection phase in an inter-dataset setup, exhibiting commendable precision, recall, F1 score, specificity, and AUC score. The results indicate that an effective preprocessing technique has the potential to improve the performance of the ML model without necessitating the development of intricate prediction models. Addressing inter-dataset discrepancies introduces a novel research avenue, enabling the amalgamation of identical features from various datasets to construct a comprehensive global dataset within a specific domain.

Establishment of a differential diagnosis method and an online prediction platform for AOSD and sepsis based on gradient boosting decision trees algorithm

OBJECTIVE

The differential diagnosis between adult-onset Still's disease (AOSD) and sepsis has always been a challenge. In this study, a machine learning model for differential diagnosis of AOSD and sepsis was developed and an online platform was developed to facilitate the clinical application of the model.

METHODS

All data were collected from 42 AOSD patients and 50 sepsis patients admitted to Affiliated Hospital of Xuzhou Medical University from December 2018 to December 2021. In addition, 5 AOSD patients and 10 sepsis patients diagnosed in our hospital after March 2022 were collected for external validation. All models were built using the scikit-learn library (version 1.0.2) in Python (version 3.9.7), and feature selection was performed using the SHAP (Shapley Additive exPlanation) package developed in Python.

RESULTS

The results showed that the gradient boosting decision tree(GBDT) optimization model based on arthralgia, ferritin × lymphocyte count, white blood cell count, ferritin × platelet count, and α1-acid glycoprotein/creatine kinase could well identify AOSD and sepsis. The training set interaction test (AUC: 0.9916, ACC: 0.9457, Sens: 0.9556, Spec: 0.9578) and the external validation also achieved satisfactory results (AUC: 0.9800, ACC: 0.9333, Sens: 0.8000, Spec: 1.000). We named this discrimination method AIADSS (AI-assisted discrimination of Still's disease and Sepsis) and created an online service platform for practical operation, the website is http://cppdd.cn/STILL1/ .

CONCLUSION

We created a method for the identification of AOSD and sepsis based on machine learning. This method can provide a reference for clinicians to formulate the next diagnosis and treatment plan.

Deep learning model to predict exercise stress test results: Optimizing the diagnostic test selection strategy and reduce wastage in suspected coronary artery disease patients

BACKGROUND

Cardiac exercise stress testing (EST) offers a non-invasive way in the management of patients with suspected coronary artery disease (CAD). However, up to 30% EST results are either inconclusive or non-diagnostic, which results in significant resource wastage. Our aim was to build machine learning (ML) based models, using patients demographic (age, sex) and pre-test clinical information (reason for performing test, medications, blood pressure, heart rate, and resting electrocardiogram), capable of predicting EST results beforehand including those with inconclusive or non-diagnostic results.

METHODS

A total of 30,710 patients (mean age 54.0 years, 69% male) were included in the study with 25% randomly sampled in the test set, and the remaining samples were split into a train and validation set with a ratio of 9:1. We constructed different ML models from pre-test variables and compared their discriminant power using the area under the receiver operating characteristic curve (AUC).

RESULTS

A network of Oblivious Decision Trees provided the best discriminant power (AUC=0.83, sensitivity=69%, specificity=0.78%) for predicting inconclusive EST results. A total of 2010 inconclusive ESTs were correctly identified in the testing set.

CONCLUSIONS

Our ML model, developed using demographic and pre-test clinical information, can accurately predict EST results and could be used to identify patients with inconclusive or non-diagnostic results beforehand. Our system could thus be used as a personalised decision support tool by clinicians for optimizing the diagnostic test selection strategy for CAD patients and to reduce healthcare expenditure by reducing nondiagnostic or inconclusive ESTs.

Scoping Meta-Review of Methods Used to Assess Artificial Intelligence-Based Medical Devices for Heart Failure

Artificial intelligence and machine learning (AI/ML) are playing increasingly important roles, permeating the field of medical devices (MDs). This rapid progress has not yet been matched by the Health Technology Assessment (HTA) process, which still needs to define a common methodology for assessing AI/ML-based MDs. To collect existing evidence from the literature about the methods used to assess AI-based MDs, with a specific focus on those used for the management of heart failure (HF), the International Federation of Medical and Biological Engineering (IFMBE) conducted a scoping meta-review. This manuscript presents the results of this search, which covered the period from January 1974 to October 2022. After careful independent screening, 21 reviews, mainly conducted in North America and Europe, were retained and included. Among the findings were that deep learning is the most commonly utilised method and that electronic health records and registries are among the most prevalent sources of data for AI/ML algorithms. Out of the 21 included reviews, 19 focused on risk prediction and/or the early diagnosis of HF. Furthermore, 10 reviews provided evidence of the impact on the incidence/progression of HF, and 13 on the length of stay. From an HTA perspective, the main areas requiring improvement are the quality assessment of studies on AI/ML (included in 11 out of 21 reviews) and their data sources, as well as the definition of the criteria used to assess the selection of the most appropriate AI/ML algorithm.

An efficient edge/cloud medical system for rapid detection of level of consciousness in emergency medicine based on explainable machine learning models

Emergency medicine (EM) is one of the attractive research fields in which researchers investigate their efforts to diagnose and treat unforeseen illnesses or injuries. There are many tests and observations are involved in EM. Detection of the level of consciousness is one of these observations, which can be detected using several methods. Among these methods, the automatic estimation of the Glasgow coma scale (GCS) is studied in this paper. The GCS is a medical score used to describe a patient’s level of consciousness. This type of scoring system requires medical examination that may not be available with the shortage of the medical expert. Therefore, the automatic medical calculation for a patient’s level of consciousness is highly needed. Artificial intelligence has been deployed in several applications and appears to have a high performance regarding providing automatic solutions. The main objective of this work is to introduce the edge/cloud system to improve the efficiency of the consciousness measurement through efficient local data processing. Moreover, an efficient machine learning (ML) model to predict the level of consciousness of a certain patient based on the patient’s demographic, vital signs, and laboratory tests is proposed, as well as maintaining the explainability issue using Shapley additive explanations (SHAP) that provides natural language explanation in a form that helps the medical expert to understand the final prediction. The developed ML model is validated using vital signs and laboratory tests extracted from the MIMIC III dataset, and it achieves superior performance (mean absolute error (MAE) = 0.269, mean square error (MSE) = 0.625, R² score = 0.964). The resulting model is accurate, medically intuitive, and trustworthy.

Decision Support System and Web-Application Using Supervised Machine Learning Algorithms for Easy Cancer Classifications

Using a decision support system (DSS) that classifies various cancers provides support to the clinicians/researchers to make better decisions that can aid in early cancer diagnosis, thereby reducing chances of incorrect disease diagnosis. Thus, this work aimed at designing a classification model that can predict accurately for 5 different cancer types comprising of 20 cancer exomes, using the mutations identified from whole exome cancer analysis. Initially, a basic model was designed using supervised machine learning classification algorithms such as K-nearest neighbor (KNN), support vector machine (SVM), decision tree, naïve bayes and random forest (RF), among which decision tree and random forest performed better in terms of preliminary model accuracy. However, output predictions were incorrect due to less training scores. Thus, 16 essential features were then selected for model improvement using 2 approaches. All imbalanced datasets were balanced using SMOTE. In the first approach, all features from 20 cancer exome datasets were trained and models were designed using decision tree and random forest. Balanced datasets for decision tree model showed an accuracy of 77%, while with the RF model, the accuracy improved to 82% where all 5 cancer types were predicted correctly. Area under the curve for RF model was closer to 1, than decision tree model. In the second approach, all 15 datasets were trained, while 5 were tested. However, only 2 cancer types were predicted correctly. To cross validate RF model, Matthew's correlation co-efficient (MCC) test was performed. For method 1, the MCC test and MCC cross validation was found to be 0.7796 and 0.9356 respectively. Likewise, for second approach, MCC was observed to be 0.9365, corroborating the accuracy of the designed model. The model was successfully deployed using Streamlit as a web application for easy use. This study presents insights for allowing easy cancer classifications.

Deep Neuro-Fuzzy System application trends, challenges, and future perspectives: a systematic survey

Deep neural networks (DNN) have remarkably progressed in applications involving large and complex datasets but have been criticized as a black-box. This downside has recently become a motivation for the research community to pursue the ideas of hybrid approaches, resulting in novel hybrid systems classified as deep neuro-fuzzy systems (DNFS). Studies regarding the implementation of DNFS have rapidly increased in the domains of computing, healthcare, transportation, and finance with high interpretability and reasonable accuracy. However, relatively few survey studies have been found in the literature to provide a comprehensive insight into this domain. Therefore, this study aims to perform a systematic review to evaluate the current progress, trends, arising issues, research gaps, challenges, and future scope related to DNFS studies. A study mapping process was prepared to guide a systematic search for publications related to DNFS published between 2015 and 2020 using five established scientific directories. As a result, a total of 105 studies were identified and critically analyzed to address research questions with the objectives: (i) to understand the concept of DNFS; (ii) to find out DNFS optimization methods; (iii) to visualize the intensity of work carried out in DNFS domain; and (iv) to highlight DNFS application subjects and domains. We believe that this study provides up-to-date guidance for future research in the DNFS domain, allowing for more effective advancement in techniques and processes. The analysis made in this review proves that DNFS-based research is actively growing with a substantial implementation and application scope in the future.

Toward an Ecologically Valid Conceptual Framework for the Use of Artificial Intelligence in Clinical Settings: Need for Systems Thinking, Accountability, Decision-making, Trust, and Patient Safety Considerations in Safeguarding the Technology and Clinicians

The health care management and the medical practitioner literature lack a descriptive conceptual framework for understanding the dynamic and complex interactions between clinicians and artificial intelligence (AI) systems. As most of the existing literature has been investigating AI's performance and effectiveness from a statistical (analytical) standpoint, there is a lack of studies ensuring AI's ecological validity. In this study, we derived a framework that focuses explicitly on the interaction between AI and clinicians. The proposed framework builds upon well-established human factors models such as the technology acceptance model and expectancy theory. The framework can be used to perform quantitative and qualitative analyses (mixed methods) to capture how clinician-AI interactions may vary based on human factors such as expectancy, workload, trust, cognitive variables related to absorptive capacity and bounded rationality, and concerns for patient safety. If leveraged, the proposed framework can help to identify factors influencing clinicians' intention to use AI and, consequently, improve AI acceptance and address the lack of AI accountability while safeguarding the patients, clinicians, and AI technology. Overall, this paper discusses the concepts, propositions, and assumptions of the multidisciplinary decision-making literature, constituting a sociocognitive approach that extends the theories of distributed cognition and, thus, will account for the ecological validity of AI.

A hybrid method for heart disease diagnosis utilizing feature selection based ensemble classifier model generation

Heart disease is one of the most complicated diseases, and it affects a large number of individuals throughout the world. In healthcare, particularly cardiology, early and accurate detection of cardiac disease is critical. The Heart Disease Data Set-UCI repository collects data on heart disease. The search space and complexity of the classification models are increased by this raw dataset, which contains redundant and inconsistent data. We need to eliminate the redundant and unnecessary elements from the data to improve classification accuracy. As a consequence, feature selection approaches might be useful for reducing the cost of diagnosis by identifying the most important qualities. This research developed an ensemble classification model based on a feature selection approach in which selected features play a role in classification. Accordingly, a classification approach was introduced using ensemble learning with a genetic algorithm, feature selection, and biomedical test values to diagnose heart disease. Based on the results, it is deduced that the benefits of using the feature selection method vary depending on the utilized machine learning technique. However, the best-proposed model based on the combination of genetic algorithm and the ensemble learning model has achieved an accuracy of 97.57% on the considered datasets. The suggested diagnosis system achieved better accuracy than previously proposed methods and can easily be implemented in healthcare to identify heart disease.

Machine Learning for Supplementing Behavioral Assessment

The Questions About Behavioral Function (QABF) has a high degree of convergent validity, but there is still a lack of agreement between the results of the assessment and the results of experimental function analysis. Machine learning (ML) may improve the validity of assessments by using data to build a mathematical model for more accurate predictions. We used published QABF and subsequent functional analyses to train ML models to identify the function of behavior. With ML models, predictions can be made from indirect assessment results based on learning from results of past experimental functional analyses. In Experiment 1, we compared the results of five algorithms to the QABF criteria using a leave-one-out cross-validation approach. All five outperformed the QABF assessment on multilabel accuracy (i.e., percentage of predictions with the presence or absence of each function indicated correctly), but false negatives remained an issue. In Experiment 2, we augmented the data with 1,000 artificial samples to train and test an artificial neural network. The artificial network outperformed other models on all measures of accuracy. The results indicated that ML could be used to inform conditions that should be present in a functional analysis. Therefore, this study represents a proof-of-concept for the application of machine learning to functional assessment.

Machine Learning for Ischemic Heart Disease Diagnosis Aided by Evolutionary Computing

Heart disease is the leading cause of death from chronic diseases in the developing countries. The difficulty of making an accurate and timely diagnosis is exacerbated by a lack of resources and professionals in some areas, which contributes to this reality. Medical professionals may benefit from technological advancements that aid in the accurate diagnosis of patients. In light of these findings, a hybrid diagnostic tool has been developed that combines several computational intelligence (machine learning) techniques capable of analyzing clinical histories and images of electrocardiogram signals and indicating whether or not the patient has ischemic heart disease with up to 97.01% accuracy. Working with medical experts and a database containing clinical data on approximately 1020 patients and their diagnoses was required for this project. Both were put to use. A picture database containing 92 images of electrocardiogram signals was also used in this project for the analysis of the Artificial Neural Network. After extensive research and testing by the medical community, which supported the project and provided positive feedback, a successful tool was developed. This demonstrated the tool's effectiveness.

An Efficient PCA-GA-HKSVM-Based Disease Diagnostic Assistant

Disease diagnosis faces challenges such as misdiagnosis, lack of diagnosis, and slow diagnosis. There are several machine learning techniques that have been applied to address these challenges, where a set of symptoms is applied to a classification model that predicts the presence or absence of a disease. To improve on the performance of these techniques, this paper presents a technique which involves feature selection using principal component analysis (PCA), a hybrid kernel-based support vector machine (HKSVM) classification model and hyperparameter optimization using genetic algorithm (GA). The HKSVM in this paper introduces a new way of combining three kernels: Radial basis function (RBF), linear, and polynomial. Combining local (RBF) and global (linear and polynomial) kernels has the effect of improved model performance. This is because the local kernels are better able to distinguish points closer to each other while the global kernels are more suited to distinguish points that are far away from each other. The PCA-GA-HKSVM is used on 7 different medical datasets, with two datasets being multiclass datasets and 5 datasets being binary. Performance evaluation metrics used were accuracy, precision, and recall. It was observed that the PCA-GA-HKSVM offered better performance than the single kernel support vector machines (SVMs).

Machine Learning: Algorithms, Real-World Applications and Research Directions

In the current age of the Fourth Industrial Revolution (4IR or Industry 4.0), the digital world has a wealth of data, such as Internet of Things (IoT) data, cybersecurity data, mobile data, business data, social media data, health data, etc. To intelligently analyze these data and develop the corresponding smart and automated applications, the knowledge of artificial intelligence (AI), particularly, machine learning (ML) is the key. Various types of machine learning algorithms such as supervised, unsupervised, semi-supervised, and reinforcement learning exist in the area. Besides, the deep learning, which is part of a broader family of machine learning methods, can intelligently analyze the data on a large scale. In this paper, we present a comprehensive view on these machine learning algorithms that can be applied to enhance the intelligence and the capabilities of an application. Thus, this study’s key contribution is explaining the principles of different machine learning techniques and their applicability in various real-world application domains, such as cybersecurity systems, smart cities, healthcare, e-commerce, agriculture, and many more. We also highlight the challenges and potential research directions based on our study. Overall, this paper aims to serve as a reference point for both academia and industry professionals as well as for decision-makers in various real-world situations and application areas, particularly from the technical point of view.

Data Science and Analytics: An Overview from Data-Driven Smart Computing, Decision-Making and Applications Perspective

The digital world has a wealth of data, such as internet of things (IoT) data, business data, health data, mobile data, urban data, security data, and many more, in the current age of the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting knowledge or useful insights from these data can be used for smart decision-making in various applications domains. In the area of data science, advanced analytics methods including machine learning modeling can provide actionable insights or deeper knowledge about data, which makes the computing process automatic and smart. In this paper, we present a comprehensive view on "Data Science" including various types of advanced analytics methods that can be applied to enhance the intelligence and capabilities of an application through smart decision-making in different scenarios. We also discuss and summarize ten potential real-world application domains including business, healthcare, cybersecurity, urban and rural data science, and so on by taking into account data-driven smart computing and decision making. Based on this, we finally highlight the challenges and potential research directions within the scope of our study. Overall, this paper aims to serve as a reference point on data science and advanced analytics to the researchers and decision-makers as well as application developers, particularly from the data-driven solution point of view for real-world problems.

Computational modeling of human reasoning processes for interpretable visual knowledge: a case study with radiographers

Visual reasoning is critical in many complex visual tasks in medicine such as radiology or pathology. It is challenging to explicitly explain reasoning processes due to the dynamic nature of real-time human cognition. A deeper understanding of such reasoning processes is necessary for improving diagnostic accuracy and computational tools. Most computational analysis methods for visual attention utilize black-box algorithms which lack explainability and are therefore limited in understanding the visual reasoning processes. In this paper, we propose a computational method to quantify and dissect visual reasoning. The method characterizes spatial and temporal features and identifies common and contrast visual reasoning patterns to extract significant gaze activities. The visual reasoning patterns are explainable and can be compared among different groups to discover strategy differences. Experiments with radiographers of varied levels of expertise on 10 levels of visual tasks were conducted. Our empirical observations show that the method can capture the temporal and spatial features of human visual attention and distinguish expertise level. The extracted patterns are further examined and interpreted to showcase key differences between expertise levels in the visual reasoning processes. By revealing task-related reasoning processes, this method demonstrates potential for explaining human visual understanding.

Convolutional Support Vector Models: Prediction of Coronavirus Disease Using Chest X-rays

The disease caused by the new coronavirus (COVID-19) has been plaguing the world for months and the number of cases are growing more rapidly as the days go by. Therefore, finding a way to identify who has the causative virus is impressive, in order to find a way to stop its proliferation. In this paper, a complete and applied study of convolutional support machines will be presented to classify patients infected with COVID-19 using X-ray data and comparing them with traditional convolutional neural network (CNN). Based on the fitted models, it was possible to observe that the convolutional support vector machine with the polynomial kernel (CSVMPol) has a better predictive performance. In addition to the results obtained based on real images, the behavior of the models studied was observed through simulated images, where it was possible to observe the advantages of support vector machine (SVM) models.

A secure remote health monitoring model for early disease diagnosis in cloud-based IoT environment

Internet of Things (IoT) and smart medical devices have improved the healthcare systems by enabling remote monitoring and screening of the patients' health conditions anywhere and anytime. Due to an unexpected and huge increasing in number of patients during coronavirus (novel COVID-19) pandemic, it is considerably indispensable to monitor patients' health condition continuously before any serious disorder or infection occur. According to transferring the huge volume of produced sensitive health data of patients who do not want their private medical information to be revealed, dealing with security issues of IoT data as a major concern and a challenging problem has remained yet. Encountering this challenge, in this paper, a remote health monitoring model that applies a lightweight block encryption method for provisioning security for health and medical data in cloud-based IoT environment is presented. In this model, the patients' health statuses are determined via predicting critical situations through data mining methods for analyzing their biological data sensed by smart medical IoT devices in which a lightweight secure block encryption technique is used to ensure the patients' sensitive data become protected. Lightweight block encryption methods have a crucial effective influence on this sort of systems due to the restricted resources in IoT platforms. Experimental outcomes show that K-star classification method achieves the best results among RF, MLP, SVM, and J48 classifiers, with accuracy of 95%, precision of 94.5%, recall of 93.5%, and f-score of 93.99%. Therefore, regarding the attained outcomes, the suggested model is successful in achieving an effective remote health monitoring model assisted by secure IoT data in cloud-based IoT platforms.

Diagnosis of Rare Diseases: a scoping review of clinical decision support systems

BACKGROUND

Rare Diseases (RDs), which are defined as diseases affecting no more than 5 out of 10,000 people, are often severe, chronic and life-threatening. A main problem is the delay in diagnosing RDs. Clinical decision support systems (CDSSs) for RDs are software systems to support clinicians in the diagnosis of patients with RDs. Due to their clinical importance, we conducted a scoping review to determine which CDSSs are available to support the diagnosis of RDs patients, whether the CDSSs are available to be used by clinicians and which functionalities and data are used to provide decision support.

METHODS

We searched PubMed for CDSSs in RDs published between December 16, 2008 and December 16, 2018. Only English articles, original peer reviewed journals and conference papers describing a clinical prototype or a routine use of CDSSs were included. For data charting, we used the data items "Objective and background of the publication/project", "System or project name", "Functionality", "Type of clinical data", "Rare Diseases covered", "Development status", "System availability", "Data entry and integration", "Last software update" and "Clinical usage".

RESULTS

The search identified 636 articles. After title and abstracting screening, as well as assessing the eligibility criteria for full-text screening, 22 articles describing 19 different CDSSs were identified. Three types of CDSSs were classified: "Analysis or comparison of genetic and phenotypic data," "machine learning" and "information retrieval". Twelve of nineteen CDSSs use phenotypic and genetic data, followed by clinical data, literature databases and patient questionnaires. Fourteen of nineteen CDSSs are fully developed systems and therefore publicly available. Data can be entered or uploaded manually in six CDSSs, whereas for four CDSSs no information for data integration was available. Only seven CDSSs allow further ways of data integration. thirteen CDSS do not provide information about clinical usage.

CONCLUSIONS

Different CDSS for various purposes are available, yet clinicians have to determine which is best for their patient. To allow a more precise usage, future research has to focus on CDSSs RDs data integration, clinical usage and updating clinical knowledge. It remains interesting which of the CDSSs will be used and maintained in the future.

Taylor Bird Swarm Algorithm Based on Deep Belief Network for Heart Disease Diagnosis

Contemporary medicine depends on a huge amount of information contained in medical databases. Thus, the extraction of valuable knowledge, and making scientific decisions for the treatment of disease, has progressively become necessary to attain effective diagnosis. The obtainability of a large amount of medical data leads to the requirement of effective data analysis tools for extracting constructive knowledge. This paper proposes a novel method for heart disease diagnosis. Here, the pre-processing of medical data is done using log-transformation that converts the data to its uniform value range. Then, the feature selection process is performed using sparse fuzzy-c-means (FCM) for selecting significant features to classify medical data. Incorporating sparse FCM for the feature selection process provides more benefits for interpreting the models, as this sparse technique provides important features for detection, and can be utilized for handling high dimensional data. Then, the selected features are given to the deep belief network (DBN), which is trained using the proposed Taylor-based bird swarm algorithm (Taylor-BSA) for detection. Here, the proposed Taylor-BSA is designed by combining the Taylor series and bird swarm algorithm (BSA). The proposed Taylor-BSA–DBN outperformed other methods, with maximal accuracy of 93.4%, maximal sensitivity of 95%, and maximal specificity of 90.3%, respectively.

Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study

Background

The development and application of clinical prediction models using machine learning in clinical decision support systems is attracting increasing attention.

Objective

The aims of this study were to develop a prediction model for cardiac arrest in the emergency department (ED) using machine learning and sequential characteristics and to validate its clinical usefulness.

Methods

This retrospective study was conducted with ED patients at a tertiary academic hospital who suffered cardiac arrest. To resolve the class imbalance problem, sampling was performed using propensity score matching. The data set was chronologically allocated to a development cohort (years 2013 to 2016) and a validation cohort (year 2017). We trained three machine learning algorithms with repeated 10-fold cross-validation.

Results

The main performance parameters were the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The random forest algorithm (AUROC 0.97; AUPRC 0.86) outperformed the recurrent neural network (AUROC 0.95; AUPRC 0.82) and the logistic regression algorithm (AUROC 0.92; AUPRC=0.72). The performance of the model was maintained over time, with the AUROC remaining at least 80% across the monitored time points during the 24 hours before event occurrence.

Conclusions

We developed a prediction model of cardiac arrest in the ED using machine learning and sequential characteristics. The model was validated for clinical usefulness by chronological visualization focused on clinical usability.

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

BACKGROUND

Artificial intelligence (AI) provides opportunities to identify the health risks of patients and thus influence patient safety outcomes.

OBJECTIVE

The purpose of this systematic literature review was to identify and analyze quantitative studies utilizing or integrating AI to address and report clinical-level patient safety outcomes.

METHODS

We restricted our search to the PubMed, PubMed Central, and Web of Science databases to retrieve research articles published in English between January 2009 and August 2019. We focused on quantitative studies that reported positive, negative, or intermediate changes in patient safety outcomes using AI apps, specifically those based on machine-learning algorithms and natural language processing. Quantitative studies reporting only AI performance but not its influence on patient safety outcomes were excluded from further review.

RESULTS

We identified 53 eligible studies, which were summarized concerning their patient safety subcategories, the most frequently used AI, and reported performance metrics. Recognized safety subcategories were clinical alarms (n=9; mainly based on decision tree models), clinical reports (n=21; based on support vector machine models), and drug safety (n=23; mainly based on decision tree models). Analysis of these 53 studies also identified two essential findings: (1) the lack of a standardized benchmark and (2) heterogeneity in AI reporting.

CONCLUSIONS

This systematic review indicates that AI-enabled decision support systems, when implemented correctly, can aid in enhancing patient safety by improving error detection, patient stratification, and drug management. Future work is still needed for robust validation of these systems in prospective and real-world clinical environments to understand how well AI can predict safety outcomes in health care settings.

Minimal Patient Clinical Variables to Accurately Predict Stress Echocardiography Outcome: Validation Study Using Machine Learning Techniques

Background

Stress echocardiography is a well-established diagnostic tool for suspected coronary artery disease (CAD). Cardiovascular risk factors are used in the assessment of the probability of CAD. The link between the outcome of stress echocardiography and patients’ variables including risk factors, current medication, and anthropometric variables has not been widely investigated.

Objective

This study aimed to use machine learning to predict significant CAD defined by positive stress echocardiography results in patients with chest pain based on anthropometrics, cardiovascular risk factors, and medication as variables. This could allow clinical prioritization of patients with likely prediction of CAD, thus saving clinician time and improving outcomes.

Methods

A machine learning framework was proposed to automate the prediction of stress echocardiography results. The framework consisted of four stages: feature extraction, preprocessing, feature selection, and classification stage. A mutual information–based feature selection method was used to investigate the amount of information that each feature carried to define the positive outcome of stress echocardiography. Two classification algorithms, support vector machine (SVM) and random forest classifiers, have been deployed. Data from 529 patients were used to train and validate the framework. Patient mean age was 61 (SD 12) years. The data consists of anthropological data and cardiovascular risk factors such as gender, age, weight, family history, diabetes, smoking history, hypertension, hypercholesterolemia, prior diagnosis of CAD, and prescribed medications at the time of the test. There were 82 positive (abnormal) and 447 negative (normal) stress echocardiography results. The framework was evaluated using the whole dataset including cases with prior diagnosis of CAD. Five-fold cross-validation was used to validate the performance of the framework. We also investigated the model in the subset of patients with no prior CAD.

Results

The feature selection methods showed that prior diagnosis of CAD, sex, and prescribed medications such as angiotensin-converting enzyme inhibitor/angiotensin receptor blocker were the features that shared the most information about the outcome of stress echocardiography. SVM classifiers showed the best trade-off between sensitivity and specificity and was achieved with three features. Using only these three features, we achieved an accuracy of 67.63% with sensitivity and specificity 72.87% and 66.67% respectively. However, for patients with no prior diagnosis of CAD, only two features (sex and angiotensin-converting enzyme inhibitor/angiotensin receptor blocker use) were needed to achieve accuracy of 70.32% with sensitivity and specificity at 70.24%.

Conclusions

This study shows that machine learning can predict the outcome of stress echocardiography based on only a few features: patient prior cardiac history, gender, and prescribed medication. Further research recruiting higher number of patients who underwent stress echocardiography could further improve the performance of the proposed algorithm with the potential of facilitating patient selection for early treatment/intervention avoiding unnecessary downstream testing.

Architecture for a Multimodal and Domain-Independent Clinical Decision Support System Software Development Kit

Digitalization of the decision-making process in healthcare has been promoted to improve clinical performance and patient outcomes. The implementation of Clinical Practice Guidelines (CPGs) using Clinical Decision Support Systems (CDSSs) is widely developed in order to achieve this purpose within clinical information systems. Nevertheless, due to several factors such as (i) incompleteness of CPG clinical knowledge, (ii) out-of-date contents, or (iii) knowledge gaps for specific clinical situations, guideline-based CDSSs may not completely satisfy clinical needs. The proposed architecture aims to cope with guideline knowledge gaps and pitfalls by harmonizing different modalities of decision support (i.e. guideline-based CDSSs, experience-based CDSSs, and data mining-based CDSSs) and information sources (i.e. CPGs and patient data) to provide the most complete, personalized, and up-to-date propositions to manage patients. We have developed a decisional event structure to retrieve all the information related to the decision-making process. This structure allows the tracking, computation, and evaluation of all the decisions made over time based on patient clinical outcomes. Finally, different user-friendly and easy-to-use authoring tools have been implemented within the proposed architecture to integrate the role of clinicians in the whole process of knowledge generation and validation. A use case based on Breast Cancer management is presented to illustrate the performance of the implemented architecture.

Cerebral Micro-Bleeding Detection Based on Densely Connected Neural Network

Cerebral micro-bleedings (CMBs) are small chronic brain hemorrhages that have many side effects. For example, CMBs can result in long-term disability, neurologic dysfunction, cognitive impairment and side effects from other medications and treatment. Therefore, it is important and essential to detect CMBs timely and in an early stage for prompt treatment. In this research, because of the limited labeled samples, it is hard to train a classifier to achieve high accuracy. Therefore, we proposed employing Densely connected neural network (DenseNet) as the basic algorithm for transfer learning to detect CMBs. To generate the subsamples for training and test, we used a sliding window to cover the whole original images from left to right and from top to bottom. Based on the central pixel of the subsamples, we could decide the target value. Considering the data imbalance, the cost matrix was also employed. Then, based on the new model, we tested the classification accuracy, and it achieved 97.71%, which provided better performance than the state of art methods.