Validation of a machine learning algorithm for early severe sepsis prediction: a retrospective study predicting severe sepsis up to 48 h in advance using a diverse dataset from 461 US hospitals

Digital health delivery in respiratory medicine: adjunct, replacement or cause for division?

Digital medicine is already well established in respiratory medicine through remote monitoring digital devices which are used in the day-to-day care of patients with asthma, COPD and sleep disorders. Image recognition software, deployed in thoracic radiology for many applications including lung cancer screening, is another application of digital medicine. Used as clinical decision support, this software will soon become part of day-to-day practice once concerns regarding generalisability have been addressed. Embodied in the electronic health record, digital medicine also plays a substantial role in the day-to-day clinical practice of respiratory medicine. Given the considerable work the electronic health record demands from clinicians, the next tangible impact of digital medicine may be artificial intelligence that aids administration, makes record keeping easier and facilitates better digital communication with patients. Future promises of digital medicine are based on their potential to analyse and characterise the large amounts of digital clinical data that are collected in routine care. Offering the potential to predict outcomes and personalise therapy, there is much to be excited by in this new epoch of innovation. However, these digital tools are by no means a silver bullet. It remains uncertain whether, let alone when, the promises of better models of personalisation and prediction will translate into clinically meaningful and cost-effective products for clinicians.

Parsimonious Waveform-derived Features consisting of Pulse Arrival Time and Heart Rate Variability Predicts the Onset of Septic Shock

Sepsis is a major public health emergency and one of the leading causes of morbidity and mortality in critically ill patients. For each hour treatment is delayed, shock-related mortality increases, so early diagnosis and intervention is of utmost importance. However, earlier recognition of shock requires active monitoring, which may be delayed due to subclinical manifestations of the disease at the early phase of onset. Machine learning systems can increase timely detection of shock onset by exploiting complex interactions among continuous physiological waveforms. We use a dataset consisting of high-resolution physiological waveforms from intensive care unit (ICU) of a tertiary hospital system. We investigate the use of mean arterial blood pressure (MAP), pulse arrival time (PAT), heart rate variability (HRV), and heart rate (HR) for the early prediction of shock onset. Using only five minutes of the aforementioned vital signals from 239 ICU patients, our developed models can accurately predict septic shock onset 6 to 36 hours prior to clinical recognition with area under the receiver operating characteristic (AUROC) of 0.84 and 0.8 respectively. This work lays foundations for a robust, efficient, accurate and early prediction of septic shock onset which may help clinicians in their decision-making processes. This study introduces machine learning models that provide fast and accurate predictions of septic shock onset times up to 36 hours in advance. BP, PAT and HR dynamics can independently predict septic shock onset with a look-back period of only 5 mins.

Machine Learning Approach with Harmonized Multinational Datasets for Enhanced Prediction of Hypothyroidism in Patients with Type 2 Diabetes

Type 2 diabetes (T2D) is a global health concern with increasing prevalence. Comorbid hypothyroidism (HT) exacerbates kidney, cardiac, neurological and other complications of T2D; these risks can be mitigated pharmacologically upon detecting HT. The current HT standard of care (SOC) screening in T2D is infrequent, delaying HT diagnosis and treatment. We present a first-to-date machine learning algorithm (MLA) clinical decision tool to classify patients as low vs. high risk for developing HT comorbid with T2D; the MLA was developed using readily available patient data from harmonized multinational datasets. The MLA was trained on data from NIH All of US (AoU) and UK Biobank (UKBB) (Combined dataset) and achieved a high negative predictive value (NPV) of 0.989 and an AUROC of 0.762 in the Combined dataset, exceeding AUROCs for the models trained on AoU or UKBB alone (0.666 and 0.622, respectively), indicating that increasing dataset diversity for MLA training improves performance. This high-NPV automated tool can supplement SOC screening and rule out T2D patients with low HT risk, allowing for the prioritization of lab-based testing for at-risk patients. Conversely, an MLA output that designates a patient to be at risk of developing HT allows for tailored clinical management and thereby promotes improved patient outcomes.

Machine Learning Approach for Improved Longitudinal Prediction of Progression from Mild Cognitive Impairment to Alzheimer's Disease

Mild cognitive impairment (MCI) is cognitive decline that can indicate future risk of Alzheimer's disease (AD). We developed and validated a machine learning algorithm (MLA), based on a gradient-boosted tree ensemble method, to analyze phenotypic data for individuals 55-88 years old (n = 493) diagnosed with MCI. Data were analyzed within multiple prediction windows and averaged to predict progression to AD within 24-48 months. The MLA outperformed the mini-mental state examination (MMSE) and three comparison models at all prediction windows on most metrics. Exceptions include sensitivity at 18 months (MLA and MMSE each achieved 0.600); and sensitivity at 30 and 42 months (MMSE marginally better). For all prediction windows, the MLA achieved AUROC ≥ 0.857 and NPV ≥ 0.800. With averaged data for the 24-48-month lookahead timeframe, the MLA outperformed MMSE on all metrics. This study demonstrates that machine learning may provide a more accurate risk assessment than the standard of care. This may facilitate care coordination, decrease healthcare expenditures, and maintain quality of life for patients at risk of progressing from MCI to AD.

Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges

The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.

Application Prospect of the SOFA Score and Related Modification Research Progress in Sepsis

In 2016, the SOFA score was proposed as the main evaluation system for diagnosis in the definition of sepsis 3.0, and the SOFA score has become a new research focus in sepsis. Some people are skeptical about diagnosing sepsis using the SOFA score. Experts and scholars from different regions have proposed different, modified versions of SOFA score to make up for the related problems with the use of the SOFA score in the diagnosis of sepsis. While synthesizing the different improved versions of SOFA proposed by experts and scholars in various regions, this paper also summarizes the relevant definitions of sepsis put forward in recent years to build a clear, improved application framework of SOFA score. In addition, the comparison between machine learning and SOFA scores related to sepsis is described and discussed in the article. Taken together, by summarizing the application of the improved SOFA score proposed in recent years in the related definition of sepsis, we believe that the SOFA score is still an effective means of diagnosing sepsis, but in the process of the continuous refinement and development of sepsis in the future, the SOFA score needs to be further refined and improved to provide more accurate coping strategies for different patient populations or application directions regarding sepsis. Against the big data background, machine learning has immeasurable value and significance, but its future applications should add more humanistic references and assistance.

Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework

OBJECTIVE

To retrieve and appraise studies of deployed artificial intelligence (AI)-based sepsis prediction algorithms using systematic methods, identify implementation barriers, enablers, and key decisions and then map these to a novel end-to-end clinical AI implementation framework.

MATERIALS AND METHODS

Systematically review studies of clinically applied AI-based sepsis prediction algorithms in regard to methodological quality, deployment and evaluation methods, and outcomes. Identify contextual factors that influence implementation and map these factors to the SALIENT implementation framework.

RESULTS

The review identified 30 articles of algorithms applied in adult hospital settings, with 5 studies reporting significantly decreased mortality post-implementation. Eight groups of algorithms were identified, each sharing a common algorithm. We identified 14 barriers, 26 enablers, and 22 decision points which were able to be mapped to the 5 stages of the SALIENT implementation framework.

DISCUSSION

Empirical studies of deployed sepsis prediction algorithms demonstrate their potential for improving care and reducing mortality but reveal persisting gaps in existing implementation guidance. In the examined publications, key decision points reflecting real-word implementation experience could be mapped to the SALIENT framework and, as these decision points appear to be AI-task agnostic, this framework may also be applicable to non-sepsis algorithms. The mapping clarified where and when barriers, enablers, and key decisions arise within the end-to-end AI implementation process.

CONCLUSIONS

A systematic review of real-world implementation studies of sepsis prediction algorithms was used to validate an end-to-end staged implementation framework that has the ability to account for key factors that warrant attention in ensuring successful deployment, and which extends on previous AI implementation frameworks.

Ethical considerations in implementing AI for mortality prediction in the emergency department: Linking theory and practice

Background

Artificial intelligence (AI) is predicted to be a solution for improving healthcare, increasing efficiency, and saving time and recourses. A lack of ethical principles for the use of AI in practice has been highlighted by several stakeholders due to the recent attention given to it. Research has shown an urgent need for more knowledge regarding the ethical implications of AI applications in healthcare. However, fundamental ethical principles may not be sufficient to describe ethical concerns associated with implementing AI applications.

Objective

The aim of this study is twofold, (1) to use the implementation of AI applications to predict patient mortality in emergency departments as a setting to explore healthcare professionals' perspectives on ethical issues in relation to ethical principles and (2) to develop a model to guide ethical considerations in AI implementation in healthcare based on ethical theory.

Methods

Semi-structured interviews were conducted with 18 participants. The abductive approach used to analyze the empirical data consisted of four steps alternating between inductive and deductive analyses.

Results

Our findings provide an ethical model demonstrating the need to address six ethical principles (autonomy, beneficence, non-maleficence, justice, explicability, and professional governance) in relation to ethical theories defined as virtue, deontology, and consequentialism when AI applications are to be implemented in clinical practice.

Conclusions

Ethical aspects of AI applications are broader than the prima facie principles of medical ethics and the principle of explicability. Ethical aspects thus need to be viewed from a broader perspective to cover different situations that healthcare professionals, in general, and physicians, in particular, may face when using AI applications in clinical practice.

Characteristics and Outcomes of Sepsis Presenting in Inpatient Pediatric Settings

OBJECTIVE

The pediatric sepsis literature lacks studies examining the inpatient setting, yet sepsis remains a leading cause of death in children's hospitals. More information is needed about sepsis arising in patients already hospitalized to improve morbidity and mortality outcomes. This study describes the clinical characteristics, process measures, and outcomes of inpatient sepsis cases compared with emergency department (ED) sepsis cases within the Improving Pediatric Sepsis Outcomes data registry from 46 hospitals that care for children.

METHODS

This retrospective cohort study included Improving Pediatric Sepsis Outcomes sepsis cases from January 2017 to December 2019 with onset in inpatient or ED. We used descriptive statistics to compare inpatient and ED sepsis metrics and describe inpatient sepsis outcomes.

RESULTS

The cohort included 26 855 cases; 8.4% were inpatient and 91.6% were ED. Inpatient cases had higher sepsis-attributable mortality (2.0% vs 1.4%, P = .025), longer length of stay after sepsis recognition (9 vs 5 days, P <.001), more intensive care admissions (57.6% vs 54.1%, P = .002), and greater average vasopressor use (18.0% vs 13.6%, P <.001) compared with ED. In the inpatient cohort, >40% of cases had a time from arrival to recognition within 12 hours. In 21% of cases, this time was >96 hours. Improved adherence to sepsis treatment bundles over time was associated with improved 30-day sepsis-attributable mortality for inpatients with sepsis.

CONCLUSIONS

Inpatient sepsis cases had longer lengths of stay, more need for intensive care, and higher vasopressor use. Sepsis-attributable mortality was significantly higher in inpatient cases compared with ED cases and improved with improved sepsis bundle adherence.

A 29-MRNA HOST RESPONSE WHOLE-BLOOD SIGNATURE IMPROVES PREDICTION OF 28-DAY MORTALITY AND 7-DAY INTENSIVE CARE UNIT CARE IN ADULTS PRESENTING TO THE EMERGENCY DEPARTMENT WITH SUSPECTED ACUTE INFECTION AND/OR SEPSIS

ABSTRACT

Background: Risk stratification of emergency department patients with suspected acute infections and/or suspected sepsis remains challenging. We prospectively validated a 29-messenger RNA host response classifier for predicting severity in these patients. Methods: We enrolled adults presenting with suspected acute infections and at least one vital sign abnormality to six emergency departments in Greece. Twenty-nine target host RNAs were quantified on NanoString nCounter and analyzed with the Inflammatix Severity 2 (IMX-SEV-2) classifier to determine risk scores as low, moderate, and high severity. Performance of IMX-SEV-2 for prediction of 28-day mortality was compared with that of lactate, procalcitonin, and quick sequential organ failure assessment (qSOFA). Results: A total of 397 individuals were enrolled; 38 individuals (9.6%) died within 28 days. Inflammatix Severity 2 classifier predicted 28-day mortality with an area under the receiver operator characteristics curve of 0.82 (95% confidence interval [CI], 0.74-0.90) compared with lactate, 0.66 (95% CI, 0.54-0.77); procalcitonin, 0.67 (95% CI, 0.57-0.78); and qSOFA, 0.81 (95% CI, 0.72-0.89). Combining qSOFA with IMX-SEV-2 improved prognostic accuracy from 0.81 to 0.89 (95% CI, 0.82-0.96). The high-severity (rule-in) interpretation band of IMX-SEV-2 demonstrated 96.9% specificity for predicting 28-day mortality, whereas the low-severity (rule-out) band had a sensitivity of 78.9%. Similarly, IMX-SEV-2 alone accurately predicted the need for day-7 intensive care unit care and further boosted overall accuracy when combined with qSOFA. Conclusions: Inflammatix Severity 2 classifier predicted 28-day mortality and 7-day intensive care unit care with high accuracy and boosted the accuracy of clinical scores when used in combination.

Artificial intelligence in perioperative medicine - a narrative review

Recent advancements in artificial intelligence (AI) techniques have enabled the development of accurate prediction models using clinical big data. AI models for perioperative risk stratification, intraoperative event prediction, biosignal analyses, and intensive care medicine have been developed in the field of perioperative medicine. Some of these models have been validated using external datasets and randomized controlled trials. Once these models are implemented in electronic health record systems or software medical devices, they could help anesthesiologists improve clinical outcomes by accurately predicting complications and suggesting optimal treatment strategies in real-time. This review provides an overview of the AI techniques used in perioperative medicine and a summary of the studies that have been published using these techniques. Understanding these techniques will aid in their appropriate application in clinical practice.

Evaluating machine learning models for sepsis prediction: A systematic review of methodologies

Studies for sepsis prediction using machine learning are developing rapidly in medical science recently. In this review, we propose a set of new evaluation criteria and reporting standards to assess 21 qualified machine learning models for quality analysis based on PRISMA. Our assessment shows that (1.) the definition of sepsis is not consistent among the studies; (2.) data sources and data preprocessing methods, machine learning models, feature engineering, and inclusion types vary widely among the studies; (3.) the closer to the onset of sepsis, the higher the value of AUROC is; (4.) the improvement in AUROC is primarily due to using machine learning as a feature engineering tool; (5.) deep neural networks coupled with Sepsis-3 diagnostic criteria tend to yield better results on the time series data collected from patients with sepsis. The new evaluation criteria and reporting standards will facilitate the development of improved machine learning models for clinical applications.

•

New evaluation/reporting standard for sepsis prediction machine learning models

•

Major limitations in the current models for sepsis prediction have been identified

•

We strongly suggest using machine learning as a feature engineering tool

•

Recommending multilayer neural networks and Sepsis 3.0 for yield better result

Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.

Machine learning in medicine: It has arrived, let's embrace it

Machine learning and artificial intelligence (AI) have arrived in medicine and the healthcare community is experiencing significant growth in their adoption across numerous patient care settings. There are countless applications for machine learning and AI in medicine ranging from patient outcome prediction, to clinical decision support, to predicting future patient therapeutic setpoints. This commentary discusses a recent application leveraging machine learning to predict one-year patient survival following orthotopic heart transplantation. This modeling approach has significant implications in terms of improving clinical decision-making, patient counseling, and ultimately organ allocation and has been shown to significantly outperform pre-existing algorithms. This commentary also discusses how adoption and advancement of this modeling approach in the future can provide increased personalization of patient care. The continued expansion of information systems and growth of electronic patient data sources in health care will continue to pave the way for increased use and adoption of data science in medicine. Personalized medicine has been a long-standing goal of the healthcare community and with machine learning and AI now being continually incorporated into clinical settings and practice, this technology is well on the pathway to make a considerable impact to greatly improve patient care in the near future.

Machine Learning and Syncope Management in the ED: The Future Is Coming

In recent years, machine learning (ML) has been promisingly applied in many fields of clinical medicine, both for diagnosis and prognosis prediction. Aims of this narrative review were to summarize the basic concepts of ML applied to clinical medicine and explore its main applications in the emergency department (ED) setting, with a particular focus on syncope management. Through an extensive literature search in PubMed and Embase, we found increasing evidence suggesting that the use of ML algorithms can improve ED triage, diagnosis, and risk stratification of many diseases. However, the lacks of external validation and reliable diagnostic standards currently limit their implementation in clinical practice. Syncope represents a challenging problem for the emergency physician both because its diagnosis is not supported by specific tests and the available prognostic tools proved to be inefficient. ML algorithms have the potential to overcome these limitations and, in the future, they could support the clinician in managing syncope patients more efficiently. However, at present only few studies have addressed this issue, albeit with encouraging results.

Immune Deregulation in Sepsis and Septic Shock: Reversing Immune Paralysis by Targeting PD-1/PD-L1 Pathway

Sepsis remains a major problem for human health worldwide, thereby manifesting high rates of morbidity and mortality. Sepsis, once understood as a monophasic sustained hyperinflammation, is currently recognized as a dysregulated host response to infection, with both hyperinflammation and immunoparalysis occurring simultaneously from the earliest stages of sepsis, involving multiple organ dysfunctions. Despite the recent progress in the understanding of the pathophysiology underlying sepsis, no specific treatment to restore immune dysregulation in sepsis has been validated in clinical trials. In recent years, treatment for immune checkpoints such as the programmed cell death protein 1/programmed death ligand (PD-1/PD-L) pathway in tumor-infiltrating T-lymphocytes has been successful in the field of cancer immune therapy. As immune-paralysis in sepsis involves exhausted T-lymphocytes, future clinical applications of checkpoint inhibitors for sepsis are expected. In addition, the functions of PD-1/PD-L on innate lymphoid cells and the role of exosomal forms of PD-L1 warrant further research. Looking back on the history of repeatedly failed clinical trials of immune modulatory therapies for sepsis, sepsis must be recognized as a difficult disease entity for performing clinical trials. A major obstacle that could prevent effective clinical trials of drug candidates is the disease complexity and heterogeneities; clinically diagnosed sepsis could contain multiple sepsis subgroups that suffer different levels of hyper-inflammation and immune-suppression in distinct organs. Thus, the selection of appropriate more homogenous sepsis subgroup is the key for testing the clinical efficacy of experimental therapies targeting specific pathways in either hyperinflammation and/or immunoparalysis. An emerging technology such as artificial intelligence (AI) may help to identify an immune paralysis subgroup who would best be treated by PD-1/PD-L1 pathway inhibitors.

Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective

Sepsis is a major cause of death worldwide. Over the past years, prediction of clinically relevant events through machine learning models has gained particular attention. In the present perspective, we provide a brief, clinician-oriented vision on the following relevant aspects concerning the use of machine learning predictive models for the early detection of sepsis in the daily practice: (i) the controversy of sepsis definition and its influence on the development of prediction models; (ii) the choice and availability of input features; (iii) the measure of the model performance, the output, and their usefulness in the clinical practice. The increasing involvement of artificial intelligence and machine learning in health care cannot be disregarded, despite important pitfalls that should be always carefully taken into consideration. In the long run, a rigorous multidisciplinary approach to enrich our understanding in the application of machine learning techniques for the early recognition of sepsis may show potential to augment medical decision-making when facing this heterogeneous and complex syndrome.

Trends in US Pediatric Hospital Admissions in 2020 Compared With the Decade Before the COVID-19 Pandemic

IMPORTANCE

In early 2020, the United States declared a public health emergency in response to coronavirus disease 2019 (COVID-19) and implemented a variety of social distancing measures. The association between the COVID-19 pandemic and the number of pediatric admissions is unclear.

OBJECTIVE

To determine the changes in patterns of pediatric admissions in 2020 compared with the prior decade.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study included 49 US hospitals contributing to the Pediatric Health Information Systems database. Inpatient admissions were transformed into time-series data, and ensemble forecasting models were generated to analyze admissions across a range of diagnoses in 2020 compared with previous years. The setting was inpatient admissions. All patients discharged between January 1, 2010, and June 30, 2020, from an inpatient hospital encounter were included.

MAIN OUTCOMES AND MEASURES

Number of hospital admissions by primary diagnosis for each encounter.

RESULTS

Of 5 424 688 inpatient encounters among 3 372 839 patients (median [interquartile range] age, 5.1 [0.7-13.3] years; 2 823 748 [52.1%] boys; 3 171 224 [58.5%] White individuals) at 49 hospitals, 213 571 (3.9%) were between January 1, 2020, and June 30, 2020. There was a decrease in the number of admissions beginning in March 2020 compared with the period from 2010 to 2019. At the nadir, admissions in April 2020 were reduced 45.4% compared with prior years (23 798 in April 2020 compared with a median [interquartile range] of 43 550 [42 110-43 946] in April 2010-2019). Inflation-adjusted hospital charges decreased 27.7% in the second quarter of 2020 compared with prior years ($4 327 580 511 in 2020 compared with a median [interquartile range] of $5 983 142 102 [$5 762 690 022-$6 324 978 456] in 2010-2019). Seasonal patterns were evident between 2010 and 2019 for a variety of common pediatric conditions, including asthma, atrial septal defects, bronchiolitis, diabetic ketoacidosis, Kawasaki syndrome, mental health admissions, and trauma. Ensemble models were able to discern seasonal patterns in admission diagnoses and accurately predicted admission rates from July 2019 until December 2019 but not from January 2020 to June 2020. All diagnoses except for birth decreased below the model 95% CIs between January 2020 and June 2020.

CONCLUSIONS AND RELEVANCE

In this cross-sectional study, pediatric admissions to US hospitals decreased in 2020 across an array of pediatric conditions. Although some conditions may have decreased in incidence, others may represent unmet needs in pediatric care during the COVID-19 pandemic.