Clinical Decision Support in the Era of Artificial Intelligence

Challenges and opportunities in the application of artificial intelligence in gastroenterology and hepatology

Artificial intelligence (AI) is an umbrella term used to describe a cluster of interrelated fields. Machine learning (ML) refers to a model that learns from past data to predict future data. Medicine and particularly gastroenterology and hepatology, are data-rich fields with extensive data repositories, and therefore fruitful ground for AI/ML-based software applications. In this study, we comprehensively review the current applications of AI/ML-based models in these fields and the opportunities that arise from their application. Specifically, we refer to the applications of AI/ML-based models in prevention, diagnosis, management, and prognosis of gastrointestinal bleeding, inflammatory bowel diseases, gastrointestinal premalignant and malignant lesions, other nonmalignant gastrointestinal lesions and diseases, hepatitis B and C infection, chronic liver diseases, hepatocellular carcinoma, cholangiocarcinoma, and primary sclerosing cholangitis. At the same time, we identify the major challenges that restrain the widespread use of these models in healthcare in an effort to explore ways to overcome them. Notably, we elaborate on the concerns regarding intrinsic biases, data protection, cybersecurity, intellectual property, liability, ethical challenges, and transparency. Even at a slower pace than anticipated, AI is infiltrating the healthcare industry. AI in healthcare will become a reality, and every physician will have to engage with it by necessity.

Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring

A new development in the practice of medicine is Artificial Intelligence-based predictive analytics that forewarn clinicians of future deterioration of their patients. This proactive opportunity, though, is different from the reactive stance that clinicians traditionally take. Implementing these tools requires new ideas about how to educate clinician users to facilitate trust and adoption and to promote sustained use. Our real-world hospital experience implementing a predictive analytics monitoring system that uses electronic health record and continuous monitoring data has taught us principles that we believe to be applicable to the implementation of other such analytics systems within the health care environment. These principles are mentioned below:• To promote trust, the science must be understandable.• To enhance uptake, the workflow should not be impacted greatly.• To maximize buy-in, engagement at all levels is important.• To ensure relevance, the education must be tailored to the clinical role and hospital culture.• To lead to clinical action, the information must integrate into clinical care.• To promote sustainability, there should be periodic support interactions after formal implementation.

Le phénotypage digital pour une pratique clinique en santé mentale mieux informée

Objectifs Cette revue trouve sa motivation dans l’observation que la prise de décision clinique en santé mentale est limitée par la nature des mesures typiquement obtenues lors de l’entretien clinique et la difficulté des cliniciens à produire des prédictions justes sur les états mentaux futurs des patients. L’objectif est de présenter un survol représentatif du potentiel du phénotypage digital couplé à l’apprentissage automatique pour répondre à cette limitation, tout en en soulignant les faiblesses actuelles.

Méthode Au travers d’une revue narrative de la littérature non systématique, nous identifions les avancées technologiques qui permettent de quantifier, instant après instant et dans le milieu de vie naturel, le phénotype humain au moyen du téléphone intelligent dans diverses populations psychiatriques. Des travaux pertinents sont également sélectionnés afin de déterminer l’utilité et les limitations de l’apprentissage automatique pour guider les prédictions et la prise de décision clinique. Finalement, la littérature est explorée pour évaluer les barrières actuelles à l’adoption de tels outils.

Résultats Bien qu’émergeant d’un champ de recherche récent, de très nombreux travaux soulignent déjà la valeur des mesures extraites des senseurs du téléphone intelligent pour caractériser le phénotype humain dans les sphères comportementale, cognitive, émotionnelle et sociale, toutes étant affectées par les troubles mentaux. L’apprentissage automatique permet d’utiles et justes prédictions cliniques basées sur ces mesures, mais souffre d’un manque d’interprétabilité qui freinera son emploi prochain dans la pratique clinique. Du reste, plusieurs barrières identifiées tant du côté du patient que du clinicien freinent actuellement l’adoption de ce type d’outils de suivi et d’aide à la décision clinique.

Conclusion Le phénotypage digital couplé à l’apprentissage automatique apparaît fort prometteur pour améliorer la pratique clinique en santé mentale. La jeunesse de ces nouveaux outils technologiques requiert cependant un nécessaire processus de maturation qui devra être encadré par les différents acteurs concernés pour que ces promesses puissent être pleinement réalisées.

Factors Related to Serious Safety Events in a Children's Hospital Patient Safety Collaborative

BACKGROUND AND OBJECTIVES

Serious safety events (SSEs) occur infrequently at individual hospitals, making it difficult to establish trends to improve patient care. Patient safety organizations, such as the Child Health Patient Safety Organization (CHILDPSO), can identify trends and support learning across children's hospitals. We aim to describe longitudinal trends in SSE rates among CHILDPSO member hospitals and describe their sources of harm.

METHODS

SSEs from 44 children's hospitals were assigned severity and reported to CHILDPSO from January 1, 2015, to December 31, 2018. SSEs were classified into groups and subgroups based on analysis. Events were then tagged with up to 3 contributing factors. Subgroups with <5 events were excluded.

RESULTS

There were 22.5 million adjusted patient days included. The 12-month rolling average SSE rate per 10 000 adjusted patient days decreased from 0.71 to 0.41 (P < .001). There were 830 SSEs reported to CHILDPSO. The median hospital volume of SSEs was 12 events (interquartile range: 6-23), or ∼3 SSEs per year. Of the 830 events, 21.0% were high severity (SSE 1-3) and approximately two-thirds (67.0%, n = 610) were patient care management events, including subgroups of missed, delayed, or wrong diagnosis or treatment; medication errors; and suboptimal care coordination. The most common contributing factor was lack of situational awareness (17.9%, n = 382), which contributed to 1 in 5 (20%) high-severity SSEs.

CONCLUSIONS

Hospitals sharing SSE data through CHILDPSO have seen a decrease in SSEs. Patient care management was the most frequently seen. Future work should focus on investigation of contributing factors and risk mitigation strategies.

Applications of machine learning to undifferentiated chest pain in the emergency department: A systematic review

Background

Chest pain is amongst the most common reason for presentation to the emergency department (ED). There are many causes of chest pain, and it is important for the emergency physician to quickly and accurately diagnose life threatening causes such as acute myocardial infarction (AMI). Multiple clinical decision tools have been developed to assist clinicians in risk stratifying patients with chest. There is growing recognition that machine learning (ML) will have a significant impact on the practice of medicine in the near future and may assist with diagnosis and risk stratification. This systematic review aims to evaluate how ML has been applied to adults presenting to the ED with undifferentiated chest pain and assess if ML models show improved performance when compared to physicians or current risk stratification techniques.

Methods and findings

We conducted a systematic review of journal articles that applied a ML technique to an adult patient presenting to an emergency department with undifferentiated chest pain. Multiple databases were searched from inception through to November 2020. In total, 3361 articles were screened, and 23 articles were included. We did not conduct a metanalysis due to a high level of heterogeneity between studies in both their methods, and reporting. The most common primary outcomes assessed were diagnosis of acute myocardial infarction (AMI) (12 studies), and prognosis of major adverse cardiovascular event (MACE) (6 studies). There were 14 retrospective studies and 5 prospective studies. Four studies reported the development of a machine learning model retrospectively then tested it prospectively. The most common machine learning methods used were artificial neural networks (14 studies), random forest (6 studies), support vector machine (5 studies), and gradient boosting (2 studies). Multiple studies achieved high accuracy in both the diagnosis of AMI in the ED setting, and in predicting mortality and composite outcomes over various timeframes. ML outperformed existing risk stratification scores in all cases, and physicians in three out of four cases. The majority of studies were single centre, retrospective, and without prospective or external validation. There were only 3 studies that were considered low risk of bias and had low applicability concerns. Two studies reported integrating the ML model into clinical practice.

Conclusions

Research on applications of ML for undifferentiated chest pain in the ED has been ongoing for decades. ML has been reported to outperform emergency physicians and current risk stratification tools to diagnose AMI and predict MACE but has rarely been integrated into practice. Many studies assessing the use of ML in undifferentiated chest pain in the ED have a high risk of bias. It is important that future studies make use of recently developed standardised ML reporting guidelines, register their protocols, and share their datasets and code. Future work is required to assess the impact of ML model implementation on clinical decision making, patient orientated outcomes, and patient and physician acceptability.

Trial registration

International Prospective Register of Systematic Reviews registration number: CRD42020184977.

Comparison of an oncology clinical decision-support system's recommendations with actual treatment decisions

Objective

IBM^(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice.

Methods

This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH’s institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage.

Results

Nearly 60% (187) of 313 treatment pairs for breast, lung, colon, and rectal cancers were identical or equally acceptable, with 70% (219) of WfO therapeutic options identical to, or acceptable alternatives to, BIH therapy. In 30% of cases (94), 1 or both treatment options were rated as unacceptable. Of 32 cases where both WfO and BIH options were acceptable, WfO was preferred in 18 cases and BIH in 14 cases. Colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; stage IV cancers demonstrated the lowest.

Conclusion

This study demonstrates that a system designed in the US to support, rather than replace, cancer-treating clinicians provides therapeutic options which are generally consistent with recommendations from oncologists outside the US.

A Machine Learning Approach in Predicting Mortality Following Emergency General Surgery

BACKGROUND

There is a significant mortality burden associated with emergency general surgery (EGS) procedures. The objective of this study was to develop and validate the use of a machine learning approach to predict mortality following EGS.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program database was queried for patients who underwent EGS between 2012 and 2017. We developed a machine learning algorithm to predict mortality following EGS and compared its performance with existing risk-prediction models of American Society of Anesthesiologists (ASA) classification, American College of Surgeon Surgical Risk Calculator (ACS-SRC), and the modified frailty index (mFI) using the area under receiver operative curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

The machine learning algorithm had a very high performance for predicting mortality following EGS, and it had superior performance compared to the ASA classification, ACS-SRC, and the mFI, as measured by the AUC, sensitivity, specificity, PPV, and NPV.

DISCUSSION

Machine learning approaches may be a promising tool to predict outcomes for EGS, aiding clinicians in surgical decision-making and counseling of patients and family, improving clinical outcomes by identifying modifiable risk factors than can be optimized, and decreasing treatment costs through resource allocation.

Effect of Interventions With a Clinical Decision Support System for Hospitalized Older Patients: Systematic Review Mapping Implementation and Design Factors

Background

Clinical decision support systems (CDSSs) form an implementation strategy that can facilitate and support health care professionals in the care of older hospitalized patients.

Objective

Our study aims to systematically review the effects of CDSS interventions in older hospitalized patients. As a secondary aim, we aim to summarize the implementation and design factors described in effective and ineffective interventions and identify gaps in the current literature.

Methods

We conducted a systematic review with a search strategy combining the categories older patients, geriatric topic, hospital, CDSS, and intervention in the databases MEDLINE, Embase, and SCOPUS. We included controlled studies, extracted data of all reported outcomes, and potentially beneficial design and implementation factors. We structured these factors using the Grol and Wensing Implementation of Change model, the GUIDES (Guideline Implementation with Decision Support) checklist, and the two-stream model. The risk of bias of the included studies was assessed using the Cochrane Collaboration’s Effective Practice and Organisation of Care risk of bias approach.

Results

Our systematic review included 18 interventions, of which 13 (72%) were effective in improving care. Among these interventions, 8 (6 effective) focused on medication review, 8 (6 effective) on delirium, 7 (4 effective) on falls, 5 (4 effective) on functional decline, 4 (3 effective) on discharge or aftercare, and 2 (0 effective) on pressure ulcers. In 77% (10/13) effective interventions, the effect was based on process-related outcomes, in 15% (2/13) interventions on both process- and patient-related outcomes, and in 8% (1/13) interventions on patient-related outcomes. The following implementation and design factors were potentially associated with effectiveness: a priori problem or performance analyses (described in 9/13, 69% effective vs 0/5, 0% ineffective interventions), multifaceted interventions (8/13, 62% vs 1/5, 20%), and consideration of the workflow (9/13, 69% vs 1/5, 20%).

Conclusions

CDSS interventions can improve the hospital care of older patients, mostly on process-related outcomes. We identified 2 implementation factors and 1 design factor that were reported more frequently in articles on effective interventions. More studies with strong designs are needed to measure the effect of CDSS on relevant patient-related outcomes, investigate personalized (data-driven) interventions, and quantify the impact of implementation and design factors on CDSS effectiveness.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews): CRD42019124470; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=124470.

Predicting survival in heart failure: a risk score based on machine-learning and change point algorithm

OBJECTIVE

Machine learning (ML) algorithm can improve risk prediction because ML can select features and segment continuous variables effectively unbiased. We generated a risk score model for mortality with ML algorithms in East-Asian patients with heart failure (HF).

METHODS

From the Korean Acute Heart Failure (KorAHF) registry, we used the data of 3683 patients with 27 continuous and 44 categorical variables. Grouped Lasso algorithm was used for the feature selection, and a novel continuous variable segmentation algorithm which is based on change-point analysis was developed for effectively segmenting the ranges of the continuous variables. Then, a risk score was assigned to each feature reflecting nonlinear relationship between features and survival times, and an integer score of maximum 100 was calculated for each patient.

RESULTS

During 3-year follow-up time, 32.8% patients died. Using grouped Lasso, we identified 15 highly significant independent clinical features. The calculated risk score of each patient ranged between 1 and 71 points with a median of 36 (interquartile range: 27-45). The 3-year survival differed according to the quintiles of the risk score, being 80% and 17% in the 1st and 5th quintile, respectively. In addition, ML risk score had higher AUCs than MAGGIC-HF score to predict 1-year mortality (0.751 vs. 0.711, P < 0.001).

CONCLUSIONS

In East-Asian patients with HF, a novel risk score model based on ML and the new continuous variable segmentation algorithm performs better for mortality prediction than conventional prediction models.

CLINICAL TRIAL REGISTRATION

Unique identifier: INCT01389843 https://clinicaltrials.gov/ct2/show/NCT01389843 .

Musculoskeletal ultrasound in hemophilia: Results and recommendations from a global survey and consensus meeting

Introduction

For persons with hemophilia, optimization of joint outcomes is an important unmet need. The aim of this initiative was to determine use of ultrasound in evaluating arthropathy in persons with hemophilia, and to move toward consensus among hemophilia care providers regarding the preferred ultrasound protocols for global adaptation.

Methods

A global survey of hemophilia treatment centers was conducted that focused on understanding how and why ultrasound was being used and endeavored to move toward consensus definitions of both point‐of‐care musculoskeletal ultrasound (POC‐MSKUS) and full diagnostic ultrasound, terminology to describe structures being assessed by ultrasound, and how these assessments should be interpreted. Next, an in‐person meeting of an international group of hemophilia health care professionals and patient representatives was held, with the objective of achieving consensus regarding the acquisition and interpretation of POC‐MSKUS and full diagnostic ultrasound for use in the assessment of musculoskeletal (MSK) pathologies in persons with hemophilia.

Results

The recommendations were that clear definitions of the types of ultrasound examinations should be adopted and that a standardized ultrasound scoring/measurement system should be developed, tested, and implemented. The scoring/measurement system should be tiered to allow for a range of complexity yet maintain the ability for comparison across levels.

Conclusion

Ultrasound is an evolving technology increasingly used for the assessment of MSK outcomes in persons with hemophilia. As adoption increases globally for clinical care and research, it will become increasingly important to establish clear guidelines for image acquisition, interpretation, and reporting to ensure accuracy, consistency, and comparability across groups.

Development and Validation of Image-Based Deep Learning Models to Predict Surgical Complexity and Complications in Abdominal Wall Reconstruction

Importance

Image-based deep learning models (DLMs) have been used in other disciplines, but this method has yet to be used to predict surgical outcomes.

Objective

To apply image-based deep learning to predict complexity, defined as need for component separation, and pulmonary and wound complications after abdominal wall reconstruction (AWR).

Design, Setting, and Participants

This quality improvement study was performed at an 874-bed hospital and tertiary hernia referral center from September 2019 to January 2020. A prospective database was queried for patients with ventral hernias who underwent open AWR by experienced surgeons and had preoperative computed tomography images containing the entire hernia defect. An 8-layer convolutional neural network was generated to analyze image characteristics. Images were batched into training (approximately 80%) or test sets (approximately 20%) to analyze model output. Test sets were blinded from the convolutional neural network until training was completed. For the surgical complexity model, a separate validation set of computed tomography images was evaluated by a blinded panel of 6 expert AWR surgeons and the surgical complexity DLM. Analysis started February 2020.

Exposures

Image-based DLM.

Main Outcomes and Measures

The primary outcome was model performance as measured by area under the curve in the receiver operating curve (ROC) calculated for each model; accuracy with accompanying sensitivity and specificity were also calculated. Measures were DLM prediction of surgical complexity using need for component separation techniques as a surrogate and prediction of postoperative surgical site infection and pulmonary failure. The DLM for predicting surgical complexity was compared against the prediction of 6 expert AWR surgeons.

Results

A total of 369 patients and 9303 computed tomography images were used. The mean (SD) age of patients was 57.9 (12.6) years, 232 (62.9%) were female, and 323 (87.5%) were White. The surgical complexity DLM performed well (ROC = 0.744; P < .001) and, when compared with surgeon prediction on the validation set, performed better with an accuracy of 81.3% compared with 65.0% (P < .001). Surgical site infection was predicted successfully with an ROC of 0.898 (P < .001). However, the DLM for predicting pulmonary failure was less effective with an ROC of 0.545 (P = .03).

Conclusions and Relevance

Image-based DLM using routine, preoperative computed tomography images was successful in predicting surgical complexity and more accurate than expert surgeon judgment. An additional DLM accurately predicted the development of surgical site infection.

Design of English hierarchical online test system based on machine learning

Large amount of data are exchanged and the internet is turning into twenty-first century Silk Road for data. Machine learning (ML) is the new area for the applications. The artificial intelligence (AI) is the field providing machines with intelligence. In the last decades, more developments have been made in the field of ML and deep learning. The technology and other advanced algorithms are implemented into more computational constrained devices. The online English test system based on ML breaks the shackles of the traditional paper English test and improves the efficiency of the English test. At the same time, it also maintains the fairness of English test and improves the marking speed. In order to realize an online English test system based on ML and facilitate the assessment of students’ college English courses, this paper mainly adopts relevant research and design on the main functional modules, key technologies, and functional realization of the online English test. The brand-new powerful teaching software and the online examination system can help schools to conduct more systematic and scientific management. The conclusion shows that as brand-new and powerful teaching software, the online examination system can help schools to conduct more systematic and scientific management.

Predicting ventilator-associated pneumonia with machine learning

Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). Existing methods for identifying VAP display low accuracy, and their use may delay antimicrobial therapy. VAP diagnostics derived from machine learning (ML) methods that utilize electronic health record (EHR) data have not yet been explored. The objective of this study is to compare the performance of a variety of ML models trained to predict whether VAP will be diagnosed during the patient stay.A retrospective study examined data from 6126 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of a diagnostic code for VAP. Five different ML models were trained to predict VAP 48 hours after initiation of mechanical ventilation. Model performance was evaluated with regard to the area under the receiver operating characteristic (AUROC) curve on a 20% hold-out test set. Feature importance was measured in terms of Shapley values.The highest performing model achieved an AUROC value of 0.854. The most important features for the best-performing model were the length of time on mechanical ventilation, the presence of antibiotics, sputum test frequency, and the most recent Glasgow Coma Scale assessment.Supervised ML using patient EHR data is promising for VAP diagnosis and warrants further validation. This tool has the potential to aid the timely diagnosis of VAP.

How to replace a physiotherapist: artificial intelligence and the redistribution of expertise

The convergence of large datasets, increased computational power, and enhanced algorithm design has led to the increased success of machine learning (ML) and artificial intelligence (AI) across a wide variety of healthcare professions but which, so far, have eluded formal discussion in physiotherapy. This is a concern as we begin to see accelerating performance improvements in AI research in general, and specifically, an increase in competence within narrow domains of practice in clinical AI. In this paper we argue that the introduction of AI-based systems within the health sector is likely to have a significant influence on physiotherapy practice, leading to the automation of tasks that we might consider to be core to the discipline. We present examples of some of these AI-based systems in clinical practice, specifically video analysis, natural language processing (NLP), robotics, personalized healthcare, expert systems, and prediction algorithms. We address some of the key ethical implications of these emerging technologies, discuss the implications for physiotherapists, and explore how the resultant changes may challenge some long-held assumptions about the status of the profession in society.

Artificial intelligence in early drug discovery enabling precision medicine

Introduction: Precision medicine is the concept of treating diseases based on environmental factors, lifestyles, and molecular profiles of patients. This approach has been found to increase success rates of clinical trials and accelerate drug approvals. However, current precision medicine applications in early drug discovery use only a handful of molecular biomarkers to make decisions, whilst clinics gear up to capture the full molecular landscape of patients in the near future. This deep multi-omics characterization demands new analysis strategies to identify appropriate treatment regimens, which we envision will be pioneered by artificial intelligence.Areas covered: In this review, the authors discuss the current state of drug discovery in precision medicine and present our vision of how artificial intelligence will impact biomarker discovery and drug design.Expert opinion: Precision medicine is expected to revolutionize modern medicine; however, its traditional form is focusing on a few biomarkers, thus not equipped to leverage the full power of molecular landscapes. For learning how the development of drugs can be tailored to the heterogeneity of patients across their molecular profiles, artificial intelligence algorithms are the next frontier in precision medicine and will enable a fully personalized approach in drug design, and thus ultimately impacting clinical practice.

Mobile application as a complementary tool for differential diagnosis in Neuro-ophthalmology: A multicenter cross-sectional study

Purpose:

Drawing differential diagnoses to a Neuro-ophthalmology clinical scenario is a difficult task for a neurology trainee. The authors conducted a study to determine if a mobile application specialized in suggesting differential diagnoses from clinical scenarios can complement clinical reasoning of a neurologist in training.

Methods:

A cross-sectional multicenter study was conducted to compare the accuracy of neurology residents versus a mobile medical app (Neurology Dx) in drawing a comprehensive list of differential diagnoses from Neuro-ophthalmology clinical vignettes. The differentials generated by residents and the App were compared with the Gold standard differential diagnoses adjudicated by experts. The prespecified primary outcome was the proportion of correctly identified high likely gold standard differential diagnosis by residents and App.

Results:

Neurology residents (n = 100) attempted 1500 Neuro-ophthalmology clinical vignettes. Frequency of correctly identified high likely differential diagnosis by residents was 19.42% versus 53.71% by the App (P < 0.0001). The first listed differential diagnosis by the residents matched with that of the first differential diagnosis adjudicated by experts (gold standard differential diagnosis) with a frequency of 26.5% versus 28.3% by the App, whereas the combined output of residents and App scored a frequency of 41.2% in identifying the first gold standard differential correctly. The residents correctly identified the first three and first five gold standard differential diagnosis with a frequency of 17.83% and 19.2%, respectively, as against 22.26% and 30.39% (P < 0.0001) by the App.

Conclusion:

A ruled based app in Neuro-ophthalmology has the potential to complement a neurology resident in drawing a comprehensive list of differential diagnoses.

Safe opioid prescribing: a prognostic machine learning approach to predicting 30-day risk after an opioid dispensation in Alberta, Canada

OBJECTIVE

To develop machine learning models employing administrative health data that can estimate risk of adverse outcomes within 30 days of an opioid dispensation for use by health departments or prescription monitoring programmes.

DESIGN, SETTING AND PARTICIPANTS

This prognostic study was conducted in Alberta, Canada between 2017 and 2018. Participants included all patients 18 years of age and older who received at least one opioid dispensation. Pregnant and cancer patients were excluded.

EXPOSURE

Each opioid dispensation served as an exposure.

MAIN OUTCOMES/MEASURES

Opioid-related adverse outcomes were identified from linked administrative health data. Machine learning algorithms were trained using 2017 data to predict risk of hospitalisation, emergency department visit and mortality within 30 days of an opioid dispensation. Two validation sets, using 2017 and 2018 data, were used to evaluate model performance. Model discrimination and calibration performance were assessed for all patients and those at higher risk. Machine learning discrimination was compared with current opioid guidelines.

RESULTS

Participants in the 2017 training set (n=275 150) and validation set (n=117 829) had similar baseline characteristics. In the 2017 validation set, c-statistics for the XGBoost, logistic regression and neural network classifiers were 0.87, 0.87 and 0.80, respectively. In the 2018 validation set (n=393 023), the corresponding c-statistics were 0.88, 0.88 and 0.82. C-statistics from the Canadian guidelines ranged from 0.54 to 0.69 while the US guidelines ranged from 0.50 to 0.62. The top five percentile of predicted risk for the XGBoost and logistic regression classifiers captured 42% of all events and translated into post-test probabilities of 13.38% and 13.45%, respectively, up from the pretest probability of 1.6%.

CONCLUSION

Machine learning classifiers, especially incorporating hospitalisation/physician claims data, have better predictive performance compared with guideline or prescription history only approaches when predicting 30-day risk of adverse outcomes. Prescription monitoring programmes and health departments with access to administrative data can use machine learning classifiers to effectively identify those at higher risk compared with current guideline-based approaches.

Artificial intelligence in nursing: Priorities and opportunities from an international invitational think-tank of the Nursing and Artificial Intelligence Leadership Collaborative

Aim

To develop a consensus paper on the central points of an international invitational think‐tank on nursing and artificial intelligence (AI).

Methods

We established the Nursing and Artificial Intelligence Leadership (NAIL) Collaborative, comprising interdisciplinary experts in AI development, biomedical ethics, AI in primary care, AI legal aspects, philosophy of AI in health, nursing practice, implementation science, leaders in health informatics practice and international health informatics groups, a representative of patients and the public, and the Chair of the ITU/WHO Focus Group on Artificial Intelligence for Health. The NAIL Collaborative convened at a 3‐day invitational think tank in autumn 2019. Activities included a pre‐event survey, expert presentations and working sessions to identify priority areas for action, opportunities and recommendations to address these. In this paper, we summarize the key discussion points and notes from the aforementioned activities.

Implications for nursing

Nursing's limited current engagement with discourses on AI and health posts a risk that the profession is not part of the conversations that have potentially significant impacts on nursing practice.

Conclusion

There are numerous gaps and a timely need for the nursing profession to be among the leaders and drivers of conversations around AI in health systems.

Impact

We outline crucial gaps where focused effort is required for nursing to take a leadership role in shaping AI use in health systems. Three priorities were identified that need to be addressed in the near future: (a) Nurses must understand the relationship between the data they collect and AI technologies they use; (b) Nurses need to be meaningfully involved in all stages of AI: from development to implementation; and (c) There is a substantial untapped and an unexplored potential for nursing to contribute to the development of AI technologies for global health and humanitarian efforts.

Discovery and Evaluation of Protein Biomarkers as a Signature of Wellness in Late-Stage Cancer Patients in Early Phase Clinical Trials

TARGET (tumour characterisation to guide experimental targeted therapy) is a cancer precision medicine programme focused on molecular characterisation of patients entering early phase clinical trials. Performance status (PS) measures a patient's ability to perform a variety of activities. However, the quality of present algorithms to assess PS is limited and based on qualitative clinician assessment. Plasma samples from patients enrolled into TARGET were analysed using the mass spectrometry (MS) technique: sequential window acquisition of all theoretical fragment ion spectra (SWATH)-MS. SWATH-MS was used on a discovery cohort of 55 patients to differentiate patients into either a good or poor prognosis by creation of a Wellness Score (WS) that showed stronger prediction of overall survival (p = 0.000551) compared to PS (p = 0.001). WS was then tested against a validation cohort of 77 patients showing significant (p = 0.000451) prediction of overall survival. WS in both sets had receiver operating characteristic curve area under the curve (AUC) values of 0.76 (p = 0.002) and 0.67 (p = 0.011): AUC of PS was 0.70 (p = 0.117) and 0.55 (p = 0.548). These signatures can now be evaluated further in larger patient populations to assess their utility in a clinical setting.

The Promise of AI in Detection, Diagnosis, and Epidemiology for Combating COVID-19: Beyond the Hype

COVID-19 has created enormous suffering, affecting lives, and causing deaths. The ease with which this type of coronavirus can spread has exposed weaknesses of many healthcare systems around the world. Since its emergence, many governments, research communities, commercial enterprises, and other institutions and stakeholders around the world have been fighting in various ways to curb the spread of the disease. Science and technology have helped in the implementation of policies of many governments that are directed toward mitigating the impacts of the pandemic and in diagnosing and providing care for the disease. Recent technological tools, artificial intelligence (AI) tools in particular, have also been explored to track the spread of the coronavirus, identify patients with high mortality risk and diagnose patients for the disease. In this paper, areas where AI techniques are being used in the detection, diagnosis and epidemiological predictions, forecasting and social control for combating COVID-19 are discussed, highlighting areas of successful applications and underscoring issues that need to be addressed to achieve significant progress in battling COVID-19 and future pandemics. Several AI systems have been developed for diagnosing COVID-19 using medical imaging modalities such as chest CT and X-ray images. These AI systems mainly differ in their choices of the algorithms for image segmentation, classification and disease diagnosis. Other AI-based systems have focused on predicting mortality rate, long-term patient hospitalization and patient outcomes for COVID-19. AI has huge potential in the battle against the COVID-19 pandemic but successful practical deployments of these AI-based tools have so far been limited due to challenges such as limited data accessibility, the need for external evaluation of AI models, the lack of awareness of AI experts of the regulatory landscape governing the deployment of AI tools in healthcare, the need for clinicians and other experts to work with AI experts in a multidisciplinary context and the need to address public concerns over data collection, privacy, and protection. Having a dedicated team with expertise in medical data collection, privacy, access and sharing, using federated learning whereby AI scientists hand over training algorithms to the healthcare institutions to train models locally, and taking full advantage of biomedical data stored in biobanks can alleviate some of problems posed by these challenges. Addressing these challenges will ultimately accelerate the translation of AI research into practical and useful solutions for combating pandemics.

Economic impact on direct healthcare costs of missing opportunities for diagnosing HIV within healthcare settings

BACKGROUND

The economic consequences of a missed opportunity for HIV testing at an earlier stage of infection within a healthcare setting are poorly described.

METHODS

For all newly diagnosed HIV patients followed at the Southern Alberta HIV/AIDS Clinic (SAC), Calgary, Canada, between 1 April 2011 and 1 April 2016, all clinical encounters occurring < 3 years prior to diagnosis within the region were obtained. The direct costs of HIV care after diagnosis to 31 March 2019 were determined from a payers' perspective and reported as mean cost per patient per month (PPPM) in 2019 Canadian dollars (CDN$). Patients with no encounters for 3 years prior to diagnosis were compared with patients with encounters, with special attention to patients with HIV clinical indicator conditions (HCICs).

RESULTS

Of 388 patients, 60% had one or more prior encounter without HIV testing; 14% had been treated for an HCIC. Females, older patients and heterosexuals were more likely to have prior encounters. At diagnosis, patients with previous encounters presented with lower CD4 counts and higher rates of AIDS. The mean PPPM costs for patients with any prior encounter or for an HCIC-based encounter were 16% and 33% higher, respectively, than for patients with no prior encounters. While mean PPPM costs for antiretroviral drugs and outpatient visits were slightly higher, in-patient costs were 10 times higher for people with HIV who had a previous HCIC encounter vs. those with no encounters (CDN$316 vs. $31, respectively).

CONCLUSIONS

Any healthcare visit, especially for an HCIC, represents relatively easy opportunities for HIV testing. Not testing can result in poorer health and higher costs. Targeted clinical testing and novel interventions to correct overlooked testing opportunities within healthcare settings may be an easy way to implement cost savings.

Artificial intelligence in oncology: Path to implementation

In recent years, the field of artificial intelligence (AI) in oncology has grown exponentially. AI solutions have been developed to tackle a variety of cancer‐related challenges. Medical institutions, hospital systems, and technology companies are developing AI tools aimed at supporting clinical decision making, increasing access to cancer care, and improving clinical efficiency while delivering safe, high‐value oncology care. AI in oncology has demonstrated accurate technical performance in image analysis, predictive analytics, and precision oncology delivery. Yet, adoption of AI tools is not widespread, and the impact of AI on patient outcomes remains uncertain. Major barriers for AI implementation in oncology include biased and heterogeneous data, data management and collection burdens, a lack of standardized research reporting, insufficient clinical validation, workflow and user‐design challenges, outdated regulatory and legal frameworks, and dynamic knowledge and data. Concrete actions that major stakeholders can take to overcome barriers to AI implementation in oncology include training and educating the oncology workforce in AI; standardizing data, model validation methods, and legal and safety regulations; funding and conducting future research; and developing, studying, and deploying AI tools through multidisciplinary collaboration.

Digital Twins for Multiple Sclerosis

An individualized innovative disease management is of great importance for people with multiple sclerosis (pwMS) to cope with the complexity of this chronic, multidimensional disease. However, an individual state of the art strategy, with precise adjustment to the patient's characteristics, is still far from being part of the everyday care of pwMS. The development of digital twins could decisively advance the necessary implementation of an individualized innovative management of MS. Through artificial intelligence-based analysis of several disease parameters - including clinical and para-clinical outcomes, multi-omics, biomarkers, patient-related data, information about the patient's life circumstances and plans, and medical procedures - a digital twin paired to the patient's characteristic can be created, enabling healthcare professionals to handle large amounts of patient data. This can contribute to a more personalized and effective care by integrating data from multiple sources in a standardized manner, implementing individualized clinical pathways, supporting physician-patient communication and facilitating a shared decision-making. With a clear display of pre-analyzed patient data on a dashboard, patient participation and individualized clinical decisions as well as the prediction of disease progression and treatment simulation could become possible. In this review, we focus on the advantages, challenges and practical aspects of digital twins in the management of MS. We discuss the use of digital twins for MS as a revolutionary tool to improve diagnosis, monitoring and therapy refining patients' well-being, saving economic costs, and enabling prevention of disease progression. Digital twins will help make precision medicine and patient-centered care a reality in everyday life.

Deep learning detects and visualizes bleeding events in electronic health records

BACKGROUND

Bleeding is associated with a significantly increased morbidity and mortality. Bleeding events are often described in the unstructured text of electronic health records, which makes them difficult to identify by manual inspection.

OBJECTIVES

To develop a deep learning model that detects and visualizes bleeding events in electronic health records.

PATIENTS/METHODS

Three hundred electronic health records with International Classification of Diseases, Tenth Revision diagnosis codes for bleeding or leukemia were extracted. Each sentence in the electronic health record was annotated as positive or negative for bleeding. The annotated sentences were used to develop a deep learning model that detects bleeding at sentence and note level.

RESULTS

On a balanced test set of 1178   sentences, the best-performing deep learning model achieved a sensitivity of 0.90, specificity of 0.90, and negative predictive value of 0.90. On a test set consisting of 700 notes, of which 49 were positive for bleeding, the model achieved a note-level sensitivity of 1.00, specificity of 0.52, and negative predictive value of 1.00. By using a sentence-level model on a note level, the model can explain its predictions by visualizing the exact sentence in a note that contains information regarding bleeding. Moreover, we found that the model performed consistently well across different types of bleedings.

CONCLUSIONS

A deep learning model can be used to detect and visualize bleeding events in the free text of electronic health records. The deep learning model can thus facilitate systematic assessment of bleeding risk, and thereby optimize patient care and safety.

How artificial intelligence can help us 'Choose Wisely'

The overuse of low value medical tests and treatments drives costs and patient harm. Efforts to address overuse, such as Choosing Wisely campaigns, typically rely on passive implementation strategies- a form of low reliability system change. Embedding guidelines into clinical decision support (CDS) software is a higher leverage approach to provide ordering suggestions through an interface embedded within the clinical workflow. Growth in computing power is increasingly enabling artificial intelligence (AI) to augment such decision making tools. This article offers a roadmap of opportunities for AI-enabled CDS to reduce overuse, which are presented according to a patient’s journey of care.

Clinical Decision Support Systems in Otolaryngology-Head and Neck Surgery: A State of the Art Review

OBJECTIVE

To offer practical, evidence-informed knowledge on clinical decision support systems (CDSSs) and their utility in improving care and reducing costs in otolaryngology-head and neck surgery. This primer on CDSSs introduces clinicians to both the capabilities and the limitations of this technology, reviews the literature on current state, and seeks to spur further progress in this area.

DATA SOURCES

PubMed/MEDLINE, Embase, and Web of Science.

REVIEW METHODS

Scoping review of CDSS literature applicable to otolaryngology clinical practice. Investigators identified articles that incorporated knowledge-based computerized CDSSs to aid clinicians in decision making and workflow. Data extraction included level of evidence, Osheroff classification of CDSS intervention type, otolaryngology subspecialty or domain, and impact on provider performance or patient outcomes.

CONCLUSIONS

Of 3191 studies retrieved, 11 articles met formal inclusion criteria. CDSS interventions included guideline or protocols support (n = 8), forms and templates (n = 5), data presentation aids (n = 2), and reactive alerts, reference information, or order sets (all n = 1); 4 studies had multiple interventions. CDSS studies demonstrated effectiveness across diverse domains, including antibiotic stewardship, cancer survivorship, guideline adherence, data capture, cost reduction, and workflow. Implementing CDSSs often involved collaboration with health information technologists.

IMPLICATIONS FOR PRACTICE

While the published literature on CDSSs in otolaryngology is finite, CDSS interventions are proliferating in clinical practice, with roles in preventing medical errors, streamlining workflows, and improving adherence to best practices for head and neck disorders. Clinicians may collaborate with information technologists and health systems scientists to develop, implement, and investigate the impact of CDSSs in otolaryngology.

The role of artificial intelligence in healthcare: a structured literature review

BACKGROUND/INTRODUCTION

Artificial intelligence (AI) in the healthcare sector is receiving attention from researchers and health professionals. Few previous studies have investigated this topic from a multi-disciplinary perspective, including accounting, business and management, decision sciences and health professions.

METHODS

The structured literature review with its reliable and replicable research protocol allowed the researchers to extract 288 peer-reviewed papers from Scopus. The authors used qualitative and quantitative variables to analyse authors, journals, keywords, and collaboration networks among researchers. Additionally, the paper benefited from the Bibliometrix R software package.

RESULTS

The investigation showed that the literature in this field is emerging. It focuses on health services management, predictive medicine, patient data and diagnostics, and clinical decision-making. The United States, China, and the United Kingdom contributed the highest number of studies. Keyword analysis revealed that AI can support physicians in making a diagnosis, predicting the spread of diseases and customising treatment paths.

CONCLUSIONS

The literature reveals several AI applications for health services and a stream of research that has not fully been covered. For instance, AI projects require skills and data quality awareness for data-intensive analysis and knowledge-based management. Insights can help researchers and health professionals understand and address future research on AI in the healthcare field.

Artificial intelligence with kidney disease: A scoping review with bibliometric analysis, PRISMA-ScR

BACKGROUND

Artificial intelligence (AI) has had a significant impact on our lives and plays many roles in various fields. By analyzing the past 30 years of AI trends in the field of nephrology, using a bibliography, we wanted to know the areas of interest and future direction of AI in research related to the kidney.

METHODS

Using the Institute for Scientific Information Web of Knowledge database, we searched for articles published from 1990 to 2019 in January 2020 using the keywords AI; deep learning; machine learning; and kidney (or renal). The selected articles were reviewed manually at the points of citation analysis.

RESULTS

From 218 related articles, we selected the top fifty with 1188 citations in total. The most-cited article was cited 84 times and the least-cited one was cited 12 times. These articles were published in 40 journals. Expert Systems with Applications (three articles) and Kidney International (three articles) were the most cited journals. Forty articles were published in the 2010s, and seven articles were published in the 2000s. The top-fifty most cited articles originated from 17 countries; the USA contributed 16 articles, followed by Turkey with four articles. The main topics in the top fifty consisted of tumors (11), acute kidney injury (10), dialysis-related (5), kidney-transplant related (4), nephrotoxicity (4), glomerular disease (4), chronic kidney disease (3), polycystic kidney disease (2), kidney stone (2), kidney image (2), renal pathology (2), and glomerular filtration rate measure (1).

CONCLUSIONS

After 2010, the interest in AI and its achievements increased enormously. To date, AIs have been investigated using data that are relatively easy to access, for example, radiologic images and laboratory results in the fields of tumor and acute kidney injury. In the near future, a deeper and wider range of information, such as genetic and personalized database, will help enrich nephrology fields with AI technology.

Identification of variation in nutritional practice in neonatal units in England and association with clinical outcomes using agnostic machine learning

We used agnostic, unsupervised machine learning to cluster a large clinical database of information on infants admitted to neonatal units in England. Our aim was to obtain insights into nutritional practice, an area of central importance in newborn care, utilising the UK National Neonatal Research Database (NNRD). We performed clustering on time-series data of daily nutritional intakes for very preterm infants born at a gestational age less than 32 weeks (n = 45,679) over a six-year period. This revealed 46 nutritional clusters heterogeneous in size, showing common interpretable clinical practices alongside rarer approaches. Nutritional clusters with similar admission profiles revealed associations between nutritional practice, geographical location and outcomes. We show how nutritional subgroups may be regarded as distinct interventions and tested for associations with measurable outcomes. We illustrate the potential for identifying relationships between nutritional practice and outcomes with two examples, discharge weight and bronchopulmonary dysplasia (BPD). We identify the well-known effect of formula milk on greater discharge weight as well as support for the plausible, but insufficiently evidenced view that human milk is protective against BPD. Our framework highlights the potential of agnostic machine learning approaches to deliver clinical practice insights and generate hypotheses using routine data.

Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI

The use of black box algorithms in medicine has raised scholarly concerns due to their opaqueness and lack of trustworthiness. Concerns about potential bias, accountability and responsibility, patient autonomy and compromised trust transpire with black box algorithms. These worries connect epistemic concerns with normative issues. In this paper, we outline that black box algorithms are less problematic for epistemic reasons than many scholars seem to believe. By outlining that more transparency in algorithms is not always necessary, and by explaining that computational processes are indeed methodologically opaque to humans, we argue that the reliability of algorithms provides reasons for trusting the outcomes of medical artificial intelligence (AI). To this end, we explain how computational reliabilism, which does not require transparency and supports the reliability of algorithms, justifies the belief that results of medical AI are to be trusted. We also argue that several ethical concerns remain with black box algorithms, even when the results are trustworthy. Having justified knowledge from reliable indicators is, therefore, necessary but not sufficient for normatively justifying physicians to act. This means that deliberation about the results of reliable algorithms is required to find out what is a desirable action. Thus understood, we argue that such challenges should not dismiss the use of black box algorithms altogether but should inform the way in which these algorithms are designed and implemented. When physicians are trained to acquire the necessary skills and expertise, and collaborate with medical informatics and data scientists, black box algorithms can contribute to improving medical care.

Challenges and Opportunities in Expanding Home Hemodialysis for 2025

The Advancing American Kidney Health Initiative has set an aggressive target for home dialysis growth in the United States, and expanding both peritoneal dialysis and home hemodialysis (HHD) will be required. While there has been a growth in HHD across the United States in the last decade, its value in controlling specific risk factors has been underappreciated and as such its appropriate utilization has lagged. Repositioning how nephrologists incorporate HHD as a critical renal replacement therapy will require overcoming a number of barriers. Advancing education of both nephrology trainees and nephrologists in practice, along with increasing patient and family education on the benefits and requirements for HHD, is essential. Implementation of a transitional care unit design coupled with an intensive patient curriculum will increase patient awareness and comfort for HHD; patients on peritoneal dialysis reaching a modality transition point will benefit from Experience the Difference programs acclimating them to HHD. In addition, the potential link between HHD program size and patient outcomes will necessitate an increase in the size of the average HHD program to more consistently deliver quality dialysis results. Addressing the implications of the nursing shortage and need for designing in scope staffing models are necessary to safeguard HHD growth. Seemingly, certain government payment policy changes and physician documentation requirements deserve further examination. Future HHD innovations must result in decreasing the burden of care for HHD patients, optimize the level of device and biometric data flow, facilitate a more functional centralized patient management care approach, and leverage computerized clinical decision support for modality assignment.

Clinician involvement in research on machine learning-based predictive clinical decision support for the hospital setting: A scoping review

OBJECTIVE

The study sought to describe the prevalence and nature of clinical expert involvement in the development, evaluation, and implementation of clinical decision support systems (CDSSs) that utilize machine learning to analyze electronic health record data to assist nurses and physicians in prognostic and treatment decision making (ie, predictive CDSSs) in the hospital.

MATERIALS AND METHODS

A systematic search of PubMed, CINAHL, and IEEE Xplore and hand-searching of relevant conference proceedings were conducted to identify eligible articles. Empirical studies of predictive CDSSs using electronic health record data for nurses or physicians in the hospital setting published in the last 5 years in peer-reviewed journals or conference proceedings were eligible for synthesis. Data from eligible studies regarding clinician involvement, stage in system design, predictive CDSS intention, and target clinician were charted and summarized.

RESULTS

Eighty studies met eligibility criteria. Clinical expert involvement was most prevalent at the beginning and late stages of system design. Most articles (95%) described developing and evaluating machine learning models, 28% of which described involving clinical experts, with nearly half functioning to verify the clinical correctness or relevance of the model (47%).

DISCUSSION

Involvement of clinical experts in predictive CDSS design should be explicitly reported in publications and evaluated for the potential to overcome predictive CDSS adoption challenges.

CONCLUSIONS

If present, clinical expert involvement is most prevalent when predictive CDSS specifications are made or when system implementations are evaluated. However, clinical experts are less prevalent in developmental stages to verify clinical correctness, select model features, preprocess data, or serve as a gold standard.

DeepAISE - An interpretable and recurrent neural survival model for early prediction of sepsis

Sepsis, a dysregulated immune system response to infection, is among the leading causes of morbidity, mortality, and cost overruns in the Intensive Care Unit (ICU). Early prediction of sepsis can improve situational awareness among clinicians and facilitate timely, protective interventions. While the application of predictive analytics in ICU patients has shown early promising results, much of the work has been encumbered by high false-alarm rates and lack of trust by the end-users due to the 'black box' nature of these models. Here, we present DeepAISE (Deep Artificial Intelligence Sepsis Expert), a recurrent neural survival model for the early prediction of sepsis. DeepAISE automatically learns predictive features related to higher-order interactions and temporal patterns among clinical risk factors that maximize the data likelihood of observed time to septic events. A comparative study of four baseline models on data from hospitalized patients at three different healthcare systems indicates that DeepAISE produces the most accurate predictions (AUCs between 0.87 and 0.90) at the lowest false alarm rates (FARs between 0.20 and 0.25) while simultaneously producing interpretable representations of the clinical time series and risk factors.

The Importance of Incorporating Human Factors in the Design and Implementation of Artificial Intelligence for Skin Cancer Diagnosis in the Real World

Artificial intelligence (AI) algorithms have been shown to diagnose skin lesions with impressive accuracy in experimental settings. The majority of the literature to date has compared AI and dermatologists as opponents in skin cancer diagnosis. However, in the real-world clinical setting, the clinician will work in collaboration with AI. Existing evidence regarding the integration of such AI diagnostic tools into clinical practice is limited. Human factors, such as cognitive style, personality, experience, preferences, and attitudes may influence clinicians' use of AI. In this review, we consider these human factors and the potential cognitive errors, biases, and unintended consequences that could arise when using an AI skin cancer diagnostic tool in the real world. Integrating this knowledge in the design and implementation of AI technology will assist in ensuring that the end product can be used effectively. Dermatologist leadership in the development of these tools will further improve their clinical relevance and safety.

Clinician checklist for assessing suitability of machine learning applications in healthcare

Machine learning algorithms are being used to screen and diagnose disease, prognosticate and predict therapeutic responses. Hundreds of new algorithms are being developed, but whether they improve clinical decision making and patient outcomes remains uncertain. If clinicians are to use algorithms, they need to be reassured that key issues relating to their validity, utility, feasibility, safety and ethical use have been addressed. We propose a checklist of 10 questions that clinicians can ask of those advocating for the use of a particular algorithm, but which do not expect clinicians, as non-experts, to demonstrate mastery over what can be highly complex statistical and computational concepts. The questions are: (1) What is the purpose and context of the algorithm? (2) How good were the data used to train the algorithm? (3) Were there sufficient data to train the algorithm? (4) How well does the algorithm perform? (5) Is the algorithm transferable to new clinical settings? (6) Are the outputs of the algorithm clinically intelligible? (7) How will this algorithm fit into and complement current workflows? (8) Has use of the algorithm been shown to improve patient care and outcomes? (9) Could the algorithm cause patient harm? and (10) Does use of the algorithm raise ethical, legal or social concerns? We provide examples where an algorithm may raise concerns and apply the checklist to a recent review of diagnostic imaging applications. This checklist aims to assist clinicians in assessing algorithm readiness for routine care and identify situations where further refinement and evaluation is required prior to large-scale use.

Risk factors analysis of COVID-19 patients with ARDS and prediction based on machine learning

COVID-19 is a newly emerging infectious disease, which is generally susceptible to human beings and has caused huge losses to people's health. Acute respiratory distress syndrome (ARDS) is one of the common clinical manifestations of severe COVID-19 and it is also responsible for the current shortage of ventilators worldwide. This study aims to analyze the clinical characteristics of COVID-19 ARDS patients and establish a diagnostic system based on artificial intelligence (AI) method to predict the probability of ARDS in COVID-19 patients. We collected clinical data of 659 COVID-19 patients from 11 regions in China. The clinical characteristics of the ARDS group and no-ARDS group of COVID-19 patients were elaborately compared and both traditional machine learning algorithms and deep learning-based method were used to build the prediction models. Results indicated that the median age of ARDS patients was 56.5 years old, which was significantly older than those with non-ARDS by 7.5 years. Male and patients with BMI > 25 were more likely to develop ARDS. The clinical features of ARDS patients included cough (80.3%), polypnea (59.2%), lung consolidation (53.9%), secondary bacterial infection (30.3%), and comorbidities such as hypertension (48.7%). Abnormal biochemical indicators such as lymphocyte count, CK, NLR, AST, LDH, and CRP were all strongly related to the aggravation of ARDS. Furthermore, through various AI methods for modeling and prediction effect evaluation based on the above risk factors, decision tree achieved the best AUC, accuracy, sensitivity and specificity in identifying the mild patients who were easy to develop ARDS, which undoubtedly helped to deliver proper care and optimize use of limited resources.

A modular approach to integrating multiple data sources into real-time clinical prediction for pediatric diarrhea

Traditional clinical prediction models focus on parameters of the individual patient. For infectious diseases, sources external to the patient, including characteristics of prior patients and seasonal factors, may improve predictive performance. We describe the development of a predictive model that integrates multiple sources of data in a principled statistical framework using a post-test odds formulation. Our method enables electronic real-time updating and flexibility, such that components can be included or excluded according to data availability. We apply this method to the prediction of etiology of pediatric diarrhea, where 'pre-test’ epidemiologic data may be highly informative. Diarrhea has a high burden in low-resource settings, and antibiotics are often over-prescribed. We demonstrate that our integrative method outperforms traditional prediction in accurately identifying cases with a viral etiology, and show that its clinical application, especially when used with an additional diagnostic test, could result in a 61% reduction in inappropriately prescribed antibiotics.

Reform and innovation of artificial intelligence technology for information service in university physical education

This article analyzes the reform of information services in university physical education based on artificial intelligence technology and conducts in-depth and innovative research on it. In-depth analysis of the relationship between big data and the development and application of information technology such as the Internet, Internet of Things, cloud computing, to clarify the difference and connection between big data, informatization and intelligence. Artificial intelligence will bring opportunities for changes in data collection, management decision-making, governance models, education and teaching, scientific research services, evaluation and evaluation of physical education in our university. At the same time, big data education management in colleges and universities faces many challenges such as the balance of privacy and freedom, data hegemony, data junk, data standards, and data security, and they have many negative effects. In accordance with the requirements of educational modernization, centering on the goal of intelligent and humanized education management, it aims existing issues in college physical education management.

Development of a Novel Clinical Decision Support System for Exercise Prescription Among Patients With Multiple Cardiovascular Disease Risk Factors

Cardiovascular disease (CVD) risk factors cluster in an individual. Exercise is universally recommended to prevent and treat CVD. Yet, clinicians lack guidance on how to design an exercise prescription (ExRx) for patients with multiple CVD risk factors. To address this unmet need, we developed a novel clinical decision support system to prescribe exercise (prioritize personalize prescribe exercise [P3-EX]) for patients with multiple CVD risk factors founded upon the evidenced-based recommendations of the American College of Sports Medicine (ACSM) and American Heart Association. To develop P3-EX, we integrated (1) the ACSM exercise preparticipation health screening recommendations; (2) an adapted American Heart Association Life's Simple 7 cardiovascular health scoring system; (3) adapted ACSM strategies for designing an ExRx for people with multiple CVD risk factors; and (4) the ACSM frequency, intensity, time, and time principle of ExRx. We have tested the clinical utility of P3-EX within a university-based online graduate program in ExRx among students that includes physicians, physical therapists, registered dietitians, exercise physiologists, kinesiologists, fitness industry professionals, and kinesiology educators in higher education. The support system P3-EX has proven to be an easy-to-use, guided, and time-efficient evidence-based approach to ExRx for patients with multiple CVD risk factors that has applicability to other chronic diseases and health conditions. Further evaluation is needed to better establish its feasibility, acceptability, and clinical utility as an ExRx tool.

Machine Learning to Predict Cardiac Death Within 1 Hour After Terminal Extubation

OBJECTIVES

Accurate prediction of time to death after withdrawal of life-sustaining therapies may improve counseling for families and help identify candidates for organ donation after cardiac death. The study objectives were to: 1) train a long short-term memory model to predict cardiac death within 1 hour after terminal extubation, 2) calculate the positive predictive value of the model and the number needed to alert among potential organ donors, and 3) examine associations between time to cardiac death and the patient's characteristics and physiologic variables using Cox regression.

DESIGN

Retrospective cohort study.

SETTING

PICU and cardiothoracic ICU in a tertiary-care academic children's hospital.

PATIENTS

Patients 0-21 years old who died after terminal extubation from 2011 to 2018 (n = 237).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The median time to death for the cohort was 0.3 hours after terminal extubation (interquartile range, 0.16-1.6 hr); 70% of patients died within 1 hour. The long short-term memory model had an area under the receiver operating characteristic curve of 0.85 and a positive predictive value of 0.81 at a sensitivity of 94% when predicting death within 1 hour of terminal extubation. About 39% of patients who died within 1 hour met organ procurement and transplantation network criteria for liver and kidney donors. The long short-term memory identified 93% of potential organ donors with a number needed to alert of 1.08, meaning that 13 of 14 prepared operating rooms would have yielded a viable organ. A Cox proportional hazard model identified independent predictors of shorter time to death including low Glasgow Coma Score, high Pao2-to-Fio2 ratio, low-pulse oximetry, and low serum bicarbonate.

CONCLUSIONS

Our long short-term memory model accurately predicted whether a child will die within 1 hour of terminal extubation and may improve counseling for families. Our model can identify potential candidates for donation after cardiac death while minimizing unnecessarily prepared operating rooms.

Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network

BACKGROUND AND OBJECTIVE

Clinical decision support assisted by prediction models usually faces the challenges of limited clinical data and a lack of labels when the model is developed with data from a single medical institution. Accordingly, research on multicenter clinical collaborative networks, which can provide external medical data, has received increasing attention. With the increasing availability of machine learning techniques such as transfer learning, leveraging large-scale patient data from multiple hospitals to build data-driven predictive models with clinical application potential provides an alternative solution to address the problem of limited patient data.

METHODS

A multicenter hybrid semi-supervised transfer learning model (MHSTL) is proposed in this study on the basis of unified common data model to ensure multicenter data standardized representation. Then the hospital-specific features, along with the co-occurrence features across domains, are aligned through a representation learning architecture that is built based on deep neural networks and the newly proposed neural decision forest model. In this process, limited patient data from the target hospital, both labeled and unlabeled, are incorporated during the feature adaptation process, thereby contributing to better model performance. Without patient-level data sharing, the proposed model learning strategy which overcomes feature misalignment and distribution divergence, enables the multi-source transfer learning process in the case of insufficient and unlabeled patient data at target hospital.

RESULTS

The effectiveness of the proposed transfer learning model was evaluated on a collaborative research network of colorectal cancer patients in the US and China. The results demonstrate that the proposed model can achieve much better performance for predicting target risk with limited resources on patient data than baseline models      . Better discrimination and calibration ability are also observed when sufficient labeled data are not available in the target hospital for prognosis prediction tasks      . Further exploratory experiments show that the proposed approach exhibits good model generalizability regardless of the data heterogeneity. With the help of the SHapley Additive exPlanations for model interpretation, the effectiveness of incorporating hospital-specific features in the transfer learning model is shown.

CONCLUSIONS

In this study, the proposed method can develop prediction models from multiple source hospitals and exhibit good performance by leveraging cross-domain hospital-specific feature information, therefore enhancing the model prediction when applied to single medical institution with limited patient data.

Development and validation of a machine learning algorithm for predicting the risk of postpartum depression among pregnant women

OBJECTIVE

There is a scarcity in tools to predict postpartum depression (PPD). We propose a machine learning framework for PPD risk prediction using data extracted from electronic health records (EHRs).

METHODS

Two EHR datasets containing data on 15,197 women from 2015 to 2018 at a single site, and 53,972 women from 2004 to 2017 at multiple sites were used as development and validation sets, respectively, to construct the PPD risk prediction model. The primary outcome was a diagnosis of PPD within 1 year following childbirth. A framework of data extraction, processing, and machine learning was implemented to select a minimal list of features from the EHR datasets to ensure model performance and to enable future point-of-care risk prediction.

RESULTS

The best-performing model uses from clinical features related to mental health history, medical comorbidity, obstetric complications, medication prescription orders, and patient demographic characteristics. The model performances as measured by area under the receiver operating characteristic curve (AUC) are 0.937 (95% CI 0.912 - 0.962) and 0.886 (95% CI 0.879-0.893) in the development and validation datasets, respectively. The model performances were consistent when tested using data ending at multiple time periods during pregnancy and at childbirth.

LIMITATIONS

The prevalence of PPD in the study data represented a treatment prevalence and is likely lower than the illness prevalence.

CONCLUSIONS

EHRs and machine learning offer the ability to identify women at risk for PPD early in their pregnancy. This may facilitate scalable and timely prevention and intervention, reducing negative outcomes and the associated burden.

Study protocol for a prospective, double-blinded, observational study investigating the diagnostic accuracy of an app-based diagnostic health care application in an emergency room setting: the eRadaR trial

INTRODUCTION

Occurrence of inaccurate or delayed diagnoses is a significant concern in patient care, particularly in emergency medicine, where decision making is often constrained by high throughput and inaccurate admission diagnoses. Artificial intelligence-based diagnostic decision support system have been developed to enhance clinical performance by suggesting differential diagnoses to a given case, based on an integrated medical knowledge base and machine learning techniques. The purpose of the study is to evaluate the diagnostic accuracy of Ada, an app-based diagnostic tool and the impact on patient outcome.

METHODS AND ANALYSIS

The eRadaR trial is a prospective, double-blinded study with patients presenting to the emergency room (ER) with abdominal pain. At initial contact in the ER, a structured interview will be performed using the Ada-App and both, patients and attending physicians, will be blinded to the proposed diagnosis lists until trial completion. Throughout the study, clinical data relating to diagnostic findings and types of therapy will be obtained and the follow-up until day 90 will comprise occurrence of complications and overall survival of patients. The primary efficacy of the trial is defined by the percentage of correct diagnoses suggested by Ada compared with the final discharge diagnosis. Further, accuracy and timing of diagnosis will be compared with decision making of classical doctor-patient interaction. Secondary objectives are complications, length of hospital stay and overall survival.

ETHICS AND DISSEMINATION

Ethical approval was received by the independent ethics committee (IEC) of the Goethe-University Frankfurt on 9 April 2020 including the patient information material and informed consent form. All protocol amendments must be reported to and adapted by the IEC. The results from this study will be submitted to peer-reviewed journals and reported at suitable national and international meetings.

TRIAL REGISTRATION NUMBER

DRKS00019098.

Divided We Stand: The Collaborative Work of Patients and Providers in an Enigmatic Chronic Disease

In chronic conditions, patients and providers need support in understanding and managing illness over time. Focusing on endometriosis, an enigmatic chronic condition, we conducted interviews with specialists and focus groups with patients to elicit their work in care specifically pertaining to dealing with an enigmatic disease, both independently and in partnership, and how technology could support these efforts. We found that the work to care for the illness, including reflecting on the illness experience and planning for care, is significantly compounded by the complex nature of the disease: enigmatic condition means uncertainty and frustration in care and management; the multi-factorial and systemic features of endometriosis without any guidance to interpret them overwhelm patients and providers; the different temporal resolutions of this chronic condition confuse both patients and provides; and patients and providers negotiate medical knowledge and expertise in an attempt to align their perspectives. We note how this added complexity demands that patients and providers work together to find common ground and align perspectives, and propose three design opportunities (considerations to construct a holistic picture of the patient, design features to reflect and make sense of the illness, and opportunities and mechanisms to correct misalignments and plan for care) and implications to support patients and providers in their care work. Specifically, the enigmatic nature of endometriosis necessitates complementary approaches from human-centered computing and artificial intelligence, and thus opens a number of future research avenues.

Artificial Intelligence and Clinical Decision Making: The New Nature of Medical Uncertainty

Estimates in a 1989 study indicated that physicians in the United States were unable to reach a diagnosis that accounted for their patient's symptoms in up to 90% of outpatient patient encounters. Many proponents of artificial intelligence (AI) see the current process of moving from clinical data gathering to medical diagnosis as being limited by human analytic capability and expect AI to be a valuable tool to refine this process. The use of AI fundamentally calls into question the extent to which uncertainty in medical decision making is tolerated. Uncertainty is perceived by some as fundamentally undesirable and thus, for them, optimal decision making should be based on minimizing uncertainty. However, uncertainty cannot be reduced to zero; thus, relative uncertainty can be used as a metric to weigh the likelihood of various diagnoses being correct and the appropriateness of treatments. Here, the authors make the argument, using as examples the experiences of 2 AI systems, IBM Watson on Jeopardy and Watson for Oncology, that medical decision making based on relative uncertainty provides a better lens for understanding the application of AI to medicine than one that minimizes uncertainty. This approach to uncertainty has significant implications for how health care leaders consider the benefits and trade-offs of AI-assisted and AI-driven decision tools and ultimately integrate AI into medical practice.