Development and Validation of a Deep Learning Algorithm for Mortality Prediction in Selecting Patients With Dementia for Earlier Palliative Care Interventions

Palliative Care in Early Dementia

BACKGROUND

Palliative care is recommended for all people with dementia from diagnosis through end-of-life. However, palliative care needs and effective elements of palliative care are not well-defined for the earlier stages of dementia.

OBJECTIVE

To systematically map current research on palliative care early in the disease trajectory of dementia.

DESIGN

Scoping review of scientific literature.

DATA SOURCES

PubMed, CINAHL, EMBASE, Cochrane, PsycINFO, Web of Science.

REVIEW METHODS

We included studies published in English over the last decade (through March 2022) that focused on palliative care in early stages of dementia and targeted outcomes in palliative care domains. Two authors independently screened abstracts and full texts and scored the quality of included studies using tools by the Joanna Briggs Institute.

RESULTS

Among the 77 papers reviewed, few addressed early stages of dementia specifically. We found that: 1) While "early" palliative care was not well-defined in the literature, evidence indicated that palliative care needs were present at or before diagnosis and across the trajectory. Notable opportunities for palliative care arise at 'tipping points' (i.e., when symptoms, functional status, or caregiving needs change). 2) Palliative care needs in early dementia include advocacy for goal-aligned care in the future, reassurance against the threat of negligence and abandonment by caregivers, planning for future scenarios of care (practical, individual, and relational needs), and establishing of long-term relationships with providers entrusted for care later in disease. 3) Elements of effective palliative care in early dementia could include dementia-specific ACP and goals of care discussions, navigation for building a network of support, provision of tools and resources for family, tailored care and knowledge of the person, and well-prepared dementia-care providers. The scarcity of palliative care studies aimed at early disease indicates a gap in the evidence in dementia care.

CONCLUSION

The literature on palliative care in early dementia is sparse. Future studies should focus on assessment tools for optimizing timing of palliative care in early dementia, gaining better understanding of patient and family needs during early phases of disease, and providing training for providers and families in long-term relationships and communication around goals of care and future planning.

Artificial intelligence prediction of In-Hospital mortality in patients with dementia: A multi-center study

BACKGROUND

Prediction of mortality is very important for care planning in hospitalized patients with dementia and artificial intelligence has the potential to serve as a solution; however, this issue remains unclear. Thus, this study was conducted to elucidate this matter.

METHODS

We identified 10,573 hospitalized patients aged ≥ 45 years with dementia from three hospitals between 2010 and 2020 for this study. Utilizing 44 feature variables extracted from electronic medical records, an artificial intelligence (AI) model was constructed to predict death during hospitalization. The data was randomly separated into 70 % training set and 30 % testing set. We compared predictive accuracy among six algorithms including logistic regression, random forest, extreme gradient boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), multilayer perceptron (MLP), and support vector machine (SVM). Additionally, another set of data collected in 2021 was used as the validation set to assess the performance of six algorithms.

RESULTS

The average age was 79.8 years, with females constituting 54.5 % of the sample. The in-hospital mortality rate was 6.7 %. LightGBM exhibited the highest area under the curve (0.991) for predicting mortality compared to other algorithms (XGBoost: 0.987, random forest: 0.985, logistic regression: 0.918, MLP: 0.898, SVM: 0.897). The accuracy, sensitivity, positive predictive value, and negative predictive value of LightGBM were 0.943, 0.944, 0.943, 0.542, and 0.996, respectively. Among the features in LightGBM, the three most important variables were the Glasgow Coma Scale, respiratory rate, and blood urea nitrogen. In the validation set, the area under the curve of LightGBM reached 0.753.

CONCLUSIONS

The AI prediction model demonstrates strong accuracy in predicting in-hospital mortality among patients with dementia, suggesting its potential implementation to enhance future care quality.

Prognostic models and factors identifying end-of-life in non-cancer chronic diseases: a systematic review

BACKGROUND

Precise prognostic information, if available, is very helpful for guiding treatment decisions and resource allocation in patients with non-cancer non-communicable chronic diseases (NCDs). This study aimed to systematically review the existing evidence, examining prognostic models and factors for identifying end-of-life non-cancer NCD patients.

METHODS

Electronic databases, including Medline, Embase, CINAHL, Cochrane Library, PsychINFO and other sources, were searched from the inception of these databases up until June 2023. Studies published in English with findings mentioning prognostic models or factors related to identifying end-of-life in non-cancer NCD patients were included. The quality of studies was assessed using the Quality in Prognosis Studies tool.

RESULTS

The analysis included data from 41 studies, with 16 focusing on chronic obstructive pulmonary diseases (COPD), 10 on dementia, 6 on heart failure and 9 on mixed NCDs. Traditional statistical modelling was predominantly used for the identified prognostic models. Common predictors in COPD models included dyspnoea, forced expiratory volume in 1 s, functional status, exacerbation history and body mass index. Models for dementia and heart failure frequently included comorbidity, age, gender, blood tests and nutritional status. Similarly, mixed NCD models commonly included functional status, age, dyspnoea, the presence of skin pressure ulcers, oral intake and level of consciousness. The identified prognostic models exhibited varying predictive accuracy, with the majority demonstrating weak to moderate discriminatory performance (area under the curve: 0.5-0.8). Additionally, most of these models lacked independent external validation, and only a few underwent internal validation.

CONCLUSION

Our review summarised the most relevant predictors for identifying end-of-life in non-cancer NCDs. However, the predictive accuracy of identified models was generally inconsistent and low, and lacked external validation. Although efforts to improve these prognostic models should continue, clinicians should recognise the possibility that disease heterogeneity may limit the utility of these models for individual prognostication; they may be more useful for population level health planning.

Identifying Psychosis Episodes in Psychiatric Admission Notes via Rule-based Methods, Machine Learning, and Pre-Trained Language Models

Early and accurate diagnosis is crucial for effective treatment and improved outcomes, yet identifying psychotic episodes presents significant challenges due to its complex nature and the varied presentation of symptoms among individuals. One of the primary difficulties lies in the underreporting and underdiagnosis of psychosis, compounded by the stigma surrounding mental health and the individuals' often diminished insight into their condition. Existing efforts leveraging Electronic Health Records (EHRs) to retrospectively identify psychosis typically rely on structured data, such as medical codes and patient demographics, which frequently lack essential information. Addressing these challenges, our study leverages Natural Language Processing (NLP) algorithms to analyze psychiatric admission notes for the diagnosis of psychosis, providing a detailed evaluation of rule-based algorithms, machine learning models, and pre-trained language models. Additionally, the study investigates the effectiveness of employing keywords to streamline extensive note data before training and evaluating the models. Analyzing 4,617 initial psychiatric admission notes (1,196 cases of psychosis versus 3,433 controls) from 2005 to 2019, we discovered that the XGBoost classifier employing Term Frequency-Inverse Document Frequency (TF-IDF) features derived from notes pre-selected by expert-curated keywords, attained the highest performance with an F1 score of 0.8881 (AUROC [95% CI]: 0.9725 [0.9717, 0.9733]). BlueBERT demonstrated comparable efficacy an F1 score of 0.8841 (AUROC [95% CI]: 0.97 [0.9580,0.9820]) on the same set of notes. Both models markedly outperformed traditional International Classification of Diseases (ICD) code-based detection methods from discharge summaries, which had an F1 score of 0.7608, thus improving the margin by 0.12. Furthermore, our findings indicate that keyword pre-selection markedly enhances the performance of both machine learning and pre-trained language models. This study illustrates the potential of NLP techniques to improve psychosis detection within admission notes and aims to serve as a foundational reference for future research on applying NLP for psychosis identification in EHR notes.

Identifying Psychosis Episodes in Psychiatric Admission Notes via Rule-based Methods, Machine Learning, and Pre-Trained Language Models

Early and accurate diagnosis is crucial for effective treatment and improved outcomes, yet identifying psychotic episodes presents significant challenges due to its complex nature and the varied presentation of symptoms among individuals. One of the primary difficulties lies in the underreporting and underdiagnosis of psychosis, compounded by the stigma surrounding mental health and the individuals' often diminished insight into their condition. Existing efforts leveraging Electronic Health Records (EHRs) to retrospectively identify psychosis typically rely on structured data, such as medical codes and patient demographics, which frequently lack essential information. Addressing these challenges, our study leverages Natural Language Processing (NLP) algorithms to analyze psychiatric admission notes for the diagnosis of psychosis, providing a detailed evaluation of rule-based algorithms, machine learning models, and pre-trained language models. Additionally, the study investigates the effectiveness of employing keywords to streamline extensive note data before training and evaluating the models. Analyzing 4,617 initial psychiatric admission notes (1,196 cases of psychosis versus 3,433 controls) from 2005 to 2019, we discovered that the XGBoost classifier employing Term Frequency-Inverse Document Frequency (TF-IDF) features derived from notes pre-selected by expert-curated keywords, attained the highest performance with an F1 score of 0.8881 (AUROC [95% CI]: 0.9725 [0.9717, 0.9733]). BlueBERT demonstrated comparable efficacy an F1 score of 0.8841 (AUROC [95% CI]: 0.97 [0.9580, 0.9820]) on the same set of notes. Both models markedly outperformed traditional International Classification of Diseases (ICD) code-based detection methods from discharge summaries, which had an F1 score of 0.7608, thus improving the margin by 0.12. Furthermore, our findings indicate that keyword pre-selection markedly enhances the performance of both machine learning and pre-trained language models. This study illustrates the potential of NLP techniques to improve psychosis detection within admission notes and aims to serve as a foundational reference for future research on applying NLP for psychosis identification in EHR notes.

Deep learning applications in vascular dementia using neuroimaging

PURPOSE OF REVIEW

Vascular dementia (VaD) is the second common cause of dementia after Alzheimer's disease, and deep learning has emerged as a critical tool in dementia research. The aim of this article is to highlight the current deep learning applications in VaD-related imaging biomarkers and diagnosis.

RECENT FINDINGS

The main deep learning technology applied in VaD using neuroimaging data is convolutional neural networks (CNN). CNN models have been widely used for lesion detection and segmentation, such as white matter hyperintensities (WMH), cerebral microbleeds (CMBs), perivascular spaces (PVS), lacunes, cortical superficial siderosis, and brain atrophy. Applications in VaD subtypes classification also showed excellent results. CNN-based deep learning models have potential for further diagnosis and prognosis of VaD.

SUMMARY

Deep learning neural networks with neuroimaging data in VaD research represent significant promise for advancing early diagnosis and treatment strategies. Ongoing research and collaboration between clinicians, data scientists, and neuroimaging experts are essential to address challenges and unlock the full potential of deep learning in VaD diagnosis and management.

Machine learning models identify predictive features of patient mortality across dementia types

BACKGROUND

Dementia care is challenging due to the divergent trajectories in disease progression and outcomes. Predictive models are needed to flag patients at risk of near-term mortality and identify factors contributing to mortality risk across different dementia types.

METHODS

Here, we developed machine-learning models predicting dementia patient mortality at four different survival thresholds using a dataset of 45,275 unique participants and 163,782 visit records from the U.S. National Alzheimer's Coordinating Center (NACC). We built multi-factorial XGBoost models using a small set of mortality predictors and conducted stratified analyses with dementiatype-specific models.

RESULTS

Our models achieved an area under the receiver operating characteristic curve (AUC-ROC) of over 0.82 utilizing nine parsimonious features for all 1-, 3-, 5-, and 10-year thresholds. The trained models mainly consisted of dementia-related predictors such as specific neuropsychological tests and were minimally affected by other age-related causes of death, e.g., stroke and cardiovascular conditions. Notably, stratified analyses revealed shared and distinct predictors of mortality across eight dementia types. Unsupervised clustering of mortality predictors grouped vascular dementia with depression and Lewy body dementia with frontotemporal lobar dementia.

CONCLUSIONS

This study demonstrates the feasibility of flagging dementia patients at risk of mortality for personalized clinical management. Parsimonious machine-learning models can be used to predict dementia patient mortality with a limited set of clinical features, and dementiatype-specific models can be applied to heterogeneous dementia patient populations.

A deep learning approach for transgender and gender diverse patient identification in electronic health records

BACKGROUND

Although accurate identification of gender identity in the electronic health record (EHR) is crucial for providing equitable health care, particularly for transgender and gender diverse (TGD) populations, it remains a challenging task due to incomplete gender information in structured EHR fields.

OBJECTIVE

Using TGD identification as a case study, this research uses NLP and deep learning to build an accurate patient gender identity predictive model, aiming to tackle the challenges of identifying relevant patient-level information from EHR data and reducing annotation work.

METHODS

This study included adult patients in a large healthcare system in Boston, MA, between 4/1/2017 to 4/1/2022. To identify relevant information from massive clinical notes, we compiled a list of gender-related keywords through expert curation, literature review, and expansion via a fine-tuned BioWordVec model. This keyword list was used to pre-screen potential TGD individuals and create two datasets for model training, testing, and validation. Dataset I was a balanced dataset that contained clinician-confirmed TGD patients and cases without keywords. Dataset II contained cases with keywords. The performance of the deep learning model was compared to traditional machine learning and rule-based algorithms.

RESULTS

The final keyword list consists of 109 keywords, of which 58 (53.2%) were expanded by the BioWordVec model. Dataset I contained 3,150 patients (50% TGD) while Dataset II contained 200 patients (90% TGD). On Dataset I the deep learning model achieved a F1 score of 0.917, sensitivity of 0.854, and a precision of 0.980; and on Dataset II a F1 score of 0.969, sensitivity of 0.967, and precision of 0.972. The deep learning model significantly outperformed rule-based algorithms.

CONCLUSION

This is the first study to show that deep learning-integrated NLP algorithms can accurately identify gender identity using EHR data. Future work should leverage and evaluate additional diverse data sources to generate more generalizable algorithms.

A comprehensive review of machine learning algorithms and their application in geriatric medicine: present and future

The increasing access to health data worldwide is driving a resurgence in machine learning research, including data-hungry deep learning algorithms. More computationally efficient algorithms now offer unique opportunities to enhance diagnosis, risk stratification, and individualised approaches to patient management. Such opportunities are particularly relevant for the management of older patients, a group that is characterised by complex multimorbidity patterns and significant interindividual variability in homeostatic capacity, organ function, and response to treatment. Clinical tools that utilise machine learning algorithms to determine the optimal choice of treatment are slowly gaining the necessary approval from governing bodies and being implemented into healthcare, with significant implications for virtually all medical disciplines during the next phase of digital medicine. Beyond obtaining regulatory approval, a crucial element in implementing these tools is the trust and support of the people that use them. In this context, an increased understanding by clinicians of artificial intelligence and machine learning algorithms provides an appreciation of the possible benefits, risks, and uncertainties, and improves the chances for successful adoption. This review provides a broad taxonomy of machine learning algorithms, followed by a more detailed description of each algorithm class, their purpose and capabilities, and examples of their applications, particularly in geriatric medicine. Additional focus is given on the clinical implications and challenges involved in relying on devices with reduced interpretability and the progress made in counteracting the latter via the development of explainable machine learning.

Characterization and prediction of individual functional outcome trajectories in schizophrenia spectrum disorders (PREDICTS study): Study protocol

Schizophrenia spectrum disorders (SSDs) are associated with significant functional impairments, disability, and low rates of personal recovery, along with tremendous economic costs linked primarily to lost productivity and premature mortality. Efforts to delineate the contributors to disability in SSDs have highlighted prominent roles for a diverse range of symptoms, physical health conditions, substance use disorders, neurobiological changes, and social factors. These findings have provided valuable advances in knowledge and helped define broad patterns of illness and outcomes across SSDs. Unsurprisingly, there have also been conflicting findings for many of these determinants that reflect the heterogeneous population of individuals with SSDs and the challenges of conceptualizing and treating SSDs as a unitary categorical construct. Presently it is not possible to identify the functional course on an individual level that would enable a personalized approach to treatment to alter the individual's functional trajectory and mitigate the ensuing disability they would otherwise experience. To address this ongoing challenge, this study aims to conduct a longitudinal multimodal investigation of a large cohort of individuals with SSDs in order to establish discrete trajectories of personal recovery, disability, and community functioning, as well as the antecedents and predictors of these trajectories. This investigation will also provide the foundation for the co-design and testing of personalized interventions that alter these functional trajectories and improve outcomes for people with SSDs.

Prediction of 3-Year Survival in Patients with Cognitive Impairment Based on Demographics, Neuropsychological Data, and Comorbidities: A Prospective Cohort Study

OBJECTIVES

Based on readily available demographic data, neuropsychological assessment results, and comorbidity data, we aimed to develop and validate a 3-year survival prediction model for patients with cognitive impairment.

METHODS

In this prospective cohort study, 616 patients with cognitive impairment were included. Demographic information, data on comorbidities, and scores of the Mini-Mental State Examination (MMSE), Instrumental Activities of Daily Living (IADL) scale, and Neuropsychiatric Inventory Questionnaire were collected. Survival status was determined via telephone interviews and further verified in the official death register in the third year. A 7:3 ratio was used to divide patients into the training and validation sets. Variables with statistical significance (p < 0.05) in the single-factor analysis were incorporated into the binary logistic regression model. A nomogram was constructed according to multivariate analysis and validated.

RESULTS

The final cohort included 587 patients, of whom 525 (89.44%) survived and 62 (10.56%) died. Younger age, higher MMSE score, lower IADL score, absence of disinhibition, and Charlson comorbidity index score ≤ 1 were all associated with 3-year survival. These predictors yielded good discrimination with C-indices of 0.80 (0.73-0.87) and 0.85 (0.77-0.94) in the training and validation cohorts, respectively. According to the Hosmer-Lemeshow test results, neither cohort displayed any statistical significance, and calibration curves displayed a good match between predictions and results.

CONCLUSIONS

Our study provided further insight into the factors contributing to the survival of patients with cognitive impairment.

CLINICAL IMPLICATIONS

Our model showed good accuracy and discrimination ability, and it can be used at community hospitals or primary care facilities that lack sophisticated equipment.

Disparities in cannabis use and documentation in electronic health records among children and young adults

The legalizations of medical and recreational cannabis have generated a great deal of interest in studying the health impacts of cannabis products. Despite increases in cannabis use, its documentation during clinical visits is not yet mainstream. This lack of information hampers efforts to study cannabis's effects on health outcomes. A clear and in-depth understanding of current trends in cannabis use documentation is necessary to develop proper guidelines to screen and document cannabis use. Here we have developed and used a natural language processing pipeline to evaluate the trends and disparities in cannabis documentation. The pipeline includes a screening step to identify clinical notes with cannabis use documentation which is then fed into a BERT-based classifier to confirm positive use. This pipeline is applied to more than 23 million notes from a large cohort of 370,087 patients seen in a high-volume multi-site pediatric and young adult clinic over a period of 21 years. Our findings show a very low but growing rate of cannabis use documentation (<2%) in electronic health records with significant demographic and socioeconomic disparities in both documentation and positive use, which requires further attention.

Machine learning algorithms for identifying predictive variables of mortality risk following dementia diagnosis: a longitudinal cohort study

Machine learning (ML) could have advantages over traditional statistical models in identifying risk factors. Using ML algorithms, our objective was to identify the most important variables associated with mortality after dementia diagnosis in the Swedish Registry for Cognitive/Dementia Disorders (SveDem). From SveDem, a longitudinal cohort of 28,023 dementia-diagnosed patients was selected for this study. Sixty variables were considered as potential predictors of mortality risk, such as age at dementia diagnosis, dementia type, sex, body mass index (BMI), mini-mental state examination (MMSE) score, time from referral to initiation of work-up, time from initiation of work-up to diagnosis, dementia medications, comorbidities, and some specific medications for chronic comorbidities (e.g., cardiovascular disease). We applied sparsity-inducing penalties for three ML algorithms and identified twenty important variables for the binary classification task in mortality risk prediction and fifteen variables to predict time to death. Area-under-ROC curve (AUC) measure was used to evaluate the classification algorithms. Then, an unsupervised clustering algorithm was applied on the set of twenty-selected variables to find two main clusters which accurately matched surviving and dead patient clusters. A support-vector-machines with an appropriate sparsity penalty provided the classification of mortality risk with accuracy = 0.7077, AUROC = 0.7375, sensitivity = 0.6436, and specificity = 0.740. Across three ML algorithms, the majority of the identified twenty variables were compatible with literature and with our previous studies on SveDem. We also found new variables which were not previously reported in literature as associated with mortality in dementia. Performance of basic dementia diagnostic work-up, time from referral to initiation of work-up, and time from initiation of work-up to diagnosis were found to be elements of the diagnostic process identified by the ML algorithms. The median follow-up time was 1053 (IQR = 516-1771) days in surviving and 1125 (IQR = 605-1770) days in dead patients. For prediction of time to death, the CoxBoost model identified 15 variables and classified them in order of importance. These highly important variables were age at diagnosis, MMSE score, sex, BMI, and Charlson Comorbidity Index with selection scores of 23%, 15%, 14%, 12% and 10%, respectively. This study demonstrates the potential of sparsity-inducing ML algorithms in improving our understanding of mortality risk factors in dementia patients and their application in clinical settings. Moreover, ML methods can be used as a complement to traditional statistical methods.

The Feasibility and Performance of Total Hip Replacement Prediction Deep Learning Algorithm with Real World Data

(1) Background: Hip degenerative disorder is a common geriatric disease is the main causes to lead to total hip replacement (THR). The surgical timing of THR is crucial for post-operative recovery. Deep learning (DL) algorithms can be used to detect anomalies in medical images and predict the need for THR. The real world data (RWD) were used to validate the artificial intelligence and DL algorithm in medicine but there was no previous study to prove its function in THR prediction. (2) Methods: We designed a sequential two-stage hip replacement prediction deep learning algorithm to identify the possibility of THR in three months of hip joints by plain pelvic radiography (PXR). We also collected RWD to validate the performance of this algorithm. (3) Results: The RWD totally included 3766 PXRs from 2018 to 2019. The overall accuracy of the algorithm was 0.9633; sensitivity was 0.9450; specificity was 1.000 and the precision was 1.000. The negative predictive value was 0.9009, the false negative rate was 0.0550, and the F1 score was 0.9717. The area under curve was 0.972 with 95% confidence interval from 0.953 to 0.987. (4) Conclusions: In summary, this DL algorithm can provide an accurate and reliable method for detecting hip degeneration and predicting the need for further THR. RWD offered an alternative support of the algorithm and validated its function to save time and cost.

Applications of Machine Learning in Palliative Care: A Systematic Review

Objective: To summarize the available literature on using machine learning (ML) for palliative care practice as well as research and to assess the adherence of the published studies to the most important ML best practices. Methods: The MEDLINE database was searched for the use of ML in palliative care practice or research, and the records were screened according to PRISMA guidelines. Results: In total, 22 publications using machine learning for mortality prediction (n = 15), data annotation (n = 5), predicting morbidity under palliative therapy (n = 1), and predicting response to palliative therapy (n = 1) were included. Publications used a variety of supervised or unsupervised models, but mostly tree-based classifiers and neural networks. Two publications had code uploaded to a public repository, and one publication uploaded the dataset. Conclusions: Machine learning in palliative care is mainly used to predict mortality. Similarly to other applications of ML, external test sets and prospective validations are the exception.

Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review

BACKGROUND

Natural Language Processing (NLP) is widely used to extract clinical insights from Electronic Health Records (EHRs). However, the lack of annotated data, automated tools, and other challenges hinder the full utilisation of NLP for EHRs. Various Machine Learning (ML), Deep Learning (DL) and NLP techniques are studied and compared to understand the limitations and opportunities in this space comprehensively.

METHODOLOGY

After screening 261 articles from 11 databases, we included 127 papers for full-text review covering seven categories of articles: (1) medical note classification, (2) clinical entity recognition, (3) text summarisation, (4) deep learning (DL) and transfer learning architecture, (5) information extraction, (6) Medical language translation and (7) other NLP applications. This study follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULT AND DISCUSSION

EHR was the most commonly used data type among the selected articles, and the datasets were primarily unstructured. Various ML and DL methods were used, with prediction or classification being the most common application of ML or DL. The most common use cases were: the International Classification of Diseases, Ninth Revision (ICD-9) classification, clinical note analysis, and named entity recognition (NER) for clinical descriptions and research on psychiatric disorders.

CONCLUSION

We find that the adopted ML models were not adequately assessed. In addition, the data imbalance problem is quite important, yet we must find techniques to address this underlining problem. Future studies should address key limitations in studies, primarily identifying Lupus Nephritis, Suicide Attempts, perinatal self-harmed and ICD-9 classification.

Predictive Models and Features of Patient Mortality across Dementia Types

Dementia care is challenging due to the divergent trajectories in disease progression and outcomes. Predictive models are needed to identify patients at risk of near-term mortality. Here, we developed machine learning models predicting survival using a dataset of 45,275 unique participants and 163,782 visit records from the U.S. National Alzheimer's Coordinating Center (NACC). Our models achieved an AUC-ROC of over 0.82 utilizing nine parsimonious features for all one-, three-, five-, and ten-year thresholds. The trained models mainly consisted of dementia-related predictors such as specific neuropsychological tests and were minimally affected by other age-related causes of death, e.g., stroke and cardiovascular conditions. Notably, stratified analyses revealed shared and distinct predictors of mortality across eight dementia types. Unsupervised clustering of mortality predictors grouped vascular dementia with depression and Lewy body dementia with frontotemporal lobar dementia. This study demonstrates the feasibility of flagging dementia patients at risk of mortality for personalized clinical management.

A Temporal Window Attention-Based Window-Dependent Long Short-Term Memory Network for Multivariate Time Series Prediction

Multivariate time series prediction models perform the required operation on a specific window length of a given input. However, capturing complex and nonlinear interdependencies in each temporal window remains challenging. The typical attention mechanisms assign a weight for a variable at the same time or the features of each previous time step to capture spatio-temporal correlations. However, it fails to directly extract each time step's relevant features that affect future values to learn the spatio-temporal pattern from a global perspective. To this end, a temporal window attention-based window-dependent long short-term memory network (TWA-WDLSTM) is proposed to enhance the temporal dependencies, which exploits the encoder-decoder framework. In the encoder, we design a temporal window attention mechanism to select relevant exogenous series in a temporal window. Furthermore, we introduce a window-dependent long short-term memory network (WDLSTM) to encode the input sequences in a temporal window into a feature representation and capture very long term dependencies. In the decoder, we use WDLSTM to generate the prediction values. We applied our model to four real-world datasets in comparison to a variety of state-of-the-art models. The experimental results suggest that TWA-WDLSTM can outperform comparison models. In addition, the temporal window attention mechanism has good interpretability. We can observe which variable contributes to the future value.

Popular deep learning algorithms for disease prediction: a review

Due to its automatic feature learning ability and high performance, deep learning has gradually become the mainstream of artificial intelligence in recent years, playing a role in many fields. Especially in the medical field, the accuracy rate of deep learning even exceeds that of doctors. This paper introduces several deep learning algorithms: Artificial Neural Network (NN), FM-Deep Learning, Convolutional NN and Recurrent NN, and expounds their theory, development history and applications in disease prediction; we analyze the defects in the current disease prediction field and give some current solutions; our paper expounds the two major trends in the future disease prediction and medical field-integrating Digital Twins and promoting precision medicine. This study can better inspire relevant researchers, so that they can use this article to understand related disease prediction algorithms and then make better related research.

Developing machine learning algorithms for dynamic estimation of progression during active surveillance for prostate cancer

Active Surveillance (AS) for prostate cancer is a management option that continually monitors early disease and considers intervention if progression occurs. A robust method to incorporate “live” updates of progression risk during follow-up has hitherto been lacking. To address this, we developed a deep learning-based individualised longitudinal survival model using Dynamic-DeepHit-Lite (DDHL) that learns data-driven distribution of time-to-event outcomes. Further refining outputs, we used a reinforcement learning approach (Actor-Critic) for temporal predictive clustering (AC-TPC) to discover groups with similar time-to-event outcomes to support clinical utility. We applied these methods to data from 585 men on AS with longitudinal and comprehensive follow-up (median 4.4 years). Time-dependent C-indices and Brier scores were calculated and compared to Cox regression and landmarking methods. Both Cox and DDHL models including only baseline variables showed comparable C-indices but the DDHL model performance improved with additional follow-up data. With 3 years of data collection and 3 years follow-up the DDHL model had a C-index of 0.79 (±0.11) compared to 0.70 (±0.15) for landmarking Cox and 0.67 (±0.09) for baseline Cox only. Model calibration was good across all models tested. The AC-TPC method further discovered 4 distinct outcome-related temporal clusters with distinct progression trajectories. Those in the lowest risk cluster had negligible progression risk while those in the highest cluster had a 50% risk of progression by 5 years. In summary, we report a novel machine learning approach to inform personalised follow-up during active surveillance which improves predictive power with increasing data input over time.

Synchrony of biomarker variability indicates a critical transition: Application to mortality prediction in hemodialysis

Critical transition theory suggests that complex systems should experience increased temporal variability just before abrupt state changes. We tested this hypothesis in 763 patients on long-term hemodialysis, using 11 biomarkers collected every two weeks and all-cause mortality as a proxy for critical transitions. We find that variability-measured by coefficients of variation (CVs)-increases before death for all 11 clinical biomarkers, and is strikingly synchronized across all biomarkers: the first axis of a principal component analysis on all CVs explains 49% of the variance. This axis then generates powerful predictions of mortality (HR95 = 9.7, p < 0.0001, where HR95 is a scale-invariant metric of hazard ratio; AUC up to 0.82) and starts to increase markedly ∼3 months prior to death. Our results provide an early warning sign of physiological collapse and, more broadly, a quantification of joint system dynamics that opens questions of how system modularity may break down before critical transitions.

Topic Modeling for Interpretable Text Classification From EHRs

The clinical notes in electronic health records have many possibilities for predictive tasks in text classification. The interpretability of these classification models for the clinical domain is critical for decision making. Using topic models for text classification of electronic health records for a predictive task allows for the use of topics as features, thus making the text classification more interpretable. However, selecting the most effective topic model is not trivial. In this work, we propose considerations for selecting a suitable topic model based on the predictive performance and interpretability measure for text classification. We compare 17 different topic models in terms of both interpretability and predictive performance in an inpatient violence prediction task using clinical notes. We find no correlation between interpretability and predictive performance. In addition, our results show that although no model outperforms the other models on both variables, our proposed fuzzy topic modeling algorithm (FLSA-W) performs best in most settings for interpretability, whereas two state-of-the-art methods (ProdLDA and LSI) achieve the best predictive performance.

Palliative Care Exposure Relative to Predicted Risk of Six-Month Mortality in Hospitalized Adults

CONTEXT

The optimal strategy for implementing mortality-predicting algorithms to facilitate clinical care, prognostic discussions, and palliative care interventions remains unknown.

OBJECTIVES

To develop and validate a real-time predictive model for 180 day mortality using routinely available clinical and laboratory admission data and determine if palliative care exposure varies with predicted mortality risk.

METHODS

Adult admissions between October 1, 2013 and October.1, 2017 were included for the model derivation. A separate cohort was collected between January 1, 2018 and July 31, 2020 for validation. Patients were followed for 180 days from discharge, and logistic regression with selected variables was used to estimate patients' risk for mortality.

RESULTS

In the model derivation cohort, 7963 events of 180 day mortality (4.5% event rate) were observed. Median age was 53.0 (IQR 24.0-66.0) with 92,734 females (52.5%). Variables with strongest association with 180 day mortality included: Braden Score (OR 0.83; 95% CI 0.82-0.84); admission Do Not Resuscitate orders (OR 2.61; 95% CI 2.43-2.79); admission service and admission status. The model yielded excellent discriminatory ability in both the derivation (c-statistic 0.873; 95% CI 0.870-0.877; Brier score 0.04) and validation cohorts (c-statistic 0.844; 95% CI 0.840-0.847; Brier score 0.072). Inpatient palliative care consultations increased from 3% of minimal-risk encounters to 41% of high-risk encounters (P < 0.01).

CONCLUSION

We developed and temporally validated a predictive mortality model for adults from a large retrospective cohort, which helps quantify the potential need for palliative care referrals based on risk strata. Machine learning algorithms for mortality require clinical interpretation, and additional studies are needed to design patient-centered and risk-specific interventions.

Intelligent Palliative Care Based on Patient-Reported Outcome Measures

CONTEXT

The growth of patient reported outcome measures data in palliative care provides an opportunity for machine learning to identify patterns in patient responses signifying different phases of illness.

OBJECTIVES

The study will explore if machine learning and network analysis can identify phases in patient palliative status through symptoms reported on the Integrated Palliative Care Outcome Scale (IPOS).

METHODS

A partly cross-sectional and partially longitudinal observational study was undertaken using the Australasian Karnofsky Performance Scale (AKPS); Integrated Palliative Care Outcome Scale (IPOS); Phase of Illness (POI). Patient palliative records (n = 1507, 65% stable, 20% unstable, 9% deteriorating, 2% terminal) from 804 adult patients enrolled in a New Zealand palliative care service were analysed using a combination of statistical, machine learning and network analysis techniques.

RESULTS

Data from IPOS showed considerable variation with phase. Also, network analysis showed clear associations between items by phase. Six machine learning techniques identified the most important variables for predicting possible transition between phases of illness. Network analysis for all patients showed that Poor Appetite and Loss of Energy were central IPOS items, with Loss of Energy linked to Drowsiness, Shortness of Breath and Lack of Mobility on the one hand, and Poor Appetite linked to Nausea, Vomiting, Constipation and Sore and Dry Mouth on the other.

CONCLUSION

These preliminary results, when coupled with the latest technological developments in mobile apps and wearable technology, could point the way to increased use of digital therapeutics in continuous palliative care monitoring.

Development and Validation of a Deep Learning Model for Earlier Detection of Cognitive Decline From Clinical Notes in Electronic Health Records

IMPORTANCE

Detecting cognitive decline earlier among older adults can facilitate enrollment in clinical trials and early interventions. Clinical notes in longitudinal electronic health records (EHRs) provide opportunities to detect cognitive decline earlier than it is noted in structured EHR fields as formal diagnoses.

OBJECTIVE

To develop and validate a deep learning model to detect evidence of cognitive decline from clinical notes in the EHR.

DESIGN, SETTING, AND PARTICIPANTS

Notes documented 4 years preceding the initial mild cognitive impairment (MCI) diagnosis were extracted from Mass General Brigham's Enterprise Data Warehouse for patients aged 50 years or older and with initial MCI diagnosis during 2019. The study was conducted from March 1, 2020, to June 30, 2021. Sections of notes for cognitive decline were labeled manually and 2 reference data sets were created. Data set I contained a random sample of 4950 note sections filtered by a list of keywords related to cognitive functions and was used for model training and testing. Data set II contained 2000 randomly selected sections without keyword filtering for assessing whether the model performance was dependent on specific keywords.

MAIN OUTCOMES AND MEASURES

A deep learning model and 4 baseline models were developed and their performance was compared using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC).

RESULTS

Data set I represented 1969 patients (1046 [53.1%] women; mean [SD] age, 76.0 [13.3] years). Data set II comprised 1161 patients (619 [53.3%] women; mean [SD] age, 76.5 [10.2] years). With some overlap of patients deleted, the unique population was 2166. Cognitive decline was noted in 1453 sections (29.4%) in data set I and 69 sections (3.45%) in data set II. Compared with the 4 baseline models, the deep learning model achieved the best performance in both data sets, with AUROC of 0.971 (95% CI, 0.967-0.976) and AUPRC of 0.933 (95% CI, 0.921-0.944) for data set I and AUROC of 0.997 (95% CI, 0.994-0.999) and AUPRC of 0.929 (95% CI, 0.870-0.969) for data set II.

CONCLUSIONS AND RELEVANCE

In this diagnostic study, a deep learning model accurately detected cognitive decline from clinical notes preceding MCI diagnosis and had better performance than keyword-based search and other machine learning models. These results suggest that a deep learning model could be used for earlier detection of cognitive decline in the EHRs.

Expected clinical utility of automatable prediction models for improving palliative and end-of-life care outcomes: Toward routine decision analysis before implementation

Objective

The study sought to evaluate the expected clinical utility of automatable prediction models for increasing goals-of-care discussions (GOCDs) among hospitalized patients at the end of life (EOL).

Materials and Methods

We built a decision model from the perspective of clinicians who aim to increase GOCDs at the EOL using an automated alert system. The alternative strategies were 4 prediction models—3 random forest models and the Modified Hospital One-year Mortality Risk model—to generate alerts for patients at a high risk of 1-year mortality. They were trained on admissions from 2011 to 2016 (70 788 patients) and tested with admissions from 2017-2018 (16 490 patients). GOCDs occurring in usual care were measured with code status orders. We calculated the expected risk difference (beneficial outcomes with alerts minus beneficial outcomes without alerts among those at the EOL), the number needed to benefit (number of alerts needed to increase benefit over usual care by 1 outcome), and the net benefit (benefit minus cost) of each strategy.

Results

Models had a C-statistic between 0.79 and 0.86. A code status order occurred during 2599 of 3773 (69%) hospitalizations at the EOL. At a risk threshold corresponding to an alert prevalence of 10%, the expected risk difference ranged from 5.4% to 10.7% and the number needed to benefit ranged from 5.4 to 10.9 alerts. Using revealed preferences, only 2 models improved net benefit over usual care. A random forest model with diagnostic predictors had the highest expected value, including in sensitivity analyses.

Discussion

Prediction models with acceptable predictive validity differed meaningfully in their ability to improve over usual decision making.

Conclusions

An evaluation of clinical utility, such as by using decision curve analysis, is recommended after validating a prediction model because metrics of model predictiveness, such as the C-statistic, are not informative of clinical value.

Use of Machine Learning and Artificial Intelligence Methods in Geriatric Mental Health Research Involving Electronic Health Record or Administrative Claims Data: A Systematic Review

Introduction: Electronic health records (EHR) and administrative healthcare data (AHD) are frequently used in geriatric mental health research to answer various health research questions. However, there is an increasing amount and complexity of data available that may lend itself to alternative analytic approaches using machine learning (ML) or artificial intelligence (AI) methods. We performed a systematic review of the current application of ML or AI approaches to the analysis of EHR and AHD in geriatric mental health. Methods: We searched MEDLINE, Embase, and PsycINFO to identify potential studies. We included all articles that used ML or AI methods on topics related to geriatric mental health utilizing EHR or AHD data. We assessed study quality either by Prediction model Risk OF Bias ASsessment Tool (PROBAST) or Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist. Results: We initially identified 391 articles through an electronic database and reference search, and 21 articles met inclusion criteria. Among the selected studies, EHR was the most used data type, and the datasets were mainly structured. A variety of ML and AI methods were used, with prediction or classification being the main application of ML or AI with the random forest as the most common ML technique. Dementia was the most common mental health condition observed. The relative advantages of ML or AI techniques compared to biostatistical methods were generally not assessed. Only in three studies, low risk of bias (ROB) was observed according to all the PROBAST domains but in none according to QUADAS-2 domains. The quality of study reporting could be further improved. Conclusion: There are currently relatively few studies using ML and AI in geriatric mental health research using EHR and AHD methods, although this field is expanding. Aside from dementia, there are few studies of other geriatric mental health conditions. The lack of consistent information in the selected studies precludes precise comparisons between them. Improving the quality of reporting of ML and AI work in the future would help improve research in the field. Other courses of improvement include using common data models to collect/organize data, and common datasets for ML model validation.

Mortality Risk Models for Persons with Dementia: A Systematic Review

BACKGROUND

Persons with dementia have higher mortality than the general population. Objective, standardized predictions of mortality risk in persons with dementia could help with planning resources for care close to the end of life.

OBJECTIVE

To systematically review prediction models for risk of death in persons with dementia.

METHODS

The Medline and PsycInfo databases were searched on November 29, 2020, for prediction models estimating the risk of death in persons with dementia. Study quality was assessed using the Prediction model Risk Of Bias ASsessment Tool.

RESULTS

The literature search identified 2,828 studies, of which 18 were included. These studies described 16 different prediction models with c statistics mostly ranging from 0.67 to 0.79. Five models were externally validated, of which four were applicable. There were two models that were both applicable and had reasonably low risk of bias. One model predicted risk of death at six months in persons with advanced dementia residing in a nursing home. The other predicted risk of death at three years in persons seen in primary care practice or a dementia specialty clinic, derived from a nationwide registry in Sweden but not externally validated.

CONCLUSION

Valid, applicable models with low risk of bias were found in two settings: advanced dementia in a nursing home and outpatient practices. The outpatient model requires external validation. Better models are needed for persons with mild to moderate dementia in nursing homes, a common demographic. These models may be useful for educating persons living with dementia and care partners and directing resources for end of life care.Registration:The study protocol is registered on PROSPERO as RD4202018076.

AI-based clinical decision-making systems in palliative medicine: ethical challenges

BACKGROUND

Improving palliative care (PC) is demanding due to the increase in people with PC needs over the next few years. An early identification of PC needs is fundamental in the care approach: it provides effective patient-centred care and could improve outcomes such as patient quality of life, reduction of the overall length of hospitalisation, survival rate prolongation, the satisfaction of both the patients and caregivers and cost-effectiveness.

METHODS

We reviewed literature with the objective of identifying and discussing the most important ethical challenges related to the implementation of AI-based data processing services in PC and advance care planning.

RESULTS

AI-based mortality predictions can signal the need for patients to obtain access to personalised communication or palliative care consultation, but they should not be used as a unique parameter to activate early PC and initiate an ACP. A number of factors must be included in the ethical decision-making process related to initiation of ACP conversations, among which are autonomy and quality of life, the risk of worsening healthcare status, the commitment by caregivers, the patients' psychosocial and spiritual distress and their wishes to initiate EOL discussions CONCLUSIONS: Despite the integration of artificial intelligence (AI)-based services into routine healthcare practice could have a positive effect of promoting early activation of ACP by means of a timely identification of PC needs, from an ethical point of view, the provision of these automated techniques raises a number of critical issues that deserve further exploration.

Interpretation of deep learning in genomics and epigenomics

Machine learning methods have been widely applied to big data analysis in genomics and epigenomics research. Although accuracy and efficiency are common goals in many modeling tasks, model interpretability is especially important to these studies towards understanding the underlying molecular and cellular mechanisms. Deep neural networks (DNNs) have recently gained popularity in various types of genomic and epigenomic studies due to their capabilities in utilizing large-scale high-throughput bioinformatics data and achieving high accuracy in predictions and classifications. However, DNNs are often challenged by their potential to explain the predictions due to their black-box nature. In this review, we present current development in the model interpretation of DNNs, focusing on their applications in genomics and epigenomics. We first describe state-of-the-art DNN interpretation methods in representative machine learning fields. We then summarize the DNN interpretation methods in recent studies on genomics and epigenomics, focusing on current data- and computing-intensive topics such as sequence motif identification, genetic variations, gene expression, chromatin interactions and non-coding RNAs. We also present the biological discoveries that resulted from these interpretation methods. We finally discuss the advantages and limitations of current interpretation approaches in the context of genomic and epigenomic studies. Contact:xiaoman@mail.ucf.edu, haihu@cs.ucf.edu.

Referral criteria to specialist palliative care for patients with dementia: A systematic review

BACKGROUND

Patients with dementia often have significant symptom burden and a progressive course of functional deterioration. Specialist palliative care referral may be helpful, but it is unclear who and when patients should be referred. We conducted a systematic review of the literature to examine referral criteria for palliative care among patients with dementia.

METHODS

We searched Ovid MEDLINE, Ovid Embase, Ovid PsycInfo, Cochrane Library, PubMed, and CINAHL databases for articles from inception to December 3, 2019, related to specialist palliative care referral for dementia. Two investigators independently reviewed the citations for inclusion, extracted the referral criteria, and categorized them thematically.

RESULTS

Of the 1788 citations, 59 articles were included in the final sample. We identified 13 categories of referral criteria, including 6 disease-based and 7 needs-based criteria. The most commonly discussed criterion was "dementia stage" (n = 43, 73%), followed by "new diagnosis of dementia" (n = 17, 29%), "medical complications of dementia" (n = 12, 20%), "prognosis" (n = 11, 19%), and "physical symptoms" (n = 11, 19%). Under dementia stage, 37/44 (84%) articles recommended a palliative care referral for advanced dementia. Pneumonia (n = 6, 10%), fall/fracture (n = 4, 7%), and decubitus ulcers (n = 4, 7%) were most commonly discussed complications to trigger a referral. Under prognosis, the time frame for referral varied from <2 years of life expectancy to <6 months. 3 (5%) of articles recommended "surprise question" as a potential trigger.

CONCLUSIONS

This systematic review highlighted the lack of consensus regarding referral criteria for palliative care in patients with dementia and the need to identify timely triggers to standardize referral.

Forecasting of the Prevalence of Dementia Using the LSTM Neural Network in Taiwan

The World Health Organization has urged countries to prioritize dementia in their public health policies. Dementia poses a tremendous socioeconomic burden, and the accurate prediction of the annual increase in prevalence is essential for establishing strategies to cope with its effects. The present study established a model based on the architecture of the long short-term memory (LSTM) neural network for predicting the number of dementia cases in Taiwan, which considers the effects of age and sex on the prevalence of dementia. The LSTM network is a variant of recurrent neural networks (RNNs), which possesses a special gate structure and avoids the problems in RNNs of gradient explosion, gradient vanishing, and long-term memory failure. A number of patients diagnosed as having dementia from 1997 to 2017 was collected in annual units from a data set extracted from the Health Insurance Database of the Ministry of Health and Welfare in Taiwan. To further verify the validity of the proposed model, the LSTM network was compared with three types of models: statistical models (exponential smoothing (ETS), autoregressive integrated moving average model (ARIMA), trigonometric seasonality, Box–Cox transformation, autoregressive moving average errors, and trend seasonal components model (TBATS)), hybrid models (support vector regression (SVR), particle swarm optimization–based support vector regression (PSOSVR)), and deep learning model (artificial neural networks (ANN)). The mean absolute percentage error (MAPE), root-mean-square error (RMSE), mean absolute error (MAE), and R-squared (R2) were used to evaluate the model performances. The results indicated that the LSTM network has higher prediction accuracy than the three types of models for forecasting the prevalence of dementia in Taiwan.

Conversing with High-Risk Patients to Determine Serious Illness Goals and Values in the Time of Covid-19

During the Covid-19 pandemic’s first surge in Boston, Brigham Health sought to ensure that patients’ health care proxies and serious illness wishes were known to care teams. The authors engaged a diverse set of team members in outreach regarding serious illness conversations. Patients enrolled in the Integrated Care Management Program (iCMP) were contacted by their own nurse care coordinator for a serious illness conversation, discussing patients’ goals and values in the context of underlying illness and the threat of Covid-19. Simultaneously, nurses, medical students, and social care team members reached out to non-iCMP primary care patients identified as being at high risk of morbidity or mortality from Covid-19 and engaged these patients in conversations regarding health care proxy documentation and social determinants of health needs. The authors’ experience demonstrates that such a population health approach can facilitate timely and well-accepted outreach regarding serious illness to patients with varied needs and profiles.

A statistically rigorous deep neural network approach to predict mortality in trauma patients admitted to the intensive care unit

BACKGROUND

Trauma patients admitted to critical care are at high risk of mortality because of their injuries. Our aim was to develop a machine learning-based model to predict mortality using Fahad-Liaqat-Ahmad Intensive Machine (FLAIM) framework. We hypothesized machine learning could be applied to critically ill patients and would outperform currently used mortality scores.

METHODS

The current Deep-FLAIM model evaluates the statistically significant risk factors and then supply these risk factors to deep neural network to predict mortality in trauma patients admitted to the intensive care unit (ICU). We analyzed adult patients (≥18 years) admitted to the trauma ICU in the publicly available database Medical Information Mart for Intensive Care III version 1.4. The first phase selection of risk factor was done using Cox-regression univariate and multivariate analyses. In the second phase, we applied deep neural network and other traditional machine learning models like Linear Discriminant Analysis, Gaussian Naïve Bayes, Decision Tree Model, and k-nearest neighbor models.

RESULTS

We identified a total of 3,041 trauma patients admitted to the trauma surgery ICU. We observed that several clinical and laboratory-based variables were statistically significant for both univariate and multivariate analyses while others were not. With most significant being serum anion gap (hazard ratio [HR], 2.46; 95% confidence interval [CI], 1.94-3.11), sodium (HR, 2.11; 95% CI, 1.61-2.77), and chloride (HR, 2.11; 95% CI, 1.69-2.64) abnormalities on laboratories, while clinical variables included the diagnosis of sepsis (HR, 2.03; 95% CI, 1.23-3.37), Quick Sequential Organ Failure Assessment score (HR, 1.52; 95% CI, 1.32-3.76). And Systemic Inflammatory Response Syndrome criteria (HR. 1.41; 95% CI, 1.24-1.26). After we used these clinically significant variables and applied various machine learning models to the data, we found out that our proposed DNN outperformed all the other methods with test set accuracy of 92.25%, sensitivity of 79.13%, and specificity of 94.16%; positive predictive value, 66.42%; negative predictive value, 96.87%; and area under the curve of the receiver-operator curve of 0.91 (1.45-1.29).

CONCLUSION

Our novel Deep-FLAIM model outperformed all other machine learning models. The model is easy to implement, user friendly and with high accuracy.

LEVEL OF EVIDENCE

Prognostic study, level II.

Topic modeling to characterize the natural history of ANCA-Associated vasculitis from clinical notes: A proof of concept study

OBJECTIVES

Clinical notes from electronic health records (EHR) are important to characterize the natural history, comorbidities, and complications of ANCA-associated vasculitis (AAV) because these details may not be captured by claims and structured data. However, labor-intensive chart review is often required to extract information from notes. We hypothesized that machine learning can automatically discover clinically-relevant themes across longitudinal notes to study AAV.

METHODS

This retrospective study included prevalent PR3- or MPO-ANCA+ AAV cases managed within the Mass General Brigham integrated health care system with providers' notes available between March 1, 1990 and August 23, 2018. We generated clinically-relevant topics mentioned in notes using latent Dirichlet allocation-based topic modeling and conducted trend analyses of those topics over the 2 years prior to and 5 years after the initiation of AAV-specific treatment.

RESULTS

The study cohort included 660 patients with AAV. We generated 90 topics using 113,048 available notes. Topics were related to the AAV diagnosis, treatment, symptoms and manifestations (e.g., glomerulonephritis), and complications (e.g., end-stage renal disease, infection). AAV-related symptoms and psychiatric symptoms were mentioned months before treatment initiation. Topics related to pulmonary and renal diseases, diabetes, and infections were common during the disease course but followed distinct temporal patterns.

CONCLUSIONS

Automated topic modeling can be used to discover clinically-relevant themes and temporal patterns related to the diagnosis, treatment, comorbidities, and complications of AAV from EHR notes. Future research might compare the temporal patterns in a non-AAV cohort and leverage clinical notes to identify possible AAV cases prospectively.

Development and Validation of a Deep Learning Model for Detection of Allergic Reactions Using Safety Event Reports Across Hospitals

IMPORTANCE

Although critical to patient safety, health care-related allergic reactions are challenging to identify and monitor.

OBJECTIVE

To develop a deep learning model to identify allergic reactions in the free-text narrative of hospital safety reports and evaluate its generalizability, efficiency, productivity, and interpretability.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study analyzed hospital safety reports filed between May 2004 and January 2019 at Brigham and Women's Hospital and between April 2006 and June 2018 at Massachusetts General Hospital in Boston. Training and validating a deep learning model involved extracting safety reports using 101 expert-curated keywords from Massachusetts General Hospital (data set I). The model was then evaluated on 3 data sets: reports without keywords (data set II), reports from a different time frame (data set III), and reports from a different hospital (Brigham and Women's Hospital; data set IV). Statistical analyses were performed between March 1, 2019, and July 18, 2020.

MAIN OUTCOMES AND MEASURES

The area under the receiver operating characteristic curve and area under the precision-recall curve were used on data set I. The precision at top-k was used on data sets II to IV.

RESULTS

A total of 299 028 safety reports with 172 854 patients were included. Of these patients, 86 544 were women (50.1%) and the median (interquartile range [IQR]) age was 59.7 (43.8-71.6) years. The deep learning model achieved an area under the receiver operating characteristic curve of 0.979 (95% CI, 0.973-0.985) and an area under the precision-recall curve of 0.809 (95% CI, 0.773-0.845). The model achieved precisions at the top 100 model-identified cases of 0.930 in data set II, 0.960 in data set III, and 0.990 in data set IV. Compared with the keyword-search approach, the deep learning model reduced the number of cases for manual review by 63.8% and identified 24.2% more cases of confirmed allergic reactions. The model highlighted important words (eg, rash, hives, and Benadryl) in prediction and extended the list of expert-curated keywords through an attention layer.

CONCLUSIONS AND RELEVANCE

This study showed that a deep learning model can accurately and efficiently identify allergic reactions using free-text narratives written by a variety of health care professionals. This model could be used to improve allergy care, potentially enabling real-time event surveillance and guidance for medical errors and system improvement.

Diverse Applications of Artificial Intelligence in Neuroradiology

Recent advances in artificial intelligence (AI) and deep learning (DL) hold promise to augment neuroimaging diagnosis for patients with brain tumors and stroke. Here, the authors review the diverse landscape of emerging neuroimaging applications of AI, including workflow optimization, lesion segmentation, and precision education. Given the many modalities used in diagnosing neurologic diseases, AI may be deployed to integrate across modalities (MR imaging, computed tomography, PET, electroencephalography, clinical and laboratory findings), facilitate crosstalk among specialists, and potentially improve diagnosis in patients with trauma, multiple sclerosis, epilepsy, and neurodegeneration. Together, there are myriad applications of AI for neuroradiology."

Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites

BACKGROUND

Automated systems that use machine learning to estimate a patient's risk of death are being developed to influence care. There remains sparse transparent reporting of model generalizability in different subpopulations especially for implemented systems.

METHODS

A prognostic study included adult admissions at a multi-site, academic medical center between 2015 and 2017. A predictive model for all-cause mortality (including initiation of hospice care) within 60 days of admission was developed. Model generalizability is assessed in temporal validation in the context of potential demographic bias. A subsequent prospective cohort study was conducted at the same sites between October 2018 and June 2019. Model performance during prospective validation was quantified with areas under the receiver operating characteristic and precision recall curves stratified by site. Prospective results include timeliness, positive predictive value, and the number of actionable predictions.

RESULTS

Three years of development data included 128,941 inpatient admissions (94,733 unique patients) across sites where patients are mostly white (61%) and female (60%) and 4.2% led to death within 60 days. A random forest model incorporating 9614 predictors produced areas under the receiver operating characteristic and precision recall curves of 87.2 (95% CI, 86.1-88.2) and 28.0 (95% CI, 25.0-31.0) in temporal validation. Performance marginally diverges within sites as the patient mix shifts from development to validation (patients of one site increases from 10 to 38%). Applied prospectively for nine months, 41,728 predictions were generated in real-time (median [IQR], 1.3 [0.9, 32] minutes). An operating criterion of 75% positive predictive value identified 104 predictions at very high risk (0.25%) where 65% (50 from 77 well-timed predictions) led to death within 60 days.

CONCLUSION

Temporal validation demonstrates good model discrimination for 60-day mortality. Slight performance variations are observed across demographic subpopulations. The model was implemented prospectively and successfully produced meaningful estimates of risk within minutes of admission.

Development of a Deep Learning Model to Identify Lymph Node Metastasis on Magnetic Resonance Imaging in Patients With Cervical Cancer

IMPORTANCE

Accurate identification of lymph node metastasis preoperatively and noninvasively in patients with cervical cancer can avoid unnecessary surgical intervention and benefit treatment planning.

OBJECTIVE

To develop a deep learning model using preoperative magnetic resonance imaging for prediction of lymph node metastasis in cervical cancer.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study developed an end-to-end deep learning model to identify lymph node metastasis in cervical cancer using magnetic resonance imaging (MRI). A total of 894 patients with stage IB to IIB cervical cancer who underwent radical hysterectomy and pelvic lymphadenectomy were reviewed. All patients underwent radical hysterectomy and pelvic lymphadenectomy, received pelvic MRI within 2 weeks before the operations, had no concurrent cancers, and received no preoperative treatment. To achieve the optimal model, the diagnostic value of 3 MRI sequences was compared, and the outcomes in the intratumoral and peritumoral regions were explored. To mine tumor information from both image and clinicopathologic levels, a hybrid model was built and its prognostic value was assessed by Kaplan-Meier analysis. The deep learning model and hybrid model were developed on a primary cohort consisting of 338 patients (218 patients from Sun Yat-sen University Cancer Center, Guangzhou, China, between January 2011 and December 2017 and 120 patients from Henan Provincial People's Hospital, Zhengzhou, China, between December 2016 and June 2018). The models then were evaluated on an independent validation cohort consisting of 141 patients from Yunnan Cancer Hospital, Kunming, China, between January 2011 and December 2017.

MAIN OUTCOMES AND MEASURES

The primary diagnostic outcome was lymph node metastasis status, with the pathologic characteristics diagnosed by lymphadenectomy. The secondary primary clinical outcome was survival. The primary diagnostic outcome was assessed by receiver operating characteristic (area under the curve [AUC]) analysis; the primary clinical outcome was assessed by Kaplan-Meier survival analysis.

RESULTS

A total of 479 patients (mean [SD] age, 49.1 [9.7] years) fulfilled the eligibility criteria and were enrolled in the primary (n = 338) and validation (n = 141) cohorts. A total of 71 patients (21.0%) in the primary cohort and 32 patients (22.7%) in the validation cohort had lymph node metastais confirmed by lymphadenectomy. Among the 3 image sequences, the deep learning model that used both intratumoral and peritumoral regions on contrast-enhanced T1-weighted imaging showed the best performance (AUC, 0.844; 95% CI, 0.780-0.907). These results were further improved in a hybrid model that combined tumor image information mined by deep learning model and MRI-reported lymph node status (AUC, 0.933; 95% CI, 0.887-0.979). Moreover, the hybrid model was significantly associated with disease-free survival from cervical cancer (hazard ratio, 4.59; 95% CI, 2.04-10.31; P < .001).

CONCLUSIONS AND RELEVANCE

The findings of this study suggest that deep learning can be used as a preoperative noninvasive tool to diagnose lymph node metastasis in cervical cancer.

Estimating real-world performance of a predictive model: a case-study in predicting mortality

OBJECTIVE

One primary consideration when developing predictive models is downstream effects on future model performance. We conduct experiments to quantify the effects of experimental design choices, namely cohort selection and internal validation methods, on (estimated) real-world model performance.

MATERIALS AND METHODS

Four years of hospitalizations are used to develop a 1-year mortality prediction model (composite of death or initiation of hospice care). Two common methods to select appropriate patient visits from their encounter history (backwards-from-outcome and forwards-from-admission) are combined with 2 testing cohorts (random and temporal validation). Two models are trained under otherwise identical conditions, and their performances compared. Operating thresholds are selected in each test set and applied to a "real-world" cohort of labeled admissions from another, unused year.

RESULTS

Backwards-from-outcome cohort selection retains 25% of candidate admissions (n = 23 579), whereas forwards-from-admission selection includes many more (n = 92 148). Both selection methods produce similar performances when applied to a random test set. However, when applied to the temporally defined "real-world" set, forwards-from-admission yields higher areas under the ROC and precision recall curves (88.3% and 56.5% vs. 83.2% and 41.6%).

DISCUSSION

A backwards-from-outcome experiment manipulates raw training data, simplifying the experiment. This manipulated data no longer resembles real-world data, resulting in optimistic estimates of test set performance, especially at high precision. In contrast, a forwards-from-admission experiment with a temporally separated test set consistently and conservatively estimates real-world performance.

CONCLUSION

Experimental design choices impose bias upon selected cohorts. A forwards-from-admission experiment, validated temporally, can conservatively estimate real-world performance.

LAY SUMMARY

The routine care of patients stands to benefit greatly from assistive technologies, including data-driven risk assessment. Already, many different machine learning and artificial intelligence applications are being developed from complex electronic health record data. To overcome challenges that arise from such data, researchers often start with simple experimental approaches to test their work. One key component is how patients (and their healthcare visits) are selected for the study from the pool of all patients seen. Another is how the group of patients used to create the risk estimator differs from the group used to evaluate how well it works. These choices complicate how the experimental setting compares to the real-world application to patients. For example, different selection approaches that depend on each patient's future outcome can simplify the experiment but are impractical upon implementation as these data are unavailable. We show that this kind of "backwards" experiment optimistically estimates how well the model performs. Instead, our results advocate for experiments that select patients in a "forwards" manner and "temporal" validation that approximates training on past data and implementing on future data. More robust results help gauge the clinical utility of recent works and aid decision-making before implementation into practice.

Transparent Reporting of Multivariable Prediction Models in Journal and Conference Abstracts: TRIPOD for Abstracts

Clear and informative reporting in titles and abstracts is essential to help readers and reviewers identify potentially relevant studies and decide whether to read the full text. Although the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) statement provides general recommendations for reporting titles and abstracts, more detailed guidance seems to be desirable. The authors present TRIPOD for Abstracts, a checklist and corresponding guidance for reporting prediction model studies in abstracts. A list of 32 potentially relevant items was the starting point for a modified Delphi procedure involving 110 experts, of whom 71 (65%) participated in the web-based survey. After 2 Delphi rounds, the experts agreed on 21 items as being essential to report in abstracts of prediction model studies. This number was reduced by merging some of the items. In a third round, participants provided feedback on a draft version of TRIPOD for Abstracts. The final checklist contains 12 items and applies to journal and conference abstracts that describe the development or external validation of a diagnostic or prognostic prediction model, or the added value of predictors to an existing model, regardless of the clinical domain or statistical approach used.

Leveraging Advances in Artificial Intelligence to Improve the Quality and Timing of Palliative Care

In recent years, research on artificial intelligence (AI) in medicine has seen great advances, especially with regards to the detection of diseases [...].

Artificial intelligence approaches to predicting and detecting cognitive decline in older adults: A conceptual review

Preserving cognition and mental capacity is critical to aging with autonomy. Early detection of pathological cognitive decline facilitates the greatest impact of restorative or preventative treatments. Artificial Intelligence (AI) in healthcare is the use of computational algorithms that mimic human cognitive functions to analyze complex medical data. AI technologies like machine learning (ML) support the integration of biological, psychological, and social factors when approaching diagnosis, prognosis, and treatment of disease. This paper serves to acquaint clinicians and other stakeholders with the use, benefits, and limitations of AI for predicting, diagnosing, and classifying mild and major neurocognitive impairments, by providing a conceptual overview of this topic with emphasis on the features explored and AI techniques employed. We present studies that fell into six categories of features used for these purposes: (1) sociodemographics; (2) clinical and psychometric assessments; (3) neuroimaging and neurophysiology; (4) electronic health records and claims; (5) novel assessments (e.g., sensors for digital data); and (6) genomics/other omics. For each category we provide examples of AI approaches, including supervised and unsupervised ML, deep learning, and natural language processing. AI technology, still nascent in healthcare, has great potential to transform the way we diagnose and treat patients with neurocognitive disorders.

Unsupervised Machine Learning of Topics Documented by Nurses about Hospitalized Patients Prior to a Rapid-Response Event

BACKGROUND

In the hospital setting, it is crucial to identify patients at risk for deterioration before it fully develops, so providers can respond rapidly to reverse the deterioration. Rapid response (RR) activation criteria include a subjective component ("worried about the patient") that is often documented in nurses' notes and is hard to capture and quantify, hindering active screening for deteriorating patients.

OBJECTIVES

We used unsupervised machine learning to automatically discover RR event risk/protective factors from unstructured nursing notes.

METHODS

In this retrospective cohort study, we obtained nursing notes of hospitalized, nonintensive care unit patients, documented from 2015 through 2018 from Partners HealthCare databases. We applied topic modeling to those notes to reveal topics (clusters of associated words) documented by nurses. Two nursing experts named each topic with a representative Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) concept. We used the concepts along with vital signs and demographics in a time-dependent covariates extended Cox model to identify risk/protective factors for RR event risk.

RESULTS

From a total of 776,849 notes of 45,299 patients, we generated 95 stable topics, of which 80 were mapped to 72 distinct SNOMED CT concepts. Compared with a model containing only demographics and vital signs, the latent topics improved the model's predictive ability from a concordance index of 0.657 to 0.720. Thirty topics were found significantly associated with RR event risk at a 0.05 level, and 11 remained significant after Bonferroni correction of the significance level to 6.94E-04, including physical examination (hazard ratio [HR] = 1.07, 95% confidence interval [CI], 1.03-1.12), informing doctor (HR = 1.05, 95% CI, 1.03-1.08), and seizure precautions (HR = 1.08, 95% CI, 1.04-1.12).

CONCLUSION

Unsupervised machine learning methods can automatically reveal interpretable and informative signals from free-text and may support early identification of patients at risk for RR events.