Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

A nursing note-aware deep neural network for predicting mortality risk after hospital discharge

BACKGROUND

ICU readmissions and post-discharge mortality pose significant challenges. Previous studies used EHRs and machine learning models, but mostly focused on structured data. Nursing records contain crucial unstructured information, but their utilization is challenging. Natural language processing (NLP) can extract structured features from clinical text. This study proposes the Crucial Nursing Description Extractor (CNDE) to predict post-ICU discharge mortality rates and identify high-risk patients for unplanned readmission by analyzing electronic nursing records.

OBJECTIVE

Developed a deep neural network (NurnaNet) with the ability to perceive nursing records, combined with a bio-clinical medicine pre-trained language model (BioClinicalBERT) to analyze the electronic health records (EHRs) in the MIMIC III dataset to predict the death of patients within six month and two year risk.

DESIGN

A cohort and system development design was used.

SETTING(S)

Based on data extracted from MIMIC-III, a database of critically ill in the US between 2001 and 2012, the results were analyzed.

PARTICIPANTS

We calculated patients' age using admission time and date of birth information from the MIMIC dataset. Patients under 18 or over 89 years old, or who died in the hospital, were excluded. We analyzed 16,973 nursing records from patients' ICU stays.

METHODS

We have developed a technology called the Crucial Nursing Description Extractor (CNDE), which extracts key content from text. We use the logarithmic likelihood ratio to extract keywords and combine BioClinicalBERT. We predict the survival of discharged patients after six months and two years and evaluate the performance of the model using precision, recall, the F1-score, the receiver operating characteristic curve (ROC curve), the area under the curve (AUC), and the precision-recall curve (PR curve).

RESULTS

The research findings indicate that NurnaNet achieved good F1-scores (0.67030, 0.70874) within six months and two years. Compared to using BioClinicalBERT alone, there was an improvement in performance of 2.05 % and 1.08 % for predictions within six months and two years, respectively.

CONCLUSIONS

CNDE can effectively reduce long-form records and extract key content. NurnaNet has a good F1-score in analyzing the data of nursing records, which helps to identify the risk of death of patients after leaving the hospital and adjust the regular follow-up and treatment plan of relevant medical care as soon as possible.

The impact of large language models on radiology: a guide for radiologists on the latest innovations in AI

The advent of Deep Learning (DL) has significantly propelled the field of diagnostic radiology forward by enhancing image analysis and interpretation. The introduction of the Transformer architecture, followed by the development of Large Language Models (LLMs), has further revolutionized this domain. LLMs now possess the potential to automate and refine the radiology workflow, extending from report generation to assistance in diagnostics and patient care. The integration of multimodal technology with LLMs could potentially leapfrog these applications to unprecedented levels.However, LLMs come with unresolved challenges such as information hallucinations and biases, which can affect clinical reliability. Despite these issues, the legislative and guideline frameworks have yet to catch up with technological advancements. Radiologists must acquire a thorough understanding of these technologies to leverage LLMs' potential to the fullest while maintaining medical safety and ethics. This review aims to aid in that endeavor.

Early identification of suspected serious infection among patients afebrile at initial presentation using neural network models and natural language processing: A development and external validation study in the emergency department

OBJECTIVE

To develop and externally validate models based on neural networks and natural language processing (NLP) to identify suspected serious infections in emergency department (ED) patients afebrile at initial presentation.

METHODS

This retrospective study included adults who visited the ED afebrile at initial presentation. We developed four models based on artificial neural networks to identify suspected serious infection. Patient demographics, vital signs, laboratory test results and information extracted from initial ED physician notes using term frequency-inverse document frequency were used as model variables. Models were trained and internally validated with data from one hospital and externally validated using data from a different hospital. Model discrimination was evaluated using area under the receiver operating characteristic curve (AUC) and 95% confidence intervals (CIs).

RESULTS

The training, internal validation, and external validation datasets comprised 150,699, 37,675, and 85,098 patients, respectively. The AUCs (95% CIs) for Models 1 (demographics + vital signs), 2 (demographics + vital signs + initial ED physician note), 3 (demographics + vital signs + laboratory tests), and 4 (demographics + vital signs + laboratory tests + initial ED physician note) in the internal validation dataset were 0.789 (0.782-0.796), 0.867 (0.862-0.872), 0.881 (0.876-0.887), and 0.911 (0.906-0.915), respectively. In the external validation dataset, the AUCs (95% CIs) of Models 1, 2, 3, and 4 were 0.824 (0.817-0.830), 0.895 (0.890-0.899), 0.879 (0.873-0.884), and 0.913 (0.909-0.917), respectively. Model 1 can be utilized immediately after ED triage, Model 2 can be utilized after the initial physician notes are recorded (median time from ED triage: 28 min), and Models 3 and 4 can be utilized after the initial laboratory tests are reported (median time from ED triage: 68 min).

CONCLUSIONS

We developed and validated models to identify suspected serious infection in the ED. Extracted information from initial ED physician notes using NLP contributed to increased model performance, permitting identification of suspected serious infection at early stages of ED visits.

Question-answering system extracts information on injection drug use from clinical notes

BACKGROUND

Injection drug use (IDU) can increase mortality and morbidity. Therefore, identifying IDU early and initiating harm reduction interventions can benefit individuals at risk. However, extracting IDU behaviors from patients' electronic health records (EHR) is difficult because there is no other structured data available, such as International Classification of Disease (ICD) codes, and IDU is most often documented in unstructured free-text clinical notes. Although natural language processing can efficiently extract this information from unstructured data, there are no validated tools.

METHODS

To address this gap in clinical information, we design a question-answering (QA) framework to extract information on IDU from clinical notes for use in clinical operations. Our framework involves two main steps: (1) generating a gold-standard QA dataset and (2) developing and testing the QA model. We use 2323 clinical notes of 1145 patients curated from the US Department of Veterans Affairs (VA) Corporate Data Warehouse to construct the gold-standard dataset for developing and evaluating the QA model. We also demonstrate the QA model's ability to extract IDU-related information from temporally out-of-distribution data.

RESULTS

Here, we show that for a strict match between gold-standard and predicted answers, the QA model achieves a 51.65% F1 score. For a relaxed match between the gold-standard and predicted answers, the QA model obtains a 78.03% F1 score, along with 85.38% Precision and 79.02% Recall scores. Moreover, the QA model demonstrates consistent performance when subjected to temporally out-of-distribution data.

CONCLUSIONS

Our study introduces a QA framework designed to extract IDU information from clinical notes, aiming to enhance the accurate and efficient detection of people who inject drugs, extract relevant information, and ultimately facilitate informed patient care.

Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions

With the advent of large language models (LLMs), the artificial intelligence revolution in medicine and radiology is now more tangible than ever. Every day, an increasingly large number of articles are published that utilize LLMs in radiology. To adopt and safely implement this new technology in the field, radiologists should be familiar with its key concepts, understand at least the technical basics, and be aware of the potential risks and ethical considerations that come with it. In this review article, the authors provide an overview of the LLMs that might be relevant to the radiology community and include a brief discussion of their short history, technical basics, ChatGPT, prompt engineering, potential applications in medicine and radiology, advantages, disadvantages and risks, ethical and regulatory considerations, and future directions.

Development of an artificial intelligence bacteremia prediction model and evaluation of its impact on physician predictions focusing on uncertainty

Prediction of bacteremia is a clinically important but challenging task. An artificial intelligence (AI) model has the potential to facilitate early bacteremia prediction, aiding emergency department (ED) physicians in making timely decisions and reducing unnecessary medical costs. In this study, we developed and externally validated a Bayesian neural network-based AI bacteremia prediction model (AI-BPM). We also evaluated its impact on physician predictive performance considering both AI and physician uncertainties using historical patient data. A retrospective cohort of 15,362 adult patients with blood cultures performed in the ED was used to develop the AI-BPM. The AI-BPM used structured and unstructured text data acquired during the early stage of ED visit, and provided both the point estimate and 95% confidence interval (CI) of its predictions. High AI-BPM uncertainty was defined as when the predetermined bacteremia risk threshold (5%) was included in the 95% CI of the AI-BPM prediction, and low AI-BPM uncertainty was when it was not included. In the temporal validation dataset (N = 8,188), the AI-BPM achieved area under the receiver operating characteristic curve (AUC) of 0.754 (95% CI 0.737-0.771), sensitivity of 0.917 (95% CI 0.897-0.934), and specificity of 0.340 (95% CI 0.330-0.351). In the external validation dataset (N = 7,029), the AI-BPM's AUC was 0.738 (95% CI 0.722-0.755), sensitivity was 0.927 (95% CI 0.909-0.942), and specificity was 0.319 (95% CI 0.307-0.330). The AUC of the post-AI physicians predictions (0.703, 95% CI 0.654-0.753) was significantly improved compared with that of the pre-AI predictions (0.639, 95% CI 0.585-0.693; p-value < 0.001) in the sampled dataset (N = 1,000). The AI-BPM especially improved the predictive performance of physicians in cases with high physician uncertainty (low subjective confidence) and low AI-BPM uncertainty. Our results suggest that the uncertainty of both the AI model and physicians should be considered for successful AI model implementation.

Integrating Structured and Unstructured EHR Data for Predicting Mortality by Machine Learning and Latent Dirichlet Allocation Method

An ICU is a critical care unit that provides advanced medical support and continuous monitoring for patients with severe illnesses or injuries. Predicting the mortality rate of ICU patients can not only improve patient outcomes, but also optimize resource allocation. Many studies have attempted to create scoring systems and models that predict the mortality of ICU patients using large amounts of structured clinical data. However, unstructured clinical data recorded during patient admission, such as notes made by physicians, is often overlooked. This study used the MIMIC-III database to predict mortality in ICU patients. In the first part of the study, only eight structured variables were used, including the six basic vital signs, the GCS, and the patient's age at admission. In the second part, unstructured predictor variables were extracted from the initial diagnosis made by physicians when the patients were admitted to the hospital and analyzed using Latent Dirichlet Allocation techniques. The structured and unstructured data were combined using machine learning methods to create a mortality risk prediction model for ICU patients. The results showed that combining structured and unstructured data improved the accuracy of the prediction of clinical outcomes in ICU patients over time. The model achieved an AUROC of 0.88, indicating accurate prediction of patient vital status. Additionally, the model was able to predict patient clinical outcomes over time, successfully identifying important variables. This study demonstrated that a small number of easily collectible structured variables, combined with unstructured data and analyzed using LDA topic modeling, can significantly improve the predictive performance of a mortality risk prediction model for ICU patients. These results suggest that initial clinical observations and diagnoses of ICU patients contain valuable information that can aid ICU medical and nursing staff in making important clinical decisions.

A comparative study of pretrained language models for long clinical text

OBJECTIVE

Clinical knowledge-enriched transformer models (eg, ClinicalBERT) have state-of-the-art results on clinical natural language processing (NLP) tasks. One of the core limitations of these transformer models is the substantial memory consumption due to their full self-attention mechanism, which leads to the performance degradation in long clinical texts. To overcome this, we propose to leverage long-sequence transformer models (eg, Longformer and BigBird), which extend the maximum input sequence length from 512 to 4096, to enhance the ability to model long-term dependencies in long clinical texts.

MATERIALS AND METHODS

Inspired by the success of long-sequence transformer models and the fact that clinical notes are mostly long, we introduce 2 domain-enriched language models, Clinical-Longformer and Clinical-BigBird, which are pretrained on a large-scale clinical corpus. We evaluate both language models using 10 baseline tasks including named entity recognition, question answering, natural language inference, and document classification tasks.

RESULTS

The results demonstrate that Clinical-Longformer and Clinical-BigBird consistently and significantly outperform ClinicalBERT and other short-sequence transformers in all 10 downstream tasks and achieve new state-of-the-art results.

DISCUSSION

Our pretrained language models provide the bedrock for clinical NLP using long texts. We have made our source code available at https://github.com/luoyuanlab/Clinical-Longformer, and the pretrained models available for public download at: https://huggingface.co/yikuan8/Clinical-Longformer.

CONCLUSION

This study demonstrates that clinical knowledge-enriched long-sequence transformers are able to learn long-term dependencies in long clinical text. Our methods can also inspire the development of other domain-enriched long-sequence transformers.

Triptychon: Usability evaluation and implementation of a web-based application for patients' lab and vital parameters

Background

A major challenge in healthcare is the interpretation of the constantly increasing amount of clinical data of interest to inpatients for diagnosis and therapy. It is vital to accurately structure and represent data from different sources to help clinicians make informed decisions.

Objective

We evaluated the usability of our tool 'Triptychon' - a three-part visualisation dashboard of essential patients' medical data provided by a direct overview of their hospitalisation information, laboratory, and vital parameters over time.

Methods

The study followed a cohort of 20 participants using the mixed-methods approach, including interviews and the usability questionnaires, Health Information Technology Usability Evaluation Scale (Health-ITUES), and User Experience Questionnaire (UEQ). The participant's interactions with the dashboard were also observed. A thematic analysis approach was applied to analyse qualitative data and the quantitative data's task completion time and success rates.

Results

The usability evaluation of the visualisation dashboard revealed issues relating to the terminology used in the user interface and colour coding in its left and middle panels. The Health-ITUES score was 3.72 (standard deviation (SD) = 1.0), and the UEQ score was 1.6 (SD = 0.74). The study demonstrated improvements in intuitive dashboard use and overall satisfaction with using the dashboard daily.

Conclusion

The Triptychon dashboard is a promising new tool for medical data presentation. We identified design and layout issues of the dashboard for improving its usability in routine clinical practice. According to users' feedback, the three panels on the dashboard provided a holistic view of a patient's hospital stay.

Predicting medical events and ICU requirements using a multimodal multiobjective transformer network

Effective utilization of premium hospital resources such as intensive care unit (ICU), operating theater (OT), mechanical ventilator, endotracheal tube, and so on plays a significant role in providing high-quality care to critically ill patients within reasonable costs. Non-availability of specialized resources can lead to dire consequences for such patients, and in the worst case, may even turn out to be fatal. However, these resources cannot be kept idle, as they are expensive to maintain. Therefore, one of the core functions of hospital management is targeted at planning and managing these critical resources in order to provide efficient and effective health-care services to the end-users. Predictive technologies play a big role in this. In this article, we present methods for predicting the length of stay in ICU as well as the need for critical interventions for a patient based on the vital signs, laboratory measurements, and the nursing notes of the patient prepared within the first 24 h of ICU stay. The model has been built and cross-validated on the publicly available Medical Information Mart for Intensive Care (MIMIC-III v1.4) data set. We show that the proposed model performs way better than most of the earlier models in the prediction of ICU stay, which had used patient vitals primarily. Experimental results also demonstrate the advantage of using a multiobjective model over independent models for the prediction of ICU stay and critical interventions. The proposed model uses Local Interpretable Model-agnostic Explanations (LIME) that help in identifying the features responsible for predictive decisions. This is very useful in building trust and confidence in the prediction model among clinical practitioners.

BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task

MOTIVATION

Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model's performance.

RESULTS

We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets-BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets.

AVAILABILITY AND IMPLEMENTATION

BioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Deep Learning Analysis of Polish Electronic Health Records for Diagnosis Prediction in Patients with Cardiovascular Diseases

Electronic health records naturally contain most of the medical information in the form of doctor’s notes as unstructured or semi-structured texts. Current deep learning text analysis approaches allow researchers to reveal the inner semantics of text information and even identify hidden consequences that can offer extra decision support to doctors. In the presented article, we offer a new automated analysis of Polish summary texts of patient hospitalizations. The presented models were found to be able to predict the final diagnosis with almost 70% accuracy based just on the patient’s medical history (only 132 words on average), with possible accuracy increases when adding further sentences from hospitalization results; even one sentence was found to improve the results by 4%, and the best accuracy of 78% was achieved with five extra sentences. In addition to detailed descriptions of the data and methodology, we present an evaluation of the analysis using more than 50,000 Polish cardiology patient texts and dive into a detailed error analysis of the approach. The results indicate that the deep analysis of just the medical history summary can suggest the direction of diagnosis with a high probability that can be further increased just by supplementing the records with further examination results.