Classification of neurologic outcomes from medical notes using natural language processing

Automated extraction of post-stroke functional outcomes from unstructured electronic health records

PURPOSE

Population level tracking of post-stroke functional outcomes is critical to guide interventions that reduce the burden of stroke-related disability. However, functional outcomes are often missing or documented in unstructured notes. We developed a natural language processing (NLP) model that reads electronic health records (EHR) notes to automatically determine the modified Rankin Scale (mRS).

METHOD

We included consecutive patients (⩾18 years) with acute stroke admitted to our center (2015-2024). mRS scores were obtained from the Get With the Guidelines registry and clinical notes (if documented), and used as the gold standard to compare against NLP-generated scores. We used text-based features from notes, along with age, sex, discharge status, and outpatient follow-up to train a logistic regression for prediction of good (0-2) versus poor (3-6) mRS, and a linear regression for the full range of mRS scores. The models were trained for prediction of mRS at hospital discharge and post-discharge. The models were externally validated in a dataset of patients with brain injuries from a different healthcare center.

FINDINGS

We included 5307 patients, 5006 in train and test and 301 in validation; average age was 69 (SD 15) and 65 (SD 17) years, respectively; 47% female. The logistic regression achieved an area under the receiver operating curve (AUROC) of 0.94 [CI 0.93-0.95] (test) and 0.94 [0.91-0.96] (validation), and the linear model a root mean squared error (RMSE) of 0.91 [0.87-0.94] (test) and 1.17 [1.06-1.28] (validation).

DISCUSSION AND CONCLUSION

The NLP-based model is suitable for use in large-scale phenotyping of stroke functional outcomes and population health research.

Advancing equity in breast cancer care: natural language processing for analysing treatment outcomes in under-represented populations

OBJECTIVE

The study aimed to develop natural language processing (NLP) algorithms to automate extracting patient-centred breast cancer treatment outcomes from clinical notes in electronic health records (EHRs), particularly for women from under-represented populations.

METHODS

The study used clinical notes from 2010 to 2021 from a tertiary hospital in the USA. The notes were processed through various NLP techniques, including vectorisation methods (term frequency-inverse document frequency (TF-IDF), Word2Vec, Doc2Vec) and classification models (support vector classification, K-nearest neighbours (KNN), random forest (RF)). Feature selection and optimisation through random search and fivefold cross-validation were also conducted.

RESULTS

The study annotated 100 out of 1000 clinical notes, using 970 notes to build the text corpus. TF-IDF and Doc2Vec combined with RF showed the highest performance, while Word2Vec was less effective. RF classifier demonstrated the best performance, although with lower recall rates, suggesting more false negatives. KNN showed lower recall due to its sensitivity to data noise.

DISCUSSION

The study highlights the significance of using NLP in analysing clinical notes to understand breast cancer treatment outcomes in under-represented populations. The TF-IDF and Doc2Vec models were more effective in capturing relevant information than Word2Vec. The study observed lower recall rates in RF models, attributed to the dataset's imbalanced nature and the complexity of clinical notes.

CONCLUSION

The study developed high-performing NLP pipeline to capture treatment outcomes for breast cancer in under-represented populations, demonstrating the importance of document-level vectorisation and ensemble methods in clinical notes analysis. The findings provide insights for more equitable healthcare strategies and show the potential for broader NLP applications in clinical settings.

Unlocking the Black Box? A Comprehensive Exploration of Large Language Models in Rehabilitation

ABSTRACT

Rehabilitation is a vital component of health care, aiming to restore function and improve the well-being of individuals with disabilities or injuries. Nevertheless, the rehabilitation process is often likened to a " black box ," with complexities that pose challenges for comprehensive analysis and optimization. The emergence of large language models offers promising solutions to better understand this " black box ." Large language models excel at comprehending and generating human-like text, making them valuable in the healthcare sector. In rehabilitation, healthcare professionals must integrate a wide range of data to create effective treatment plans, akin to selecting the best ingredients for the " black box. " Large language models enhance data integration, communication, assessment, and prediction.This article delves into the ground-breaking use of large language models as a tool to further understand the rehabilitation process. Large language models address current rehabilitation issues, including data bias, contextual comprehension, and ethical concerns. Collaboration with healthcare experts and rigorous validation is crucial when deploying large language models. Integrating large language models into rehabilitation yields insights into this intricate process, enhancing data-driven decision making, refining clinical practices, and predicting rehabilitation outcomes. Although challenges persist, large language models represent a significant stride in rehabilitation, underscoring the importance of ethical use and collaboration.

Large Language Models in Neurology Research and Future Practice

Recent advancements in generative artificial intelligence, particularly using large language models (LLMs), are gaining increased public attention. We provide a perspective on the potential of LLMs to analyze enormous amounts of data from medical records and gain insights on specific topics in neurology. In addition, we explore use cases for LLMs, such as early diagnosis, supporting patient and caregivers, and acting as an assistant for clinicians. We point to the potential ethical and technical challenges raised by LLMs, such as concerns about privacy and data security, potential biases in the data for model training, and the need for careful validation of results. Researchers must consider these challenges and take steps to address them to ensure that their work is conducted in a safe and responsible manner. Despite these challenges, LLMs offer promising opportunities for improving care and treatment of various neurologic disorders.

AI and machine learning in resuscitation: Ongoing research, new concepts, and key challenges

Aim

Artificial intelligence (AI) and machine learning (ML) are important areas of computer science that have recently attracted attention for their application to medicine. However, as techniques continue to advance and become more complex, it is increasingly challenging for clinicians to stay abreast of the latest research. This overview aims to translate research concepts and potential concerns to healthcare professionals interested in applying AI and ML to resuscitation research but who are not experts in the field.

Main text

We present various research including prediction models using structured and unstructured data, exploring treatment heterogeneity, reinforcement learning, language processing, and large-scale language models. These studies potentially offer valuable insights for optimizing treatment strategies and clinical workflows. However, implementing AI and ML in clinical settings presents its own set of challenges. The availability of high-quality and reliable data is crucial for developing accurate ML models. A rigorous validation process and the integration of ML into clinical practice is essential for practical implementation. We furthermore highlight the potential risks associated with self-fulfilling prophecies and feedback loops, emphasizing the importance of transparency, interpretability, and trustworthiness in AI and ML models. These issues need to be addressed in order to establish reliable and trustworthy AI and ML models.

Conclusion

In this article, we overview concepts and examples of AI and ML research in the resuscitation field. Moving forward, appropriate understanding of ML and collaboration with relevant experts will be essential for researchers and clinicians to overcome the challenges and harness the full potential of AI and ML in resuscitation.