Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review

From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality?

Deep learning (DL) has substantially enhanced natural language processing (NLP) in healthcare research. However, the increasing complexity of DL-based NLP necessitates transparent model interpretability, or at least explainability, for reliable decision-making. This work presents a thorough scoping review of explainable and interpretable DL in healthcare NLP. The term "eXplainable and Interpretable Artificial Intelligence" (XIAI) is introduced to distinguish XAI from IAI. Different models are further categorized based on their functionality (model-, input-, output-based) and scope (local, global). Our analysis shows that attention mechanisms are the most prevalent emerging IAI technique. The use of IAI is growing, distinguishing it from XAI. The major challenges identified are that most XIAI does not explore "global" modelling processes, the lack of best practices, and the lack of systematic evaluation and benchmarks. One important opportunity is to use attention mechanisms to enhance multi-modal XIAI for personalized medicine. Additionally, combining DL with causal logic holds promise. Our discussion encourages the integration of XIAI in Large Language Models (LLMs) and domain-specific smaller models. In conclusion, XIAI adoption in healthcare requires dedicated in-house expertise. Collaboration with domain experts, end-users, and policymakers can lead to ready-to-use XIAI methods across NLP and medical tasks. While challenges exist, XIAI techniques offer a valuable foundation for interpretable NLP algorithms in healthcare.

Using real-world electronic health record data to predict the development of 12 cancer-related symptoms in the context of multimorbidity

Objective

This study uses electronic health record (EHR) data to predict 12 common cancer symptoms, assessing the efficacy of machine learning (ML) models in identifying symptom influencers.

Materials and Methods

We analyzed EHR data of 8156 adults diagnosed with cancer who underwent cancer treatment from 2017 to 2020. Structured and unstructured EHR data were sourced from the Enterprise Data Warehouse for Research at the University of Iowa Hospital and Clinics. Several predictive models, including logistic regression, random forest (RF), and XGBoost, were employed to forecast symptom development. The performances of the models were evaluated by F1-score and area under the curve (AUC) on the testing set. The SHapley Additive exPlanations framework was used to interpret these models and identify the predictive risk factors associated with fatigue as an exemplar.

Results

The RF model exhibited superior performance with a macro average AUC of 0.755 and an F1-score of 0.729 in predicting a range of cancer-related symptoms. For instance, the RF model achieved an AUC of 0.954 and an F1-score of 0.914 for pain prediction. Key predictive factors identified included clinical history, cancer characteristics, treatment modalities, and patient demographics depending on the symptom. For example, the odds ratio (OR) for fatigue was significantly influenced by allergy (OR = 2.3, 95% CI: 1.8-2.9) and colitis (OR = 1.9, 95% CI: 1.5-2.4).

Discussion

Our research emphasizes the critical integration of multimorbidity and patient characteristics in modeling cancer symptoms, revealing the considerable influence of chronic conditions beyond cancer itself.

Conclusion

We highlight the potential of ML for predicting cancer symptoms, suggesting a pathway for integrating such models into clinical systems to enhance personalized care and symptom management.

Inductive thematic analysis of healthcare qualitative interviews using open-source large language models: How does it compare to traditional methods?

BACKGROUND

Large language models (LLMs) are generative artificial intelligence that have ignited much interest and discussion about their utility in clinical and research settings. Despite this interest there is sparse analysis of their use in qualitative thematic analysis comparing their current ability to that of human coding and analysis. In addition, there has been no published analysis of their use in real-world, protected health information.

OBJECTIVE

Here we fill that gap in the literature by comparing an LLM to standard human thematic analysis in real-world, semi-structured interviews of both patients and clinicians within a psychiatric setting.

METHODS

Using a 70 billion parameter open-source LLM running on local hardware and advanced prompt engineering techniques, we produced themes that summarized a full corpus of interviews in minutes. Subsequently we used three different evaluation methods for quantifying similarity between themes produced by the LLM and those produced by humans.

RESULTS

These revealed similarities ranging from moderate to substantial (Jaccard similarity coefficients 0.44-0.69), which are promising preliminary results.

CONCLUSION

Our study demonstrates that open-source LLMs can effectively generate robust themes from qualitative data, achieving substantial similarity to human-generated themes. The validation of LLMs in thematic analysis, coupled with evaluation methodologies, highlights their potential to enhance and democratize qualitative research across diverse fields.

Methods for pragmatic randomized clinical trials of pain therapies: IMMPACT statement

ABSTRACT

Pragmatic, randomized, controlled trials hold the potential to directly inform clinical decision making and health policy regarding the treatment of people experiencing pain. Pragmatic trials are designed to replicate or are embedded within routine clinical care and are increasingly valued to bridge the gap between trial research and clinical practice, especially in multidimensional conditions, such as pain and in nonpharmacological intervention research. To maximize the potential of pragmatic trials in pain research, the careful consideration of each methodological decision is required. Trials aligned with routine practice pose several challenges, such as determining and enrolling appropriate study participants, deciding on the appropriate level of flexibility in treatment delivery, integrating information on concomitant treatments and adherence, and choosing comparator conditions and outcome measures. Ensuring data quality in real-world clinical settings is another challenging goal. Furthermore, current trials in the field would benefit from analysis methods that allow for a differentiated understanding of effects across patient subgroups and improved reporting of methods and context, which is required to assess the generalizability of findings. At the same time, a range of novel methodological approaches provide opportunities for enhanced efficiency and relevance of pragmatic trials to stakeholders and clinical decision making. In this study, best-practice considerations for these and other concerns in pragmatic trials of pain treatments are offered and a number of promising solutions discussed. The basis of these recommendations was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) meeting organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks.

Analyzing pain patterns in the emergency department: Leveraging clinical text deep learning models for real-world insights

OBJECTIVE

To determine the incidence of patients presenting in pain to a large Australian inner-city emergency department (ED) using a clinical text deep learning algorithm.

MATERIALS AND METHODS

A fine-tuned, domain-specific, transformer-based clinical text deep learning model was used to interpret free-text nursing assessments in the electronic medical records of 235,789 adult presentations to the ED over a three-year period. The model classified presentations according to whether the patient had pain on arrival at the ED. Interrupted time series analysis was used to determine the incidence of pain in patients on arrival over time. We described the changes in the population characteristics and incidence of patients with pain on arrival occurring with the start of the Covid-19 pandemic.

RESULTS

55.16% (95%CI 54.95%-55.36%) of all patients presenting to this ED had pain on arrival. There were differences in demographics and arrival and departure patterns between patients with and without pain. The Covid-19 pandemic initially precipitated a decrease followed by a sharp, sustained rise in pain on arrival, with concurrent changes to the population arriving in pain and their treatment.

DISCUSSION

Applying a clinical text deep learning model has successfully identified the incidence of pain on arrival. It represents an automated, reproducible mechanism to identify pain from routinely collected medical records. The description of this population and their treatment forms the basis of intervention to improve care for patients with pain. The combination of the clinical text deep learning models and interrupted time series analysis has reported on the effects of the Covid-19 pandemic on pain care in the ED, outlining a methodology to assess the impact of significant events or interventions on pain care in the ED.

CONCLUSION

Applying a novel deep learning approach to identifying pain guides methodological approaches to evaluating pain care interventions in the ED, giving previously unavailable population-level insights.

Impact of clinical note format on diagnostic accuracy and efficiency

BACKGROUND

Clinician notes are structured in a variety of ways. This research pilot tested an innovative study design and explored the impact of note formats on diagnostic accuracy and documentation review time.

OBJECTIVE

To compare two formats for clinical documentation (narrative format vs. list of findings) on clinician diagnostic accuracy and documentation review time.

METHOD

Participants diagnosed written clinical cases, half in narrative format, and half in list format. Diagnostic accuracy (defined as including correct case diagnosis among top three diagnoses) and time spent processing the case scenario were measured for each format. Generalised linear mixed regression models and bias-corrected bootstrap percentile confidence intervals for mean paired differences were used to analyse the primary research questions.

RESULTS

Odds of correctly diagnosing list format notes were 26% greater than with narrative notes. However, there is insufficient evidence that this difference is significant (75% CI 0.8-1.99). On average the list format notes required 85.6 more seconds to process and arrive at a diagnosis compared to narrative notes (95% CI -162.3, -2.77). Of cases where participants included the correct diagnosis, on average the list format notes required 94.17 more seconds compared to narrative notes (75% CI -195.9, -8.83).

CONCLUSION

This study offers note format considerations for those interested in improving clinical documentation and suggests directions for future research. Balancing the priority of clinician preference with value of structured data may be necessary.

IMPLICATIONS

This study provides a method and suggestive results for further investigation in usability of electronic documentation formats.

Toward Reliable Symptom Coding in Electronic Health Records for Symptom Assessment and Research: Identification and Categorization of International Classification of Diseases, Ninth Revision, Clinical Modification Symptom Codes

To date, symptom documentation has mostly relied on clinical notes in electronic health records or patient-reported outcomes using disease-specific symptom inventories. To provide a common and precise language for symptom recording, assessment, and research, a comprehensive list of symptom codes is needed. The International Classification of Diseases, Ninth Revision or its clinical modification ( International Classification of Diseases, Ninth Revision, Clinical Modification ) has a range of codes designated for symptoms, but it does not contain codes for all possible symptoms, and not all codes in that range are symptom related. This study aimed to identify and categorize the first list of International Classification of Diseases, Ninth Revision, Clinical Modification symptom codes for a general population and demonstrate their use to characterize symptoms of patients with type 2 diabetes mellitus in the Cerner database. A list of potential symptom codes was automatically extracted from the Unified Medical Language System Metathesaurus. Two clinical experts in symptom science and diabetes manually reviewed this list to identify and categorize codes as symptoms. A total of 1888 International Classification of Diseases, Ninth Revision, Clinical Modification symptom codes were identified and categorized into 65 categories. The symptom characterization using the newly obtained symptom codes and categories was found to be more reasonable than that using the previous symptom codes and categories on the same Cerner diabetes cohort.

Prediction of intra-abdominal injury using natural language processing of electronic medical record data

BACKGROUND

This study aimed to use natural language processing to predict the presence of intra-abdominal injury using unstructured data from electronic medical records.

METHODS

This was a random-sample retrospective observational cohort study leveraging unstructured data from injured patients taken to one of 9 acute care hospitals in an integrated health system between 2015 and 2021. Patients with International Classification of Diseases External Cause of Morbidity codes were identified. History and physical, consult, progress, and radiology report text from the first 8 hours of care were abstracted. Annotator dyads independently annotated encounters' text files to establish ground truth regarding whether intra-abdominal injury occurred. Features were extracted from text using natural language processing techniques, bag of words, and principal component analysis. We tested logistic regression, random forests, and gradient boosting machine to determine accuracy, recall, and precision of natural language processing to predict intra-abdominal injury.

RESULTS

A random sample of 7,000 patient encounters of 177,127 was annotated. Only 2,951 had sufficient information to determine whether an intra-abdominal injury was present. Among those, 84 (2.9%) had an intra-abdominal injury. The concordance between annotators was 0.989. Logistic regression of features identified with bag of words and principal component analysis had the best predictive ability, with an area under the receiver operating characteristic curve of 0.9, recall of 0.73, and precision of 0.17. Text features with greatest importance included "abdomen," "pelvis," "spleen," and "hematoma."

CONCLUSION

Natural language processing could be a screening decision support tool, which, if paired with human clinical assessment, can maximize precision of intra-abdominal injury identification.

Natural Language Processing Accurately Differentiates Cancer Symptom Information in Electronic Health Record Narratives

PURPOSE

Identifying cancer symptoms in electronic health record (EHR) narratives is feasible with natural language processing (NLP). However, more efficient NLP systems are needed to detect various symptoms and distinguish observed symptoms from negated symptoms and medication-related side effects. We evaluated the accuracy of NLP in (1) detecting 14 symptom groups (ie, pain, fatigue, swelling, depressed mood, anxiety, nausea/vomiting, pruritus, headache, shortness of breath, constipation, numbness/tingling, decreased appetite, impaired memory, disturbed sleep) and (2) distinguishing observed symptoms in EHR narratives among patients with cancer.

METHODS

We extracted 902,508 notes for 11,784 unique patients diagnosed with cancer and developed a gold standard corpus of 1,112 notes labeled for presence or absence of 14 symptom groups. We trained an embeddings-augmented NLP system integrating human and machine intelligence and conventional machine learning algorithms. NLP metrics were calculated on a gold standard corpus subset for testing.

RESULTS

The interannotator agreement for labeling the gold standard corpus was excellent at 92%. The embeddings-augmented NLP model achieved the best performance (F1 score = 0.877). The highest NLP accuracy was observed in pruritus (F1 score = 0.937) while the lowest accuracy was in swelling (F1 score = 0.787). After classifying the entire data set with embeddings-augmented NLP, we found that 41% of the notes included symptom documentation. Pain was the most documented symptom (29% of all notes) while impaired memory was the least documented (0.7% of all notes).

CONCLUSION

We illustrated the feasibility of detecting 14 symptom groups in EHR narratives and showed that an embeddings-augmented NLP system outperforms conventional machine learning algorithms in detecting symptom information and differentiating observed symptoms from negated symptoms and medication-related side effects.

Towards the automatic calculation of the EQUAL Candida Score: Extraction of CVC-related information from EMRs of critically ill patients with candidemia in Intensive Care Units

OBJECTIVES

Candidemia is the most frequent invasive fungal disease and the fourth most frequent bloodstream infection in hospitalized patients. Its optimal management is crucial for improving patients' survival. The quality of candidemia management can be assessed with the EQUAL Candida Score. The objective of this work is to support its automatic calculation by extracting central venous catheter-related information from Italian text in clinical notes of electronic medical records.

MATERIALS AND METHODS

The sample includes 4,787 clinical notes of 108 patients hospitalized between January 2018 to December 2020 in the Intensive Care Units of the IRCCS San Martino Polyclinic Hospital in Genoa (Italy). The devised pipeline exploits natural language processing (NLP) to produce numerical representations of clinical notes used as input of machine learning (ML) algorithms to identify CVC presence and removal. It compares the performances of (i) rule-based method, (ii) count-based method together with a ML algorithm, and (iii) a transformers-based model.

RESULTS

Results, obtained with three different approaches, were evaluated in terms of weighted F1 Score. The random forest classifier showed the higher performance in both tasks reaching 82.35%.

CONCLUSION

The present work constitutes a first step towards the automatic calculation of the EQUAL Candida Score from unstructured daily collected data by combining ML and NLP methods. The automatic calculation of the EQUAL Candida Score could provide crucial real-time feedback on the quality of candidemia management, aimed at further improving patients' health.

Improving biomedical entity linking for complex entity mentions with LLM-based text simplification

Large amounts of important medical information are captured in free-text documents in biomedical research and within healthcare systems, which can be made accessible through natural language processing (NLP). A key component in most biomedical NLP pipelines is entity linking, i.e. grounding textual mentions of named entities to a reference of medical concepts, usually derived from a terminology system, such as the Systematized Nomenclature of Medicine Clinical Terms. However, complex entity mentions, spanning multiple tokens, are notoriously hard to normalize due to the difficulty of finding appropriate candidate concepts. In this work, we propose an approach to preprocess such mentions for candidate generation, building upon recent advances in text simplification with generative large language models. We evaluate the feasibility of our method in the context of the entity linking track of the BioCreative VIII SympTEMIST shared task. We find that instructing the latest Generative Pre-trained Transformer model with a few-shot prompt for text simplification results in mention spans that are easier to normalize. Thus, we can improve recall during candidate generation by 2.9 percentage points compared to our baseline system, which achieved the best score in the original shared task evaluation. Furthermore, we show that this improvement in recall can be fully translated into top-1 accuracy through careful initialization of a subsequent reranking model. Our best system achieves an accuracy of 63.6% on the SympTEMIST test set. The proposed approach has been integrated into the open-source xMEN toolkit, which is available online via https://github.com/hpi-dhc/xmen.

Risk factors for prediabetes in community-dwelling adults: A generalized estimating equation logistic regression approach with natural language processing insights

The global prevalence of prediabetes is expected to reach 8.3% (587 million people) by 2045, with 70% of people with prediabetes developing diabetes during their lifetimes. We aimed to classify community-dwelling adults with a high risk for prediabetes based on prediabetes-related symptoms and to identify their characteristics, which might be factors associated with prediabetes. We analyzed homecare nursing records (n = 26,840) of 1628 patients aged over 20 years. Using a natural language processing algorithm, we classified each nursing episode as either low-risk or high-risk for prediabetes based on the detected number and category of prediabetes-symptom words. To identify differences between the risk groups, we employed t-tests, chi-square tests, and data visualization. Risk factors for prediabetes were identified using multiple logistic regression models with generalized estimating equations. A total of 3270 episodes (12.18%) were classified as potentially high-risk for prediabetes. There were significant differences in the personal, social, and clinical factors between groups. Results revealed that female sex, age, cancer coverage as part of homecare insurance coverage, and family caregivers were significantly associated with an increased risk of prediabetes. Although prediabetes is not a life-threatening disease, uncontrolled blood glucose can cause unfavorable outcomes for other major diseases. Thus, medical professionals should consider the associated symptoms and risk factors of prediabetes. Moreover, the proposed algorithm may support the detection of individuals at a high risk for prediabetes. Implementing this approach could facilitate proactive monitoring and early intervention, leading to reduced healthcare expenses and better health outcomes for community-dwelling adults.

Advancing equity in breast cancer care: natural language processing for analysing treatment outcomes in under-represented populations

OBJECTIVE

The study aimed to develop natural language processing (NLP) algorithms to automate extracting patient-centred breast cancer treatment outcomes from clinical notes in electronic health records (EHRs), particularly for women from under-represented populations.

METHODS

The study used clinical notes from 2010 to 2021 from a tertiary hospital in the USA. The notes were processed through various NLP techniques, including vectorisation methods (term frequency-inverse document frequency (TF-IDF), Word2Vec, Doc2Vec) and classification models (support vector classification, K-nearest neighbours (KNN), random forest (RF)). Feature selection and optimisation through random search and fivefold cross-validation were also conducted.

RESULTS

The study annotated 100 out of 1000 clinical notes, using 970 notes to build the text corpus. TF-IDF and Doc2Vec combined with RF showed the highest performance, while Word2Vec was less effective. RF classifier demonstrated the best performance, although with lower recall rates, suggesting more false negatives. KNN showed lower recall due to its sensitivity to data noise.

DISCUSSION

The study highlights the significance of using NLP in analysing clinical notes to understand breast cancer treatment outcomes in under-represented populations. The TF-IDF and Doc2Vec models were more effective in capturing relevant information than Word2Vec. The study observed lower recall rates in RF models, attributed to the dataset's imbalanced nature and the complexity of clinical notes.

CONCLUSION

The study developed high-performing NLP pipeline to capture treatment outcomes for breast cancer in under-represented populations, demonstrating the importance of document-level vectorisation and ensemble methods in clinical notes analysis. The findings provide insights for more equitable healthcare strategies and show the potential for broader NLP applications in clinical settings.

Does nurse use of a standardized flowsheet to document communication with advanced providers provide a mechanism to detect pulse oximetry failures? A retrospective study of electronic health record data

BACKGROUND

Pulse oximetry guides clinical decisions, yet does not uniformly identify hypoxemia. We hypothesized that nursing documentation of notifying providers, facilitated by a standardized flowsheet for documenting communication to providers (physicians, nurse practitioners, and physician assistants), may increase when hypoxemia is present, but undetected by the pulse oximeter, in events termed "occult hypoxemia."

OBJECTIVE

To compare nurse documentation of provider notification in the 4 h preceding cases of occult hypoxemia, normal oxygenation, and evident hypoxemia confirmed by an arterial blood gas reading.

METHODS

We conducted a retrospective study using electronic health record data from patients with COVID-19 at five hospitals in a healthcare system with paired SpO2 and SaO2 readings (measurements within 10 min of oxygen saturation levels in arterial blood, SaO2, and by pulse oximetry, SpO2). We applied multivariate logistic regression to assess if having any nursing documentation of provider notification in the 4 h prior to a paired reading confirming occult hypoxemia was more likely compared to a paired reading confirming normal oxygen status, adjusting for characteristics significantly associated with nursing documentation. We applied conditional logistic regression to assess if having any nursing documentation of provider notification was more likely in the 4-hour window preceding a paired reading compared to the 4-hour window 24 h earlier separately for occult hypoxemia, visible hypoxemia, and normal oxygenation.

RESULTS

There were data from 1910 patients hospitalized with COVID-19 who had 44,972 paired readings and an average of 26.5 (34.5) nursing documentation of provider notification events. The mean age was 63.4 (16.2). Almost half (866/1910, 45.3 %) were White, 701 (36.7 %) were Black, and 239 (12.5 %) were Hispanic. Having any nursing documentation of provider notification was 46 % more common in the 4 h before an occult hypoxemia paired reading compared to a normal oxygen status paired reading (OR 1.46, 95 % CI: 1.28-1.67). Comparing the 4 h immediately before the reading to the 4 h one day preceding the paired reading, there was a higher likelihood of having any nursing documentation of provider notification for both evident (OR 1.45, 95 % CI 1.24-1.68) and occult paired readings (OR 1.26, 95 % CI 1.04-1.53).

CONCLUSION

This study finds that nursing documentation of provider notification significantly increases prior to confirmed occult hypoxemia, which has potential in proactively identifying occult hypoxemia and other clinical issues. There is potential value to encouraging standardized documentation of nurse concern, including communication to providers, to facilitate its inclusion in clinical decision-making.

A Framework for Developing and Assessing Custom Case Definitions: A Demonstration Applied to Opioid Overdose in Maryland

CONTEXT

Public health epidemiologists monitor data sources for disease outbreaks and other events of public health concern, but manual review of records to identify cases of interest is slow and labor-intensive and may not reflect evolving data practices. To automatically identify cases from electronic data sources, epidemiologists must use "case definitions" or formal logic that captures the criteria used to identify a record as a case of interest.

OBJECTIVE

To establish a methodology for development and evaluation of case definitions. A logical evaluation framework to approach case definitions will allow jurisdictions the flexibility to implement a case definition tailored to their goals and available data.

DESIGN

Case definition development is explained as a process with multiple logical components combining free-text and categorical data fields. The process is illustrated with the development of a case definition to identify emergency medical services (EMS) call records related to opioid overdoses in Maryland.

SETTING

The Maryland Department of Health (MDH) installation of the Electronic Surveillance System for Early Notification of Community-Based Epidemics (ESSENCE), which began capturing EMS call records in ESSENCE in 2019 to improve statewide coverage of all-hazards health issues.

RESULTS

We describe a case definition evaluation framework and demonstrate its application through development of an opioid overdose case definition to be used in MDH ESSENCE. We show the iterative process of development, from defining how a case can be identified conceptually to examining each component of the conceptual definition and then exploring how to capture that component using available data.

CONCLUSION

We present a framework for developing and qualitatively assessing case definitions and demonstrate an application of the framework to identifying opioid overdose incidents from MDH EMS data. We discuss guidelines to support jurisdictions in applying this framework to their own data and public health challenges to improve local surveillance capability.

Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review

Objective

Natural language processing (NLP) can enhance research on activities of daily living (ADL) by extracting structured information from unstructured electronic health records (EHRs) notes. This review aims to give insight into the state-of-the-art, usability, and performance of NLP systems to extract information on ADL from EHRs.

Materials and Methods

A systematic review was conducted based on searches in Pubmed, Embase, Cinahl, Web of Science, and Scopus. Studies published between 2017 and 2022 were selected based on predefined eligibility criteria.

Results

The review identified 22 studies. Most studies (65%) used NLP for classifying unstructured EHR data on 1 or 2 ADL. Deep learning, combined with a ruled-based method or machine learning, was the approach most commonly used. NLP systems varied widely in terms of the pre-processing and algorithms. Common performance evaluation methods were cross-validation and train/test datasets, with F1, precision, and sensitivity as the most frequently reported evaluation metrics. Most studies reported relativity high overall scores on the evaluation metrics.

Discussion

NLP systems are valuable for the extraction of unstructured EHR data on ADL. However, comparing the performance of NLP systems is difficult due to the diversity of the studies and challenges related to the dataset, including restricted access to EHR data, inadequate documentation, lack of granularity, and small datasets.

Conclusion

This systematic review indicates that NLP is promising for deriving information on ADL from unstructured EHR notes. However, what the best-performing NLP system is, depends on characteristics of the dataset, research question, and type of ADL.

Collecting routine and timely cancer stage at diagnosis by implementing a cancer staging tiered framework: the Western Australian Cancer Registry experience

BACKGROUND

Current processes collecting cancer stage data in population-based cancer registries (PBCRs) lack standardisation, resulting in difficulty utilising diverse data sources and incomplete, low-quality data. Implementing a cancer staging tiered framework aims to improve stage collection and facilitate inter-PBCR benchmarking.

OBJECTIVE

Demonstrate the application of a cancer staging tiered framework in the Western Australian Cancer Staging Project to establish a standardised method for collecting cancer stage at diagnosis data in PBCRs.

METHODS

The tiered framework, developed in collaboration with a Project Advisory Group and applied to breast, colorectal, and melanoma cancers, provides business rules - procedures for stage collection. Tier 1 represents the highest staging level, involving complete American Joint Committee on Cancer (AJCC) tumour-node-metastasis (TNM) data collection and other critical staging information. Tier 2 (registry-derived stage) relies on supplementary data, including hospital admission data, to make assumptions based on data availability. Tier 3 (pathology stage) solely uses pathology reports.

FINDINGS

The tiered framework promotes flexible utilisation of staging data, recognising various levels of data completeness. Tier 1 is suitable for all purposes, including clinical and epidemiological applications. Tiers 2 and 3 are recommended for epidemiological analysis alone. Lower tiers provide valuable insights into disease patterns, risk factors, and overall disease burden for public health planning and policy decisions. Capture of staging at each tier depends on data availability, with potential shifts to higher tiers as new data sources are acquired.

CONCLUSIONS

The tiered framework offers a dynamic approach for PBCRs to record stage at diagnosis, promoting consistency in population-level staging data and enabling practical use for benchmarking across jurisdictions, public health planning, policy development, epidemiological analyses, and assessing cancer outcomes. Evolution with staging classifications and data variable changes will futureproof the tiered framework. Its adaptability fosters continuous refinement of data collection processes and encourages improvements in data quality.

Natural Language Processing Application in Nursing Research: A Study Using Text Network Analysis and Topic Modeling

Although the potential of natural language processing and an increase in its application in nursing research is evident, there is a lack of understanding of the research trends. This study conducts text network analysis and topic modeling to uncover the underlying knowledge structures, research trends, and emergent research themes within nursing literature related to natural language processing. In addition, this study aims to provide a foundation for future scholarly inquiries and enhance the integration of natural language processing in the analysis of nursing research. We analyzed 443 literature abstracts and performed core keyword analysis and topic modeling based on frequency and centrality. The following topics emerged: (1) Term Identification and Communication; (2) Application of Machine Learning; (3) Exploration of Health Outcome Factors; (4) Intervention and Participant Experience; and (5) Disease-Related Algorithms. Nursing meta-paradigm elements were identified within the core keyword analysis, which led to understanding and expanding the meta-paradigm. Although still in its infancy in nursing research with limited topics and research volumes, natural language processing can potentially enhance research efficiency and nursing quality. The findings emphasize the possibility of integrating natural language processing in nursing-related subjects, validating nursing value, and fostering the exploration of essential paradigms in nursing science.

Clinical indicators to monitor health care in low back pain: a scoping review

Clinical care indicators for low back pain can be used to monitor healthcare practices and consequently be used to evaluate success of strategies to improve care quality. The aim of this study was to identify the clinical care indicators that have been used to measure appropriateness of health care for patients with low back pain. We conducted a systematic search of five electronic databases and Google to identify clinical care indicators that have been used to measure any aspect of care for people with low back pain. Care indicators were narratively described according to their type (i.e. structure, process, or outcomes) and categorized by their purpose (e.g. to measure aspects related to assessment, imaging requests, treatment/prevention, and outcomes). A total of 3562 and 2180 records were retrieved from electronic databases and Google searches, respectively. We identified 280 indicators related to low back pain care from 40 documents and publications. Most quality indicators were process indicators (n = 213, 76%), followed by structure (n = 41, 15%) and outcome indicators (n = 26, 9%). The most common indicators were related to imaging requests (n = 41, 15%), referral to healthcare providers (n = 30, 11%), and shared decision-making (n = 21, 7%). Our review identified a range of clinical care indicators that have been used to measure the quality of health care for people with low back pain. Our findings will support a Delphi study to reach international consensus on what would be the most important and feasible indicators for a minimum dataset to be collected globally.

Applications of natural language processing tools in the surgical journey

Background

Natural language processing tools are becoming increasingly adopted in multiple industries worldwide. They have shown promising results however their use in the field of surgery is under-recognised. Many trials have assessed these benefits in small settings with promising results before large scale adoption can be considered in surgery. This study aims to review the current research and insights into the potential for implementation of natural language processing tools into surgery.

Methods

A narrative review was conducted following a computer-assisted literature search on Medline, EMBASE and Google Scholar databases. Papers related to natural language processing tools and consideration into their use for surgery were considered.

Results

Current applications of natural language processing tools within surgery are limited. From the literature, there is evidence of potential improvement in surgical capability and service delivery, such as through the use of these technologies to streamline processes including surgical triaging, data collection and auditing, surgical communication and documentation. Additionally, there is potential to extend these capabilities to surgical academia to improve processes in surgical research and allow innovation in the development of educational resources. Despite these outcomes, the evidence to support these findings are challenged by small sample sizes with limited applicability to broader settings.

Conclusion

With the increasing adoption of natural language processing technology, such as in popular forms like ChatGPT, there has been increasing research in the use of these tools within surgery to improve surgical workflow and efficiency. This review highlights multifaceted applications of natural language processing within surgery, albeit with clear limitations due to the infancy of the infrastructure available to leverage these technologies. There remains room for more rigorous research into broader capability of natural language processing technology within the field of surgery and the need for cross-sectoral collaboration to understand the ways in which these algorithms can best be integrated.

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

Background

There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date.

Objective

We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey.

Methods

Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis.

Results

The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve > 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI.

Conclusion

Mining EHR notes with NLP holds promise for improving early IEI patient detection.

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

Characterizing atrial fibrillation symptom improvement following de novo catheter ablation

AIMS

Atrial fibrillation (AF) symptom relief is a primary indication for catheter ablation, but AF symptom resolution is not well characterized. The study objective was to describe AF symptom documentation in electronic health records (EHRs) pre- and post-ablation and identify correlates of post-ablation symptoms.

METHODS AND RESULTS

We conducted a retrospective cohort study using EHRs of patients with AF (n = 1293), undergoing ablation in a large, urban health system from 2010 to 2020. We extracted symptom data from clinical notes using a natural language processing algorithm (F score: 0.81). We used Cochran's Q tests with post-hoc McNemar's tests to determine differences in symptom prevalence pre- and post-ablation. We used logistic regression models to estimate the adjusted odds of symptom resolution by personal or clinical characteristics at 6 and 12 months post-ablation. In fully adjusted models, at 12 months post-ablation patients, patients with heart failure had significantly lower odds of dyspnoea resolution [odds ratio (OR) 0.38, 95% confidence interval (CI) 0.25-0.57], oedema resolution (OR 0.37, 95% CI 0.25-0.56), and fatigue resolution (OR 0.54, 95% CI 0.34-0.85), but higher odds of palpitations resolution (OR 1.90, 95% CI 1.25-2.89) compared with those without heart failure. Age 65 and older, female sex, Black or African American race, smoking history, and antiarrhythmic use were also associated with lower odds of resolution of specific symptoms at 6 and 12 months.

CONCLUSION

The post-ablation symptom patterns are heterogeneous. Findings warrant confirmation with larger, more representative data sets, which may be informative for patients whose primary goal for undergoing an ablation is symptom relief.

Using natural language processing to analyze unstructured patient-reported outcomes data derived from electronic health records for cancer populations: a systematic review

INTRODUCTION

Patient-reported outcomes (PROs; symptoms, functional status, quality-of-life) expressed in the 'free-text' or 'unstructured' format within clinical notes from electronic health records (EHRs) offer valuable insights beyond biological and clinical data for medical decision-making. However, a comprehensive assessment of utilizing natural language processing (NLP) coupled with machine learning (ML) methods to analyze unstructured PROs and their clinical implementation for individuals affected by cancer remains lacking.

AREAS COVERED

This study aimed to systematically review published studies that used NLP techniques to extract and analyze PROs in clinical narratives from EHRs for cancer populations. We examined the types of NLP (with and without ML) techniques and platforms for data processing, analysis, and clinical applications.

EXPERT OPINION

Utilizing NLP methods offers a valuable approach for processing and analyzing unstructured PROs among cancer patients and survivors. These techniques encompass a broad range of applications, such as extracting or recognizing PROs, categorizing, characterizing, or grouping PROs, predicting or stratifying risk for unfavorable clinical results, and evaluating connections between PROs and adverse clinical outcomes. The employment of NLP techniques is advantageous in converting substantial volumes of unstructured PRO data within EHRs into practical clinical utilities for individuals with cancer.

Design and Development of a Web-Based Prospective Nationwide Registry for Ocular Inflammatory Diseases: UVEITE.PT - The Portuguese Ocular Inflammation Registry

Uveitis is a heterogeneous collection of infrequent diseases, which poses significant challenges to cost-effective research in the field. Medical registries are being increasingly recognized as crucial tools to provide high-quality data, thus enabling prospective clinical research. This paper describes the design and technical structure development of an innovative countrywide electronic medical record for uveitis, Uveite.pt, and gives an overview of the cohort registered since its foundation, March 2020.Uveite.pt is an electronic medical record platform developed by the Portuguese Ocular Inflammation Group (POIG), a scientific committee of the Portuguese Ophthalmology Society. This is a nationwide customized web-based platform for uveitis patients useful for both clinical practice and real-world-based research, working as a central repository and reporting tool for uveitis. This paper describes the technical principles, the design and the development of a web-based interoperable registry for uveitis in Portugal and provides an overview of more than 400 patients registered in the first 18 months since inception.In infrequent diseases, the existence of registries enables to gather evidence and increase research possibilities to clinicians. The adoption of this platform enables standardization and improvement of clinical practice in uveitis. It is useful to apprehend the repercussion of medical and surgical treatments in uveitis and scleritis, supporting clinicians in the strict monitoring of drug adverse reactions and surgical outcomes.

Information heterogeneity between progress notes by physicians and nurses for inpatients with digestive system diseases

This study focused on the heterogeneity in progress notes written by physicians or nurses. A total of 806 days of progress notes written by physicians or nurses from 83 randomly selected patients hospitalized in the Gastroenterology Department at Kagawa University Hospital from January to December 2021 were analyzed. We extracted symptoms as the International Classification of Diseases (ICD) Chapter 18 (R00-R99, hereinafter R codes) from each progress note using MedNER-J natural language processing software and counted the days one or more symptoms were extracted to calculate the extraction rate. The R-code extraction rate was significantly higher from progress notes by nurses than by physicians (physicians 68.5% vs. nurses 75.2%; p = 0.00112), regardless of specialty. By contrast, the R-code subcategory R10-R19 for digestive system symptoms (44.2 vs. 37.5%, respectively; p = 0.00299) and many chapters of ICD codes for disease names, as represented by Chapter 11 K00-K93 (68.4 vs. 30.9%, respectively; p < 0.001), were frequently extracted from the progress notes by physicians, reflecting their specialty. We believe that understanding the information heterogeneity of medical documents, which can be the basis of medical artificial intelligence, is crucial, and this study is a pioneering step in that direction.

MetaTron: advancing biomedical annotation empowering relation annotation and collaboration

BACKGROUND

The constant growth of biomedical data is accompanied by the need for new methodologies to effectively and efficiently extract machine-readable knowledge for training and testing purposes. A crucial aspect in this regard is creating large, often manually or semi-manually, annotated corpora vital for developing effective and efficient methods for tasks like relation extraction, topic recognition, and entity linking. However, manual annotation is expensive and time-consuming especially if not assisted by interactive, intuitive, and collaborative computer-aided tools. To support healthcare experts in the annotation process and foster annotated corpora creation, we present MetaTron. MetaTron is an open-source and free-to-use web-based annotation tool to annotate biomedical data interactively and collaboratively; it supports both mention-level and document-level annotations also integrating automatic built-in predictions. Moreover, MetaTron enables relation annotation with the support of ontologies, functionalities often overlooked by off-the-shelf annotation tools.

RESULTS

We conducted a qualitative analysis to compare MetaTron with a set of manual annotation tools including TeamTat, INCEpTION, LightTag, MedTAG, and brat, on three sets of criteria: technical, data, and functional. A quantitative evaluation allowed us to assess MetaTron performances in terms of time and number of clicks to annotate a set of documents. The results indicated that MetaTron fulfills almost all the selected criteria and achieves the best performances.

CONCLUSIONS

MetaTron stands out as one of the few annotation tools targeting the biomedical domain supporting the annotation of relations, and fully customizable with documents in several formats-PDF included, as well as abstracts retrieved from PubMed, Semantic Scholar, and OpenAIRE. To meet any user need, we released MetaTron both as an online instance and as a Docker image locally deployable.

Enhancing pain care with the American Pain Society Patient Outcome Questionnaire for use in the emergency department (APS-POQ-RED): validating a patient-reported outcome measure

BACKGROUND

In general, the quality of pain care in emergency departments (ED) is poor, despite up to 80% of all ED patients presenting with pain. This may be due to the lack of well-validated patient-reported outcome measures (PROMs) of pain care in the ED setting. The American Pain Society-Patient Outcome Questionnaire-Revised Edition (APS-POQ-R), with slight modification for ED patients, is a potentially useful PROM for the adult ED, however it is yet to be completely validated.

METHODS

Adult patients, who had presented with moderate to severe acute pain, were recruited at two large inner-city EDs in Australia. A modified version of the APS-POQ-R was administered at the completion of their ED care. Responses were randomly split into three groups and underwent multiple rounds of exploratory and confirmatory factor analysis with testing for construct, convergent, divergent validity and internal consistency.

RESULTS

A total of 646 ED patients (55.6% female), with a median age of 48.3 years, and moderate to severe pain on arrival, completed the ED-modified APS-POQ-R. Psychometric evaluation resulted in a reduced nine-question tool, which measures three constructs (pain relief and satisfaction (α=0.891), affective distress (α=0.823) and pain interference (α=0.908)) and demonstrated construct, convergent, divergent validity, and internal consistency.

CONCLUSIONS

This new tool, which we refer to as the American Pain Society-Patient Outcome Questionnaire-Revised for the ED (APS-POQ-RED), should form the basis for reporting patient-reported outcomes of ED pain care in future quality improvement and research.

Distinguishing cardiac catheter ablation energy modalities by applying natural language processing to electronic health records

Aim: Catheter ablation is used to treat symptomatic atrial fibrillation (AF) and is performed using either cryoballoon (CB) or radiofrequency (RF) ablation. There is limited real world data of CB and RF in the US as healthcare codes are agnostic of energy modality. An alternative method is to analyze patients' electronic health records (EHRs) using Optum's EHR database. Objective: To determine the feasibility of using patients' EHRs with natural language processing (NLP) to distinguish CB versus RF ablation procedures. Data Source: Optum® de-identified EHR dataset, Optum® Cardiac Ablation NLP Table. Methods: This was a retrospective analysis of existing de-identified EHR data. Medical codes were used to create an ablation validation table. Frequency analysis was used to assess ablation procedures and their associated note terms. Two cohorts were created (1) index procedures, (2) multiple procedures. Possible note term combinations included (1) cryoablation (2) radiofrequency (3) ablation, or (4) both. Results: Of the 40,810 validated cardiac ablations, 3777 (9%) index ablation procedures had available and matching NLP note terms. Of these, 22% (n = 844) were classified as ablation, 27% (n = 1016) as cryoablation, 49% (n = 1855) as radiofrequency ablation, and 1.6% (n = 62) as both. In the multiple procedures analysis, 5691 (14%) procedures had matching note terms. 24% (n = 1362) were classified as ablation, 27% as cryoablation, 47% as radiofrequency ablation, and 2% as both. Conclusion: NLP has potential to evaluate the frequency of cardiac ablation by type, however, for this to be a reliable real-world data source, mandatory data entry by providers and standardized electronic health reporting must occur.

Parkinson Disease Genetics Extended to African and Hispanic Ancestries in the VA Million Veteran Program

Background and Objectives

Nearly all genetic analyses of Parkinson disease (PD) have been in populations of European ancestry. We sought to test the ability of a machine learning method to extract accurate PD diagnoses from an electronic medical record (EMR) system, to see whether genetic variants identified in European populations generalize to individuals of African and Hispanic ancestries, and to compare the rates of PD across ancestries.

Methods

A machine learning method using natural language processing was applied to EMRs of US veterans participating in the VA Million Veteran Program (MVP) to identify individuals with PD. These putative cases were vetted via blind chart review by a movement disorder specialist. A polygenic risk score (PRS) of 90 established genetic variants whose genotypes were imputed from a customized Axiom Biobank Array was evaluated in different case groups.

Results

The EMR prediction scores had a distinct trimodal distribution, with 97% of the high group and only 30% of the middle group having a credible diagnosis of PD. Using the 3,542 cases from the high group matched 4:1 to controls, the PRS was highly predictive in individuals of European ancestry (n = 3,137 cases; OR = 1.82; p = 8.01E-48), and nearly identical effect sizes were seen in individuals of African (n = 184; OR = 2.07; p = 3.4E-4) and Hispanic ancestries (n = 221; OR = 2.13; p = 3.9E-6). The PRS was much less predictive for the 2,757 European ancestry cases who had an ICD code for PD but for whom the machine learning method had a lower confidence in their diagnosis. No novel ancestry-specific genetic variants were identified. Individuals with African ancestry had one-quarter the rate of PD compared with European or Hispanic ancestries aged 60-70 years and one half the rate in the 70-80 years age range. African American cases had a higher proportion of their DNA originating in Europe compared with African American controls.

Discussion

Machine learning can reliably classify PD using data from a large EMR. Larger studies of non-European populations are required to confirm the generalizability of PD risk variants identified in populations of European ancestry and the increased risk coming from a higher proportion of European DNA in African Americans.

Artificial Intelligence in Urology: Current Status and Future Perspectives

Surgical fields, especially urology, have shifted increasingly toward the use of artificial intelligence (AI). Advancements in AI have created massive improvements in diagnostics, outcome predictions, and robotic surgery. For robotic surgery to progress from assisting surgeons to eventually reaching autonomous procedures, there must be advancements in machine learning, natural language processing, and computer vision. Moreover, barriers such as data availability, interpretability of autonomous decision-making, Internet connection and security, and ethical concerns must be overcome.

Anesthesia Patient Monitoring 2050

The monitoring of vital signs in patients undergoing anesthesia began with the very first case of anesthesia and has evolved alongside the development of anesthesiology ever since. Patient monitoring started out as a manually performed, intermittent, and qualitative assessment of the patient's general well-being in the operating room. In its evolution, patient monitoring development has responded to the clinical need, for example, when critical incident studies in the 1980s found that many anesthesia adverse events could be prevented by improved monitoring, especially respiratory monitoring. It also facilitated and perhaps even enabled increasingly complex surgeries in increasingly higher-risk patients. For example, it would be very challenging to perform and provide anesthesia care during some of the very complex cardiovascular surgeries that are almost routine today without being able to simultaneously and reliably monitor multiple pressures in a variety of places in the circulatory system. Of course, anesthesia patient monitoring itself is enabled by technological developments in the world outside of the operating room. Throughout its history, anesthesia patient monitoring has taken advantage of advancements in material science (when nonthrombogenic polymers allowed the design of intravascular catheters, for example), in electronics and transducers, in computers, in displays, in information technology, and so forth. Slower product life cycles in medical devices mean that by carefully observing technologies such as consumer electronics, including user interfaces, it is possible to peek ahead and estimate with confidence the foundational technologies that will be used by patient monitors in the near future. Just as the discipline of anesthesiology has, the patient monitoring that accompanies it has come a long way from its beginnings in the mid-19th century. Extrapolating from careful observations of the prevailing trends that have shaped anesthesia patient monitoring historically, patient monitoring in the future will use noncontact technologies, will predict the trajectory of a patient's vital signs, will add regional vital signs to the current systemic ones, and will facilitate directed and supervised anesthesia care over the broader scope that anesthesia will be responsible for.

Optimizing classification of diseases through language model analysis of symptoms

This paper investigated the use of language models and deep learning techniques for automating disease prediction from symptoms. Specifically, we explored the use of two Medical Concept Normalization-Bidirectional Encoder Representations from Transformers (MCN-BERT) models and a Bidirectional Long Short-Term Memory (BiLSTM) model, each optimized with a different hyperparameter optimization method, to predict diseases from symptom descriptions. In this paper, we utilized two distinct dataset called Dataset-1, and Dataset-2. Dataset-1 consists of 1,200 data points, with each point representing a unique combination of disease labels and symptom descriptions. While, Dataset-2 is designed to identify Adverse Drug Reactions (ADRs) from Twitter data, comprising 23,516 rows categorized as ADR (1) or Non-ADR (0) tweets. The results indicate that the MCN-BERT model optimized with AdamP achieved 99.58% accuracy for Dataset-1 and 96.15% accuracy for Dataset-2. The MCN-BERT model optimized with AdamW performed well with 98.33% accuracy for Dataset-1 and 95.15% for Dataset-2, while the BiLSTM model optimized with Hyperopt achieved 97.08% accuracy for Dataset-1 and 94.15% for Dataset-2. Our findings suggest that language models and deep learning techniques have promise for supporting earlier detection and more prompt treatment of diseases, as well as expanding remote diagnostic capabilities. The MCN-BERT and BiLSTM models demonstrated robust performance in accurately predicting diseases from symptoms, indicating the potential for further related research.

Identifying Symptoms Prior to Pancreatic Ductal Adenocarcinoma Diagnosis in Real-World Care Settings: Natural Language Processing Approach

BACKGROUND

Pancreatic cancer is the third leading cause of cancer deaths in the United States. Pancreatic ductal adenocarcinoma (PDAC) is the most common form of pancreatic cancer, accounting for up to 90% of all cases. Patient-reported symptoms are often the triggers of cancer diagnosis and therefore, understanding the PDAC-associated symptoms and the timing of symptom onset could facilitate early detection of PDAC.

OBJECTIVE

This paper aims to develop a natural language processing (NLP) algorithm to capture symptoms associated with PDAC from clinical notes within a large integrated health care system.

METHODS

We used unstructured data within 2 years prior to PDAC diagnosis between 2010 and 2019 and among matched patients without PDAC to identify 17 PDAC-related symptoms. Related terms and phrases were first compiled from publicly available resources and then recursively reviewed and enriched with input from clinicians and chart review. A computerized NLP algorithm was iteratively developed and fine-trained via multiple rounds of chart review followed by adjudication. Finally, the developed algorithm was applied to the validation data set to assess performance and to the study implementation notes.

RESULTS

A total of 408,147 and 709,789 notes were retrieved from 2611 patients with PDAC and 10,085 matched patients without PDAC, respectively. In descending order, the symptom distribution of the study implementation notes ranged from 4.98% for abdominal or epigastric pain to 0.05% for upper extremity deep vein thrombosis in the PDAC group, and from 1.75% for back pain to 0.01% for pale stool in the non-PDAC group. Validation of the NLP algorithm against adjudicated chart review results of 1000 notes showed that precision ranged from 98.9% (jaundice) to 84% (upper extremity deep vein thrombosis), recall ranged from 98.1% (weight loss) to 82.8% (epigastric bloating), and F1-scores ranged from 0.97 (jaundice) to 0.86 (depression).

CONCLUSIONS

The developed and validated NLP algorithm could be used for the early detection of PDAC.

Extracting symptoms from free-text responses using ChatGPT among COVID-19 cases in Hong Kong

OBJECTIVES

To investigate the feasibility and performance of Chat Generative Pretrained Transformer (ChatGPT) in converting symptom narratives into structured symptom labels.

METHODS

We extracted symptoms from 300 deidentified symptom narratives of COVID-19 patients by a computer-based matching algorithm (the standard), and prompt engineering in ChatGPT. Common symptoms were those with a prevalence >10% according to the standard, and similarly less common symptoms were those with a prevalence of 2-10%. The precision of ChatGPT was compared with the standard using sensitivity and specificity with 95% exact binomial CIs (95% binCIs). In ChatGPT, we prompted without examples (zero-shot prompting) and with examples (few-shot prompting).

RESULTS

In zero-shot prompting, GPT-4 achieved high specificity (0.947 [95% binCI: 0.894-0.978]-1.000 [95% binCI: 0.965-0.988, 1.000]) for all symptoms, high sensitivity for common symptoms (0.853 [95% binCI: 0.689-0.950]-1.000 [95% binCI: 0.951-1.000]), and moderate sensitivity for less common symptoms (0.200 [95% binCI: 0.043-0.481]-1.000 [95% binCI: 0.590-0.815, 1.000]). Few-shot prompting increased the sensitivity and specificity. GPT-4 outperformed GPT-3.5 in response accuracy and consistent labelling.

DISCUSSION

This work substantiates ChatGPT's role as a research tool in medical fields. Its performance in converting symptom narratives to structured symptom labels was encouraging, saving time and effort in compiling the task-specific training data. It potentially accelerates free-text data compilation and synthesis in future disease outbreaks and improves the accuracy of symptom checkers. Focused prompt training addressing ambiguous descriptions impacts medical research positively.

Application of a Machine Learning Algorithm in Prediction of Abusive Head Trauma in Children

PURPOSE

We explored the application of a machine learning algorithm for the timely detection of potential abusive head trauma (AHT) using the first free-text note of an encounter and demographic information.

METHODS

First free-text physician notes and demographic information were collected for children under 5 years of age at a Level 1 Trauma Center. The control group, which included patients with head/neck injury, was compared to those with AHT diagnosed by the Child Protective Team. Differential scores accounted for words overrepresented in AHT patient vs. control notes. Sentiment scores were reflective of note positivity/negativity and subjectivity scores accounted for note subjectivity/objectivity. The composite scores reflected the patient's differential score modified by the subjectivity score. Composite, sentiment, and subjectivity scores combined with demographic information trained a Random Forest (RF) machine learning algorithm to predict AHT.

RESULTS

Final composite scores with demographic information were highly associated with AHT in a test dataset. The control group included 587 patients and the test group included 193 patients. Combining composite scores with demographic information into the RF model improved AHT classification area under the curve (AUC) from 0.68 to 0.78, with an overall accuracy of 84%. Feature importance analysis of our RF model revealed that composite score, sentiment, age, and subjectivity were the most impactful predictors of AHT. The sentiment was not significantly different between control and AHT notes (p = 0.87), while subjectivity trended higher for AHT notes (p = 0.081).

CONCLUSION

We conclude that a machine learning algorithm can recognize patterns within free-text notes and demographic information that aid in AHT detection in children.

LEVEL OF EVIDENCE

III.

Natural Language Processing Applied to Clinical Documentation in Post-acute Care Settings: A Scoping Review

OBJECTIVES

To determine the scope of the application of natural language processing to free-text clinical notes in post-acute care and provide a foundation for future natural language processing-based research in these settings.

DESIGN

Scoping review; reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.

SETTING AND PARTICIPANTS

Post-acute care (ie, home health care, long-term care, skilled nursing facilities, and inpatient rehabilitation facilities).

METHODS

PubMed, Cumulative Index of Nursing and Allied Health Literature, and Embase were searched in February 2023. Eligible studies had quantitative designs that used natural language processing applied to clinical documentation in post-acute care settings. The quality of each study was appraised.

RESULTS

Twenty-one studies were included. Almost all studies were conducted in home health care settings. Most studies extracted data from electronic health records to examine the risk for negative outcomes, including acute care utilization, medication errors, and suicide mortality. About half of the studies did not report age, sex, race, or ethnicity data or use standardized terminologies. Only 8 studies included variables from socio-behavioral domains. Most studies fulfilled all quality appraisal indicators.

CONCLUSIONS AND IMPLICATIONS

The application of natural language processing is nascent in post-acute care settings. Future research should apply natural language processing using standardized terminologies to leverage free-text clinical notes in post-acute care to promote timely, comprehensive, and equitable care. Natural language processing could be integrated with predictive models to help identify patients who are at risk of negative outcomes. Future research should incorporate socio-behavioral determinants and diverse samples to improve health equity in informatics tools.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

Patient Information Summarization in Clinical Settings: Scoping Review

BACKGROUND

Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field.

OBJECTIVE

This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties.

METHODS

A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework "collect-synthesize-communicate" referring to information gathering from data, its synthesis, and communication to the end user.

RESULTS

Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings.

CONCLUSIONS

The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the "collect-synthesize-communicate" framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary.

Long Covid symptoms and diagnosis in primary care: A cohort study using structured and unstructured data in The Health Improvement Network primary care database

BACKGROUND

Long Covid is a widely recognised consequence of COVID-19 infection, but little is known about the burden of symptoms that patients present with in primary care, as these are typically recorded only in free text clinical notes.

AIMS

To compare symptoms in patients with and without a history of COVID-19, and investigate symptoms associated with a Long Covid diagnosis.

METHODS

We used primary care electronic health record data until the end of December 2020 from The Health Improvement Network (THIN), a Cegedim database. We included adults registered with participating practices in England, Scotland or Wales. We extracted information about 89 symptoms and 'Long Covid' diagnoses from free text using natural language processing. We calculated hazard ratios (adjusted for age, sex, baseline medical conditions and prior symptoms) for each symptom from 12 weeks after the COVID-19 diagnosis.

RESULTS

We compared 11,015 patients with confirmed COVID-19 and 18,098 unexposed controls. Only 20% of symptom records were coded, with 80% in free text. A wide range of symptoms were associated with COVID-19 at least 12 weeks post-infection, with strongest associations for fatigue (adjusted hazard ratio (aHR) 3.46, 95% confidence interval (CI) 2.87, 4.17), shortness of breath (aHR 2.89, 95% CI 2.48, 3.36), palpitations (aHR 2.59, 95% CI 1.86, 3.60), and phlegm (aHR 2.43, 95% CI 1.65, 3.59). However, a limited subset of symptoms were recorded within 7 days prior to a Long Covid diagnosis in more than 20% of cases: shortness of breath, chest pain, pain, fatigue, cough, and anxiety / depression.

CONCLUSIONS

Numerous symptoms are reported to primary care at least 12 weeks after COVID-19 infection, but only a subset are commonly associated with a GP diagnosis of Long Covid.

Adverse event signal extraction from cancer patients' narratives focusing on impact on their daily-life activities

Adverse event (AE) management is important to improve anti-cancer treatment outcomes, but it is known that some AE signals can be missed during clinical visits. In particular, AEs that affect patients' activities of daily living (ADL) need careful monitoring as they may require immediate medical intervention. This study aimed to build deep-learning (DL) models for extracting signals of AEs limiting ADL from patients' narratives. The data source was blog posts written in Japanese by breast cancer patients. After pre-processing and annotation for AE signals, three DL models (BERT, ELECTRA, and T5) were trained and tested in three different approaches for AE signal identification. The performances of the trained models were evaluated in terms of precision, recall, and F1 scores. From 2,272 blog posts, 191 and 702 articles were identified as describing AEs limiting ADL or not limiting ADL, respectively. Among tested DL modes and approaches, T5 showed the best F1 scores to identify articles with AE limiting ADL or all AE: 0.557 and 0.811, respectively. The most frequent AE signals were "pain or numbness", "fatigue" and "nausea". Our results suggest that this AE monitoring scheme focusing on patients' ADL has potential to reinforce current AE management provided by medical staff.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Approach to machine learning for extraction of real-world data variables from electronic health records

Background: As artificial intelligence (AI) continues to advance with breakthroughs in natural language processing (NLP) and machine learning (ML), such as the development of models like OpenAI's ChatGPT, new opportunities are emerging for efficient curation of electronic health records (EHR) into real-world data (RWD) for evidence generation in oncology. Our objective is to describe the research and development of industry methods to promote transparency and explainability. Methods: We applied NLP with ML techniques to train, validate, and test the extraction of information from unstructured documents (e.g., clinician notes, radiology reports, lab reports, etc.) to output a set of structured variables required for RWD analysis. This research used a nationwide electronic health record (EHR)-derived database. Models were selected based on performance. Variables curated with an approach using ML extraction are those where the value is determined solely based on an ML model (i.e. not confirmed by abstraction), which identifies key information from visit notes and documents. These models do not predict future events or infer missing information. Results: We developed an approach using NLP and ML for extraction of clinically meaningful information from unstructured EHR documents and found high performance of output variables compared with variables curated by manually abstracted data. These extraction methods resulted in research-ready variables including initial cancer diagnosis with date, advanced/metastatic diagnosis with date, disease stage, histology, smoking status, surgery status with date, biomarker test results with dates, and oral treatments with dates. Conclusion: NLP and ML enable the extraction of retrospective clinical data in EHR with speed and scalability to help researchers learn from the experience of every person with cancer.

Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review

BACKGROUND

Natural Language Processing (NLP) applications have developed over the past years in various fields including its application to clinical free text for named entity recognition and relation extraction. However, there has been rapid developments the last few years that there's currently no overview of it. Moreover, it is unclear how these models and tools have been translated into clinical practice. We aim to synthesize and review these developments.

METHODS

We reviewed literature from 2010 to date, searching PubMed, Scopus, the Association of Computational Linguistics (ACL), and Association of Computer Machinery (ACM) libraries for studies of NLP systems performing general-purpose (i.e., not disease- or treatment-specific) information extraction and relation extraction tasks in unstructured clinical text (e.g., discharge summaries).

RESULTS

We included in the review 94 studies with 30 studies published in the last three years. Machine learning methods were used in 68 studies, rule-based in 5 studies, and both in 22 studies. 63 studies focused on Named Entity Recognition, 13 on Relation Extraction and 18 performed both. The most frequently extracted entities were "problem", "test" and "treatment". 72 studies used public datasets and 22 studies used proprietary datasets alone. Only 14 studies defined clearly a clinical or information task to be addressed by the system and just three studies reported its use outside the experimental setting. Only 7 studies shared a pre-trained model and only 8 an available software tool.

DISCUSSION

Machine learning-based methods have dominated the NLP field on information extraction tasks. More recently, Transformer-based language models are taking the lead and showing the strongest performance. However, these developments are mostly based on a few datasets and generic annotations, with very few real-world use cases. This may raise questions about the generalizability of findings, translation into practice and highlights the need for robust clinical evaluation.

Generalizability and portability of natural language processing system to extract individual social risk factors

OBJECTIVE

The objective of this study is to validate and report on portability and generalizability of a Natural Language Processing (NLP) method to extract individual social factors from clinical notes, which was originally developed at a different institution.

MATERIALS AND METHODS

A rule-based deterministic state machine NLP model was developed to extract financial insecurity and housing instability using notes from one institution and was applied on all notes written during 6 months at another institution. 10% of positively-classified notes by NLP and the same number of negatively-classified notes were manually annotated. The NLP model was adjusted to accommodate notes at the new site. Accuracy, positive predictive value, sensitivity, and specificity were calculated.

RESULTS

More than 6 million notes were processed at the receiving site by the NLP model, which resulted in about 13,000 and 19,000 classified as positive for financial insecurity and housing instability, respectively. The NLP model showed excellent performance on the validation dataset with all measures over 0.87 for both social factors.

DISCUSSION

Our study illustrated the need to accommodate institution-specific note-writing templates as well as clinical terminology of emergent diseases when applying NLP model for social factors. A state machine is relatively simple to port effectively across institutions. Our study. showed superior performance to similar generalizability studies for extracting social factors.

CONCLUSION

Rule-based NLP model to extract social factors from clinical notes showed strong portability and generalizability across organizationally and geographically distinct institutions. With only relatively simple modifications, we obtained promising performance from an NLP-based model.

Characterising symptom clusters in patients with atrial fibrillation undergoing catheter ablation

OBJECTIVE

This study aims to leverage natural language processing (NLP) and machine learning clustering analyses to (1) identify co-occurring symptoms of patients undergoing catheter ablation for atrial fibrillation (AF) and (2) describe clinical and sociodemographic correlates of symptom clusters.

METHODS

We conducted a cross-sectional retrospective analysis using electronic health records data. Adults who underwent AF ablation between 2010 and 2020 were included. Demographic, comorbidity and medication information was extracted using structured queries. Ten AF symptoms were extracted from unstructured clinical notes (n=13 416) using a validated NLP pipeline (F-score=0.81). We used the unsupervised machine learning approach known as Ward's hierarchical agglomerative clustering to characterise and identify subgroups of patients representing different clusters. Fisher's exact tests were used to investigate subgroup differences based on age, gender, race and heart failure (HF) status.

RESULTS

A total of 1293 patients were included in our analysis (mean age 65.5 years, 35.2% female, 58% white). The most frequently documented symptoms were dyspnoea (64%), oedema (62%) and palpitations (57%). We identified six symptom clusters: generally symptomatic, dyspnoea and oedema, chest pain, anxiety, fatigue and palpitations, and asymptomatic (reference). The asymptomatic cluster had a significantly higher prevalence of male, white and comorbid HF patients.

CONCLUSIONS

We applied NLP and machine learning to a large dataset to identify symptom clusters, which may signify latent biological underpinnings of symptom experiences and generate implications for clinical care. AF patients' symptom experiences vary widely. Given prior work showing that AF symptoms predict adverse outcomes, future work should investigate associations between symptom clusters and postablation outcomes.

Text-based predictions of COVID-19 diagnosis from self-reported chemosensory descriptions

BACKGROUND

There is a prevailing view that humans' capacity to use language to characterize sensations like odors or tastes is poor, providing an unreliable source of information.

METHODS

Here, we developed a machine learning method based on Natural Language Processing (NLP) using Large Language Models (LLM) to predict COVID-19 diagnosis solely based on text descriptions of acute changes in chemosensation, i.e., smell, taste and chemesthesis, caused by the disease. The dataset of more than 1500 subjects was obtained from survey responses early in the COVID-19 pandemic, in Spring 2020.

RESULTS

When predicting COVID-19 diagnosis, our NLP model performs comparably (AUC ROC ~ 0.65) to models based on self-reported changes in function collected via quantitative rating scales. Further, our NLP model could attribute importance of words when performing the prediction; sentiment and descriptive words such as "smell", "taste", "sense", had strong contributions to the predictions. In addition, adjectives describing specific tastes or smells such as "salty", "sweet", "spicy", and "sour" also contributed considerably to predictions.

CONCLUSIONS

Our results show that the description of perceptual symptoms caused by a viral infection can be used to fine-tune an LLM model to correctly predict and interpret the diagnostic status of a subject. In the future, similar models may have utility for patient verbatims from online health portals or electronic health records.

Towards Medical Billing Automation: NLP for Outpatient Clinician Note Classification

Objectives

Our primary objective was to develop a natural language processing approach that accurately predicts outpatient Evaluation and Management (E/M) level of service (LoS) codes using clinicians' notes from a health system electronic health record. A secondary objective was to investigate the impact of clinic note de-identification on document classification performance.

Methods

We used retrospective outpatient office clinic notes from four medical and surgical specialties. Classification models were fine-tuned on the clinic notes datasets and stratified by subspecialty. The success criteria for the classification tasks were the classification accuracy and F1-scores on internal test data. For the secondary objective, the dataset was de-identified using Named Entity Recognition (NER) to remove protected health information (PHI), and models were retrained.

Results

The models demonstrated similar predictive performance across different specialties, except for internal medicine, which had the lowest classification accuracy across all model architectures. The models trained on the entire note corpus achieved an E/M LoS CPT code classification accuracy of 74.8% (CI 95: 74.1-75.6). However, the de-identified note corpus showed a markedly lower classification accuracy of 48.2% (CI 95: 47.7-48.6) compared to the model trained on the identified notes.

Conclusion

The study demonstrates the potential of NLP-based document classifiers to accurately predict E/M LoS CPT codes using clinical notes from various medical and procedural specialties. The models' performance suggests that the classification task's complexity merits further investigation. The de-identification experiment demonstrated that de-identification may negatively impact classifier performance. Further research is needed to validate the performance of our NLP classifiers in different healthcare settings and patient populations and to investigate the potential implications of de-identification on model performance.

Learning Representations from Medical Text for Effective Diagnoses and Knowledge Discovery

Discovering knowledge and effectively predicting target events are two main goals of medical text mining. However, few models can achieve them simultaneously. In this study, we investigated the possibility of discovering knowledge and predicting diagnosis at once via raw medical text. We proposed the Enhanced Neural Topic Model (ENTM), a variant of the neural topic model, to learn interpretable representations. We introduced the auxiliary loss set to improve the effectiveness of learned representations. Then, we used learned representations to train a softmax regression model to predict target events. As each element in representations learned by the ENTM has an explicit semantic meaning, weights in softmax regression represent potential knowledge of whether an element is a significant factor in predicting diagnosis. We adopted two independent medical text datasets to evaluate our ENTM model. Results indicate that our model performed better than the latest pretrained neural language models. Meanwhile, analysis of model parameters indicates that our model has the potential discover knowledge from data.Clinical relevance- This work provides a model that can effectively predict patient diagnosis and has the potential to discover knowledge from medical text.