Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review

ChatGPT in Oncology Diagnosis and Treatment: Applications, Legal and Ethical Challenges

PURPOSE OF REVIEW

This study aims to systematically review the trajectory of artificial intelligence (AI) development in the medical field, with a particular emphasis on ChatGPT, a cutting-edge tool that is transforming oncology's diagnosis and treatment practices.

RECENT FINDINGS

Recent advancements have demonstrated that ChatGPT can be effectively utilized in various areas, including collecting medical histories, conducting radiological & pathological diagnoses, generating electronic medical record (EMR), providing nutritional support, participating in Multidisciplinary Team (MDT) and formulating personalized, multidisciplinary treatment plans. However, some significant challenges related to data privacy and legal issues that need to be addressed for the safe and effective integration of ChatGPT into clinical practice. ChatGPT, an emerging AI technology, opens up new avenues and viewpoints for oncology diagnosis and treatment. If current technological and legal challenges can be overcome, ChatGPT is expected to play a more significant role in oncology diagnosis and treatment in the future, providing better treatment options and improving the quality of medical services.

Neuradicon: Operational representation learning of neuroimaging reports

BACKGROUND AND OBJECTIVE

Radiological reports typically summarize the content and interpretation of imaging studies in unstructured form that precludes quantitative analysis. This limits the monitoring of radiological services to throughput undifferentiated by content, impeding specific, targeted operational optimization. Here we present Neuradicon, a natural language processing (NLP) framework for quantitative analysis of neuroradiological reports.

METHODS

Our framework is a hybrid of rule-based and machine-learning models to represent neurological reports in succinct, quantitative form optimally suited to operational guidance. These include probabilistic models for text classification and tagging tasks, alongside auto-encoders for learning latent representations and statistical mapping of the latent space.

RESULTS

We demonstrate the application of Neuradicon to operational phenotyping of a corpus of 336,569 reports, and report excellent generalizability across time and two independent healthcare institutions. In particular, we report pathology classification metrics with f1-scores of 0.96 on prospective data, and semantic means of interrogating the phenotypes surfaced via latent space representations.

CONCLUSION

Neuradicon allows the segmentation, analysis, classification, representation and interrogation of neuroradiological reports structure and content. It offers a blueprint for the extraction of rich, quantitative, actionable signals from unstructured text data in an operational context.

Evaluating Use of Generative Artificial Intelligence in Clinical Pathology Practice: Opportunities and the Way Forward

CONTEXT.—

Generative artificial intelligence (GAI) technologies are likely to dramatically impact health care workflows in clinical pathology (CP). Applications in CP include education, data mining, decision support, result summaries, and patient trend assessments.

OBJECTIVE.—

To review use cases of GAI in CP, with a particular focus on large language models. Specific examples are provided for the applications of GAI in the subspecialties of clinical chemistry, microbiology, hematopathology, and molecular diagnostics. Additionally, the review addresses potential pitfalls of GAI paradigms.

DATA SOURCES.—

Current literature on GAI in health care was reviewed broadly. The use case scenarios for each CP subspecialty review common data sources generated in each subspecialty. The potential for utilization of CP data in the GAI context was subsequently assessed, focusing on issues such as future reporting paradigms, impact on quality metrics, and potential for translational research activities.

CONCLUSIONS.—

GAI is a powerful tool with the potential to revolutionize health care for patients and practitioners alike. However, GAI must be implemented with much caution considering various shortcomings of the technology such as biases, hallucinations, practical challenges of implementing GAI in existing CP workflows, and end-user acceptance. Human-in-the-loop models of GAI implementation have the potential to revolutionize CP by delivering deeper, meaningful insights into patient outcomes both at an individual and a population level.

Estimating the Severity of Oral Lesions Via Analysis of Cone Beam Computed Tomography Reports: A Proposed Deep Learning Model

OBJECTIVES

Several factors such as unavailability of specialists, dental phobia, and financial difficulties may lead to a delay between receiving an oral radiology report and consulting a dentist. The primary aim of this study was to distinguish between high-risk and low-risk oral lesions according to the radiologist's reports of cone beam computed tomography (CBCT) images. Such a facility may be employed by dentist or his/her assistant to make the patient aware of the severity and the grade of the oral lesion and referral for immediate treatment or other follow-up care.

METHODS

A total number of 1134 CBCT radiography reports owned by Shiraz University of Medical Sciences were collected. The severity level of each sample was specified by three experts, and an annotation was carried out accordingly. After preprocessing the data, a deep learning model, referred to as CNN-LSTM, was developed, which aims to detect the degree of severity of the problem based on analysis of the radiologist's report. Unlike traditional models which usually use a simple collection of words, the proposed deep model uses words embedded in dense vector representations, which empowers it to effectively capture semantic similarities.

RESULTS

The results indicated that the proposed model outperformed its counterparts in terms of precision, recall, and F1 criteria. This suggests its potential as a reliable tool for early estimation of the severity of oral lesions.

CONCLUSIONS

This study shows the effectiveness of deep learning in the analysis of textual reports and accurately distinguishing between high-risk and low-risk lesions. Employing the proposed model which can Provide timely warnings about the need for follow-up and prompt treatment can shield the patient from the risks associated with delays.

CLINICAL SIGNIFICANCE

Our collaboratively collected and expert-annotated dataset serves as a valuable resource for exploratory research. The results demonstrate the pivotal role of our deep learning model could play in assessing the severity of oral lesions in dental reports.

Safety-netting advice documentation in out-of-hours primary care: a retrospective cohort from 2013 to 2020

BACKGROUND

Providing safety-netting advice (SNA) in out-of-hours (OOH) primary care is a recognised standard of safe care, but it is not known how frequently this occurs in practice.

AIM

Assess the frequency and type of SNA documented in OOH primary care and explore factors associated with its presence.

DESIGN AND SETTING

This was a retrospective cohort study using the Birmingham Out-of-hours general practice Research Database.

METHOD

A stratified sample of 30 adult consultation records per month from July 2013 to February 2020 were assessed using a safety-netting coding tool. Associations were tested using linear and logistic regression.

RESULTS

The overall frequency of SNA per consultation was 78.0% (1472/1886), increasing from 75.7% (224/296) in 2014 to 81.5% (220/270) in 2019. The proportion of specific SNA and the average number of symptoms patients were told to look out for increased with time. The most common symptom to look out for was if the patients' condition worsened followed by if their symptoms persisted, but only one in five consultations included a timeframe to reconsult for persistent symptoms. SNA was more frequently documented in face-to-face treatment-centre encounters compared with telephone consultations (odds ratio [OR] 1.77, 95% confidence interval [CI] = 1.09 to 2.85, P = 0.02), for possible infections (OR 1.53, 95% CI = 1.13 to 2.07, P = 0.006), and less frequently for mental (versus physical) health consultations (OR 0.33, 95% CI = 0.17 to 0.66, P = 0.002) and where follow-up was planned (OR 0.34, 95% CI = 0.25 to 0.46, P<0.001).

CONCLUSION

The frequency of SNA documented in OOH primary care was higher than previously reported during in-hours care. Over time, the frequency of SNA and proportion that contained specific advice increased, however, this study highlights potential consultations where SNA could be improved, such as mental health and telephone consultations.

Development of Artificial Intelligence Systems for Chronic Kidney Disease

Chronic kidney disease (CKD) is a complex disease that is related not only to dialysis but also to the onset of cardiovascular disease and life prognosis. As renal function declines with age and depending on lifestyle, the number of patients with CKD is rapidly increasing in Japan. Accurate prognosis prediction for patients with CKD in clinical settings is important for selecting treatment methods and screening patients with high-risk. In recent years, big databases on CKD and dialysis have been constructed through the use of data science technology, and the pathology of CKD is being elucidated. Therefore, we developed an artificial intelligence (AI) system that can accurately predict the prognosis of CKD such as its progression, the timing of dialysis introduction, and death. Aiming for its social implementation, the prognosis prediction system developed for patients with CKD was released on the website. We then developed a clinical practice guideline creation support system called Doctor K as an AI system. When creating clinical practice guidelines, huge amounts of manpower and time are required to conduct a systematic review of thousands of papers. Therefore, we developed a natural language processing (NLP) AI system to significantly improve work efficiency. Doctor K was used in the preparation of the guidelines of the Japanese Society of Nephrology. Furthermore, by comparing and analyzing the medical word virtual space constructed by the NLP AI system based on patient big data, we proved using the latest mathematical theory (category theory) that this system reflects the pathology of CKD. This suggests the possibility that the NLP AI system can predict patient prognosis. We hope that, through these studies, the use of AI based on big data will lead to the development of new treatments and improvement in patient prognosis.

Identifying people with post-COVID condition using linked, population-based administrative health data from Manitoba, Canada: prevalence and predictors in a cohort of COVID-positive individuals

OBJECTIVE

Many individuals exposed to SARS-CoV-2 experience long-term symptoms as part of a syndrome called post-COVID condition (PCC). Research on PCC is still emerging but is urgently needed to support diagnosis, clinical treatment guidelines and health system resource allocation. In this study, we developed a method to identify PCC cases using administrative health data and report PCC prevalence and predictive factors in Manitoba, Canada.

DESIGN

Cohort study.

SETTING

Manitoba, Canada.

PARTICIPANTS

All Manitobans who tested positive for SARS-CoV-2 during population-wide PCR testing from March 2020 to December 2021 (n=66 365) and were subsequently deemed to have PCC based on International Classification of Disease-9/10 diagnostic codes and prescription drug codes (n=11 316). Additional PCC cases were identified using predictive modelling to assess patterns of health service use, including physician visits, emergency department visits and hospitalisation for any reason (n=4155).

OUTCOMES

We measured PCC prevalence as % PCC cases among Manitobans with positive tests and identified predictive factors associated with PCC by calculating odds ratios with 95% confidence intervals, adjusted for sociodemographic and clinical characteristics (aOR).

RESULTS

Among 66 365 Manitobans with positive tests, we identified 15 471 (23%) as having PCC. Being female (aOR 1.64, 95% CI 1.58 to 1.71), being age 60-79 (aOR 1.33, 95% CI 1.25 to 1.41) or age 80+ (aOR 1.62, 95% CI 1.46 to 1.80), being hospitalised within 14 days of COVID-19 infection (aOR 1.95, 95% CI 1.80 to 2.10) and having a Charlson Comorbidity Index of 1+ (aOR 1.95, 95% CI 1.78 to 2.14) were predictive of PCC. Receiving 1+ doses of the COVID-19 vaccine (one dose, aOR 0.80, 95% CI 0.74 to 0.86; two doses, aOR 0.29, 95% CI 0.22 to 0.31) decreased the odds of PCC.

CONCLUSIONS

This data-driven approach expands our understanding of the prevalence and epidemiology of PCC and may be applied in other jurisdictions with population-based data. The study provides additional insights into risk and protective factors for PCC to inform health system planning and service delivery.

Twenty-Five Years of Evolution and Hurdles in Electronic Health Records and Interoperability in Medical Research: Comprehensive Review

BACKGROUND

Electronic health records (EHRs) facilitate the accessibility and sharing of patient data among various health care providers, contributing to more coordinated and efficient care.

OBJECTIVE

This study aimed to summarize the evolution of secondary use of EHRs and their interoperability in medical research over the past 25 years.

METHODS

We conducted an extensive literature search in the PubMed, Scopus, and Web of Science databases using the keywords Electronic health record and Electronic medical record in the title or abstract and Medical research in all fields from 2000 to 2024. Specific terms were applied to different time periods.

RESULTS

The review yielded 2212 studies, all of which were then screened and processed in a structured manner. Of these 2212 studies, 2102 (93.03%) were included in the review analysis, of which 1079 (51.33%) studies were from 2000 to 2009, 582 (27.69%) were from 2010 to 2019, 251 (11.94%) were from 2020 to 2023, and 190 (9.04%) were from 2024.

CONCLUSIONS

The evolution of EHRs marks an important milestone in health care's journey toward integrating technology and medicine. From early documentation practices to the sophisticated use of artificial intelligence and big data analytics today, EHRs have become central to improving patient care, enhancing public health surveillance, and advancing medical research.

The potential of Generative Pre-trained Transformer 4 (GPT-4) to analyse medical notes in three different languages: a retrospective model-evaluation study

BACKGROUND

Patient notes contain substantial information but are difficult for computers to analyse due to their unstructured format. Large-language models (LLMs), such as Generative Pre-trained Transformer 4 (GPT-4), have changed our ability to process text, but we do not know how effectively they handle medical notes. We aimed to assess the ability of GPT-4 to answer predefined questions after reading medical notes in three different languages.

METHODS

For this retrospective model-evaluation study, we included eight university hospitals from four countries (ie, the USA, Colombia, Singapore, and Italy). Each site submitted seven de-identified medical notes related to seven separate patients to the coordinating centre between June 1, 2023, and Feb 28, 2024. Medical notes were written between Feb 1, 2020, and June 1, 2023. One site provided medical notes in Spanish, one site provided notes in Italian, and the remaining six sites provided notes in English. We included admission notes, progress notes, and consultation notes. No discharge summaries were included in this study. We advised participating sites to choose medical notes that, at time of hospital admission, were for patients who were male or female, aged 18-65 years, had a diagnosis of obesity, had a diagnosis of COVID-19, and had submitted an admission note. Adherence to these criteria was optional and participating sites randomly chose which medical notes to submit. When entering information into GPT-4, we prepended each medical note with an instruction prompt and a list of 14 questions that had been chosen a priori. Each medical note was individually given to GPT-4 in its original language and in separate sessions; the questions were always given in English. At each site, two physicians independently validated responses by GPT-4 and responded to all 14 questions. Each pair of physicians evaluated responses from GPT-4 to the seven medical notes from their own site only. Physicians were not masked to responses from GPT-4 before providing their own answers, but were masked to responses from the other physician.

FINDINGS

We collected 56 medical notes, of which 42 (75%) were in English, seven (13%) were in Italian, and seven (13%) were in Spanish. For each medical note, GPT-4 responded to 14 questions, resulting in 784 responses. In 622 (79%, 95% CI 76-82) of 784 responses, both physicians agreed with GPT-4. In 82 (11%, 8-13) responses, only one physician agreed with GPT-4. In the remaining 80 (10%, 8-13) responses, neither physician agreed with GPT-4. Both physicians agreed with GPT-4 more often for medical notes written in Spanish (86 [88%, 95% CI 79-93] of 98 responses) and Italian (82 [84%, 75-90] of 98 responses) than in English (454 [77%, 74-80] of 588 responses).

INTERPRETATION

The results of our model-evaluation study suggest that GPT-4 is accurate when analysing medical notes in three different languages. In the future, research should explore how LLMs can be integrated into clinical workflows to maximise their use in health care.

FUNDING

None.

Automated Identification of Heart Failure With Reduced Ejection Fraction Using Deep Learning-Based Natural Language Processing

BACKGROUND

The lack of automated tools for measuring care quality limits the implementation of a national program to assess guideline-directed care in heart failure with reduced ejection fraction (HFrEF).

OBJECTIVES

The authors aimed to automate the identification of patients with HFrEF at hospital discharge, an opportunity to evaluate and improve the quality of care.

METHODS

The authors developed a novel deep-learning language model for identifying patients with HFrEF from discharge summaries of hospitalizations with heart failure at Yale New Haven Hospital during 2015 to 2019. HFrEF was defined by left ventricular ejection fraction <40% on antecedent echocardiography. The authors externally validated the model at Northwestern Medicine, community hospitals of Yale, and the MIMIC-III (Medical Information Mart for Intensive Care III) database.

RESULTS

A total of 13,251 notes from 5,392 unique individuals (age 73 ± 14 years, 48% women), including 2,487 patients with HFrEF (46.1%), were used for model development (train/held-out: 70%/30%). The model achieved an area under receiver-operating characteristic curve (AUROC) of 0.97 and area under precision recall curve (AUPRC) of 0.97 in detecting HFrEF on the held-out set. The model had high performance in identifying HFrEF with AUROC = 0.94 and AUPRC = 0.91 on 19,242 notes from Northwestern Medicine, AUROC = 0.95 and AUPRC = 0.96 on 139 manually abstracted notes from Yale community hospitals, and AUROC = 0.91 and AUPRC = 0.92 on 146 manually reviewed notes from MIMIC-III. Model-based predictions of HFrEF corresponded to a net reclassification improvement of 60.2% ± 1.9% compared with diagnosis codes (P < 0.001).

CONCLUSIONS

The authors developed a language model that identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment for individuals with HFrEF.

Consensus statements on the current landscape of artificial intelligence applications in endoscopy, addressing roadblocks, and advancing artificial intelligence in gastroenterology

BACKGROUND AND AIMS

The American Society for Gastrointestinal Endoscopy (ASGE) AI Task Force along with experts in endoscopy, technology space, regulatory authorities, and other medical subspecialties initiated a consensus process that analyzed the current literature, highlighted potential areas, and outlined the necessary research in artificial intelligence (AI) to allow a clearer understanding of AI as it pertains to endoscopy currently.

METHODS

A modified Delphi process was used to develop these consensus statements.

RESULTS

Statement 1: Current advances in AI allow for the development of AI-based algorithms that can be applied to endoscopy to augment endoscopist performance in detection and characterization of endoscopic lesions. Statement 2: Computer vision-based algorithms provide opportunities to redefine quality metrics in endoscopy using AI, which can be standardized and can reduce subjectivity in reporting quality metrics. Natural language processing-based algorithms can help with the data abstraction needed for reporting current quality metrics in GI endoscopy effortlessly. Statement 3: AI technologies can support smart endoscopy suites, which may help optimize workflows in the endoscopy suite, including automated documentation. Statement 4: Using AI and machine learning helps in predictive modeling, diagnosis, and prognostication. High-quality data with multidimensionality are needed for risk prediction, prognostication of specific clinical conditions, and their outcomes when using machine learning methods. Statement 5: Big data and cloud-based tools can help advance clinical research in gastroenterology. Multimodal data are key to understanding the maximal extent of the disease state and unlocking treatment options. Statement 6: Understanding how to evaluate AI algorithms in the gastroenterology literature and clinical trials is important for gastroenterologists, trainees, and researchers, and hence education efforts by GI societies are needed. Statement 7: Several challenges regarding integrating AI solutions into the clinical practice of endoscopy exist, including understanding the role of human-AI interaction. Transparency, interpretability, and explainability of AI algorithms play a key role in their clinical adoption in GI endoscopy. Developing appropriate AI governance, data procurement, and tools needed for the AI lifecycle are critical for the successful implementation of AI into clinical practice. Statement 8: For payment of AI in endoscopy, a thorough evaluation of the potential value proposition for AI systems may help guide purchasing decisions in endoscopy. Reliable cost-effectiveness studies to guide reimbursement are needed. Statement 9: Relevant clinical outcomes and performance metrics for AI in gastroenterology are currently not well defined. To improve the quality and interpretability of research in the field, steps need to be taken to define these evidence standards. Statement 10: A balanced view of AI technologies and active collaboration between the medical technology industry, computer scientists, gastroenterologists, and researchers are critical for the meaningful advancement of AI in gastroenterology.

CONCLUSIONS

The consensus process led by the ASGE AI Task Force and experts from various disciplines has shed light on the potential of AI in endoscopy and gastroenterology. AI-based algorithms have shown promise in augmenting endoscopist performance, redefining quality metrics, optimizing workflows, and aiding in predictive modeling and diagnosis. However, challenges remain in evaluating AI algorithms, ensuring transparency and interpretability, addressing governance and data procurement, determining payment models, defining relevant clinical outcomes, and fostering collaboration between stakeholders. Addressing these challenges while maintaining a balanced perspective is crucial for the meaningful advancement of AI in gastroenterology.

Using digital health technologies to optimise antimicrobial use globally

Digital health technology (DHT) describes tools and devices that generate or process health data. The application of DHTs could improve the diagnosis, treatment, and surveillance of bacterial infection and the prevention of antimicrobial resistance (AMR). DHTs to optimise antimicrobial use are rapidly being developed. To support the global adoption of DHTs and the opportunities offered to optimise antimicrobial use consensus is needed on what data are required to support antimicrobial decision making. This Series paper will explore bacterial AMR in humans and the need to optimise antimicrobial use in response to this global threat. It will also describe state-of-the-art DHTs to optimise antimicrobial prescribing in high-income and low-income and middle-income countries, and consider what fundamental data are ideally required for and from such technologies to support optimised antimicrobial use.

Artificial Intelligence in Human Reproduction

The use of artificial intelligence (AI) in human reproduction is a rapidly evolving field with both exciting possibilities and ethical considerations. This technology has the potential to improve success rates and reduce the emotional and financial burden of infertility. However, it also raises ethical and privacy concerns. This paper presents an overview of the current and potential applications of AI in human reproduction. It explores the use of AI in various aspects of reproductive medicine, including fertility tracking, assisted reproductive technologies, management of pregnancy complications, and laboratory automation. In addition, we discuss the need for robust ethical frameworks and regulations to ensure the responsible and equitable use of AI in reproductive medicine.

Assessing the use of unstructured electronic health record data to identify exposure to firearm violence

Objectives

Research on firearm violence is largely limited to people who experienced acute bodily trauma and death which is readily gathered from Inpatient and Emergency Department settings and mortality data. Exposures to firearm violence, such as witnessing firearm violence or losing a loved one to firearm violence, are not routinely collected in health care. As a result, the true public health burden of firearm violence is underestimated. Clinical notes from electronic health records (EHRs) are a promising source of data that may expand our understanding of the impact of firearm violence on health. Pilot work was conducted on a sample of clinical notes to assess how firearm terms present in unstructured clinical notes as part of a larger initiative to develop a natural language processing (NLP) model to identify firearm exposure and injury in ambulatory care data.

Materials and Methods

We used EHR data from 2012 to 2022 from a large multistate network of primary care and behavioral health clinics. A text string search of broad, gun-only, and shooting terms was applied to 9,598 patients with either/both an ICD-10 or an OCHIN-developed structured data field indicating exposure to firearm violence. A sample of clinical notes from 90 patients was reviewed to ascertain the meaning of terms.

Results

Among the 90 clinical patient notes, 13 (14%) notes reflect documentation of exposure to firearm violence or injury from firearms. Results from this study identified refinements that should be considered for NLP text classification.

Conclusion

Unstructured clinical notes from primary and behavioral health clinics have potential to expand understanding of firearm violence.

Artificial Intelligence as a Tool for Creating Patient Visit Summary: A Scoping Review and Guide to Implementation in an Erectile Dysfunction Clinic

PURPOSE OF REVIEW

In modern healthcare, the integration of artificial intelligence (AI) has revolutionized clinical practices, particularly in data management and patient visit summary creation. Manual creation of patient summary is repetitive, time-consuming, prone to errors, and increases clinicians' workload. AI, through voice recognition and Natural Language Processing (NLP), can automate this task more accurately and efficiently. Erectile dysfunction (ED) clinics, which deal with specific pattern of conditions together with an involvement of broader systemic issues, can greatly benefit from AI-driven patient summary. This scoping review examined the evidence on AI-generated patient summary and evaluated their implementation in ED clinics.

RECENT FINDINGS

A total of 381 articles were initially identified, 11 studies were included for the analysis. These studies showcased various methodologies, such as AI-assisted clinical notes and NLP algorithms. Most studies have demonstrated the ability of AI to be used in real life clinical scenarios. Major electronic health record platforms are also integrating AI to their system. However, to date, no studies have specifically addressed AI for patient summary creation in ED clinics.

Enhancing early detection of cognitive decline in the elderly: a comparative study utilizing large language models in clinical notes

BACKGROUND

Large language models (LLMs) have shown promising performance in various healthcare domains, but their effectiveness in identifying specific clinical conditions in real medical records is less explored. This study evaluates LLMs for detecting signs of cognitive decline in real electronic health record (EHR) clinical notes, comparing their error profiles with traditional models. The insights gained will inform strategies for performance enhancement.

METHODS

This study, conducted at Mass General Brigham in Boston, MA, analysed clinical notes from the four years prior to a 2019 diagnosis of mild cognitive impairment in patients aged 50 and older. We developed prompts for two LLMs, Llama 2 and GPT-4, on Health Insurance Portability and Accountability Act (HIPAA)-compliant cloud-computing platforms using multiple approaches (e.g., hard prompting, retrieval augmented generation, and error analysis-based instructions) to select the optimal LLM-based method. Baseline models included a hierarchical attention-based neural network and XGBoost. Subsequently, we constructed an ensemble of the three models using a majority vote approach. Confusion-matrix-based scores were used for model evaluation.

FINDINGS

We used a randomly annotated sample of 4949 note sections from 1969 patients (women: 1046 [53.1%]; age: mean, 76.0 [SD, 13.3] years), filtered with keywords related to cognitive functions, for model development. For testing, a random annotated sample of 1996 note sections from 1161 patients (women: 619 [53.3%]; age: mean, 76.5 [SD, 10.2] years) without keyword filtering was utilised. GPT-4 demonstrated superior accuracy and efficiency compared to Llama 2, but did not outperform traditional models. The ensemble model outperformed the individual models in terms of all evaluation metrics with statistical significance (p < 0.01), achieving a precision of 90.2% [95% CI: 81.9%-96.8%], a recall of 94.2% [95% CI: 87.9%-98.7%], and an F1-score of 92.1% [95% CI: 86.8%-96.4%]. Notably, the ensemble model showed a significant improvement in precision, increasing from a range of 70%-79% to above 90%, compared to the best-performing single model. Error analysis revealed that 63 samples were incorrectly predicted by at least one model; however, only 2 cases (3.2%) were mutual errors across all models, indicating diverse error profiles among them.

INTERPRETATION

LLMs and traditional machine learning models trained using local EHR data exhibited diverse error profiles. The ensemble of these models was found to be complementary, enhancing diagnostic performance. Future research should investigate integrating LLMs with smaller, localised models and incorporating medical data and domain knowledge to enhance performance on specific tasks.

FUNDING

This research was supported by the National Institute on Aging grants (R44AG081006, R01AG080429) and National Library of Medicine grant (R01LM014239).

SEETrials: Leveraging large language models for safety and efficacy extraction in oncology clinical trials

Background

Initial insights into oncology clinical trial outcomes are often gleaned manually from conference abstracts. We aimed to develop an automated system to extract safety and efficacy information from study abstracts with high precision and fine granularity, transforming them into computable data for timely clinical decision-making.

Methods

We collected clinical trial abstracts from key conferences and PubMed (2012-2023). The SEETrials system was developed with three modules: preprocessing, prompt engineering with knowledge ingestion, and postprocessing. We evaluated the system's performance qualitatively and quantitatively and assessed its generalizability across different cancer types- multiple myeloma (MM), breast, lung, lymphoma, and leukemia. Furthermore, the efficacy and safety of innovative therapies, including CAR-T, bispecific antibodies, and antibody-drug conjugates (ADC), in MM were analyzed across a large scale of clinical trial studies.

Results

SEETrials achieved high precision (0.964), recall (sensitivity) (0.988), and F1 score (0.974) across 70 data elements present in the MM trial studies Generalizability tests on four additional cancers yielded precision, recall, and F1 scores within the 0.979-0.992 range. Variation in the distribution of safety and efficacy-related entities was observed across diverse therapies, with certain adverse events more common in specific treatments. Comparative performance analysis using overall response rate (ORR) and complete response (CR) highlighted differences among therapies: CAR-T (ORR: 88 %, 95 % CI: 84-92 %; CR: 95 %, 95 % CI: 53-66 %), bispecific antibodies (ORR: 64 %, 95 % CI: 55-73 %; CR: 27 %, 95 % CI: 16-37 %), and ADC (ORR: 51 %, 95 % CI: 37-65 %; CR: 26 %, 95 % CI: 1-51 %). Notable study heterogeneity was identified (>75 % I 2 heterogeneity index scores) across several outcome entities analyzed within therapy subgroups.

Conclusion

SEETrials demonstrated highly accurate data extraction and versatility across different therapeutics and various cancer domains. Its automated processing of large datasets facilitates nuanced data comparisons, promoting the swift and effective dissemination of clinical insights.

Qualitative Health-Related Quality of Life and Natural Language Processing: Characteristics, Implications, and Challenges

OBJECTIVES

This article focuses on describing the main characteristics of the application of NLP in the qualitative assessment of quality of life, as well as its implications and challenges.

METHODS

The qualitative methodology allows analysing patient comments in unstructured free text and obtaining valuable information through manual analysis of these data. However, large amounts of data are a healthcare challenge since it would require a high number of staff and time resources that are not available in most healthcare organizations.

RESULTS

One potential solution to mitigate the resource constraints of qualitative analysis is the use of machine learning and artificial intelligence, specifically methodologies based on natural language processing.

Scalable incident detection via natural language processing and probabilistic language models

Post marketing safety surveillance depends in part on the ability to detect concerning clinical events at scale. Spontaneous reporting might be an effective component of safety surveillance, but it requires awareness and understanding among healthcare professionals to achieve its potential. Reliance on readily available structured data such as diagnostic codes risks under-coding and imprecision. Clinical textual data might bridge these gaps, and natural language processing (NLP) has been shown to aid in scalable phenotyping across healthcare records in multiple clinical domains. In this study, we developed and validated a novel incident phenotyping approach using unstructured clinical textual data agnostic to Electronic Health Record (EHR) and note type. It's based on a published, validated approach (PheRe) used to ascertain social determinants of health and suicidality across entire healthcare records. To demonstrate generalizability, we validated this approach on two separate phenotypes that share common challenges with respect to accurate ascertainment: (1) suicide attempt; (2) sleep-related behaviors. With samples of 89,428 records and 35,863 records for suicide attempt and sleep-related behaviors, respectively, we conducted silver standard (diagnostic coding) and gold standard (manual chart review) validation. We showed Area Under the Precision-Recall Curve of ~ 0.77 (95% CI 0.75-0.78) for suicide attempt and AUPR ~ 0.31 (95% CI 0.28-0.34) for sleep-related behaviors. We also evaluated performance by coded race and demonstrated differences in performance by race differed across phenotypes. Scalable phenotyping models, like most healthcare AI, require algorithmovigilance and debiasing prior to implementation.

Validation of large language models for detecting pathologic complete response in breast cancer using population-based pathology reports

AIMS

The primary goal of this study is to evaluate the capabilities of Large Language Models (LLMs) in understanding and processing complex medical documentation. We chose to focus on the identification of pathologic complete response (pCR) in narrative pathology reports. This approach aims to contribute to the advancement of comprehensive reporting, health research, and public health surveillance, thereby enhancing patient care and breast cancer management strategies.

METHODS

The study utilized two analytical pipelines, developed with open-source LLMs within the healthcare system's computing environment. First, we extracted embeddings from pathology reports using 15 different transformer-based models and then employed logistic regression on these embeddings to classify the presence or absence of pCR. Secondly, we fine-tuned the Generative Pre-trained Transformer-2 (GPT-2) model by attaching a simple feed-forward neural network (FFNN) layer to improve the detection performance of pCR from pathology reports.

RESULTS

In a cohort of 351 female breast cancer patients who underwent neoadjuvant chemotherapy (NAC) and subsequent surgery between 2010 and 2017 in Calgary, the optimized method displayed a sensitivity of 95.3% (95%CI: 84.0-100.0%), a positive predictive value of 90.9% (95%CI: 76.5-100.0%), and an F1 score of 93.0% (95%CI: 83.7-100.0%). The results, achieved through diverse LLM integration, surpassed traditional machine learning models, underscoring the potential of LLMs in clinical pathology information extraction.

CONCLUSIONS

The study successfully demonstrates the efficacy of LLMs in interpreting and processing digital pathology data, particularly for determining pCR in breast cancer patients post-NAC. The superior performance of LLM-based pipelines over traditional models highlights their significant potential in extracting and analyzing key clinical data from narrative reports. While promising, these findings highlight the need for future external validation to confirm the reliability and broader applicability of these methods.

US public health surveillance, reimagined

Introduction

This study presents two novel concepts for standardizing electronic health records (EHR)-based public health surveillance through utilization of existing informatics methods and data platforms.

Methods

Drawing from the collective experience in applied epidemiology, health services research and health informatics, the author presents a vision for an alternative path to public health surveillance by repurposing existing tools and resources, such as (1) computable phenotypes which have already been created and validated for a variety of chronic diseases of interest to public health and (2) large data platforms/collaboratives, such as All of Us Research Program and National COVID Cohort Collaborative. Opportunities and challenges are discussed regarding EHR-based chronic disease surveillance, as well as the concept of phenotype definitions and large data platforms reuse for public health needs.

Results/Framework

Reusing of computable phenotypes for EHR-based public health surveillance would require secure data platforms and nationally representative data. Standardization metrics for reuse of previously developed and validated computable phenotypes are also necessary and are currently being developed by the author. This study presents a reimagined Learning Health System framework by incorporating Public Health and two novel concept sets of solutions into the healthcare ecosystem.

Conclusion/Next Steps

Alternative approaches to limited resources and current infrastructure of the US Public Health System, especially as applied to disease surveillance, are needed and may be possible when repurposing the resources and methodologies across the Learning Health System.

Machine learning applications in preventive healthcare: A systematic literature review on predictive analytics of disease comorbidity from multiple perspectives

Artificial intelligence is constantly revolutionizing biomedical research and healthcare management. Disease comorbidity is a major threat to the quality of life for susceptible groups, especially middle-aged and elderly patients. The presence of multiple chronic diseases makes precision diagnosis challenging to realize and imposes a heavy burden on the healthcare system and economy. Given an enormous amount of accumulated health data, machine learning techniques show their capability in handling this puzzle. The present study conducts a review to uncover current research efforts in applying these methods to understanding comorbidity mechanisms and making clinical predictions considering these complex patterns. A descriptive metadata analysis of 791 unique publications aims to capture the overall research progression between January 2012 and June 2023. To delve into comorbidity-focused research, 61 of these scientific papers are systematically assessed. Four predictive analytics of tasks are detected: disease comorbidity data extraction, clustering, network, and risk prediction. It is observed that some machine learning-driven applications address inherent data deficiencies in healthcare datasets and provide a model interpretation that identifies significant risk factors of comorbidity development. Based on insights, both technical and practical, gained from relevant literature, this study intends to guide future interests in comorbidity research and draw conclusions about chronic disease prevention and diagnosis with managerial implications.

Text mining method to unravel long COVID's clinical condition in hospitalized patients

Long COVID is characterized by persistent that extends symptoms beyond established timeframes. Its varied presentation across different populations and healthcare systems poses significant challenges in understanding its clinical manifestations and implications. In this study, we present a novel application of text mining technique to automatically extract unstructured data from a long COVID survey conducted at a prominent university hospital in São Paulo, Brazil. Our phonetic text clustering (PTC) method enables the exploration of unstructured Electronic Healthcare Records (EHR) data to unify different written forms of similar terms into a single phonemic representation. We used n-gram text analysis to detect compound words and negated terms in Portuguese-BR, focusing on medical conditions and symptoms related to long COVID. By leveraging text mining, we aim to contribute to a deeper understanding of this chronic condition and its implications for healthcare systems globally. The model developed in this study has the potential for scalability and applicability in other healthcare settings, thereby supporting broader research efforts and informing clinical decision-making for long COVID patients.

Advancements in AI based healthcare techniques with FOCUS ON diagnostic techniques

Since the past decade, the interest towards more precise and efficient healthcare techniques with special emphasis on diagnostic techniques has increased. Artificial Intelligence has proved to be instrumental in development of various such techniques. The various types of AI like ML, NLP, RPA etc. are being used, which have streamlined and organised the Electronic Health Records (EHR) along with aiding the healthcare provider with decision making and sample and data analysis. This article also deals with the 3 major categories of diagnostic techniques - Imaging based, Pathology based and Preventive diagnostic techniques and what all changes and modifications were brought upon them, due to use of AI. Due to such a high demand, the investment in AI based healthcare techniques has increased substantially, with predicted market size of almost 188 billon USD by 2030. In India itself, AI in healthcare is expected to raise the GDP by 25 billion USD by 2028. But there are also several challenges associated with this like unavailability of quality data, black box issue etc. One of the major challenges is the ethical considerations and issues during use of medical records as it is a very sensitive document. Due to this, there is several trust issues associated with adoption of AI by many organizations. These challenges have also been discussed in this article. Need for further development in the AI based diagnostic techniques is also done in the article. Alongside, the production of such techniques and devices which are easy to use and simple to incorporate into the daily workflows have immense scope in the upcoming times. The increasing scope of Clinical Decision Support System, Telemedicine etc. make AI a promising field in the healthcare and diagnostics arena. Concluding the article, it can be said that despite the presence of various challenges to the implementation and usage, the future prospects for AI in healthcare is immense and work needs to be done in order to ensure the availability of resources for same so that high level of accuracy can be achieved and better health outcomes can be provided to patients. Ethical concerns need to be addressed for smooth implementation and to reduce the burden of the developers, which has been discussed in this narrative review article.

Toward Reliable Symptom Coding in Electronic Health Records for Symptom Assessment and Research: Identification and Categorization of International Classification of Diseases, Ninth Revision, Clinical Modification Symptom Codes

To date, symptom documentation has mostly relied on clinical notes in electronic health records or patient-reported outcomes using disease-specific symptom inventories. To provide a common and precise language for symptom recording, assessment, and research, a comprehensive list of symptom codes is needed. The International Classification of Diseases, Ninth Revision or its clinical modification ( International Classification of Diseases, Ninth Revision, Clinical Modification ) has a range of codes designated for symptoms, but it does not contain codes for all possible symptoms, and not all codes in that range are symptom related. This study aimed to identify and categorize the first list of International Classification of Diseases, Ninth Revision, Clinical Modification symptom codes for a general population and demonstrate their use to characterize symptoms of patients with type 2 diabetes mellitus in the Cerner database. A list of potential symptom codes was automatically extracted from the Unified Medical Language System Metathesaurus. Two clinical experts in symptom science and diabetes manually reviewed this list to identify and categorize codes as symptoms. A total of 1888 International Classification of Diseases, Ninth Revision, Clinical Modification symptom codes were identified and categorized into 65 categories. The symptom characterization using the newly obtained symptom codes and categories was found to be more reasonable than that using the previous symptom codes and categories on the same Cerner diabetes cohort.

Using machine learning methods to predict all-cause somatic hospitalizations in adults: A systematic review

AIM

In this review, we investigated how Machine Learning (ML) was utilized to predict all-cause somatic hospital admissions and readmissions in adults.

METHODS

We searched eight databases (PubMed, Embase, Web of Science, CINAHL, ProQuest, OpenGrey, WorldCat, and MedNar) from their inception date to October 2023, and included records that predicted all-cause somatic hospital admissions and readmissions of adults using ML methodology. We used the CHARMS checklist for data extraction, PROBAST for bias and applicability assessment, and TRIPOD for reporting quality.

RESULTS

We screened 7,543 studies of which 163 full-text records were read and 116 met the review inclusion criteria. Among these, 45 predicted admission, 70 predicted readmission, and one study predicted both. There was a substantial variety in the types of datasets, algorithms, features, data preprocessing steps, evaluation, and validation methods. The most used types of features were demographics, diagnoses, vital signs, and laboratory tests. Area Under the ROC curve (AUC) was the most used evaluation metric. Models trained using boosting tree-based algorithms often performed better compared to others. ML algorithms commonly outperformed traditional regression techniques. Sixteen studies used Natural language processing (NLP) of clinical notes for prediction, all studies yielded good results. The overall adherence to reporting quality was poor in the review studies. Only five percent of models were implemented in clinical practice. The most frequently inadequately addressed methodological aspects were: providing model interpretations on the individual patient level, full code availability, performing external validation, calibrating models, and handling class imbalance.

CONCLUSION

This review has identified considerable concerns regarding methodological issues and reporting quality in studies investigating ML to predict hospitalizations. To ensure the acceptability of these models in clinical settings, it is crucial to improve the quality of future studies.

The Use of Deep Learning and Machine Learning on Longitudinal Electronic Health Records for the Early Detection and Prevention of Diseases: Scoping Review

BACKGROUND

Electronic health records (EHRs) contain patients' health information over time, including possible early indicators of disease. However, the increasing amount of data hinders clinicians from using them. There is accumulating evidence suggesting that machine learning (ML) and deep learning (DL) can assist clinicians in analyzing these large-scale EHRs, as algorithms thrive on high volumes of data. Although ML has become well developed, studies mainly focus on engineering but lack medical outcomes.

OBJECTIVE

This study aims for a scoping review of the evidence on how the use of ML on longitudinal EHRs can support the early detection and prevention of disease. The medical insights and clinical benefits that have been generated were investigated by reviewing applications in a variety of diseases.

METHODS

This study was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search was performed in 2022 in collaboration with a medical information specialist in the following databases: PubMed, Embase, Web of Science Core Collection (Clarivate Analytics), and IEEE Xplore Digital Library and computer science bibliography. Studies were eligible when longitudinal EHRs were used that aimed for the early detection of disease via ML in a prevention context. Studies with a technical focus or using imaging or hospital admission data were beyond the scope of this review. Study screening and selection and data extraction were performed independently by 2 researchers.

RESULTS

In total, 20 studies were included, mainly published between 2018 and 2022. They showed that a variety of diseases could be detected or predicted, particularly diabetes; kidney diseases; diseases of the circulatory system; and mental, behavioral, and neurodevelopmental disorders. Demographics, symptoms, procedures, laboratory test results, diagnoses, medications, and BMI were frequently used EHR data in basic recurrent neural network or long short-term memory techniques. By developing and comparing ML and DL models, medical insights such as a high diagnostic performance, an earlier detection, the most important predictors, and additional health indicators were obtained. A clinical benefit that has been evaluated positively was preliminary screening. If these models are applied in practice, patients might also benefit from personalized health care and prevention, with practical benefits such as workload reduction and policy insights.

CONCLUSIONS

Longitudinal EHRs proved to be helpful for support in health care. Current ML models on EHRs can support the detection of diseases in terms of accuracy and offer preliminary screening benefits. Regarding the prevention of diseases, ML and specifically DL models can accurately predict or detect diseases earlier than current clinical diagnoses. Adding personally responsible factors allows targeted prevention interventions. While ML models based on textual EHRs are still in the developmental stage, they have high potential to support clinicians and the health care system and improve patient outcomes.

Natural Language Processing Accurately Differentiates Cancer Symptom Information in Electronic Health Record Narratives

PURPOSE

Identifying cancer symptoms in electronic health record (EHR) narratives is feasible with natural language processing (NLP). However, more efficient NLP systems are needed to detect various symptoms and distinguish observed symptoms from negated symptoms and medication-related side effects. We evaluated the accuracy of NLP in (1) detecting 14 symptom groups (ie, pain, fatigue, swelling, depressed mood, anxiety, nausea/vomiting, pruritus, headache, shortness of breath, constipation, numbness/tingling, decreased appetite, impaired memory, disturbed sleep) and (2) distinguishing observed symptoms in EHR narratives among patients with cancer.

METHODS

We extracted 902,508 notes for 11,784 unique patients diagnosed with cancer and developed a gold standard corpus of 1,112 notes labeled for presence or absence of 14 symptom groups. We trained an embeddings-augmented NLP system integrating human and machine intelligence and conventional machine learning algorithms. NLP metrics were calculated on a gold standard corpus subset for testing.

RESULTS

The interannotator agreement for labeling the gold standard corpus was excellent at 92%. The embeddings-augmented NLP model achieved the best performance (F1 score = 0.877). The highest NLP accuracy was observed in pruritus (F1 score = 0.937) while the lowest accuracy was in swelling (F1 score = 0.787). After classifying the entire data set with embeddings-augmented NLP, we found that 41% of the notes included symptom documentation. Pain was the most documented symptom (29% of all notes) while impaired memory was the least documented (0.7% of all notes).

CONCLUSION

We illustrated the feasibility of detecting 14 symptom groups in EHR narratives and showed that an embeddings-augmented NLP system outperforms conventional machine learning algorithms in detecting symptom information and differentiating observed symptoms from negated symptoms and medication-related side effects.

A Comprehensive Review of Artificial Intelligence in Prostate Cancer Care: State-of-the-Art Diagnostic Tools and Future Outlook

Prostate cancer remains a significant global health challenge, characterized by high incidence and substantial morbidity and mortality rates. Early detection is critical for improving patient outcomes, yet current diagnostic methods have limitations in accuracy and reliability. Artificial intelligence (AI) has emerged as a promising tool to address these challenges in prostate cancer care. AI technologies, including machine learning algorithms and advanced imaging techniques, offer potential solutions to enhance diagnostic accuracy, optimize treatment strategies, and personalize patient care. This review explores the current landscape of AI applications in prostate cancer diagnostics, highlighting state-of-the-art tools and their clinical implications. By synthesizing recent advancements and discussing future directions, the review underscores the transformative potential of AI in revolutionizing prostate cancer diagnosis and management. Ultimately, integrating AI into clinical practice can potentially improve outcomes and quality of life for patients affected by prostate cancer.

Pre-trained language models in medicine: A survey

With the rapid progress in Natural Language Processing (NLP), Pre-trained Language Models (PLM) such as BERT, BioBERT, and ChatGPT have shown great potential in various medical NLP tasks. This paper surveys the cutting-edge achievements in applying PLMs to various medical NLP tasks. Specifically, we first brief PLMS and outline the research of PLMs in medicine. Next, we categorise and discuss the types of tasks in medical NLP, covering text summarisation, question-answering, machine translation, sentiment analysis, named entity recognition, information extraction, medical education, relation extraction, and text mining. For each type of task, we first provide an overview of the basic concepts, the main methodologies, the advantages of applying PLMs, the basic steps of applying PLMs application, the datasets for training and testing, and the metrics for task evaluation. Subsequently, a summary of recent important research findings is presented, analysing their motivations, strengths vs weaknesses, similarities vs differences, and discussing potential limitations. Also, we assess the quality and influence of the research reviewed in this paper by comparing the citation count of the papers reviewed and the reputation and impact of the conferences and journals where they are published. Through these indicators, we further identify the most concerned research topics currently. Finally, we look forward to future research directions, including enhancing models' reliability, explainability, and fairness, to promote the application of PLMs in clinical practice. In addition, this survey also collect some download links of some model codes and the relevant datasets, which are valuable references for researchers applying NLP techniques in medicine and medical professionals seeking to enhance their expertise and healthcare service through AI technology.

Towards the automatic calculation of the EQUAL Candida Score: Extraction of CVC-related information from EMRs of critically ill patients with candidemia in Intensive Care Units

OBJECTIVES

Candidemia is the most frequent invasive fungal disease and the fourth most frequent bloodstream infection in hospitalized patients. Its optimal management is crucial for improving patients' survival. The quality of candidemia management can be assessed with the EQUAL Candida Score. The objective of this work is to support its automatic calculation by extracting central venous catheter-related information from Italian text in clinical notes of electronic medical records.

MATERIALS AND METHODS

The sample includes 4,787 clinical notes of 108 patients hospitalized between January 2018 to December 2020 in the Intensive Care Units of the IRCCS San Martino Polyclinic Hospital in Genoa (Italy). The devised pipeline exploits natural language processing (NLP) to produce numerical representations of clinical notes used as input of machine learning (ML) algorithms to identify CVC presence and removal. It compares the performances of (i) rule-based method, (ii) count-based method together with a ML algorithm, and (iii) a transformers-based model.

RESULTS

Results, obtained with three different approaches, were evaluated in terms of weighted F1 Score. The random forest classifier showed the higher performance in both tasks reaching 82.35%.

CONCLUSION

The present work constitutes a first step towards the automatic calculation of the EQUAL Candida Score from unstructured daily collected data by combining ML and NLP methods. The automatic calculation of the EQUAL Candida Score could provide crucial real-time feedback on the quality of candidemia management, aimed at further improving patients' health.

Applied artificial intelligence in dentistry: emerging data modalities and modeling approaches

Artificial intelligence (AI) is increasingly applied across all disciplines of medicine, including dentistry. Oral health research is experiencing a rapidly increasing use of machine learning (ML), the branch of AI that identifies inherent patterns in data similarly to how humans learn. In contemporary clinical dentistry, ML supports computer-aided diagnostics, risk stratification, individual risk prediction, and decision support to ultimately improve clinical oral health care efficiency, outcomes, and reduce disparities. Further, ML is progressively used in dental and oral health research, from basic and translational science to clinical investigations. With an ML perspective, this review provides a comprehensive overview of how dental medicine leverages AI for diagnostic, prognostic, and generative tasks. The spectrum of available data modalities in dentistry and their compatibility with various methods of applied AI are presented. Finally, current challenges and limitations as well as future possibilities and considerations for AI application in dental medicine are summarized.

LSTM-Based Prediction Model for Tuberculosis Among HIV-Infected Patients Using Structured Electronic Medical Records: A Retrospective Machine Learning Study

Background

Both HIV and TB are chronic infectious diseases requiring long-term treatment and follow-up, resulting in extensive electronic medical records. With the exponential growth of health and medical big data, effectively extracting and analyzing these data has become the research hotspot. As a fundamental aspect of artificial intelligence, machine learning has been extensively applied in medical research, encompassing diagnosis, treatment, patient monitoring, drug development, and epidemiological investigations. This significantly enhances medical information systems and facilitates the interoperability of medical data.

Methods

In our study, we analyzed longitudinal data from the electronic health records of 4540 patients, gathered from the National Clinical Research Center for Infectious Diseases in Shenzhen, China, spanning from 2017 to 2021. Initially, we employed the fine-tuned ChatGLM to structure the electronic medical records. Subsequently, we utilized a multi-layer perceptron to classify each patient and determined the presence of tuberculosis in HIV patients. Using machine learning-based natural language processing, we structured these records to build a specialized database for HIV and TB co-infection. We studied the epidemiological characteristics, focusing on incidence patterns, patient characteristics, and influencing factors, to uncover the transmission characteristics of these diseases in Shenzhen. Additionally, we used Long Short-Term Memory to create a predictive model for TB co-infection among HIV patients, based on their medical records. This model predicted the risk of TB co-infection, providing scientific evidence for clinical decision-making and enabling early detection and precise intervention.

Results

Based on the refined ChatGLM model tailored for structured electronic health records, the accuracy of symptom extraction consistently surpassed 0.95 precision. Key symptoms such as diarrhea and normal showed precision rates exceeding 0.90. High scores were also achieved in recall and F1 scores. Among 4540 HIV patients, 758 were diagnosed with concurrent tuberculosis, indicating a 16.7% co-infection rate, while syphilis co-infection affected 25.1%, underscoring the prevalence of concurrent infections among HIV patients. Utilizing electronic health records, a Multilayer Perceptron classifier was developed as a benchmark against Long Short-Term Memory to predict high-risk groups for HIV and tuberculosis co-infections. The Multilayer Perceptron classifier demonstrated predictive ability with AUROC values ranging from 0.616 to 0.682 on the test set, suggesting opportunities for further optimization and generalization despite its accuracy in identifying HIV-TB co-infections. In tuberculosis intelligent diagnosis based on laboratory results, the Long Short-Term Memory showed consistent performance across 5-fold cross-validation, with AUROC values ranging from 0.827 to 0.850, indicating reliability and consistency in tuberculosis prediction. Furthermore, by optimizing classification thresholds, the model achieved an overall accuracy of 81.18% in distinguishing HIV co-infected tuberculosis from simple HIV infection.

Conclusion

Combining the Multilayer Perceptron classifier with Long Short-Term Memory represented an advanced approach for effectively extracting electronic health records and utilizing it for disease prediction. This underscored the superior performance of deep learning techniques in managing both structured and unstructured medical data. Models leveraging laboratory time-series data demonstrated notably better performance compared to those relying solely on electronic health records for predicting tuberculosis incidence. This emphasized the benefits of deep learning in handling intricate medical data and provided valuable insights for healthcare providers exploring the use of deep learning in disease prediction and management.

The use of natural language processing for the identification of ageing syndromes including sarcopenia, frailty and falls in electronic healthcare records: a systematic review

BACKGROUND

Recording and coding of ageing syndromes in hospital records is known to be suboptimal. Natural Language Processing algorithms may be useful to identify diagnoses in electronic healthcare records to improve the recording and coding of these ageing syndromes, but the feasibility and diagnostic accuracy of such algorithms are unclear.

METHODS

We conducted a systematic review according to a predefined protocol and in line with Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. Searches were run from the inception of each database to the end of September 2023 in PubMed, Medline, Embase, CINAHL, ACM digital library, IEEE Xplore and Scopus. Eligible studies were identified via independent review of search results by two coauthors and data extracted from each study to identify the computational method, source of text, testing strategy and performance metrics. Data were synthesised narratively by ageing syndrome and computational method in line with the Studies Without Meta-analysis guidelines.

RESULTS

From 1030 titles screened, 22 studies were eligible for inclusion. One study focussed on identifying sarcopenia, one frailty, twelve falls, five delirium, five dementia and four incontinence. Sensitivity (57.1%-100%) of algorithms compared with a reference standard was reported in 20 studies, and specificity (84.0%-100%) was reported in only 12 studies. Study design quality was variable with results relevant to diagnostic accuracy not always reported, and few studies undertaking external validation of algorithms.

CONCLUSIONS

Current evidence suggests that Natural Language Processing algorithms can identify ageing syndromes in electronic health records. However, algorithms require testing in rigorously designed diagnostic accuracy studies with appropriate metrics reported.

Natural Language Processing Algorithm to Extract Multiple Myeloma Stage From Oncology Notes in the Veterans Affairs Healthcare System

PURPOSE

Stage in multiple myeloma (MM) is an essential measure of disease risk, but its measurement in large databases is often lacking. We aimed to develop and validate a natural language processing (NLP) algorithm to extract oncologists' documentation of stage in the national Veterans Affairs (VA) Healthcare System.

METHODS

Using nationwide electronic health record (EHR) and cancer registry data from the VA Corporate Data Warehouse, we developed and validated a rule-based NLP algorithm to extract oncologist-determined MM stage. To that end, a clinician annotated MM stage within over 5,000 short snippets of clinical notes, and annotated MM stage at MM treatment initiation for 200 patients. These were allocated into snippet- and patient-level development and validation sets. We developed MM stage extraction and roll-up algorithms within the development sets. After the algorithms were finalized, we validated them using standard measures in held-out validation sets.

RESULTS

We developed algorithms for three different MM staging systems that have been in widespread use (Revised International Staging System [R-ISS], International Staging System [ISS], and Durie-Salmon [DS]) and for stage reported without a clearly defined system. Precision and recall were uniformly high for MM stage at the snippet level, ranging from 0.92 to 0.99 for the different MM staging systems. Performance in identifying for MM stage at treatment initiation at the patient level was also excellent, with precision of 0.92, 0.96, 0.90, and 0.86 and recall of 0.99, 0.98, 0.94, and 0.92 for R-ISS, ISS, DS, and unclear stage, respectively.

CONCLUSION

Our MM stage extraction algorithm uses rule-based NLP and data aggregation to accurately measure MM stage documented in oncology notes and pathology reports in VA's national EHR system. It may be adapted to other systems where MM stage is recorded in clinical notes.

Using natural language processing in emergency medicine health service research: A systematic review and meta-analysis

OBJECTIVES

Natural language processing (NLP) represents one of the adjunct technologies within artificial intelligence and machine learning, creating structure out of unstructured data. This study aims to assess the performance of employing NLP to identify and categorize unstructured data within the emergency medicine (EM) setting.

METHODS

We systematically searched publications related to EM research and NLP across databases including MEDLINE, Embase, Scopus, CENTRAL, and ProQuest Dissertations & Theses Global. Independent reviewers screened, reviewed, and evaluated article quality and bias. NLP usage was categorized into syndromic surveillance, radiologic interpretation, and identification of specific diseases/events/syndromes, with respective sensitivity analysis reported. Performance metrics for NLP usage were calculated and the overall area under the summary of receiver operating characteristic curve (SROC) was determined.

RESULTS

A total of 27 studies underwent meta-analysis. Findings indicated an overall mean sensitivity (recall) of 82%-87%, specificity of 95%, with the area under the SROC at 0.96 (95% CI 0.94-0.98). Optimal performance using NLP was observed in radiologic interpretation, demonstrating an overall mean sensitivity of 93% and specificity of 96%.

CONCLUSIONS

Our analysis revealed a generally favorable performance accuracy in using NLP within EM research, particularly in the realm of radiologic interpretation. Consequently, we advocate for the adoption of NLP-based research to augment EM health care management.

Artificial Intelligence in Cardiovascular Disease Prevention: Is it Ready for Prime Time?

PURPOSE OF REVIEW

This review evaluates how Artificial Intelligence (AI) enhances atherosclerotic cardiovascular disease (ASCVD) risk assessment, allows for opportunistic screening, and improves adherence to guidelines through the analysis of unstructured clinical data and patient-generated data. Additionally, it discusses strategies for integrating AI into clinical practice in preventive cardiology.

RECENT FINDINGS

AI models have shown superior performance in personalized ASCVD risk evaluations compared to traditional risk scores. These models now support automated detection of ASCVD risk markers, including coronary artery calcium (CAC), across various imaging modalities such as dedicated ECG-gated CT scans, chest X-rays, mammograms, coronary angiography, and non-gated chest CT scans. Moreover, large language model (LLM) pipelines are effective in identifying and addressing gaps and disparities in ASCVD preventive care, and can also enhance patient education. AI applications are proving invaluable in preventing and managing ASCVD and are primed for clinical use, provided they are implemented within well-regulated, iterative clinical pathways.

Characterizing Myasthenia Gravis Symptoms, Exacerbations, and Crises From Neurologist's Clinical Notes Using Natural Language Processing

Background Myasthenia gravis (MG) is a rare, autoantibody neuromuscular disorder characterized by fatigable weakness. Real-world evidence based on administrative and structured datasets regarding MG may miss important details related to the clinical encounter. Examination of free-text clinical progress notes has the potential to illuminate aspects of MG care. Objective The primary objective was to examine and characterize neurologist progress notes in the care of individuals with MG regarding the prevalence of documentation of clinical subtypes, antibody status, symptomatology, and MG deteriorations, including exacerbations and crises. The secondary objectives were to categorize MG deteriorations into practical, objective states as well as examine potential sources of clinical inertia in MG care. Methods We performed a retrospective, cross-sectional analysis of de-identified neurologist clinical notes from 2017 to 2022. A qualitative analysis of physician descriptions of MG deteriorations and a discussion of risks in MG care (risk for adverse effects, risk for clinical decompensation, etc.) was performed. Results Of the 3,085 individuals with MG, clinical subtypes and antibody status identified included gMG (n = 400; 13.0%), ocular MG (n = 253; 8.2%), MG unspecified (2,432; 78.8%), seropositivity for acetylcholine receptor antibody (n = 441; 14.3%), and MuSK antibody (n = 29; 0.9%). The most common gMG manifestations were dysphagia (n = 712; 23.0%), dyspnea (n = 626; 20.3%), and dysarthria (n = 514; 16.7%). In MG crisis patients, documentation of difficulties with MG standard therapies was common (n = 62; 45.2%). The qualitative analysis of MG deterioration types includes symptom fluctuation, symptom worsening with treatment intensification, MG deterioration with rescue therapy, and MG crisis. Qualitative analysis of MG-related risks included the toxicity of new therapies and concern for worsening MG because of changing therapies. Conclusions This study of neurologist progress notes demonstrates the potential for real-world evidence generation in the care of individuals with MG. MG patients suffer fluctuating symptomatology and a spectrum of clinical deteriorations. Adverse effects of MG therapies are common, highlighting the need for effective, less toxic treatments.

Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review

Objective

Natural language processing (NLP) can enhance research on activities of daily living (ADL) by extracting structured information from unstructured electronic health records (EHRs) notes. This review aims to give insight into the state-of-the-art, usability, and performance of NLP systems to extract information on ADL from EHRs.

Materials and Methods

A systematic review was conducted based on searches in Pubmed, Embase, Cinahl, Web of Science, and Scopus. Studies published between 2017 and 2022 were selected based on predefined eligibility criteria.

Results

The review identified 22 studies. Most studies (65%) used NLP for classifying unstructured EHR data on 1 or 2 ADL. Deep learning, combined with a ruled-based method or machine learning, was the approach most commonly used. NLP systems varied widely in terms of the pre-processing and algorithms. Common performance evaluation methods were cross-validation and train/test datasets, with F1, precision, and sensitivity as the most frequently reported evaluation metrics. Most studies reported relativity high overall scores on the evaluation metrics.

Discussion

NLP systems are valuable for the extraction of unstructured EHR data on ADL. However, comparing the performance of NLP systems is difficult due to the diversity of the studies and challenges related to the dataset, including restricted access to EHR data, inadequate documentation, lack of granularity, and small datasets.

Conclusion

This systematic review indicates that NLP is promising for deriving information on ADL from unstructured EHR notes. However, what the best-performing NLP system is, depends on characteristics of the dataset, research question, and type of ADL.

Collecting routine and timely cancer stage at diagnosis by implementing a cancer staging tiered framework: the Western Australian Cancer Registry experience

BACKGROUND

Current processes collecting cancer stage data in population-based cancer registries (PBCRs) lack standardisation, resulting in difficulty utilising diverse data sources and incomplete, low-quality data. Implementing a cancer staging tiered framework aims to improve stage collection and facilitate inter-PBCR benchmarking.

OBJECTIVE

Demonstrate the application of a cancer staging tiered framework in the Western Australian Cancer Staging Project to establish a standardised method for collecting cancer stage at diagnosis data in PBCRs.

METHODS

The tiered framework, developed in collaboration with a Project Advisory Group and applied to breast, colorectal, and melanoma cancers, provides business rules - procedures for stage collection. Tier 1 represents the highest staging level, involving complete American Joint Committee on Cancer (AJCC) tumour-node-metastasis (TNM) data collection and other critical staging information. Tier 2 (registry-derived stage) relies on supplementary data, including hospital admission data, to make assumptions based on data availability. Tier 3 (pathology stage) solely uses pathology reports.

FINDINGS

The tiered framework promotes flexible utilisation of staging data, recognising various levels of data completeness. Tier 1 is suitable for all purposes, including clinical and epidemiological applications. Tiers 2 and 3 are recommended for epidemiological analysis alone. Lower tiers provide valuable insights into disease patterns, risk factors, and overall disease burden for public health planning and policy decisions. Capture of staging at each tier depends on data availability, with potential shifts to higher tiers as new data sources are acquired.

CONCLUSIONS

The tiered framework offers a dynamic approach for PBCRs to record stage at diagnosis, promoting consistency in population-level staging data and enabling practical use for benchmarking across jurisdictions, public health planning, policy development, epidemiological analyses, and assessing cancer outcomes. Evolution with staging classifications and data variable changes will futureproof the tiered framework. Its adaptability fosters continuous refinement of data collection processes and encourages improvements in data quality.

Vocabulary Matters: An Annotation Pipeline and Four Deep Learning Algorithms for Enzyme Named Entity Recognition

Enzymes are indispensable in many biological processes, and with biomedical literature growing exponentially, effective literature review becomes increasingly challenging. Natural language processing methods offer solutions to streamline this process. This study aims to develop an annotated enzyme corpus for training and evaluating enzyme named entity recognition (NER) models. A novel pipeline, combining dictionary matching and rule-based keyword searching, automatically annotated enzyme entities in >4800 full-text publications. Four deep learning NER models were created with different vocabularies (BioBERT/SciBERT) and architectures (BiLSTM/transformer) and evaluated on 526 manually annotated full-text publications. The annotation pipeline achieved an F1-score of 0.86 (precision = 1.00, recall = 0.76), surpassed by fine-tuned transformers for F1-score (BioBERT: 0.89, SciBERT: 0.88) and recall (0.86) with BiLSTM models having higher precision (0.94) than transformers (0.92). The annotation pipeline runs in seconds on standard laptops with almost perfect precision, but was outperformed by fine-tuned transformers in terms of F1-score and recall, demonstrating generalizability beyond the training data. In comparison, SciBERT-based models exhibited higher precision, and BioBERT-based models exhibited higher recall, highlighting the importance of vocabulary and architecture. These models, representing the first enzyme NER algorithms, enable more effective enzyme text mining and information extraction. Codes for automated annotation and model generation are available from https://github.com/omicsNLP/enzymeNER and https://zenodo.org/doi/10.5281/zenodo.10581586.

Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models

BACKGROUND

Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms.

OBJECTIVES

This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes.

METHODS

The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level.

RESULTS

A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model.

CONCLUSIONS

The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.

Collaborative and privacy-enhancing workflows on a clinical data warehouse: an example developing natural language processing pipelines to detect medical conditions

OBJECTIVE

To develop and validate a natural language processing (NLP) pipeline that detects 18 conditions in French clinical notes, including 16 comorbidities of the Charlson index, while exploring a collaborative and privacy-enhancing workflow.

MATERIALS AND METHODS

The detection pipeline relied both on rule-based and machine learning algorithms, respectively, for named entity recognition and entity qualification, respectively. We used a large language model pre-trained on millions of clinical notes along with annotated clinical notes in the context of 3 cohort studies related to oncology, cardiology, and rheumatology. The overall workflow was conceived to foster collaboration between studies while respecting the privacy constraints of the data warehouse. We estimated the added values of the advanced technologies and of the collaborative setting.

RESULTS

The pipeline reached macro-averaged F1-score positive predictive value, sensitivity, and specificity of 95.7 (95%CI 94.5-96.3), 95.4 (95%CI 94.0-96.3), 96.0 (95%CI 94.0-96.7), and 99.2 (95%CI 99.0-99.4), respectively. F1-scores were superior to those observed using alternative technologies or non-collaborative settings. The models were shared through a secured registry.

CONCLUSIONS

We demonstrated that a community of investigators working on a common clinical data warehouse could efficiently and securely collaborate to develop, validate and use sensitive artificial intelligence models. In particular, we provided an efficient and robust NLP pipeline that detects conditions mentioned in clinical notes.

SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials

Background

Initial insights into oncology clinical trial outcomes are often gleaned manually from conference abstracts. We aimed to develop an automated system to extract safety and efficacy information from study abstracts with high precision and fine granularity, transforming them into computable data for timely clinical decision-making.

Methods

We collected clinical trial abstracts from key conferences and PubMed (2012-2023). The SEETrials system was developed with four modules: preprocessing, prompt modeling, knowledge ingestion and postprocessing. We evaluated the system's performance qualitatively and quantitatively and assessed its generalizability across different cancer types- multiple myeloma (MM), breast, lung, lymphoma, and leukemia. Furthermore, the efficacy and safety of innovative therapies, including CAR-T, bispecific antibodies, and antibody-drug conjugates (ADC), in MM were analyzed across a large scale of clinical trial studies.

Results

SEETrials achieved high precision (0.958), recall (sensitivity) (0.944), and F1 score (0.951) across 70 data elements present in the MM trial studies Generalizability tests on four additional cancers yielded precision, recall, and F1 scores within the 0.966-0.986 range. Variation in the distribution of safety and efficacy-related entities was observed across diverse therapies, with certain adverse events more common in specific treatments. Comparative performance analysis using overall response rate (ORR) and complete response (CR) highlighted differences among therapies: CAR-T (ORR: 88%, 95% CI: 84-92%; CR: 95%, 95% CI: 53-66%), bispecific antibodies (ORR: 64%, 95% CI: 55-73%; CR: 27%, 95% CI: 16-37%), and ADC (ORR: 51%, 95% CI: 37-65%; CR: 26%, 95% CI: 1-51%). Notable study heterogeneity was identified (>75% I 2 heterogeneity index scores) across several outcome entities analyzed within therapy subgroups.

Conclusion

SEETrials demonstrated highly accurate data extraction and versatility across different therapeutics and various cancer domains. Its automated processing of large datasets facilitates nuanced data comparisons, promoting the swift and effective dissemination of clinical insights.

AssistMED project: Transforming cardiology cohort characterisation from electronic health records through natural language processing - Algorithm design, preliminary results, and field prospects

INTRODUCTION

Electronic health records (EHR) are of great value for clinical research. However, EHR consists primarily of unstructured text which must be analysed by a human and coded into a database before data analysis- a time-consuming and costly process limiting research efficiency. Natural language processing (NLP) can facilitate data retrieval from unstructured text. During AssistMED project, we developed a practical, NLP tool that automatically provides comprehensive clinical characteristics of patients from EHR, that is tailored to clinical researchers needs.

MATERIAL AND METHODS

AssistMED retrieves patient characteristics regarding clinical conditions, medications with dosage, and echocardiographic parameters with clinically oriented data structure and provides researcher-friendly database output. We validate the algorithm performance against manual data retrieval and provide critical quantitative and qualitative analysis.

RESULTS

AssistMED analysed the presence of 56 clinical conditions, medications from 16 drug groups with dosage and 15 numeric echocardiographic parameters in a sample of 400 patients hospitalized in the cardiology unit. No statistically significant differences between algorithm and human retrieval were noted. Qualitative analysis revealed that disagreements with manual annotation were primarily accounted to random algorithm errors, erroneous human annotation and lack of advanced context awareness of our tool.

CONCLUSIONS

Current NLP approaches are feasible to acquire accurate and detailed patient characteristics tailored to clinical researchers' needs from EHR. We present an in-depth description of an algorithm development and validation process, discuss obstacles and pinpoint potential solutions, including opportunities arising with recent advancements in the field of NLP, such as large language models.

Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease

Aims

ICD codes are used for classification of hospitalizations. The codes are used for administrative, financial, and research purposes. It is known, however, that errors occur. Natural language processing (NLP) offers promising solutions for optimizing the process. To investigate methods for automatic classification of disease in unstructured medical records using NLP and to compare these to conventional ICD coding.

Methods and results

Two datasets were used: the open-source Medical Information Mart for Intensive Care (MIMIC)-III dataset (n = 55.177) and a dataset from a hospital in Belgium (n = 12.706). Automated searches using NLP algorithms were performed for the diagnoses 'atrial fibrillation (AF)' and 'heart failure (HF)'. Four methods were used: rule-based search, logistic regression, term frequency-inverse document frequency (TF-IDF), Extreme Gradient Boosting (XGBoost), and Bio-Bidirectional Encoder Representations from Transformers (BioBERT). All algorithms were developed on the MIMIC-III dataset. The best performing algorithm was then deployed on the Belgian dataset. After preprocessing a total of 1438 reports was retained in the Belgian dataset. XGBoost on TF-IDF matrix resulted in an accuracy of 0.94 and 0.92 for AF and HF, respectively. There were 211 mismatches between algorithm and ICD codes. One hundred and three were due to a difference in data availability or differing definitions. In the remaining 108 mismatches, 70% were due to incorrect labelling by the algorithm and 30% were due to erroneous ICD coding (2% of total hospitalizations).

Conclusion

A newly developed NLP algorithm attained a high accuracy for classifying disease in medical records. XGBoost outperformed the deep learning technique BioBERT. NLP algorithms could be used to identify ICD-coding errors and optimize and support the ICD-coding process.

Bridging information gaps in menopause status classification through natural language processing

Objective

To use natural language processing (NLP) of clinical notes to augment existing structured electronic health record (EHR) data for classification of a patient's menopausal status.

Materials and methods

A rule-based NLP system was designed to capture evidence of a patient's menopause status including dates of a patient's last menstrual period, reproductive surgeries, and postmenopause diagnosis as well as their use of birth control and menstrual interruptions. NLP-derived output was used in combination with structured EHR data to classify a patient's menopausal status. NLP processing and patient classification were performed on a cohort of 307 512 female Veterans receiving healthcare at the US Department of Veterans Affairs (VA).

Results

NLP was validated at 99.6% precision. Including the NLP-derived data into a menopause phenotype increased the number of patients with data relevant to their menopausal status by 118%. Using structured codes alone, 81 173 (27.0%) are able to be classified as postmenopausal or premenopausal. However, with the inclusion of NLP, this number increased 167 804 (54.6%) patients. The premenopausal category grew by 532.7% with the inclusion of NLP data.

Discussion

By employing NLP, it became possible to identify documented data elements that predate VA care, originate outside VA networks, or have no corresponding structured field in the VA EHR that would be otherwise inaccessible for further analysis.

Conclusion

NLP can be used to identify concepts relevant to a patient's menopausal status in clinical notes. Adding NLP-derived data to an algorithm classifying a patient's menopausal status significantly increases the number of patients classified using EHR data, ultimately enabling more detailed assessments of the impact of menopause on health outcomes.

Bolstering Advance Care Planning Measurement Using Natural Language Processing

Despite its growth as a clinical activity and research topic, the complex dynamic nature of advance care planning (ACP) has posed serious challenges for researchers hoping to quantitatively measure it. Methods for measurement have traditionally depended on lengthy manual chart abstractions or static documents (e.g., advance directive forms) even though completion of such documents is only one aspect of ACP. Natural language processing (NLP), in the form of an assisted electronic health record (EHR) review, is a technological advancement that may help researchers better measure ACP activity. In this article, we aim to show how NLP-assisted EHR review supports more accurate and robust measurement of ACP. We do so by presenting three example applications that illustrate how using NLP for this purpose supports (1) measurement in research, (2) detailed insights into ACP in quality improvement, and (3) identification of current limitations of ACP in clinical settings.