1
|
Danna G, Garg R, Buchheit J, Patel R, Zhan T, Ellyn A, Maqbool F, Yala L, Moklyak Y, Frydman J, Kho A, Kong N, Furmanchuk A, Lundberg A, Stey AM. Prediction of intra-abdominal injury using natural language processing of electronic medical record data. Surgery 2024:S0039-6060(24)00373-8. [PMID: 38972771 DOI: 10.1016/j.surg.2024.05.042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Revised: 05/12/2024] [Accepted: 05/28/2024] [Indexed: 07/09/2024]
Abstract
BACKGROUND This study aimed to use natural language processing to predict the presence of intra-abdominal injury using unstructured data from electronic medical records. METHODS This was a random-sample retrospective observational cohort study leveraging unstructured data from injured patients taken to one of 9 acute care hospitals in an integrated health system between 2015 and 2021. Patients with International Classification of Diseases External Cause of Morbidity codes were identified. History and physical, consult, progress, and radiology report text from the first 8 hours of care were abstracted. Annotator dyads independently annotated encounters' text files to establish ground truth regarding whether intra-abdominal injury occurred. Features were extracted from text using natural language processing techniques, bag of words, and principal component analysis. We tested logistic regression, random forests, and gradient boosting machine to determine accuracy, recall, and precision of natural language processing to predict intra-abdominal injury. RESULTS A random sample of 7,000 patient encounters of 177,127 was annotated. Only 2,951 had sufficient information to determine whether an intra-abdominal injury was present. Among those, 84 (2.9%) had an intra-abdominal injury. The concordance between annotators was 0.989. Logistic regression of features identified with bag of words and principal component analysis had the best predictive ability, with an area under the receiver operating characteristic curve of 0.9, recall of 0.73, and precision of 0.17. Text features with greatest importance included "abdomen," "pelvis," "spleen," and "hematoma." CONCLUSION Natural language processing could be a screening decision support tool, which, if paired with human clinical assessment, can maximize precision of intra-abdominal injury identification.
Collapse
Affiliation(s)
- Giovanna Danna
- Chicago Medical School, Rosalind Franklin University, Chicago, IL
| | - Ravi Garg
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Joanna Buchheit
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Radha Patel
- Chicago Medical School, Rosalind Franklin University, Chicago, IL
| | - Tiannan Zhan
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Alexander Ellyn
- Chicago Medical School, Rosalind Franklin University, Chicago, IL
| | - Farhan Maqbool
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Linda Yala
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Yuriy Moklyak
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - James Frydman
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Abel Kho
- Feinberg School of Medicine, Northwestern University, Chicago, IL. https://www.twitter.com/Abelkho
| | - Nan Kong
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN
| | - Alona Furmanchuk
- Feinberg School of Medicine, Northwestern University, Chicago, IL
| | | | - Anne M Stey
- Feinberg School of Medicine, Northwestern University, Chicago, IL.
| |
Collapse
|
2
|
Miller M, Jorm L, Partyka C, Burns B, Habig K, Oh C, Immens S, Ballard N, Gallego B. Identifying prehospital trauma patients from ambulance patient care records; comparing two methods using linked data in New South Wales, Australia. Injury 2024; 55:111570. [PMID: 38664086 DOI: 10.1016/j.injury.2024.111570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Revised: 04/11/2024] [Accepted: 04/14/2024] [Indexed: 06/16/2024]
Abstract
BACKGROUND Linked datasets for trauma system monitoring should ideally follow patients from the prehospital scene to hospital admission and post-discharge. Having a well-defined cohort when using administrative datasets is essential because they must capture the representative population. Unlike hospital electronic health records (EHR), ambulance patient-care records lack access to sources beyond immediate clinical notes. Relying on a limited set of variables to define a study population might result in missed patient inclusion. We aimed to compare two methods of identifying prehospital trauma patients: one using only those documented under a trauma protocol and another incorporating additional data elements from ambulance patient care records. METHODS We analyzed data from six routinely collected administrative datasets from 2015 to 2018, including ambulance patient-care records, aeromedical data, emergency department visits, hospitalizations, rehabilitation outcomes, and death records. Three prehospital trauma cohorts were created: an Extended-T-protocol cohort (patients transported under a trauma protocol and/or patients with prespecified criteria from structured data fields), T-protocol cohort (only patients documented as transported under a trauma protocol) and non-T-protocol (extended-T-protocol population not in the T-protocol cohort). Patient-encounter characteristics, mortality, clinical and post-hospital discharge outcomes were compared. A conservative p-value of 0.01 was considered significant RESULTS: Of 1 038 263 patient-encounters included in the extended-T-population 814 729 (78.5 %) were transported, with 438 893 (53.9 %) documented as a T-protocol patient. Half (49.6 %) of the non-T-protocol sub-cohort had an International Classification of Disease 10th edition injury or external cause code, indicating 79644 missed patients when a T-protocol-only definition was used. The non-T-protocol sub-cohort also identified additional patients with intubation, prehospital blood transfusion and positive eFAST. A higher proportion of non-T protocol patients than T-protocol patients were admitted to the ICU (4.6% vs 3.6 %), ventilated (1.8% vs 1.3 %), received in-hospital transfusion (7.9 vs 6.8 %) or died (1.8% vs 1.3 %). Urgent trauma surgery was similar between groups (1.3% vs 1.4 %). CONCLUSION The extended-T-population definition identified 50 % more admitted patients with an ICD-10-AM code consistent with an injury, including patients with severe trauma. Developing an EHR phenotype incorporating multiple data fields of ambulance-transported trauma patients for use with linked data may avoid missing these patients.
Collapse
Affiliation(s)
- Matthew Miller
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia; Department of Anesthesia, St George Hospital, Kogarah, NSW 2217 Australia; Centre for Big Data Research in Health at UNSW Sydney, Kensington, NSW 2052, Australia.
| | - Louisa Jorm
- Foundation Director of the Centre for Big Data Research in Health at UNSW Sydney, Kensington 2052, Australia
| | - Chris Partyka
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia; Department of Emergency Medicine, Royal North Shore Hospital, St Leonards, NSW 2065, Australia
| | - Brian Burns
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia; Royal North Shore Hospital, St Leonards, NSW 2065, Australia; Faculty of Medicine & Health, University of Sydney, Camperdown, NSW 2050, Australia
| | - Karel Habig
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia
| | - Carissa Oh
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia; Department of Emergency Medicine, St George Hospital, Kogarah, NSW 2217 Australia
| | - Sam Immens
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia
| | - Neil Ballard
- Aeromedical Operations, New South Wales Ambulance, Rozelle, NSW 2039, Australia; Department of Paediatric Emergency Medicine, Sydney Children's Hospital, Randwick, NSW 2031, Australia; Department of Emergency Medicine, Royal Prince Alfred Hospital, Camperdown, NSW 2050, Australia
| | - Blanca Gallego
- Clinical analytics and machine learning unit, Centre for Big Data Research in Health at UNSW Sydney, Kensington 2052, Australia
| |
Collapse
|
3
|
Karway GK, Koyner JL, Caskey J, Spicer AB, Carey KA, Gilbert ER, Dligach D, Mayampurath A, Afshar M, Churpek MM. Development and external validation of multimodal postoperative acute kidney injury risk machine learning models. JAMIA Open 2023; 6:ooad109. [PMID: 38144168 PMCID: PMC10746378 DOI: 10.1093/jamiaopen/ooad109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 11/18/2023] [Accepted: 12/11/2023] [Indexed: 12/26/2023] Open
Abstract
Objectives To develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings. Materials and Methods Data for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong's test for statistical differences. Results The study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]). Discussion A multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models. Conclusion These findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.
Collapse
Affiliation(s)
- George K Karway
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
| | - Jay L Koyner
- Section of Nephrology, Department of Medicine, University of Chicago, Chicago, IL 60637, United States
| | - John Caskey
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
| | - Alexandra B Spicer
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
| | - Kyle A Carey
- Section of Nephrology, Department of Medicine, University of Chicago, Chicago, IL 60637, United States
| | - Emily R Gilbert
- Department of Medicine, Loyola University Chicago, Chicago, IL 60153, United States
| | - Dmitriy Dligach
- Department of Computer Science, Loyola University Chicago, Chicago, IL 60626, United States
| | - Anoop Mayampurath
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53726, United States
| | - Majid Afshar
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53726, United States
| | - Matthew M Churpek
- Department of Medicine, University of Wisconsin-Madison, Madison, WI 53792, United States
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53726, United States
| |
Collapse
|
4
|
Piliuk K, Tomforde S. Artificial intelligence in emergency medicine. A systematic literature review. Int J Med Inform 2023; 180:105274. [PMID: 37944275 DOI: 10.1016/j.ijmedinf.2023.105274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 10/21/2023] [Accepted: 10/26/2023] [Indexed: 11/12/2023]
Abstract
Motivation and objective: Emergency medicine is becoming a popular application area for artificial intelligence methods but remains less investigated than other healthcare branches. The need for time-sensitive decision-making on the basis of high data volumes makes the use of quantitative technologies inevitable. However, the specifics of healthcare regulations impose strict requirements for such applications. Published contributions cover separate parts of emergency medicine and use disparate data and algorithms. This study aims to systematize the relevant contributions, investigate the main obstacles to artificial intelligence applications in emergency medicine, and propose directions for further studies. METHODS The contributions selection process was conducted with systematic electronic databases querying and filtering with respect to established exclusion criteria. Among the 380 papers gathered from IEEE Xplore, ACM Digital Library, Springer Library, ScienceDirect, and Nature databases 116 were considered to be a part of the survey. The main features of the selected papers are the focus on emergency medicine and the use of machine learning or deep learning algorithms. FINDINGS AND DISCUSSION The selected papers were classified into two branches: diagnostics-specific and triage-specific. The former ones are focused on either diagnosis prediction or decision support. The latter covers such applications as mortality, outcome, admission prediction, condition severity estimation, and urgent care prediction. The observed contributions are highly specialized within a single disease or medical operation and often use privately collected retrospective data, making them incomparable. These and other issues can be addressed by creating an end-to-end solution based on human-machine interaction. CONCLUSION Artificial intelligence applications are finding their place in emergency medicine, while most of the corresponding studies remain isolated and lack higher generalization and more sophisticated methodology, which can be a matter of forthcoming improvements.
Collapse
Affiliation(s)
| | - Sven Tomforde
- Christian-Albrechts-Universität zu Kiel, 24118 Kiel, Germany
| |
Collapse
|
5
|
Bobba PS, Sailer A, Pruneski JA, Beck S, Mozayan A, Mozayan S, Arango J, Cohan A, Chheang S. Natural language processing in radiology: Clinical applications and future directions. Clin Imaging 2023; 97:55-61. [PMID: 36889116 DOI: 10.1016/j.clinimag.2023.02.014] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 02/10/2023] [Accepted: 02/20/2023] [Indexed: 03/07/2023]
Abstract
Natural language processing (NLP) is a wide range of techniques that allows computers to interact with human text. Applications of NLP in everyday life include language translation aids, chat bots, and text prediction. It has been increasingly utilized in the medical field with increased reliance on electronic health records. As findings in radiology are primarily communicated via text, the field is particularly suited to benefit from NLP based applications. Furthermore, rapidly increasing imaging volume will continue to increase burden on clinicians, emphasizing the need for improvements in workflow. In this article, we highlight the numerous non-clinical, provider focused, and patient focused applications of NLP in radiology. We also comment on challenges associated with development and incorporation of NLP based applications in radiology as well as potential future directions.
Collapse
Affiliation(s)
- Pratheek S Bobba
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Anne Sailer
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | | | - Spencer Beck
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Ali Mozayan
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Sara Mozayan
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Jennifer Arango
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Arman Cohan
- Department of Computer Science, Yale University, New Haven, CT, United States
| | - Sophie Chheang
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States.
| |
Collapse
|
6
|
Han R, Zhang Z, Wei H, Yin D. Chinese medical event detection based on event frequency distribution ratio and document consistency. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:11063-11080. [PMID: 37322971 DOI: 10.3934/mbe.2023489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Structured information especially medical events extracted from electronic medical records has extremely practical application value and play a basic role in various intelligent diagnosis and treatment systems. Fine-grained Chinese medical event detection is crucial in the process of structuring Chinese Electronic Medical Record (EMR). The current methods for detecting fine-grained Chinese medical events primarily rely on statistical machine learning and deep learning. However, they have two shortcomings: 1) they neglect to take into account the distribution characteristics of these fine-grained medical events. 2) they overlook the consistency in the distribution of medical events within each individual document. Therefore, this paper presents a fine-grained Chinese medical event detection method, which is based on event frequency distribution ratio and document consistency. To start with, a significant number of Chinese EMR texts are used to adapt the Chinese pre-training model BERT to the domain. Second, based on the fundamental features, the Event Frequency - Event Distribution Ratio (EF-DR) is devised to select distinct event information as supplementary features, taking into account the distribution of events within the EMR. Finally, using EMR document consistency within the model improves the outcome of event detection. Our experiments demonstrate that the proposed method significantly outperforms the baseline model.
Collapse
Affiliation(s)
- Ruirui Han
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Zhichang Zhang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Hao Wei
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Deyue Yin
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| |
Collapse
|
7
|
Bombieri M, Rospocher M, Ponzetto SP, Fiorini P. Machine understanding surgical actions from intervention procedure textbooks. Comput Biol Med 2023; 152:106415. [PMID: 36527782 DOI: 10.1016/j.compbiomed.2022.106415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 11/23/2022] [Accepted: 12/04/2022] [Indexed: 12/12/2022]
Abstract
The automatic extraction of procedural surgical knowledge from surgery manuals, academic papers or other high-quality textual resources, is of the utmost importance to develop knowledge-based clinical decision support systems, to automatically execute some procedure's step or to summarize the procedural information, spread throughout the texts, in a structured form usable as a study resource by medical students. In this work, we propose a first benchmark on extracting detailed surgical actions from available intervention procedure textbooks and papers. We frame the problem as a Semantic Role Labeling task. Exploiting a manually annotated dataset, we apply different Transformer-based information extraction methods. Starting from RoBERTa and BioMedRoBERTa pre-trained language models, we first investigate a zero-shot scenario and compare the obtained results with a full fine-tuning setting. We then introduce a new ad-hoc surgical language model, named SurgicBERTa, pre-trained on a large collection of surgical materials, and we compare it with the previous ones. In the assessment, we explore different dataset splits (one in-domain and two out-of-domain) and we investigate also the effectiveness of the approach in a few-shot learning scenario. Performance is evaluated on three correlated sub-tasks: predicate disambiguation, semantic argument disambiguation and predicate-argument disambiguation. Results show that the fine-tuning of a pre-trained domain-specific language model achieves the highest performance on all splits and on all sub-tasks. All models are publicly released.
Collapse
Affiliation(s)
- Marco Bombieri
- Department of Computer Science, University of Verona, Verona, Italy.
| | - Marco Rospocher
- Department of Foreign Languages and Literatures, University of Verona, Verona, Italy
| | | | - Paolo Fiorini
- Department of Computer Science, University of Verona, Verona, Italy
| |
Collapse
|
8
|
Baur D, Gehlen T, Scherer J, Back DA, Tsitsilonis S, Kabir K, Osterhoff G. Decision support by machine learning systems for acute management of severely injured patients: A systematic review. Front Surg 2022; 9:924810. [PMID: 36299574 PMCID: PMC9589228 DOI: 10.3389/fsurg.2022.924810] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 08/31/2022] [Indexed: 11/07/2022] Open
Abstract
Introduction Treating severely injured patients requires numerous critical decisions within short intervals in a highly complex situation. The coordination of a trauma team in this setting has been shown to be associated with multiple procedural errors, even of experienced care teams. Machine learning (ML) is an approach that estimates outcomes based on past experiences and data patterns using a computer-generated algorithm. This systematic review aimed to summarize the existing literature on the value of ML for the initial management of severely injured patients. Methods We conducted a systematic review of the literature with the goal of finding all articles describing the use of ML systems in the context of acute management of severely injured patients. MESH search of Pubmed/Medline and Web of Science was conducted. Studies including fewer than 10 patients were excluded. Studies were divided into the following main prediction groups: (1) injury pattern, (2) hemorrhage/need for transfusion, (3) emergency intervention, (4) ICU/length of hospital stay, and (5) mortality. Results Thirty-six articles met the inclusion criteria; among these were two prospective and thirty-four retrospective case series. Publication dates ranged from 2000 to 2020 and included 32 different first authors. A total of 18,586,929 patients were included in the prediction models. Mortality was the most represented main prediction group (n = 19). ML models used were artificial neural network ( n = 15), singular vector machine (n = 3), Bayesian network (n = 7), random forest (n = 6), natural language processing (n = 2), stacked ensemble classifier [SuperLearner (SL), n = 3], k-nearest neighbor (n = 1), belief system (n = 1), and sequential minimal optimization (n = 2) models. Thirty articles assessed results as positive, five showed moderate results, and one article described negative results to their implementation of the respective prediction model. Conclusions While the majority of articles show a generally positive result with high accuracy and precision, there are several requirements that need to be met to make the implementation of such models in daily clinical work possible. Furthermore, experience in dealing with on-site implementation and more clinical trials are necessary before the implementation of ML techniques in clinical care can become a reality.
Collapse
Affiliation(s)
- David Baur
- Department for Orthopedics and Traumatology, University Hospital Leipzig, Leipzig, Germany
| | - Tobias Gehlen
- Center for Musculoskeletal Surgery, Charité University Medicine Berlin, Berlin, Germany
| | - Julian Scherer
- Clinic for Traumatology, University Hospital Zurich, Zurich, Switzerland
| | - David Alexander Back
- Center for Musculoskeletal Surgery, Charité University Medicine Berlin, Berlin, Germany,Clinic for Traumatology and Orthopedics, Bundeswehr Hospital Berlin, Berlin, Germany
| | - Serafeim Tsitsilonis
- Center for Musculoskeletal Surgery, Charité University Medicine Berlin, Berlin, Germany
| | - Koroush Kabir
- Department of Orthopaedics and Trauma Surgery, University Hospital Bonn, Bonn, Germany
| | - Georg Osterhoff
- Department for Orthopedics, Traumatology and Plastic Surgery, University Hospital Leipzig, Leipzig, Germany,Correspondence: Georg Osterhoff
| |
Collapse
|
9
|
Masukawa K, Aoyama M, Yokota S, Nakamura J, Ishida R, Nakayama M, Miyashita M. Machine learning models to detect social distress, spiritual pain, and severe physical psychological symptoms in terminally ill patients with cancer from unstructured text data in electronic medical records. Palliat Med 2022; 36:1207-1216. [PMID: 35773973 DOI: 10.1177/02692163221105595] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
BACKGROUND Few studies have developed automatic systems for identifying social distress, spiritual pain, and severe physical and phycological symptoms from text data in electronic medical records. AIM To develop models to detect social distress, spiritual pain, and severe physical and psychological symptoms in terminally ill patients with cancer from unstructured text data contained in electronic medical records. DESIGN A retrospective study of 1,554,736 narrative clinical records was analyzed 1 month before patients died. Supervised machine learning models were trained to detect comprehensive symptoms, and the performance of the models was tested using the area under the receiver operating characteristic curve (AUROC) and precision recall curve (AUPRC). SETTING/PARTICIPANTS A total of 808 patients was included in the study using records obtained from a university hospital in Japan between January 1, 2018 and December 31, 2019. As training data, we used medical records labeled for detecting social distress (n = 10,000) and spiritual pain (n = 10,000), and records that could be combined with the Support Team Assessment Schedule (based on date) for detecting severe physical/psychological symptoms (n = 5409). RESULTS Machine learning models for detecting social distress had AUROC and AUPRC values of 0.98 and 0.61, respectively; values for spiritual pain, were 0.90 and 0.58, respectively. The machine learning models accurately identified severe symptoms (pain, dyspnea, nausea, insomnia, and anxiety) with a high level of discrimination (AUROC > 0.8). CONCLUSION The machine learning models could detect social distress, spiritual pain, and severe symptoms in terminally ill patients with cancer from text data contained in electronic medical records.
Collapse
Affiliation(s)
- Kento Masukawa
- Department of Palliative Nursing, Health Sciences, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Maho Aoyama
- Department of Palliative Nursing, Health Sciences, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Shinichiroh Yokota
- Faculty of Medicine, The University of Tokyo, Hongo, Tokyo, Japan.,Department of Healthcare Information Management, The University of Tokyo Hospital, Hongo, Tokyo, Japan
| | - Jyunya Nakamura
- Department of Palliative Nursing, Health Sciences, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Ryoka Ishida
- Department of Palliative Nursing, Health Sciences, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Masaharu Nakayama
- Department of Medical Informatics, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Mitsunori Miyashita
- Department of Palliative Nursing, Health Sciences, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| |
Collapse
|
10
|
Boonstra A, Laven M. Influence of artificial intelligence on the work design of emergency department clinicians a systematic literature review. BMC Health Serv Res 2022; 22:669. [PMID: 35585603 PMCID: PMC9118875 DOI: 10.1186/s12913-022-08070-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 05/11/2022] [Indexed: 11/30/2022] Open
Abstract
Objective This systematic literature review aims to demonstrate how Artificial Intelligence (AI) is currently used in emergency departments (ED) and how it alters the work design of ED clinicians. AI is still new and unknown to many healthcare professionals in emergency care, leading to unfamiliarity with its capabilities. Method Various criteria were used to establish the suitability of the articles to answer the research question. This study was based on 34 selected peer-reviewed papers on the use of Artificial Intelligence (AI) in the Emergency Department (ED), published in the last five years. Drawing on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, all articles were scanned, read full-text, and analyzed afterward. Results The majority of the AI applications consisted of AI-based tools to aid with clinical decisions and to relieve overcrowded EDs of their burden. AI support was mostly offered during triage, the moment that sets the patient trajectory. There is ample evidence that AI-based applications could improve the clinical decision-making process. Conclusion The use of AI in EDs is still in its nascent stages. Many studies focus on the question of whether AI has clinical utility, such as decision support, improving resource allocation, reducing diagnostic errors, and promoting proactivity. Some studies suggest that AI-based tools essentially have the ability to outperform human skills. However, it is evident from the literature that current technology does not have the aims or power to do so. Nevertheless, AI-based tools can impact clinician work design in the ED by providing support with clinical decisions, which could ultimately help alleviate a portion of the increasing clinical burden. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-022-08070-7.
Collapse
Affiliation(s)
- Albert Boonstra
- Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands.
| | - Mente Laven
- Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands
| |
Collapse
|
11
|
Ren G, Yu K, Xie Z, Liu L, Wang P, Zhang W, Wang Y, Wu X. Differentiation of lumbar disc herniation and lumbar spinal stenosis using natural language processing–based machine learning based on positive symptoms. Neurosurg Focus 2022; 52:E7. [DOI: 10.3171/2022.1.focus21561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Accepted: 01/20/2022] [Indexed: 11/06/2022]
Abstract
OBJECTIVE
The purpose of this study was to develop natural language processing (NLP)–based machine learning algorithms to automatically differentiate lumbar disc herniation (LDH) and lumbar spinal stenosis (LSS) based on positive symptoms in free-text admission notes. The secondary purpose was to compare the performance of the deep learning algorithm with the ensemble model on the current task.
METHODS
In total, 1921 patients whose principal diagnosis was LDH or LSS between June 2013 and June 2020 at Zhongda Hospital, affiliated with Southeast University, were retrospectively analyzed. The data set was randomly divided into a training set and testing set at a 7:3 ratio. Long Short-Term Memory (LSTM) and extreme gradient boosting (XGBoost) models were developed in this study. NLP algorithms were assessed on the testing set by the following metrics: receiver operating characteristic (ROC) curve, area under the curve (AUC), accuracy score, recall score, F1 score, and precision score.
RESULTS
In the testing set, the LSTM model achieved an AUC of 0.8487, accuracy score of 0.7818, recall score of 0.9045, F1 score of 0.8108, and precision score of 0.7347. In comparison, the XGBoost model achieved an AUC of 0.7565, accuracy score of 0.6961, recall score of 0.7387, F1 score of 0.7153, and precision score of 0.6934.
CONCLUSIONS
NLP-based machine learning algorithms were a promising auxiliary to the electronic health record in spine disease diagnosis. LSTM, the deep learning model, showed better capacity compared with the widely used ensemble model, XGBoost, in differentiation of LDH and LSS using positive symptoms. This study presents a proof of concept for the application of NLP in prediagnosis of spine disease.
Collapse
Affiliation(s)
- GuanRui Ren
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - Kun Yu
- Nanjing Jiangbei Hospital, Nanjing, Jiangsu, China
| | - ZhiYang Xie
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - Lei Liu
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - PeiYang Wang
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - Wei Zhang
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - YunTao Wang
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| | - XiaoTao Wu
- Zhongda Hospital, Medical College, Southeast University, Nanjing; and
| |
Collapse
|
12
|
Lee DY, Kim C, Lee S, Son SJ, Cho SM, Cho YH, Lim J, Park RW. Psychosis Relapse Prediction Leveraging Electronic Health Records Data and Natural Language Processing Enrichment Methods. Front Psychiatry 2022; 13:844442. [PMID: 35479497 PMCID: PMC9037331 DOI: 10.3389/fpsyt.2022.844442] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Accepted: 03/09/2022] [Indexed: 12/30/2022] Open
Abstract
BACKGROUND Identifying patients at a high risk of psychosis relapse is crucial for early interventions. A relevant psychiatric clinical context is often recorded in clinical notes; however, the utilization of unstructured data remains limited. This study aimed to develop psychosis-relapse prediction models using various types of clinical notes and structured data. METHODS Clinical data were extracted from the electronic health records of the Ajou University Medical Center in South Korea. The study population included patients with psychotic disorders, and outcome was psychosis relapse within 1 year. Using only structured data, we developed an initial prediction model, then three natural language processing (NLP)-enriched models using three types of clinical notes (psychological tests, admission notes, and initial nursing assessment) and one complete model. Latent Dirichlet Allocation was used to cluster the clinical context into similar topics. All models applied the least absolute shrinkage and selection operator logistic regression algorithm. We also performed an external validation using another hospital database. RESULTS A total of 330 patients were included, and 62 (18.8%) experienced psychosis relapse. Six predictors were used in the initial model and 10 additional topics from Latent Dirichlet Allocation processing were added in the enriched models. The model derived from all notes showed the highest value of the area under the receiver operating characteristic (AUROC = 0.946) in the internal validation, followed by models based on the psychological test notes, admission notes, initial nursing assessments, and structured data only (0.902, 0.855, 0.798, and 0.784, respectively). The external validation was performed using only the initial nursing assessment note, and the AUROC was 0.616. CONCLUSIONS We developed prediction models for psychosis relapse using the NLP-enrichment method. Models using clinical notes were more effective than models using only structured data, suggesting the importance of unstructured data in psychosis prediction.
Collapse
Affiliation(s)
- Dong Yun Lee
- Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, South Korea
| | - Chungsoo Kim
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, South Korea
| | - Seongwon Lee
- Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, South Korea.,Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, South Korea
| | - Sang Joon Son
- Department of Psychiatry, Ajou University School of Medicine, Suwon, South Korea
| | - Sun-Mi Cho
- Department of Psychiatry, Ajou University School of Medicine, Suwon, South Korea
| | - Yong Hyuk Cho
- Department of Psychiatry, Ajou University School of Medicine, Suwon, South Korea
| | - Jaegyun Lim
- Department of Laboratory Medicine, Myongji Hospital, Hanyang University College of Medicine, Goyang, South Korea
| | - Rae Woong Park
- Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, South Korea.,Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, South Korea
| |
Collapse
|
13
|
Kulshrestha S, Dligach D, Joyce C, Gonzalez R, O'Rourke AP, Glazer JM, Stey A, Kruser JM, Churpek MM, Afshar M. Comparison and interpretability of machine learning models to predict severity of chest injury. JAMIA Open 2021; 4:ooab015. [PMID: 33709067 PMCID: PMC7935500 DOI: 10.1093/jamiaopen/ooab015] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2020] [Revised: 02/08/2021] [Accepted: 02/12/2021] [Indexed: 11/15/2022] Open
Abstract
Objective Trauma quality improvement programs and registries improve care and outcomes for injured patients. Designated trauma centers calculate injury scores using dedicated trauma registrars; however, many injuries arrive at nontrauma centers, leaving a substantial amount of data uncaptured. We propose automated methods to identify severe chest injury using machine learning (ML) and natural language processing (NLP) methods from the electronic health record (EHR) for quality reporting. Materials and Methods A level I trauma center was queried for patients presenting after injury between 2014 and 2018. Prediction modeling was performed to classify severe chest injury using a reference dataset labeled by certified registrars. Clinical documents from trauma encounters were processed into concept unique identifiers for inputs to ML models: logistic regression with elastic net (EN) regularization, extreme gradient boosted (XGB) machines, and convolutional neural networks (CNN). The optimal model was identified by examining predictive and face validity metrics using global explanations. Results Of 8952 encounters, 542 (6.1%) had a severe chest injury. CNN and EN had the highest discrimination, with an area under the receiver operating characteristic curve of 0.93 and calibration slopes between 0.88 and 0.97. CNN had better performance across risk thresholds with fewer discordant cases. Examination of global explanations demonstrated the CNN model had better face validity, with top features including “contusion of lung” and “hemopneumothorax.” Discussion The CNN model featured optimal discrimination, calibration, and clinically relevant features selected. Conclusion NLP and ML methods to populate trauma registries for quality analyses are feasible.
Collapse
Affiliation(s)
- Sujay Kulshrestha
- Burn and Shock Trauma Research Institute, Loyola University Chicago, Maywood, Illinois, USA.,Department of Surgery, Loyola University Medical Center, Maywood, Illinois, USA
| | - Dmitriy Dligach
- Center for Health Outcomes and Informatics Research, Health Sciences Division, Loyola University Chicago, Maywood, Illinois, USA.,Department of Public Health Sciences, Stritch School of Medicine, Loyola University Chicago, Maywood, Illinois, USA.,Department of Computer Science, Loyola University Chicago, Chicago, Illinois, USA
| | - Cara Joyce
- Center for Health Outcomes and Informatics Research, Health Sciences Division, Loyola University Chicago, Maywood, Illinois, USA.,Department of Public Health Sciences, Stritch School of Medicine, Loyola University Chicago, Maywood, Illinois, USA
| | - Richard Gonzalez
- Burn and Shock Trauma Research Institute, Loyola University Chicago, Maywood, Illinois, USA.,Department of Surgery, Loyola University Medical Center, Maywood, Illinois, USA
| | - Ann P O'Rourke
- Department of Surgery, University of Wisconsin, Madison, Wisconsin, USA
| | - Joshua M Glazer
- Department of Emergency Medicine, University of Wisconsin, Madison, Wisconsin, USA
| | - Anne Stey
- Department of Surgery, Northwestern University, Chicago, Illinois, USA
| | | | - Matthew M Churpek
- Department of Medicine, University of Wisconsin, Madison, Wisconsin, USA
| | - Majid Afshar
- Department of Medicine, University of Wisconsin, Madison, Wisconsin, USA
| |
Collapse
|