51
|
Preston S, Wei M, Rao R, Tinn R, Usuyama N, Lucas M, Gu Y, Weerasinghe R, Lee S, Piening B, Tittel P, Valluri N, Naumann T, Bifulco C, Poon H. Toward structuring real-world data: Deep learning for extracting oncology information from clinical text with patient-level supervision. PATTERNS (NEW YORK, N.Y.) 2023; 4:100726. [PMID: 37123439 PMCID: PMC10140604 DOI: 10.1016/j.patter.2023.100726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Revised: 11/11/2022] [Accepted: 03/14/2023] [Indexed: 05/02/2023]
Abstract
Most detailed patient information in real-world data (RWD) is only consistently available in free-text clinical documents. Manual curation is expensive and time consuming. Developing natural language processing (NLP) methods for structuring RWD is thus essential for scaling real-world evidence generation. We propose leveraging patient-level supervision from medical registries, which are often readily available and capture key patient information, for general RWD applications. We conduct an extensive study on 135,107 patients from the cancer registry of a large integrated delivery network (IDN) comprising healthcare systems in five western US states. Our deep-learning methods attain test area under the receiver operating characteristic curve (AUROC) values of 94%-99% for key tumor attributes and comparable performance on held-out data from separate health systems and states. Ablation results demonstrate the superiority of these advanced deep-learning methods. Error analysis shows that our NLP system sometimes even corrects errors in registrar labels.
Collapse
Affiliation(s)
| | - Mu Wei
- Microsoft Research, Redmond, WA, USA
| | | | | | | | | | - Yu Gu
- Microsoft Research, Redmond, WA, USA
| | | | - Soohee Lee
- Providence St Joseph’s Health, Portland, OR, USA
| | - Brian Piening
- Providence Genomics & Earle A. Chiles Research Institute, Portland, OR, USA
| | - Paul Tittel
- Providence Genomics & Earle A. Chiles Research Institute, Portland, OR, USA
| | | | | | - Carlo Bifulco
- Providence Genomics & Earle A. Chiles Research Institute, Portland, OR, USA
- Corresponding author
| | - Hoifung Poon
- Microsoft Research, Redmond, WA, USA
- Corresponding author
| |
Collapse
|
52
|
Richter-Pechanski P, Wiesenbach P, Schwab DM, Kiriakou C, He M, Allers MM, Tiefenbacher AS, Kunz N, Martynova A, Spiller N, Mierisch J, Borchert F, Schwind C, Frey N, Dieterich C, Geis NA. A distributable German clinical corpus containing cardiovascular clinical routine doctor's letters. Sci Data 2023; 10:207. [PMID: 37059736 PMCID: PMC10104831 DOI: 10.1038/s41597-023-02128-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 03/31/2023] [Indexed: 04/16/2023] Open
Abstract
We present CARDIO:DE, the first freely available and distributable large German clinical corpus from the cardiovascular domain. CARDIO:DE encompasses 500 clinical routine German doctor's letters from Heidelberg University Hospital, which were manually annotated. Our prospective study design complies well with current data protection regulations and allows us to keep the original structure of clinical documents consistent. In order to ease access to our corpus, we manually de-identified all letters. To enable various information extraction tasks the temporal information in the documents was preserved. We added two high-quality manual annotation layers to CARDIO:DE, (1) medication information and (2) CDA-compliant section classes. To the best of our knowledge, CARDIO:DE is the first freely available and distributable German clinical corpus in the cardiovascular domain. In summary, our corpus offers unique opportunities for collaborative and reproducible research on natural language processing models for German clinical texts.
Collapse
Affiliation(s)
- Phillip Richter-Pechanski
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany.
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany.
- German Center for Cardiovascular Research (DZHK) - Partner site Heidelberg/Mannheim, Heidelberg, DE, Germany.
- Informatics for Life, Heidelberg, DE, Germany.
| | - Philipp Wiesenbach
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
- Informatics for Life, Heidelberg, DE, Germany
| | - Dominic M Schwab
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
| | - Christina Kiriakou
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
| | - Mingyang He
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
| | - Michael M Allers
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Anna S Tiefenbacher
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Nicola Kunz
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Anna Martynova
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Noemie Spiller
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Julian Mierisch
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
| | - Florian Borchert
- Digital Health Center, Hasso Plattner Institute, University of Potsdam, Potsdam, DE, Germany
| | - Charlotte Schwind
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
| | - Norbert Frey
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
- German Center for Cardiovascular Research (DZHK) - Partner site Heidelberg/Mannheim, Heidelberg, DE, Germany
- Informatics for Life, Heidelberg, DE, Germany
| | - Christoph Dieterich
- Section of Bioinformatics and Systems Cardiology, Klaus Tschira Institute for Integrative Computational Cardiology, Heidelberg, DE, Germany
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
- German Center for Cardiovascular Research (DZHK) - Partner site Heidelberg/Mannheim, Heidelberg, DE, Germany
- Informatics for Life, Heidelberg, DE, Germany
| | - Nicolas A Geis
- Department of Internal Medicine III, University Hospital Heidelberg, Heidelberg, DE, Germany
- Informatics for Life, Heidelberg, DE, Germany
| |
Collapse
|
53
|
Bignami EG, Vittori A, Lanza R, Compagnone C, Cascella M, Bellini V. The Clinical Researcher Journey in the Artificial Intelligence Era: The PAC-MAN’s Challenge. Healthcare (Basel) 2023; 11:healthcare11070975. [PMID: 37046900 PMCID: PMC10093965 DOI: 10.3390/healthcare11070975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Revised: 03/22/2023] [Accepted: 03/23/2023] [Indexed: 03/31/2023] Open
Abstract
Artificial intelligence (AI) is a powerful tool that can assist researchers and clinicians in various settings. However, like any technology, it must be used with caution and awareness as there are numerous potential pitfalls. To provide a creative analogy, we have likened research to the PAC-MAN classic arcade video game. Just as the protagonist of the game is constantly seeking data, researchers are constantly seeking information that must be acquired and managed within the constraints of the research rules. In our analogy, the obstacles that researchers face are represented by “ghosts”, which symbolize major ethical concerns, low-quality data, legal issues, and educational challenges. In short, clinical researchers need to meticulously collect and analyze data from various sources, often navigating through intricate and nuanced challenges to ensure that the data they obtain are both precise and pertinent to their research inquiry. Reflecting on this analogy can foster a deeper comprehension of the significance of employing AI and other powerful technologies with heightened awareness and attentiveness.
Collapse
Affiliation(s)
- Elena Giovanna Bignami
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Alessandro Vittori
- Department of Anesthesia and Critical Care, ARCO ROMA, Ospedale Pediatrico Bambino Gesù IRCCS, Piazza S. Onofrio 4, 00165 Rome, Italy
- Correspondence: or ; Tel.: +39-0668592397
| | - Roberto Lanza
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Christian Compagnone
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Marco Cascella
- Department of Anesthesia and Critical Care, Istituto Nazionale Tumori—IRCCS, Fondazione Pascale, 80131 Naples, Italy
| | - Valentina Bellini
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| |
Collapse
|
54
|
Tyagi N, Bhushan B. Demystifying the Role of Natural Language Processing (NLP) in Smart City Applications: Background, Motivation, Recent Advances, and Future Research Directions. WIRELESS PERSONAL COMMUNICATIONS 2023; 130:857-908. [PMID: 37168438 PMCID: PMC10019426 DOI: 10.1007/s11277-023-10312-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 02/25/2023] [Indexed: 05/13/2023]
Abstract
Smart cities provide an efficient infrastructure for the enhancement of the quality of life of the people by aiding in fast urbanization and resource management through sustainable and scalable innovative solutions. The penetration of Information and Communication Technology (ICT) in smart cities has been a major contributor to keeping up with the agility and pace of their development. In this paper, we have explored Natural Language Processing (NLP) which is one such technical discipline that has great potential in optimizing ICT processes and has so far been kept away from the limelight. Through this study, we have established the various roles that NLP plays in building smart cities after thoroughly analyzing its architecture, background, and scope. Subsequently, we present a detailed description of NLP's recent applications in the domain of smart healthcare, smart business, and industry, smart community, smart media, smart research, and development as well as smart education accompanied by NLP's open challenges at the very end. This work aims to throw light on the potential of NLP as one of the pillars in assisting the technical advancement and realization of smart cities.
Collapse
Affiliation(s)
- Nemika Tyagi
- Department of Computer Science and Engineering School of Engineering and Technology, Sharda University, Greater Noida, Uttar Pradesh 201310 India
| | - Bharat Bhushan
- Department of Computer Science and Engineering School of Engineering and Technology, Sharda University, Greater Noida, Uttar Pradesh 201310 India
| |
Collapse
|
55
|
Kreuzthaler M, Brochhausen M, Zayas C, Blobel B, Schulz S. Linguistic and ontological challenges of multiple domains contributing to transformed health ecosystems. Front Med (Lausanne) 2023; 10:1073313. [PMID: 37007792 PMCID: PMC10050682 DOI: 10.3389/fmed.2023.1073313] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 02/13/2023] [Indexed: 03/17/2023] Open
Abstract
This paper provides an overview of current linguistic and ontological challenges which have to be met in order to provide full support to the transformation of health ecosystems in order to meet precision medicine (5 PM) standards. It highlights both standardization and interoperability aspects regarding formal, controlled representations of clinical and research data, requirements for smart support to produce and encode content in a way that humans and machines can understand and process it. Starting from the current text-centered communication practices in healthcare and biomedical research, it addresses the state of the art in information extraction using natural language processing (NLP). An important aspect of the language-centered perspective of managing health data is the integration of heterogeneous data sources, employing different natural languages and different terminologies. This is where biomedical ontologies, in the sense of formal, interchangeable representations of types of domain entities come into play. The paper discusses the state of the art of biomedical ontologies, addresses their importance for standardization and interoperability and sheds light to current misconceptions and shortcomings. Finally, the paper points out next steps and possible synergies of both the field of NLP and the area of Applied Ontology and Semantic Web to foster data interoperability for 5 PM.
Collapse
Affiliation(s)
- Markus Kreuzthaler
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria
| | - Mathias Brochhausen
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, United States
| | - Cilia Zayas
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, United States
| | - Bernd Blobel
- Medical Faculty, University of Regensburg, Regensburg, Germany
- eHealth Competence Center Bavaria, Deggendorf Institute of Technology, Deggendorf, Germany
- First Medical Faculty, Charles University Prague, Prague, Czechia
| | - Stefan Schulz
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria
- Averbis GmbH, Freiburg, Germany
- *Correspondence: Stefan Schulz,
| |
Collapse
|
56
|
Hossain E, Rana R, Higgins N, Soar J, Barua PD, Pisani AR, Turner K. Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review. Comput Biol Med 2023; 155:106649. [PMID: 36805219 DOI: 10.1016/j.compbiomed.2023.106649] [Citation(s) in RCA: 25] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 01/04/2023] [Accepted: 02/07/2023] [Indexed: 02/12/2023]
Abstract
BACKGROUND Natural Language Processing (NLP) is widely used to extract clinical insights from Electronic Health Records (EHRs). However, the lack of annotated data, automated tools, and other challenges hinder the full utilisation of NLP for EHRs. Various Machine Learning (ML), Deep Learning (DL) and NLP techniques are studied and compared to understand the limitations and opportunities in this space comprehensively. METHODOLOGY After screening 261 articles from 11 databases, we included 127 papers for full-text review covering seven categories of articles: (1) medical note classification, (2) clinical entity recognition, (3) text summarisation, (4) deep learning (DL) and transfer learning architecture, (5) information extraction, (6) Medical language translation and (7) other NLP applications. This study follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULT AND DISCUSSION EHR was the most commonly used data type among the selected articles, and the datasets were primarily unstructured. Various ML and DL methods were used, with prediction or classification being the most common application of ML or DL. The most common use cases were: the International Classification of Diseases, Ninth Revision (ICD-9) classification, clinical note analysis, and named entity recognition (NER) for clinical descriptions and research on psychiatric disorders. CONCLUSION We find that the adopted ML models were not adequately assessed. In addition, the data imbalance problem is quite important, yet we must find techniques to address this underlining problem. Future studies should address key limitations in studies, primarily identifying Lupus Nephritis, Suicide Attempts, perinatal self-harmed and ICD-9 classification.
Collapse
Affiliation(s)
- Elias Hossain
- School of Engineering & Physical Sciences, North South University, Dhaka 1229, Bangladesh.
| | - Rajib Rana
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Niall Higgins
- School of Management and Enterprise, University of Southern Queensland, Darling Heights QLD 4350, Australia; School of Nursing, Queensland University of Technology, Kelvin Grove, Brisbane, QLD 4000, Australia; Metro North Mental Health, Herston QLD 4029, Australia
| | - Jeffrey Soar
- School of Business, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Prabal Datta Barua
- School of Business, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Anthony R Pisani
- Center for the Study and Prevention of Suicide, University of Rochester, Rochester, NY, United States
| | - Kathryn Turner
- School of Nursing, Queensland University of Technology, Kelvin Grove, Brisbane, QLD 4000, Australia
| |
Collapse
|
57
|
McDermott MBA, Nestor B, Szolovits P. Clinical Artificial Intelligence: Design Principles and Fallacies. Clin Lab Med 2023; 43:29-46. [PMID: 36764807 DOI: 10.1016/j.cll.2022.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Clinical artificial intelligence (AI)/machine learning (ML) is anticipated to offer new abilities in clinical decision support, diagnostic reasoning, precision medicine, clinical operational support, and clinical research, but careful concern is needed to ensure these technologies work effectively in the clinic. Here, we detail the clinical ML/AI design process, identifying several key questions and detailing several common forms of issues that arise with ML tools, as motivated by real-world examples, such that clinicians and researchers can better anticipate and correct for such issues in their own use of ML/AI techniques.
Collapse
Affiliation(s)
| | - Bret Nestor
- Department of Computer Science, University of Toronto, 40 St George St, Toronto, ON M5S 2E4, Canada
| | | |
Collapse
|
58
|
Attri-VAE: Attribute-based interpretable representations of medical images with variational autoencoders. Comput Med Imaging Graph 2023; 104:102158. [PMID: 36638626 DOI: 10.1016/j.compmedimag.2022.102158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 12/06/2022] [Accepted: 12/06/2022] [Indexed: 12/13/2022]
Abstract
Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in the reasoning behind the medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attributes. Attribute-based methods enforcing attribute disentanglement have been proposed in the literature for classical computer vision tasks in benchmark data. In this paper, we propose a VAE approach, the Attri-VAE, that includes an attribute regularization term to associate clinical and medical imaging attributes with different regularized dimensions in the generated latent space, enabling a better-disentangled interpretation of the attributes. Furthermore, the generated attention maps explained the attribute encoding in the regularized latent space dimensions. Using the Attri-VAE approach we analyzed healthy and myocardial infarction patients with clinical, cardiac morphology, and radiomics attributes. The proposed model provided an excellent trade-off between reconstruction fidelity, disentanglement, and interpretability, outperforming state-of-the-art VAE approaches according to several quantitative metrics. The resulting latent space allowed the generation of realistic synthetic data in the trajectory between two distinct input samples or along a specific attribute dimension to better interpret changes between different cardiac conditions.
Collapse
|
59
|
Lee RY, Kross EK, Torrence J, Li KS, Sibley J, Cohen T, Lober WB, Engelberg RA, Curtis JR. Assessment of Natural Language Processing of Electronic Health Records to Measure Goals-of-Care Discussions as a Clinical Trial Outcome. JAMA Netw Open 2023; 6:e231204. [PMID: 36862411 PMCID: PMC9982698 DOI: 10.1001/jamanetworkopen.2023.1204] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/03/2023] Open
Abstract
IMPORTANCE Many clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies. OBJECTIVE To evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system. MAIN OUTCOMES AND MEASURES Main outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation. RESULTS A total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set from identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations. CONCLUSIONS AND RELEVANCE In this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.
Collapse
Affiliation(s)
- Robert Y. Lee
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Erin K. Kross
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Janaki Torrence
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Kevin S. Li
- Division of Biomedical and Health Informatics, Department of Biomedical Informatics and Medical Education, University of Washington, Seattle
| | - James Sibley
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Department of Biobehavioral Nursing and Health Informatics, University of Washington, Seattle
| | - Trevor Cohen
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Biomedical and Health Informatics, Department of Biomedical Informatics and Medical Education, University of Washington, Seattle
| | - William B. Lober
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Biomedical and Health Informatics, Department of Biomedical Informatics and Medical Education, University of Washington, Seattle
- Department of Biobehavioral Nursing and Health Informatics, University of Washington, Seattle
- Department of Global Health, University of Washington, Seattle
| | - Ruth A. Engelberg
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - J. Randall Curtis
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
- Department of Biobehavioral Nursing and Health Informatics, University of Washington, Seattle
- Department of Health Systems and Population Health, University of Washington, Seattle
| |
Collapse
|
60
|
Jantscher M, Gunzer F, Kern R, Hassler E, Tschauner S, Reishofer G. Information extraction from German radiological reports for general clinical text and language understanding. Sci Rep 2023; 13:2353. [PMID: 36759679 PMCID: PMC9911592 DOI: 10.1038/s41598-023-29323-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Accepted: 02/02/2023] [Indexed: 02/11/2023] Open
Abstract
Recent advances in deep learning and natural language processing (NLP) have opened many new opportunities for automatic text understanding and text processing in the medical field. This is of great benefit as many clinical downstream tasks rely on information from unstructured clinical documents. However, for low-resource languages like German, the use of modern text processing applications that require a large amount of training data proves to be difficult, as only few data sets are available mainly due to legal restrictions. In this study, we present an information extraction framework that was initially pre-trained on real-world computed tomographic (CT) reports of head examinations, followed by domain adaptive fine-tuning on reports from different imaging examinations. We show that in the pre-training phase, the semantic and contextual meaning of one clinical reporting domain can be captured and effectively transferred to foreign clinical imaging examinations. Moreover, we introduce an active learning approach with an intrinsic strategic sampling method to generate highly informative training data with low human annotation cost. We see that the model performance can be significantly improved by an appropriate selection of the data to be annotated, without the need to train the model on a specific downstream task. With a general annotation scheme that can be used not only in the radiology field but also in a broader clinical setting, we contribute to a more consistent labeling and annotation process that also facilitates the verification and evaluation of language models in the German clinical setting.
Collapse
Affiliation(s)
| | - Felix Gunzer
- Division of Neuroradiology, Vascular and Interventional Radiology, Department of Radiology, Medical University Graz, 8036, Graz, Austria
| | | | - Eva Hassler
- Division of Neuroradiology, Vascular and Interventional Radiology, Department of Radiology, Medical University Graz, 8036, Graz, Austria
| | - Sebastian Tschauner
- Division of Pediatric Radiology, Department of Radiology, Medical University Graz, 8036, Graz, Austria
| | - Gernot Reishofer
- Department of Radiology, Medical University Graz, 8036, Graz, Austria. .,BioTechMed-Graz, 8010, Graz, Austria.
| |
Collapse
|
61
|
Seong D, Choi YH, Shin SY, Yi BK. Deep learning approach to detection of colonoscopic information from unstructured reports. BMC Med Inform Decis Mak 2023; 23:28. [PMID: 36750932 PMCID: PMC9903463 DOI: 10.1186/s12911-023-02121-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Accepted: 01/23/2023] [Indexed: 02/09/2023] Open
Abstract
BACKGROUND Colorectal cancer is a leading cause of cancer deaths. Several screening tests, such as colonoscopy, can be used to find polyps or colorectal cancer. Colonoscopy reports are often written in unstructured narrative text. The information embedded in the reports can be used for various purposes, including colorectal cancer risk prediction, follow-up recommendation, and quality measurement. However, the availability and accessibility of unstructured text data are still insufficient despite the large amounts of accumulated data. We aimed to develop and apply deep learning-based natural language processing (NLP) methods to detect colonoscopic information. METHODS This study applied several deep learning-based NLP models to colonoscopy reports. Approximately 280,668 colonoscopy reports were extracted from the clinical data warehouse of Samsung Medical Center. For 5,000 reports, procedural information and colonoscopic findings were manually annotated with 17 labels. We compared the long short-term memory (LSTM) and BioBERT model to select the one with the best performance for colonoscopy reports, which was the bidirectional LSTM with conditional random fields. Then, we applied pre-trained word embedding using large unlabeled data (280,668 reports) to the selected model. RESULTS The NLP model with pre-trained word embedding performed better for most labels than the model with one-hot encoding. The F1 scores for colonoscopic findings were: 0.9564 for lesions, 0.9722 for locations, 0.9809 for shapes, 0.9720 for colors, 0.9862 for sizes, and 0.9717 for numbers. CONCLUSIONS This study applied deep learning-based clinical NLP models to extract meaningful information from colonoscopy reports. The method in this study achieved promising results that demonstrate it can be applied to various practical purposes.
Collapse
Affiliation(s)
- Donghyeong Seong
- grid.264381.a0000 0001 2181 989XSamsung Advanced Institute for Health Sciences and Technology (SAIHST), Sungkyunkwan University, Seoul, 06355 Republic of Korea
| | - Yoon Ho Choi
- grid.264381.a0000 0001 2181 989XDepartment of Digital Health, SAIHST, Sungkyunkwan University, Seoul, 06355 Republic of Korea
| | - Soo-Yong Shin
- grid.264381.a0000 0001 2181 989XDepartment of Digital Health, SAIHST, Sungkyunkwan University, Seoul, 06355 Republic of Korea ,grid.414964.a0000 0001 0640 5613Research Institute for Future Medicine, Samsung Medical Center, Seoul, 06351 Republic of Korea
| | - Byoung-Kee Yi
- Department of Artificial Intelligence Convergence, Kangwon National University, 1 Kangwondaehak-Gil, Chuncheon-si, Gangwon-do, 24341, Republic of Korea.
| |
Collapse
|
62
|
Pethani F, Dunn AG. Natural language processing for clinical notes in dentistry: A systematic review. J Biomed Inform 2023; 138:104282. [PMID: 36623780 DOI: 10.1016/j.jbi.2023.104282] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 12/01/2022] [Accepted: 01/04/2023] [Indexed: 01/09/2023]
Abstract
OBJECTIVE To identify and synthesise research on applications of natural language processing (NLP) for information extraction and retrieval from clinical notes in dentistry. MATERIALS AND METHODS A predefined search strategy was applied in EMBASE, CINAHL and Medline. Studies eligible for inclusion were those that that described, evaluated, or applied NLP to clinical notes containing either human or simulated patient information. Quality of the study design and reporting was independently assessed based on a set of questions derived from relevant tools including CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). A narrative synthesis was conducted to present the results. RESULTS Of the 17 included studies, 10 developed and evaluated NLP methods and 7 described applications of NLP-based information retrieval methods in dental records. Studies were published between 2015 and 2021, most were missing key details needed for reproducibility, and there was no consistency in design or reporting. The 10 studies developing or evaluating NLP methods used document classification or entity extraction, and 4 compared NLP methods to non-NLP methods. The quality of reporting on NLP studies in dentistry has modestly improved over time. CONCLUSIONS Study design heterogeneity and incomplete reporting of studies currently limits our ability to synthesise NLP applications in dental records. Standardisation of reporting and improved connections between NLP methods and applied NLP in dentistry may improve how we can make use of clinical notes from dentistry in population health or decision support systems. PROTOCOL REGISTRATION PROSPERO CRD42021227823.
Collapse
Affiliation(s)
- Farhana Pethani
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, the University of Sydney, Sydney, Australia
| | - Adam G Dunn
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, the University of Sydney, Sydney, Australia.
| |
Collapse
|
63
|
Fine-tuning BERT for automatic ADME semantic labeling in FDA drug labeling to enhance product-specific guidance assessment. J Biomed Inform 2023; 138:104285. [PMID: 36632860 DOI: 10.1016/j.jbi.2023.104285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Revised: 10/25/2022] [Accepted: 01/07/2023] [Indexed: 01/11/2023]
Abstract
Product-specific guidances (PSGs) recommended by the United States Food and Drug Administration (FDA) are instrumental to promote and guide generic drug product development. To assess a PSG, the FDA assessor needs to take extensive time and effort to manually retrieve supportive drug information of absorption, distribution, metabolism, and excretion (ADME) from the reference listed drug labeling. In this work, we leveraged the state-of-the-art pre-trained language models to automatically label the ADME paragraphs in the pharmacokinetics section from the FDA-approved drug labeling to facilitate PSG assessment. We applied a transfer learning approach by fine-tuning the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model to develop a novel application of ADME semantic labeling, which can automatically retrieve ADME paragraphs from drug labeling instead of manual work. We demonstrate that fine-tuning the pre-trained BERT model can outperform conventional machine learning techniques, achieving up to 12.5% absolute F1 improvement. To our knowledge, we were the first to successfully apply BERT to solve the ADME semantic labeling task. We further assessed the relative contribution of pre-training and fine-tuning to the overall performance of the BERT model in the ADME semantic labeling task using a series of analysis methods, such as attention similarity and layer-based ablations. Our analysis revealed that the information learned via fine-tuning is focused on task-specific knowledge in the top layers of the BERT, whereas the benefit from the pre-trained BERT model is from the bottom layers.
Collapse
|
64
|
Hernigou P, Barbier O, Chenaie P. Hip arthroplasty dislocation risk calculator: evaluation of one million primary implants and twenty-five thousand dislocations with deep learning artificial intelligence in a systematic review of reviews. INTERNATIONAL ORTHOPAEDICS 2023; 47:557-571. [PMID: 36445413 DOI: 10.1007/s00264-022-05644-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 11/19/2022] [Indexed: 11/30/2022]
Abstract
PURPOSE This paper aims to provide an overview of the possibility regarding the artificial intelligence application in orthopaedics to predict dislocation with a calculator according to the type of implant (hemiarthroplasty, standard total hip arthroplasty, dual mobility, constrained cups) after primary arthroplasty. MATERIAL AND METHODS Among 75 results for primary arthroplasties, 26 articles were reviews on dislocation after hemiarthroplasty, 40 after standard total hip arthroplasty, seven about primary dual-mobility arthroplasty (DM THA), and two reviews about constrained implants. Although our search method for systematic reviews covers ten years (2012-2022), none for dual mobility was published before 2016, showing a recent explosion of original articles on this subject. A total of 1,069,565 implants and 26,488 dislocations in primary arthroplasties are included in these 75 reviews. We used a supervised learning model in which models assign objects to groups as input and artificial neural network (ANN) with nodes, hidden layers, and output layers. We considered only four implant types as the input layer. We considered the patient's factors (indication for THA, demographics, spine surgery, and neurologic disease) as the second input values (hidden layer). We considered the implant position as the third input (hidden layer) property including head size, combined anteversion, or spinopelvic alignment. Surgery-related factors, approach, capsule repair, etc. were the fourth input values (hidden layer). The output was a post-operative dislocation or not within three months. RESULTS The accuracy for predicting dislocation with this systematic review was 95%. Dislocation risk, based on the type of implant, was wide-ranging, from 0 to 3.9% (mean 0.31%) for the 3045 DM THA, from 0.2 to 1.2% (overall 0.91%) for the 457 constrained liners, from 1.76 to 4.2% (mean 2.1%) for 895,734 conventional total hip arthroplasties, and from 0.76 to 12.2% (mean 4.5%) for 170,329 hemiarthroplasties. In the conventional THA group, many factors increase the risk of dislocation according to the calculator, and only a few (big head, anterior approach) decrease the risk, but not very significantly. In the hemiarthroplasty group, many factors can increase the risk of dislocation until 30%, but none could decrease the risk. According to the calculator, the DM THA and the constrained liner markedly decreased the risk and were not affected by implant position, spine surgery, and spinopelvic position. CONCLUSION To our knowledge, this study is the first to yield an implant-specific dislocation risk calculator that incorporates the patient's comorbidities, the position of components, and surgery factors affecting instability risk.
Collapse
|
65
|
Nunez JJ, Leung B, Ho C, Bates AT, Ng RT. Predicting the Survival of Patients With Cancer From Their Initial Oncology Consultation Document Using Natural Language Processing. JAMA Netw Open 2023; 6:e230813. [PMID: 36848085 PMCID: PMC9972192 DOI: 10.1001/jamanetworkopen.2023.0813] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/01/2023] Open
Abstract
IMPORTANCE Predicting short- and long-term survival of patients with cancer may improve their care. Prior predictive models either use data with limited availability or predict the outcome of only 1 type of cancer. OBJECTIVE To investigate whether natural language processing can predict survival of patients with general cancer from a patient's initial oncologist consultation document. DESIGN, SETTING, AND PARTICIPANTS This retrospective prognostic study used data from 47 625 of 59 800 patients who started cancer care at any of the 6 BC Cancer sites located in the province of British Columbia between April 1, 2011, and December 31, 2016. Mortality data were updated until April 6, 2022, and data were analyzed from update until September 30, 2022. All patients with a medical or radiation oncologist consultation document generated within 180 days of diagnosis were included; patients seen for multiple cancers were excluded. EXPOSURES Initial oncologist consultation documents were analyzed using traditional and neural language models. MAIN OUTCOMES AND MEASURES The primary outcome was the performance of the predictive models, including balanced accuracy and receiver operating characteristics area under the curve (AUC). The secondary outcome was investigating what words the models used. RESULTS Of the 47 625 patients in the sample, 25 428 (53.4%) were female and 22 197 (46.6%) were male, with a mean (SD) age of 64.9 (13.7) years. A total of 41 447 patients (87.0%) survived 6 months, 31 143 (65.4%) survived 36 months, and 27 880 (58.5%) survived 60 months, calculated from their initial oncologist consultation. The best models achieved a balanced accuracy of 0.856 (AUC, 0.928) for predicting 6-month survival, 0.842 (AUC, 0.918) for 36-month survival, and 0.837 (AUC, 0.918) for 60-month survival, on a holdout test set. Differences in what words were important for predicting 6- vs 60-month survival were found. CONCLUSIONS AND RELEVANCE These findings suggest that models performed comparably with or better than previous models predicting cancer survival and that they may be able to predict survival using readily available data without focusing on 1 cancer type.
Collapse
Affiliation(s)
- John-Jose Nunez
- BC Cancer, Vancouver, British Columbia, Canada
- Department of Computer Science, University of British Columbia, Vancouver, British Columbia, Canada
- Department of Psychiatry, University of British Columbia, Vancouver, British Columbia, Canada
| | | | - Cheryl Ho
- BC Cancer, Vancouver, British Columbia, Canada
| | - Alan T. Bates
- BC Cancer, Vancouver, British Columbia, Canada
- Department of Psychiatry, University of British Columbia, Vancouver, British Columbia, Canada
| | - Raymond T. Ng
- Department of Computer Science, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
66
|
Azuma I, Mizuno T, Kusuhara H. NRBdMF: A Recommendation Algorithm for Predicting Drug Effects Considering Directionality. J Chem Inf Model 2023; 63:474-483. [PMID: 36635231 DOI: 10.1021/acs.jcim.2c01210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023]
Abstract
Predicting the novel effects of drugs based on information about approved drugs can be regarded as a recommendation system. Matrix factorization is one of the most used recommendation systems, and various algorithms have been devised for it. A literature survey and summary of existing algorithms for predicting drug effects demonstrated that most such methods, including neighborhood regularized logistic matrix factorization, which was the best performer in benchmark tests, used a binary matrix that considers only the presence or absence of interactions. However, drug effects are known to have two opposite aspects, such as side effects and therapeutic effects. In the present study, we proposed using neighborhood regularized bidirectional matrix factorization (NRBdMF) to predict drug effects by incorporating bidirectionality, which is a characteristic property of drug effects. We used this proposed method for predicting side effects using a matrix that considered the bidirectionality of drug effects, in which known side effects were assigned a positive (+1) label and known treatment effects were assigned a negative (-1) label. The NRBdMF model, which utilizes drug bidirectional information, achieved enrichment of side effects at the top and indications at the bottom of the prediction list. This first attempt to consider the bidirectional nature of drug effects using NRBdMF showed that it reduced false positives and produced a highly interpretable output.
Collapse
Affiliation(s)
- Iori Azuma
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, Bunkyo-ku, Tokyo113-0033, Japan
| | - Tadahaya Mizuno
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, Bunkyo-ku, Tokyo113-0033, Japan
| | - Hiroyuki Kusuhara
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, Bunkyo-ku, Tokyo113-0033, Japan
| |
Collapse
|
67
|
Yang S, Varghese P, Stephenson E, Tu K, Gronsbell J. Machine learning approaches for electronic health records phenotyping: a methodical review. J Am Med Inform Assoc 2023; 30:367-381. [PMID: 36413056 PMCID: PMC9846699 DOI: 10.1093/jamia/ocac216] [Citation(s) in RCA: 23] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 09/27/2022] [Accepted: 10/27/2022] [Indexed: 11/23/2022] Open
Abstract
OBJECTIVE Accurate and rapid phenotyping is a prerequisite to leveraging electronic health records for biomedical research. While early phenotyping relied on rule-based algorithms curated by experts, machine learning (ML) approaches have emerged as an alternative to improve scalability across phenotypes and healthcare settings. This study evaluates ML-based phenotyping with respect to (1) the data sources used, (2) the phenotypes considered, (3) the methods applied, and (4) the reporting and evaluation methods used. MATERIALS AND METHODS We searched PubMed and Web of Science for articles published between 2018 and 2022. After screening 850 articles, we recorded 37 variables on 100 studies. RESULTS Most studies utilized data from a single institution and included information in clinical notes. Although chronic conditions were most commonly considered, ML also enabled the characterization of nuanced phenotypes such as social determinants of health. Supervised deep learning was the most popular ML paradigm, while semi-supervised and weakly supervised learning were applied to expedite algorithm development and unsupervised learning to facilitate phenotype discovery. ML approaches did not uniformly outperform rule-based algorithms, but deep learning offered a marginal improvement over traditional ML for many conditions. DISCUSSION Despite the progress in ML-based phenotyping, most articles focused on binary phenotypes and few articles evaluated external validity or used multi-institution data. Study settings were infrequently reported and analytic code was rarely released. CONCLUSION Continued research in ML-based phenotyping is warranted, with emphasis on characterizing nuanced phenotypes, establishing reporting and evaluation standards, and developing methods to accommodate misclassified phenotypes due to algorithm errors in downstream applications.
Collapse
Affiliation(s)
- Siyue Yang
- Department of Statistical Sciences, University of Toronto, Toronto, Ontario, Canada
| | | | - Ellen Stephenson
- Department of Family & Community Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Karen Tu
- Department of Family & Community Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Jessica Gronsbell
- Department of Statistical Sciences, University of Toronto, Toronto, Ontario, Canada
- Department of Family & Community Medicine, University of Toronto, Toronto, Ontario, Canada
- Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
68
|
Chenais G, Gil-Jardiné C, Touchais H, Avalos Fernandez M, Contrand B, Tellier E, Combes X, Bourdois L, Revel P, Lagarde E. Deep Learning Transformer Models for Building a Comprehensive and Real-time Trauma Observatory: Development and Validation Study. JMIR AI 2023; 2:e40843. [PMID: 38875539 PMCID: PMC11041521 DOI: 10.2196/40843] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 10/14/2022] [Accepted: 10/29/2022] [Indexed: 06/16/2024]
Abstract
BACKGROUND Public health surveillance relies on the collection of data, often in near-real time. Recent advances in natural language processing make it possible to envisage an automated system for extracting information from electronic health records. OBJECTIVE To study the feasibility of setting up a national trauma observatory in France, we compared the performance of several automatic language processing methods in a multiclass classification task of unstructured clinical notes. METHODS A total of 69,110 free-text clinical notes related to visits to the emergency departments of the University Hospital of Bordeaux, France, between 2012 and 2019 were manually annotated. Among these clinical notes, 32.5% (22,481/69,110) were traumas. We trained 4 transformer models (deep learning models that encompass attention mechanism) and compared them with the term frequency-inverse document frequency associated with the support vector machine method. RESULTS The transformer models consistently performed better than the term frequency-inverse document frequency and a support vector machine. Among the transformers, the GPTanam model pretrained with a French corpus with an additional autosupervised learning step on 306,368 unlabeled clinical notes showed the best performance with a micro F1-score of 0.969. CONCLUSIONS The transformers proved efficient at the multiclass classification of narrative and medical data. Further steps for improvement should focus on the expansion of abbreviations and multioutput multiclass classification.
Collapse
Affiliation(s)
- Gabrielle Chenais
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
| | - Cédric Gil-Jardiné
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
- Emergency Department, Bordeaux University Hospital, Bordeaux, France
| | - Hélène Touchais
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
| | - Marta Avalos Fernandez
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
- Statistics in Systems Biology and Translational Medicine Team, University of Bordeaux, Institut National de Recherche en Sciences et Technologies du Numérique, Talence, France
| | - Benjamin Contrand
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
| | - Eric Tellier
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
- Emergency Department, Bordeaux University Hospital, Bordeaux, France
| | - Xavier Combes
- Emergency Department, Bordeaux University Hospital, Bordeaux, France
| | - Loick Bourdois
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
| | - Philippe Revel
- Emergency Department, Bordeaux University Hospital, Bordeaux, France
| | - Emmanuel Lagarde
- Unit 1219, Bordeaux Public Health Center, Institut National de la Santé et de la Recherche Médicale, Bordeaux, France
| |
Collapse
|
69
|
Tamang S, Humbert-Droz M, Gianfrancesco M, Izadi Z, Schmajuk G, Yazdany J. Practical Considerations for Developing Clinical Natural Language Processing Systems for Population Health Management and Measurement. JMIR Med Inform 2023; 11:e37805. [PMID: 36595345 PMCID: PMC9846439 DOI: 10.2196/37805] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 09/02/2022] [Accepted: 11/09/2022] [Indexed: 11/11/2022] Open
Abstract
Experts have noted a concerning gap between clinical natural language processing (NLP) research and real-world applications, such as clinical decision support. To help address this gap, in this viewpoint, we enumerate a set of practical considerations for developing an NLP system to support real-world clinical needs and improve health outcomes. They include determining (1) the readiness of the data and compute resources for NLP, (2) the organizational incentives to use and maintain the NLP systems, and (3) the feasibility of implementation and continued monitoring. These considerations are intended to benefit the design of future clinical NLP projects and can be applied across a variety of settings, including large health systems or smaller clinical practices that have adopted electronic medical records in the United States and globally.
Collapse
Affiliation(s)
- Suzanne Tamang
- Division of Immunology and Rheumatology, Stanford University School of Medicine, Stanford, CA, United States
- Department of Veterans Affairs, Office of Mental Health and Suicide Prevention, Program Evaluation Resource Center, Palo Alto, CA, United States
| | - Marie Humbert-Droz
- Division of Immunology and Rheumatology, Stanford University School of Medicine, Stanford, CA, United States
| | - Milena Gianfrancesco
- Division of Rheumatology, University of California, San Francisco, San Francisco, CA, United States
| | - Zara Izadi
- Division of Rheumatology, University of California, San Francisco, San Francisco, CA, United States
| | - Gabriela Schmajuk
- Division of Rheumatology, University of California, San Francisco, San Francisco, CA, United States
| | - Jinoos Yazdany
- Division of Rheumatology, University of California, San Francisco, San Francisco, CA, United States
| |
Collapse
|
70
|
Li N, Feng L, Hu J, Jiang L, Wang J, Han J, Gan L, He Z, Wang G. Using deeply time-series semantics to assess depressive symptoms based on clinical interview speech. Front Psychiatry 2023; 14:1104190. [PMID: 36865077 PMCID: PMC9971220 DOI: 10.3389/fpsyt.2023.1104190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 01/31/2023] [Indexed: 02/18/2023] Open
Abstract
INTRODUCTION Depression is an affective disorder that contributes to a significant global burden of disease. Measurement-Based Care (MBC) is advocated during the full course management, with symptom assessment being an important component. Rating scales are widely used as convenient and powerful assessment tool, but they are influenced by the subjectivity and consistency of the raters. The assessment of depressive symptoms is usually conducted with a clear purpose and restricted content, such as clinical interviews based on the Hamilton Depression Rating Scale (HAMD), so that the results are easy to obtain and quantify. Artificial Intelligence (AI) techniques are used due to their objective, stable and consistent performance, and are suitable for assessing depressive symptoms. Therefore, this study applied Deep Learning (DL)-based Natural Language Processing (NLP) techniques to assess depressive symptoms during clinical interviews; thus, we proposed an algorithm model, explored the feasibility of the techniques, and evaluated their performance. METHODS The study included 329 patients with Major Depressive Episode. Clinical interviews based on the HAMD-17 were conducted by trained psychiatrists, whose speech was simultaneously recorded. A total of 387 audio recordings were included in the final analysis. A deeply time-series semantics model for the assessment of depressive symptoms based on multi-granularity and multi-task joint training (MGMT) is proposed. RESULTS The performance of MGMT is acceptable for assessing depressive symptoms with an F1 score (a metric of model performance, the harmonic mean of precision and recall) of 0.719 in classifying the four-level severity of depression and an F1 score of 0.890 in identifying the presence of depressive symptoms. DISSCUSSION This study demonstrates the feasibility of the DL and the NLP techniques applied to the clinical interview and the assessment of depressive symptoms. However, there are limitations to this study, including the lack of adequate samples, and the fact that using speech content alone to assess depressive symptoms loses the information gained through observation. A multi-dimensional model combing semantics with speech voice, facial expression, and other valuable information, as well as taking into account personalized information, is a possible direction in the future.
Collapse
Affiliation(s)
- Nanxi Li
- Beijing Key Laboratory of Mental Disorders, National Clinical Research Center for Mental Disorders and National Center for Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Lei Feng
- Beijing Key Laboratory of Mental Disorders, National Clinical Research Center for Mental Disorders and National Center for Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China.,Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China
| | - Jiaxue Hu
- Anhui iFLYTEK Health Co., Ltd., Hefei, China
| | - Lei Jiang
- Anhui iFLYTEK Health Co., Ltd., Hefei, China
| | - Jing Wang
- Anhui iFLYTEK Health Co., Ltd., Hefei, China
| | - Jiali Han
- Beijing Key Laboratory of Mental Disorders, National Clinical Research Center for Mental Disorders and National Center for Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Lu Gan
- Anhui iFLYTEK Health Co., Ltd., Hefei, China
| | - Zhiyang He
- Anhui iFLYTEK Health Co., Ltd., Hefei, China
| | - Gang Wang
- Beijing Key Laboratory of Mental Disorders, National Clinical Research Center for Mental Disorders and National Center for Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China.,Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China
| |
Collapse
|
71
|
Ponthongmak W, Thammasudjarit R, McKay GJ, Attia J, Theera-Ampornpunt N, Thakkinstian A. Development and external validation of automated ICD-10 coding from discharge summaries using deep learning approaches. INFORMATICS IN MEDICINE UNLOCKED 2023. [DOI: 10.1016/j.imu.2023.101227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023] Open
|
72
|
Saha A, Burns L, Kulkarni AM. A scoping review of natural language processing of radiology reports in breast cancer. Front Oncol 2023; 13:1160167. [PMID: 37124523 PMCID: PMC10130381 DOI: 10.3389/fonc.2023.1160167] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Accepted: 03/28/2023] [Indexed: 05/02/2023] Open
Abstract
Various natural language processing (NLP) algorithms have been applied in the literature to analyze radiology reports pertaining to the diagnosis and subsequent care of cancer patients. Applications of this technology include cohort selection for clinical trials, population of large-scale data registries, and quality improvement in radiology workflows including mammography screening. This scoping review is the first to examine such applications in the specific context of breast cancer. Out of 210 identified articles initially, 44 met our inclusion criteria for this review. Extracted data elements included both clinical and technical details of studies that developed or evaluated NLP algorithms applied to free-text radiology reports of breast cancer. Our review illustrates an emphasis on applications in diagnostic and screening processes over treatment or therapeutic applications and describes growth in deep learning and transfer learning approaches in recent years, although rule-based approaches continue to be useful. Furthermore, we observe increased efforts in code and software sharing but not with data sharing.
Collapse
Affiliation(s)
- Ashirbani Saha
- Department of Oncology, McMaster University, Hamilton, ON, Canada
- Hamilton Health Sciences and McMaster University, Escarpment Cancer Research Institute, Hamilton, ON, Canada
- *Correspondence: Ashirbani Saha,
| | - Levi Burns
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | | |
Collapse
|
73
|
Joseph JW, Leventhal EL, Grossestreuer AV, Chen PC, White BA, Nathanson LA, Elhadad N, Sanchez LD. Machine Learning Methods for Predicting Patient-Level Emergency Department Workload. J Emerg Med 2023; 64:83-92. [PMID: 36450614 DOI: 10.1016/j.jemermed.2022.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 08/19/2022] [Accepted: 10/11/2022] [Indexed: 11/30/2022]
Abstract
BACKGROUND Work Relative Value Units (wRVUs) are a component of many compensation models, and a proxy for the effort required to care for a patient. Accurate prediction of wRVUs generated per patient at triage could facilitate real-time load balancing between physicians and provide many practical operational and clinical benefits. OBJECTIVE We examined whether deep-learning approaches could predict the wRVUs generated by a patient's visit using data commonly available at triage. METHODS Adult patients presenting to an urban, academic emergency department from July 1, 2016-March 1, 2020 were included. Deidentified triage information included structured data (age, sex, vital signs, Emergency Severity Index score, language, race, standardized chief complaint) and unstructured data (free-text chief complaint) with wRVUs as outcome. Five models were examined: average wRVUs per chief complaint, linear regression, neural network and gradient-boosted tree on structured data, and neural network on unstructured textual data. Models were evaluated using mean absolute error. RESULTS We analyzed 204,064 visits between July 1, 2016 and March 1, 2020. The median wRVUs were 3.80 (interquartile range 2.56-4.21), with significant effects of age, gender, and race. Models demonstrated lower error as complexity increased. Predictions using averages from chief complaints alone demonstrated a mean error of 2.17 predicted wRVUs per visit (95% confidence interval [CI] 2.07-2.27), the linear regression model: 1.00 wRVUs (95% CI 0.97-1.04), gradient-boosted tree: 0.85 wRVUs (95% CI 0.84-0.86), neural network with structured data: 0.86 wRVUs (95% CI 0.85-0.87), and neural network with unstructured data: 0.78 wRVUs (95% CI 0.76-0.80). CONCLUSIONS Chief complaints are a poor predictor of the effort needed to evaluate a patient; however, deep-learning techniques show promise. These algorithms have the potential to provide many practical applications, including balancing workloads and compensation between emergency physicians, quantify crowding and mobilizing resources, and reducing bias in the triage process.
Collapse
Affiliation(s)
- Joshua W Joseph
- Department of Emergency Medicine, Brigham and Women's Hospital, Boston, Massachusetts; Harvard Medical School, Boston, Massachusetts
| | - Evan L Leventhal
- Harvard Medical School, Boston, Massachusetts; Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Anne V Grossestreuer
- Harvard Medical School, Boston, Massachusetts; Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Paul C Chen
- Department of Emergency Medicine, Brigham and Women's Hospital, Boston, Massachusetts; Harvard Medical School, Boston, Massachusetts
| | - Benjamin A White
- Harvard Medical School, Boston, Massachusetts; Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - Larry A Nathanson
- Harvard Medical School, Boston, Massachusetts; Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Noémie Elhadad
- Departments of Biomedical Informatics and Computer Science, Columbia University, New York, New York
| | - Leon D Sanchez
- Department of Emergency Medicine, Brigham and Women's Hospital, Boston, Massachusetts; Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
74
|
Morris MX, Song EY, Rajesh A, Kass N, Asaad M, Phillips BT. New Frontiers of Natural Language Processing in Surgery. Am Surg 2023; 89:43-48. [PMID: 35969539 DOI: 10.1177/00031348221117039] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
The vast and ever-growing volume of electronic health records (EHR) have generated a wealth of information-rich data. Traditional, non-machine learning data extraction techniques are error-prone and laborious, hindering the analytical potential of these massive data sources. Equipped with natural language processing (NLP) tools, surgeons are better able to automate, and customize their review to investigate and implement surgical solutions. We identify current perioperative applications of NLP algorithms as well as research limitations and future avenues to outline the impact and potential of this technology for progressing surgical innovation.
Collapse
Affiliation(s)
- Miranda X Morris
- 12277Duke University School of Medicine, Durham, NC, USA.,Duke Pratt School of Engineering, Durham, NC, USA
| | - Ethan Y Song
- Division of Plastic, Maxillofacial, and Oral Surgery, Department of Surgery, 22957Duke University, Durham, NC, USA
| | - Aashish Rajesh
- Department of Surgery, 14742University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| | - Nicolas Kass
- 12317University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | - Malke Asaad
- Department of Plastic Surgery, 6595University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Brett T Phillips
- Division of Plastic, Maxillofacial, and Oral Surgery, Department of Surgery, 22957Duke University, Durham, NC, USA
| |
Collapse
|
75
|
Bombieri M, Rospocher M, Ponzetto SP, Fiorini P. Machine understanding surgical actions from intervention procedure textbooks. Comput Biol Med 2023; 152:106415. [PMID: 36527782 DOI: 10.1016/j.compbiomed.2022.106415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 11/23/2022] [Accepted: 12/04/2022] [Indexed: 12/12/2022]
Abstract
The automatic extraction of procedural surgical knowledge from surgery manuals, academic papers or other high-quality textual resources, is of the utmost importance to develop knowledge-based clinical decision support systems, to automatically execute some procedure's step or to summarize the procedural information, spread throughout the texts, in a structured form usable as a study resource by medical students. In this work, we propose a first benchmark on extracting detailed surgical actions from available intervention procedure textbooks and papers. We frame the problem as a Semantic Role Labeling task. Exploiting a manually annotated dataset, we apply different Transformer-based information extraction methods. Starting from RoBERTa and BioMedRoBERTa pre-trained language models, we first investigate a zero-shot scenario and compare the obtained results with a full fine-tuning setting. We then introduce a new ad-hoc surgical language model, named SurgicBERTa, pre-trained on a large collection of surgical materials, and we compare it with the previous ones. In the assessment, we explore different dataset splits (one in-domain and two out-of-domain) and we investigate also the effectiveness of the approach in a few-shot learning scenario. Performance is evaluated on three correlated sub-tasks: predicate disambiguation, semantic argument disambiguation and predicate-argument disambiguation. Results show that the fine-tuning of a pre-trained domain-specific language model achieves the highest performance on all splits and on all sub-tasks. All models are publicly released.
Collapse
Affiliation(s)
- Marco Bombieri
- Department of Computer Science, University of Verona, Verona, Italy.
| | - Marco Rospocher
- Department of Foreign Languages and Literatures, University of Verona, Verona, Italy
| | | | - Paolo Fiorini
- Department of Computer Science, University of Verona, Verona, Italy
| |
Collapse
|
76
|
Taira RK, Garlid AO, Speier W. Design considerations for a hierarchical semantic compositional framework for medical natural language understanding. PLoS One 2023; 18:e0282882. [PMID: 36928721 PMCID: PMC10019629 DOI: 10.1371/journal.pone.0282882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Accepted: 02/24/2023] [Indexed: 03/18/2023] Open
Abstract
Medical natural language processing (NLP) systems are a key enabling technology for transforming Big Data from clinical report repositories to information used to support disease models and validate intervention methods. However, current medical NLP systems fall considerably short when faced with the task of logically interpreting clinical text. In this paper, we describe a framework inspired by mechanisms of human cognition in an attempt to jump the NLP performance curve. The design centers on a hierarchical semantic compositional model (HSCM), which provides an internal substrate for guiding the interpretation process. The paper describes insights from four key cognitive aspects: semantic memory, semantic composition, semantic activation, and hierarchical predictive coding. We discuss the design of a generative semantic model and an associated semantic parser used to transform a free-text sentence into a logical representation of its meaning. The paper discusses supportive and antagonistic arguments for the key features of the architecture as a long-term foundational framework.
Collapse
Affiliation(s)
- Ricky K. Taira
- Medical and Imaging Informatics (MII) Group, Department of Radiological Sciences, University of California, Los Angeles, Los Angeles, California, United States of America
- * E-mail:
| | - Anders O. Garlid
- Medical and Imaging Informatics (MII) Group, Department of Radiological Sciences, University of California, Los Angeles, Los Angeles, California, United States of America
| | - William Speier
- Medical and Imaging Informatics (MII) Group, Department of Radiological Sciences, University of California, Los Angeles, Los Angeles, California, United States of America
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, California, United States of America
| |
Collapse
|
77
|
Zack T, Dhaliwal G, Geha R, Margaretten M, Murray S, Hong JC. A Clinical Reasoning-Encoded Case Library Developed through Natural Language Processing. J Gen Intern Med 2023; 38:5-11. [PMID: 36071325 PMCID: PMC9849536 DOI: 10.1007/s11606-022-07758-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 07/29/2022] [Indexed: 01/22/2023]
Abstract
IMPORTANCE Case reports that externalize expert diagnostic reasoning are utilized for clinical reasoning instruction but are difficult to search based on symptoms, final diagnosis, or differential diagnosis construction. Computational approaches that uncover how experienced diagnosticians analyze the medical information in a case as they formulate a differential diagnosis can guide educational uses of case reports. OBJECTIVE To develop a "reasoning-encoded" case database for advanced clinical reasoning instruction by applying natural language processing (NLP), a sub-field of artificial intelligence, to a large case report library. DESIGN We collected 2525 cases from the New England Journal of Medicine (NEJM) Clinical Pathological Conference (CPC) from 1965 to 2020 and used NLP to analyze the medical terminology in each case to derive unbiased (not prespecified) categories of analysis used by the clinical discussant. We then analyzed and mapped the degree of category overlap between cases. RESULTS Our NLP algorithms identified clinically relevant categories that reflected the relationships between medical terms (which included symptoms, signs, test results, pathophysiology, and diagnoses). NLP extracted 43,291 symptoms across 2525 cases and physician-annotated 6532 diagnoses (both primary and related diagnoses). Our unsupervised learning computational approach identified 12 categories of medical terms that characterized the differential diagnosis discussions within individual cases. We used these categories to derive a measure of differential diagnosis similarity between cases and developed a website ( universeofcpc.com ) to allow visualization and exploration of 55 years of NEJM CPC case series. CONCLUSIONS Applying NLP to curated instances of diagnostic reasoning can provide insight into how expert clinicians correlate and coordinate disease categories and processes when creating a differential diagnosis. Our reasoning-encoded CPC case database can be used by clinician-educators to design a case-based curriculum and by physicians to direct their lifelong learning efforts.
Collapse
Affiliation(s)
- Travis Zack
- Division of Hematology/Oncology, Department of Medicine, University of California, San Francisco, CA, USA.
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, 94158, USA.
| | - Gurpreet Dhaliwal
- San Francisco VA Medical Center, San Francisco, CA, USA
- Department of Medicine, University of California, San Francisco, CA, USA
| | - Rabih Geha
- San Francisco VA Medical Center, San Francisco, CA, USA
- Department of Medicine, University of California, San Francisco, CA, USA
| | - Mary Margaretten
- Division of Rheumatology, Department of Medicine, University of California, San Francisco, CA, USA
| | - Sara Murray
- Division of Hospital Medicine, Department of Medicine, University of California, San Francisco, CA, USA
| | - Julian C Hong
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, 94158, USA
- Department of Radiation Oncology, University of California, San Francisco, CA, USA
| |
Collapse
|
78
|
Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, Dong H, Poon MTC, Fitzpatrick N, Levine AP, Slater LT, Handy A, Karwath A, Gkoutos GV, Chelala C, Shah AD, Stewart R, Collier N, Alex B, Whiteley W, Sudlow C, Roberts A, Dobson RJB. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med 2022; 5:186. [PMID: 36544046 PMCID: PMC9770568 DOI: 10.1038/s41746-022-00730-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.
Collapse
Affiliation(s)
- Honghan Wu
- Institute of Health Informatics, University College London, London, UK.
| | - Minhong Wang
- Institute of Health Informatics, University College London, London, UK
| | - Jinge Wu
- Institute of Health Informatics, University College London, London, UK
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Farah Francis
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Yun-Hsuan Chang
- Institute of Health Informatics, University College London, London, UK
| | - Alex Shavick
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Hang Dong
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Department of Computer Science, University of Oxford, Oxford, UK
| | | | | | - Adam P Levine
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Luke T Slater
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Alex Handy
- Institute of Health Informatics, University College London, London, UK
- University College London Hospitals NHS Trust, London, UK
| | - Andreas Karwath
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Georgios V Gkoutos
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Claude Chelala
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Anoop Dinesh Shah
- Institute of Health Informatics, University College London, London, UK
| | - Robert Stewart
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK
- South London and Maudsley NHS Foundation Trust, London, UK
| | - Nigel Collier
- Theoretical and Applied Linguistics, Faculty of Modern & Medieval Languages & Linguistics, University of Cambridge, Cambridge, UK
| | - Beatrice Alex
- Edinburgh Futures Institute, University of Edinburgh, Edinburgh, UK
| | | | - Cathie Sudlow
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Angus Roberts
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London, UK
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| |
Collapse
|
79
|
YAN J, WEN Z, ZOU B. Heterogeneous graph construction and node representation learning method of Treatise on Febrile Diseases based on graph convolutional network. DIGITAL CHINESE MEDICINE 2022. [DOI: 10.1016/j.dcmed.2022.12.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2023] Open
|
80
|
Dino MJS, Davidson PM, Dion KW, Szanton SL, Ong IL. Nursing and human-computer interaction in healthcare robots for older people: An integrative review. INTERNATIONAL JOURNAL OF NURSING STUDIES ADVANCES 2022; 4:100072. [PMID: 38745638 PMCID: PMC11080351 DOI: 10.1016/j.ijnsa.2022.100072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 03/01/2022] [Accepted: 03/02/2022] [Indexed: 10/18/2022] Open
Abstract
Objectives This study examined the published works related to healthcare robotics for older people using the attributes of health, nursing, and the human-computer interaction framework. Design An integrative literature review. Methods A search strategy captured 55 eligible articles from databases (CINAHL, Embase, IEEE Xplore, and PubMed) and hand-searching approaches. Bibliometric and content analyses grounded on the health and nursing attributes and human-computer interaction framework were performed using MAXQDA. Finally, results were verified using critical friend feedback by a second reviewer. Results Most articles were from multiple authorship, published in non-nursing journals, and originating from developed economies. They primarily focused on applying healthcare robots in practice settings, physical health, and communication tasks. Using the human-computer interaction framework, it was found that older adults frequently served as the primary users while nurses, healthcare providers, and researchers functioned as secondary users and operators. Research articles focused on the usability, functionality, and acceptability of robotic systems. At the same time, theoretical papers explored the frameworks and the value of empathy and emotion in robots, human-computer interaction and nursing models and theories supporting healthcare practice, and gerontechnology. Current robotic systems are less anthropomorphic, operated through real-time direct and supervisory inputs, and mainly equipped with visual and auditory sensors and actuators with limited capability in performing health assessments. Conclusion Results communicate the need for technological competency among nurses, advancements in increasing healthcare robot humanness, and the importance of conscientious efforts from an interdisciplinary research team in improving robotic system usability and utility for the care of older adults.
Collapse
Affiliation(s)
- Michael Joseph S. Dino
- School of Nursing, Johns Hopkins University, 525 N Wolfe St, Baltimore, MD 21205, USA
- Our Lady of Fatima University, 120 McArthur Highway, Marulas, Valenzuela City 1440, Philippines
| | - Patricia M. Davidson
- School of Nursing, Johns Hopkins University, 525 N Wolfe St, Baltimore, MD 21205, USA
- University of Wollongong, The Vice-Chancellor's Unit Building 36, University of Wollongong, NSW 2522, Australia
| | - Kenneth W. Dion
- School of Nursing, Johns Hopkins University, 525 N Wolfe St, Baltimore, MD 21205, USA
| | - Sarah L. Szanton
- School of Nursing, Johns Hopkins University, 525 N Wolfe St, Baltimore, MD 21205, USA
| | - Irvin L. Ong
- School of Nursing, Johns Hopkins University, 525 N Wolfe St, Baltimore, MD 21205, USA
- Our Lady of Fatima University, 120 McArthur Highway, Marulas, Valenzuela City 1440, Philippines
| |
Collapse
|
81
|
Horng H, Steinkamp J, Kahn CE, Cook TS. Ensemble Approaches to Recognize Protected Health Information in Radiology Reports. J Digit Imaging 2022; 35:1694-1698. [PMID: 35715655 PMCID: PMC9712864 DOI: 10.1007/s10278-022-00673-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Revised: 06/02/2022] [Accepted: 06/07/2022] [Indexed: 10/18/2022] Open
Abstract
Natural language processing (NLP) techniques for electronic health records have shown great potential to improve the quality of medical care. The text of radiology reports frequently constitutes a large fraction of EHR data, and can provide valuable information about patients' diagnoses, medical history, and imaging findings. The lack of a major public repository for radiological reports severely limits the development, testing, and application of new NLP tools. De-identification of protected health information (PHI) presents a major challenge to building such repositories, as many automated tools for de-identification were trained or designed for clinical notes and do not perform sufficiently well to build a public database of radiology reports. We developed and evaluated six ensemble models based on three publically available de-identification tools: MIT de-id, NeuroNER, and Philter. A set of 1023 reports was set aside as the testing partition. Two individuals with medical training annotated the test set for PHI; differences were resolved by consensus. Ensemble methods included simple voting schemes (1-Vote, 2-Votes, and 3-Votes), a decision tree, a naïve Bayesian classifier, and Adaboost boosting. The 1-Vote ensemble achieved recall of 998 / 1043 (95.7%); the 3-Votes ensemble had precision of 1035 / 1043 (99.2%). F1 scores were: 93.4% for the decision tree, 71.2% for the naïve Bayesian classifier, and 87.5% for the boosting method. Basic voting algorithms and machine learning classifiers incorporating the predictions of multiple tools can outperform each tool acting alone in de-identifying radiology reports. Ensemble methods hold substantial potential to improve automated de-identification tools for radiology reports to make such reports more available for research use to improve patient care and outcomes.
Collapse
Affiliation(s)
- Hannah Horng
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Jackson Steinkamp
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, USA
| | - Charles E Kahn
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, USA.
- Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA.
| | - Tessa S Cook
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
82
|
Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations. ATS Sch 2022; 3:548-560. [PMID: 36726701 PMCID: PMC9886197 DOI: 10.34197/ats-scholar.2022-0010oc] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Accepted: 08/08/2022] [Indexed: 02/04/2023] Open
Abstract
Background Oral case presentation is a crucial skill of physicians and a key component of team-based care. However, consistent and objective assessment and feedback on presentations during training are infrequent. Objective To determine the potential value of applying natural language processing, computer software that extracts meaning from text, to transcripts of oral case presentations as a strategy to assess their quality automatically and objectively. Methods We transcribed a collection of simulated oral case presentations. The presentations were from eight critical care fellows and one critical care attending. They were instructed to review the medical charts of 11 real intensive care unit patient cases and to audio record themselves, presenting each case as if they were doing so on morning rounds. We then used natural language processing to convert the transcripts from human-readable text into machine-readable numbers. These numbers represent details of the presentation style and content. The distance between the numeric representation of two different transcripts negatively correlates with the similarity of those two transcripts. We ranked fellows on the basis of how similar their presentations were to the attending's presentations. Results The 99 presentations included 260 minutes of audio (mean length: 2.6 ± 1.24 min per case). On average, 23.88 ± 2.65 sentences were spoken, and each sentence had 14.10 ± 0.67 words, 3.62 ± 0.15 medical concepts, and 0.75 ± 0.09 medical adjectives. When ranking fellows on the basis of how similar their presentations were to the attending's presentation, we found a gap between the five fellows with the most similar presentations and the three fellows with the least similar presentations (average group similarity scores of 0.62 ± 0.01 and 0.53 ± 0.01, respectively). Rankings were sensitive to whether presentation style or content information were weighted more heavily when calculating transcript similarity. Conclusion Natural language processing enabled the ranking of case presentations on the basis of how similar they were to a reference presentation. Although additional work is needed to convert these rankings, and underlying similarity scores, into actionable feedback for trainees, these methods may support new tools for improving medical education.
Collapse
|
83
|
Wang C, Xu T, Yu W, Li T, Han H, Zhang M, Tao M. Early diagnosis of Alzheimer's disease and mild cognitive impairment based on electroencephalography: From the perspective of event related potentials and deep learning. Int J Psychophysiol 2022; 182:182-189. [DOI: 10.1016/j.ijpsycho.2022.10.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 10/21/2022] [Accepted: 10/24/2022] [Indexed: 11/06/2022]
|
84
|
Uthman OA, Court R, Enderby J, Al-Khudairy L, Nduka C, Mistry H, Melendez-Torres GJ, Taylor-Phillips S, Clarke A. Increasing comprehensiveness and reducing workload in a systematic review of complex interventions using automated machine learning. Health Technol Assess 2022:10.3310/UDIR6682. [PMID: 36562494 PMCID: PMC10068584 DOI: 10.3310/udir6682] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/05/2022] Open
Abstract
BACKGROUND As part of our ongoing systematic review of complex interventions for the primary prevention of cardiovascular diseases, we have developed and evaluated automated machine-learning classifiers for title and abstract screening. The aim was to develop a high-performing algorithm comparable to human screening. METHODS We followed a three-phase process to develop and test an automated machine learning-based classifier for screening potential studies on interventions for primary prevention of cardiovascular disease. We labelled a total of 16,611 articles during the first phase of the project. In the second phase, we used the labelled articles to develop a machine learning-based classifier. After that, we examined the performance of the classifiers in correctly labelling the papers. We evaluated the performance of the five deep-learning models [i.e. parallel convolutional neural network ( CNN ), stacked CNN , parallel-stacked CNN , recurrent neural network ( RNN ) and CNN-RNN]. The models were evaluated using recall, precision and work saved over sampling at no less than 95% recall. RESULTS We labelled a total of 16,611 articles, of which 676 (4.0%) were tagged as 'relevant' and 15,935 (96%) were tagged as 'irrelevant'. The recall ranged from 51.9% to 96.6%. The precision ranged from 64.6% to 99.1%. The work saved over sampling ranged from 8.9% to as high as 92.1%. The best-performing model was parallel CNN , yielding a 96.4% recall, as well as 99.1% precision, and a potential workload reduction of 89.9%. FUTURE WORK AND LIMITATIONS We used words from the title and the abstract only. More work needs to be done to look into possible changes in performance, such as adding features such as full document text. The approach might also not be able to be used for other complex systematic reviews on different topics. CONCLUSION Our study shows that machine learning has the potential to significantly aid the labour-intensive screening of abstracts in systematic reviews of complex interventions. Future research should concentrate on enhancing the classifier system and determining how it can be integrated into the systematic review workflow. FUNDING This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in Health Technology Assessment. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
| | - Rachel Court
- Warwick Medical School, University of Warwick, Coventry, UK
| | - Jodie Enderby
- Warwick Medical School, University of Warwick, Coventry, UK
| | | | - Chidozie Nduka
- Warwick Medical School, University of Warwick, Coventry, UK
| | - Hema Mistry
- Warwick Medical School, University of Warwick, Coventry, UK
| | - G J Melendez-Torres
- Peninsula Technology Assessment Group (PenTAG), College of Medicine and Health, University of Exeter, Exeter, UK
| | | | - Aileen Clarke
- Warwick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
85
|
Cheng C, Duan S, He H, Li X, Chen Y. A Generalized Robot Navigation Analysis Platform (RoNAP) with Visual Results Using Multiple Navigation Algorithms. SENSORS (BASEL, SWITZERLAND) 2022; 22:9036. [PMID: 36501739 PMCID: PMC9737631 DOI: 10.3390/s22239036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Revised: 11/11/2022] [Accepted: 11/17/2022] [Indexed: 06/17/2023]
Abstract
The robotic navigation task is to find a collision-free path among a mass of stationary or migratory obstacles. Various well-established algorithms have been applied to solve navigation tasks. It is necessary to test the performance of designed navigation algorithms in practice. However, it seems an extremely unwise choice to implement them in a real environment directly unless their performance is guaranteed to be acceptable. Otherwise, it takes time to test navigation algorithms because of a long training process, and imperfect performance may cause damage if the robot collides with obstacles. Hence, it is of key significance to develop a mobile robot analysis platform to simulate the real environment which has the ability to replicate the exact application scenario and be operated in a simple manner. This paper introduces a brand new analysis platform named robot navigation analysis platform (RoNAP), which is an open-source platform developed using the Python environment. A user-friendly interface supports its realization for the evaluation of various navigation algorithms. A variety of existing algorithms were able to achieve desired test results on this platform, indicating its feasibility and efficiency for navigation algorithm analysis.
Collapse
Affiliation(s)
- Chuanxin Cheng
- School of Mechanical and Electrical Engineering, Soochow University, Suzhou 215031, China
| | - Shuang Duan
- School of Mechanical and Electrical Engineering, Soochow University, Suzhou 215031, China
| | - Haidong He
- School of Mechanical and Electrical Engineering, Soochow University, Suzhou 215031, China
| | - Xinlin Li
- Department of Digital Media, Soochow University, Suzhou 215031, China
| | - Yiyang Chen
- School of Mechanical and Electrical Engineering, Soochow University, Suzhou 215031, China
| |
Collapse
|
86
|
Lentzen M, Madan S, Lage-Rupprecht V, Kühnel L, Fluck J, Jacobs M, Mittermaier M, Witzenrath M, Brunecker P, Hofmann-Apitius M, Weber J, Fröhlich H. Critical assessment of transformer-based AI models for German clinical notes. JAMIA Open 2022; 5:ooac087. [PMID: 36380848 PMCID: PMC9663939 DOI: 10.1093/jamiaopen/ooac087] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 10/02/2022] [Accepted: 10/25/2022] [Indexed: 11/17/2022] Open
Abstract
Objective Healthcare data such as clinical notes are primarily recorded in an unstructured manner. If adequately translated into structured data, they can be utilized for health economics and set the groundwork for better individualized patient care. To structure clinical notes, deep-learning methods, particularly transformer-based models like Bidirectional Encoder Representations from Transformers (BERT), have recently received much attention. Currently, biomedical applications are primarily focused on the English language. While general-purpose German-language models such as GermanBERT and GottBERT have been published, adaptations for biomedical data are unavailable. This study evaluated the suitability of existing and novel transformer-based models for the German biomedical and clinical domain. Materials and Methods We used 8 transformer-based models and pre-trained 3 new models on a newly generated biomedical corpus, and systematically compared them with each other. We annotated a new dataset of clinical notes and used it with 4 other corpora (BRONCO150, CLEF eHealth 2019 Task 1, GGPONC, and JSynCC) to perform named entity recognition (NER) and document classification tasks. Results General-purpose language models can be used effectively for biomedical and clinical natural language processing (NLP) tasks, still, our newly trained BioGottBERT model outperformed GottBERT on both clinical NER tasks. However, training new biomedical models from scratch proved ineffective. Discussion The domain-adaptation strategy’s potential is currently limited due to a lack of pre-training data. Since general-purpose language models are only marginally inferior to domain-specific models, both options are suitable for developing German-language biomedical applications. Conclusion General-purpose language models perform remarkably well on biomedical and clinical NLP tasks. If larger corpora become available in the future, domain-adapting these models may improve performances.
Collapse
Affiliation(s)
- Manuel Lentzen
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, Sankt Augustin, Germany,Bonn-Aachen International Center for Information Technology (B-IT), University of Bonn, Bonn, Germany
| | - Sumit Madan
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, Sankt Augustin, Germany,Institute of Computer Science, University of Bonn, Bonn, Germany
| | - Vanessa Lage-Rupprecht
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, Sankt Augustin, Germany
| | - Lisa Kühnel
- Knowledge Management, ZB MED – Information Centre for Life Sciences, Cologne, Germany,Graduate School DILS, Bielefeld Institute for Bioinformatics Infrastructure (BIBI), Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Juliane Fluck
- Knowledge Management, ZB MED – Information Centre for Life Sciences, Cologne, Germany,The Agricultural Faculty, University of Bonn, Bonn, Germany
| | - Marc Jacobs
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, Sankt Augustin, Germany
| | - Mirja Mittermaier
- Department of Infectious Diseases and Respiratory Medicine, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany,Berlin Institute of Health (BIH) at Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Martin Witzenrath
- Department of Infectious Diseases and Respiratory Medicine, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany,German Center for Lung Research (DZL), Partner Site Charité, Berlin, Germany
| | - Peter Brunecker
- Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Core Facility Research IT, Berlin, Germany
| | - Martin Hofmann-Apitius
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, Sankt Augustin, Germany,Bonn-Aachen International Center for Information Technology (B-IT), University of Bonn, Bonn, Germany
| | - Joachim Weber
- Berlin Institute of Health (BIH) at Charité – Universitätsmedizin Berlin, Berlin, Germany,Charité – Universitätsmedizin Berlin, Center for Stroke Research Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany,Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Holger Fröhlich
- Corresponding Author: Prof. Dr. Holger Fröhlich, Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Schloss Birlinghoven, 53757 Sankt Augustin, Germany;
| |
Collapse
|
87
|
Huang BB, Huang J, Swong KN. Natural Language Processing in Spine Surgery: A Systematic Review of Applications, Bias, and Reporting Transparency. World Neurosurg 2022; 167:156-164.e6. [PMID: 36049723 DOI: 10.1016/j.wneu.2022.08.109] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 08/22/2022] [Accepted: 08/23/2022] [Indexed: 10/31/2022]
Abstract
BACKGROUND Natural language processing (NLP) is a discipline of machine learning concerned with the analysis of language and text. Although NLP has been applied to various forms of clinical text, the applications and utility of NLP in spine surgery remain poorly characterized. Here, we systematically reviewed studies that use NLP for spine surgery applications, and analyzed applications, bias, and reporting transparency of the studies. METHODS We performed a literature search using the PubMed, Scopus, and Embase databases. Data extraction was performed after appropriate screening. The risk of bias and reporting quality were assessed using the PROBAST and TRIPOD tools. RESULTS A total of 12 full-text articles were included. The most common diseases represented include spondylolisthesis (25%), scoliosis (17%), and lumbar disk herniation (17%). The most common procedures included spinal fusion (42%), imaging (e.g. magnetic resonance, X-ray) (25%), and scoliosis correction (17%). Reported outcomes were diverse and included incidental durotomy, venous thromboembolism, and the tone of social media posts regarding scoliosis surgery. Common sources of bias identified included the use of older methods that do not capture the nuance of a text, and not using a prespecified or standard outcome measure when evaluating NLP methods. CONCLUSIONS Although the application of NLP to spine surgery is expanding, current studies face limitations and none are indicated as ready for clinical use. Thus, for future studies we recommend an emphasis on transparent reporting and collaboration with NLP experts to incorporate the latest developments to improve models and contribute to further innovation.
Collapse
Affiliation(s)
- Bonnie B Huang
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Jonathan Huang
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Kevin N Swong
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA.
| |
Collapse
|
88
|
Xu X, Qin L, Ding L, Wang C, Wang M, Li Z, Li J. Identifying stroke diagnosis-related features from medical imaging reports to improve clinical decision-making support. BMC Med Inform Decis Mak 2022; 22:275. [PMID: 36266650 PMCID: PMC9583470 DOI: 10.1186/s12911-022-02012-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 09/30/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Medical imaging reports play an important role in communication of diagnostic information between radiologists and clinicians. Head magnetic resonance imaging (MRI) reports can provide evidence that is widely used in the diagnosis and treatment of ischaemic stroke. The high-signal regions of diffusion-weighted imaging (DWI) images in MRI reports are key evidence. Correctly identifying high-signal regions of DWI images is helpful for the treatment of ischaemic stroke patients. Since most of the multiple signals recorded in head MRI reports appear in the same part, it is challenging to identify high-signal regions of DWI images from MRI reports. METHODS We developed a deep learning model to automatically identify high-signal regions of DWI images from head MRI reports. We proposed a fine-grained entity typing model based on machine reading comprehension that transformed the traditional two-step fine-grained entity typing task into a question-answering task. RESULTS To prove the validity of the model proposed, we compared it with the fine-grained entity typing model, of which the F1 measure was 5.9% and 3.2% higher than the F1 measures of the models based on LSTM and BERT, respectively. CONCLUSION In this study, we explore the automatic identification of high-signal regions of DWI images from the description part of a head MRI report. We transformed the identification of high-signal regions of DWI images to an FET task and proposed an MRC-FET model. Compared with the traditional two-step FET method, the model we proposed not only simplifies the task but also has better performance. The comparable result shows that the work in this study can contribute to improving the clinical decision support system.
Collapse
Affiliation(s)
- Xiaowei Xu
- Institute of Medical Information, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China
| | - Lu Qin
- Institute of Medical Information, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China
| | - Lingling Ding
- Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Chunjuan Wang
- Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Meng Wang
- Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Zixiao Li
- Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Jiao Li
- Institute of Medical Information, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China.
| |
Collapse
|
89
|
Deep learning accurately predicts food categories and nutrients based on ingredient statements. Food Chem 2022; 391:133243. [PMID: 35623276 DOI: 10.1016/j.foodchem.2022.133243] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 05/06/2022] [Accepted: 05/16/2022] [Indexed: 11/20/2022]
Abstract
Determining attributes such as classification, creating taxonomies and nutrients for foods can be a challenging and resource-intensive task, albeit important for a better understanding of foods. In this study, a novel dataset, 134 k BFPD, was collected from USDA Branded Food Products Database with modification and labeled with three food taxonomy and nutrient values and became an artificial intelligence (AI) dataset that covered the largest food types to date. Overall, the Multi-Layer Perceptron (MLP)-TF-SE method obtained the highest learning efficiency for food natural language processing tasks using AI, which achieved up to 99% accuracy for food classification and 0.98 R2 for calcium estimation (0.93 ∼ 0.97 for calories, protein, sodium, total carbohydrate, total lipids, etc.). The deep learning approach has great potential to be embedded in other food classification and regression tasks and as an extension to other applications in the food and nutrient scope.
Collapse
|
90
|
Topaz M, Song J, Davoudi A, McDonald M, Taylor J, Sittig S, Bowles K. Home Healthcare Clinicians Use Judgment Language More Frequently for Black and Hispanic Patients: A Natural Language Processing Study (Preprint). JMIR Nurs 2022; 6:e42552. [PMID: 37067893 PMCID: PMC10152333 DOI: 10.2196/42552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 12/05/2022] [Accepted: 03/16/2023] [Indexed: 03/18/2023] Open
Abstract
BACKGROUND A clinician's biased behavior toward patients can affect the quality of care. Recent literature reviews report on widespread implicit biases among clinicians. Although emerging studies in hospital settings show racial biases in the language used in clinical documentation within electronic health records, no studies have yet investigated the extent of judgment language in home health care. OBJECTIVE We aimed to examine racial differences in judgment language use and the relationship between judgment language use and the amount of time clinicians spent on home visits as a reflection of care quality in home health care. METHODS This study is a retrospective observational cohort study. Study data were extracted from a large urban home health care organization in the Northeastern United States. Study data set included patients (N=45,384) who received home health care services between January 1 and December 31, 2019. The study applied a natural language processing algorithm to automatically detect the language of judgment in clinical notes. RESULTS The use of judgment language was observed in 38% (n=17,141) of the patients. The highest use of judgment language was found in Hispanic (7,167/66,282, 10.8% of all clinical notes), followed by Black (7,010/65,628, 10.7%), White (10,206/107,626, 9.5%), and Asian (1,756/22,548, 7.8%) patients. Black and Hispanic patients were 14% more likely to have notes with judgment language than White patients. The length of a home health care visit was reduced by 21 minutes when judgment language was used. CONCLUSIONS Racial differences were identified in judgment language use. When judgment language is used, clinicians spend less time at patients' homes. Because the language clinicians use in documentation is associated with the time spent providing care, further research is needed to study the impact of using judgment language on quality of home health care. Policy, education, and clinical practice improvements are needed to address the biases behind judgment language.
Collapse
Affiliation(s)
- Maxim Topaz
- Columbia University School of Nursing, New York, NY, United States
- Data Science Institute, Columbia University, New York, NY, United States
- Center for Home Care Policy & Research, VNS Health, New York, NY, United States
| | - Jiyoun Song
- Columbia University School of Nursing, New York, NY, United States
| | - Anahita Davoudi
- Center for Home Care Policy & Research, VNS Health, New York, NY, United States
| | - Margaret McDonald
- Center for Home Care Policy & Research, VNS Health, New York, NY, United States
| | - Jacquelyn Taylor
- Columbia University School of Nursing, New York, NY, United States
| | - Scott Sittig
- Department of Health Sciences, University of Louisiana at Lafayette, Lafayette, LA, United States
| | - Kathryn Bowles
- Center for Home Care Policy & Research, VNS Health, New York, NY, United States
- Department of Biobehavioral Health Sciences, University of Pennsylvania School of Nursing, Philadelphia, PA, United States
| |
Collapse
|
91
|
Li Z, Wang X, Xu M, Li Y, Wang Y, Chen Y, Li S, Li Z, Yang J, Tang C, Xiong F, Jian W, He P, Zhan Y, Zheng J, Ye F. Development and clinical application of an electronic health record quality control system for pulmonary aspergillosis based on guidelines and natural language processing technology. J Thorac Dis 2022; 14:3398-3407. [PMID: 36245604 PMCID: PMC9562533 DOI: 10.21037/jtd-22-532] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 08/19/2022] [Indexed: 11/26/2022]
Abstract
Background There are considerable differences in the diagnosis and treatment of pulmonary aspergillosis (PA) between specialized hospitals and primary hospitals or developed areas and underdeveloped areas in China. There is a lack of electronic systems that assist respiratory physicians in standardizing the diagnosis and treatment of PA. Methods We extracted 26 quality control points from the latest guidelines related to PA, and developed a PA quality control system of electronic health record (EHR) based on natural language processing (NLP) techniques. We obtained PA patient records in the Department of Respiratory Medicine of the First Affiliated Hospital of Guangzhou Medical University to verify the effectiveness of the system comparing with manually evaluation of respiratory experts. Results We successfully developed quality control system of PA; 699 PA medical records from EHR of the First Affiliated Hospital of Guangzhou Medical University between January 2015 and March 2020 were obtained and assessed by the system; 162 defects were found, which included 19 medical records with diagnostic defects, 76 medical records with examination defects, and 80 medical records with treatment defects; 200 medical records were sampled for validation, and found that the sensitivity and accuracy of quality control system for pulmonary aspergillosis (QCSA) were 0.99 and 0.96, F1 value was 0.85, and the recall rate was 0.77 compared with experts' evaluation. Conclusions Our system successfully uses medical guidelines and NLP technology to detect defects in the diagnosis and treatment of PA, which helps to improve the management quality of PA patients.
Collapse
Affiliation(s)
- Zhengtu Li
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Xidong Wang
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Mengke Xu
- Guangzhou Tianpeng Technology Co., Ltd., Guangzhou, China
| | - Yongming Li
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Yinguang Wang
- Guangzhou Tianpeng Technology Co., Ltd., Guangzhou, China
| | - Yijun Chen
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Shaoqiang Li
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Zhun Li
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Jinglu Yang
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Chun Tang
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Fangshu Xiong
- Guangzhou Tianpeng Technology Co., Ltd., Guangzhou, China
| | - Wenhua Jian
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Peimei He
- Guangzhou Tianpeng Technology Co., Ltd., Guangzhou, China
| | - Yangqing Zhan
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Jinping Zheng
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Feng Ye
- State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| |
Collapse
|
92
|
Zhang B, Fan T. Knowledge structure and emerging trends in the application of deep learning in genetics research: A bibliometric analysis [2000–2021]. Front Genet 2022; 13:951939. [PMID: 36081985 PMCID: PMC9445221 DOI: 10.3389/fgene.2022.951939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 07/13/2022] [Indexed: 11/13/2022] Open
Abstract
Introduction: Deep learning technology has been widely used in genetic research because of its characteristics of computability, statistical analysis, and predictability. Herein, we aimed to summarize standardized knowledge and potentially innovative approaches for deep learning applications of genetics by evaluating publications to encourage more research.Methods: The Science Citation Index Expanded TM (SCIE) database was searched for deep learning applications for genomics-related publications. Original articles and reviews were considered. In this study, we derived a clustered network from 69,806 references that were cited by the 1,754 related manuscripts identified. We used CiteSpace and VOSviewer to identify countries, institutions, journals, co-cited references, keywords, subject evolution, path, current characteristics, and emerging topics.Results: We assessed the rapidly increasing publications concerned about deep learning applications of genomics approaches and identified 1,754 articles that published reports focusing on this subject. Among these, a total of 101 countries and 2,487 institutes contributed publications, The United States of America had the most publications (728/1754) and the highest h-index, and the US has been in close collaborations with China and Germany. The reference clusters of SCI articles were clustered into seven categories: deep learning, logic regression, variant prioritization, random forests, scRNA-seq (single-cell RNA-seq), genomic regulation, and recombination. The keywords representing the research frontiers by year were prediction (2016–2021), sequence (2017–2021), mutation (2017–2021), and cancer (2019–2021).Conclusion: Here, we summarized the current literature related to the status of deep learning for genetics applications and analyzed the current research characteristics and future trajectories in this field. This work aims to provide resources for possible further intensive exploration and encourages more researchers to overcome the research of deep learning applications in genetics.
Collapse
Affiliation(s)
- Bijun Zhang
- Department of Clinical Genetics, Shengjing Hospital of China Medical University, Shenyang, China
| | - Ting Fan
- Department of Computer, School of Intelligent Medicine, China Medical University, Shenyang, China
- *Correspondence: Ting Fan,
| |
Collapse
|
93
|
Chen JS, Baxter SL. Applications of natural language processing in ophthalmology: present and future. Front Med (Lausanne) 2022; 9:906554. [PMID: 36004369 PMCID: PMC9393550 DOI: 10.3389/fmed.2022.906554] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 05/31/2022] [Indexed: 11/13/2022] Open
Abstract
Advances in technology, including novel ophthalmic imaging devices and adoption of the electronic health record (EHR), have resulted in significantly increased data available for both clinical use and research in ophthalmology. While artificial intelligence (AI) algorithms have the potential to utilize these data to transform clinical care, current applications of AI in ophthalmology have focused mostly on image-based deep learning. Unstructured free-text in the EHR represents a tremendous amount of underutilized data in big data analyses and predictive AI. Natural language processing (NLP) is a type of AI involved in processing human language that can be used to develop automated algorithms using these vast quantities of available text data. The purpose of this review was to introduce ophthalmologists to NLP by (1) reviewing current applications of NLP in ophthalmology and (2) exploring potential applications of NLP. We reviewed current literature published in Pubmed and Google Scholar for articles related to NLP and ophthalmology, and used ancestor search to expand our references. Overall, we found 19 published studies of NLP in ophthalmology. The majority of these publications (16) focused on extracting specific text such as visual acuity from free-text notes for the purposes of quantitative analysis. Other applications included: domain embedding, predictive modeling, and topic modeling. Future ophthalmic applications of NLP may also focus on developing search engines for data within free-text notes, cleaning notes, automated question-answering, and translating ophthalmology notes for other specialties or for patients, especially with a growing interest in open notes. As medicine becomes more data-oriented, NLP offers increasing opportunities to augment our ability to harness free-text data and drive innovations in healthcare delivery and treatment of ophthalmic conditions.
Collapse
Affiliation(s)
- Jimmy S. Chen
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, United States
- Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA, United States
| | - Sally L. Baxter
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, United States
- Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA, United States
| |
Collapse
|
94
|
Guo Y, Ge Y, Yang YC, Al-Garadi MA, Sarker A. Comparison of Pretraining Models and Strategies for Health-Related Social Media Text Classification. Healthcare (Basel) 2022; 10:healthcare10081478. [PMID: 36011135 PMCID: PMC9408372 DOI: 10.3390/healthcare10081478] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Revised: 07/29/2022] [Accepted: 08/02/2022] [Indexed: 11/24/2022] Open
Abstract
Pretrained contextual language models proposed in the recent past have been reported to achieve state-of-the-art performances in many natural language processing (NLP) tasks, including those involving health-related social media data. We sought to evaluate the effectiveness of different pretrained transformer-based models for social media-based health-related text classification tasks. An additional objective was to explore and propose effective pretraining strategies to improve machine learning performance on such datasets and tasks. We benchmarked six transformer-based models that were pretrained with texts from different domains and sources—BERT, RoBERTa, BERTweet, TwitterBERT, BioClinical_BERT, and BioBERT—on 22 social media-based health-related text classification tasks. For the top-performing models, we explored the possibility of further boosting performance by comparing several pretraining strategies: domain-adaptive pretraining (DAPT), source-adaptive pretraining (SAPT), and a novel approach called topic specific pretraining (TSPT). We also attempted to interpret the impacts of distinct pretraining strategies by visualizing document-level embeddings at different stages of the training process. RoBERTa outperformed BERTweet on most tasks, and better than others. BERT, TwitterBERT, BioClinical_BERT and BioBERT consistently underperformed. For pretraining strategies, SAPT performed better or comparable to the off-the-shelf models, and significantly outperformed DAPT. SAPT + TSPT showed consistently high performance, with statistically significant improvement in three tasks. Our findings demonstrate that RoBERTa and BERTweet are excellent off-the-shelf models for health-related social media text classification, and extended pretraining using SAPT and TSPT can further improve performance.
Collapse
Affiliation(s)
- Yuting Guo
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
- Correspondence:
| | - Yao Ge
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| | - Yuan-Chi Yang
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| | - Mohammed Ali Al-Garadi
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37240, USA
| | - Abeed Sarker
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
95
|
Chaichulee S, Promchai C, Kaewkomon T, Kongkamol C, Ingviya T, Sangsupawanich P. Multi-label classification of symptom terms from free-text bilingual adverse drug reaction reports using natural language processing. PLoS One 2022; 17:e0270595. [PMID: 35925971 PMCID: PMC9352066 DOI: 10.1371/journal.pone.0270595] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 06/14/2022] [Indexed: 11/25/2022] Open
Abstract
Allergic reactions to medication range from mild to severe or even life-threatening. Proper documentation of patient allergy information is critical for safe prescription, avoiding drug interactions, and reducing healthcare costs. Allergy information is regularly obtained during the medical interview, but is often poorly documented in electronic health records (EHRs). While many EHRs allow for structured adverse drug reaction (ADR) reporting, a free-text entry is still common. The resulting information is neither interoperable nor easily reusable for other applications, such as clinical decision support systems and prescription alerts. Current approaches require pharmacists to review and code ADRs documented by healthcare professionals. Recently, the effectiveness of machine algorithms in natural language processing (NLP) has been widely demonstrated. Our study aims to develop and evaluate different NLP algorithms that can encode unstructured ADRs stored in EHRs into institutional symptom terms. Our dataset consists of 79,712 pharmacist-reviewed drug allergy records. We evaluated three NLP techniques: Naive Bayes—Support Vector Machine (NB-SVM), Universal Language Model Fine-tuning (ULMFiT), and Bidirectional Encoder Representations from Transformers (BERT). We tested different general-domain pre-trained BERT models, including mBERT, XLM-RoBERTa, and WanchanBERTa, as well as our domain-specific AllergyRoBERTa, which was pre-trained from scratch on our corpus. Overall, BERT models had the highest performance. NB-SVM outperformed ULMFiT and BERT for several symptom terms that are not frequently coded. The ensemble model achieved an exact match ratio of 95.33%, a F1 score of 98.88%, and a mean average precision of 97.07% for the 36 most frequently coded symptom terms. The model was then further developed into a symptom term suggestion system and achieved a Krippendorff’s alpha agreement coefficient of 0.7081 in prospective testing with pharmacists. Some degree of automation could both accelerate the availability of allergy information and reduce the efforts for human coding.
Collapse
Affiliation(s)
- Sitthichok Chaichulee
- Department of Biomedical Sciences and Biomedical Engineering, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
- Division of Digital Innovation and Data Analytics, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
- * E-mail:
| | - Chissanupong Promchai
- Department of Pharmacy, Songklanagarind Hospital, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
| | - Tanyamai Kaewkomon
- Department of Pharmacy, Songklanagarind Hospital, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
| | - Chanon Kongkamol
- Department of Family and Preventive Medicine, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
- Division of Digital Innovation and Data Analytics, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
| | - Thammasin Ingviya
- Department of Family and Preventive Medicine, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
- Division of Digital Innovation and Data Analytics, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
| | - Pasuree Sangsupawanich
- Department of Pediatrics, Faculty of Medicine, Prince of Songkla University, Hatyai, Songkhla, Thailand
| |
Collapse
|
96
|
Liu J, Capurro D, Nguyen A, Verspoor K. "Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks. J Biomed Inform 2022; 133:104149. [PMID: 35878821 DOI: 10.1016/j.jbi.2022.104149] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/28/2022] [Accepted: 07/19/2022] [Indexed: 10/17/2022]
Abstract
One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.
Collapse
Affiliation(s)
- Jinghui Liu
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Australian e-Health Research Centre, CSIRO, Brisbane, Australia.
| | - Daniel Capurro
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, Melbourne Medical School, The University of Melbourne, Victoria, Australia.
| | - Anthony Nguyen
- Australian e-Health Research Centre, CSIRO, Brisbane, Australia.
| | - Karin Verspoor
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, Melbourne Medical School, The University of Melbourne, Victoria, Australia; School of Computing Technologies, RMIT University, Victoria, Australia.
| |
Collapse
|
97
|
Khan DZ, Luengo I, Barbarisi S, Addis C, Culshaw L, Dorward NL, Haikka P, Jain A, Kerr K, Koh CH, Layard Horsfall H, Muirhead W, Palmisciano P, Vasey B, Stoyanov D, Marcus HJ. Automated operative workflow analysis of endoscopic pituitary surgery using machine learning: development and preclinical evaluation (IDEAL stage 0). J Neurosurg 2022; 137:51-58. [PMID: 34740198 PMCID: PMC10243668 DOI: 10.3171/2021.6.jns21923] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Accepted: 06/15/2021] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Surgical workflow analysis involves systematically breaking down operations into key phases and steps. Automatic analysis of this workflow has potential uses for surgical training, preoperative planning, and outcome prediction. Recent advances in machine learning (ML) and computer vision have allowed accurate automated workflow analysis of operative videos. In this Idea, Development, Exploration, Assessment, Long-term study (IDEAL) stage 0 study, the authors sought to use Touch Surgery for the development and validation of an ML-powered analysis of phases and steps in the endoscopic transsphenoidal approach (eTSA) for pituitary adenoma resection, a first for neurosurgery. METHODS The surgical phases and steps of 50 anonymized eTSA operative videos were labeled by expert surgeons. Forty videos were used to train a combined convolutional and recurrent neural network model by Touch Surgery. Ten videos were used for model evaluation (accuracy, F1 score), comparing the phase and step recognition of surgeons to the automatic detection of the ML model. RESULTS The longest phase was the sellar phase (median 28 minutes), followed by the nasal phase (median 22 minutes) and the closure phase (median 14 minutes). The longest steps were step 5 (tumor identification and excision, median 17 minutes); step 3 (posterior septectomy and removal of sphenoid septations, median 14 minutes); and step 4 (anterior sellar wall removal, median 10 minutes). There were substantial variations within the recorded procedures in terms of video appearances, step duration, and step order, with only 50% of videos containing all 7 steps performed sequentially in numerical order. Despite this, the model was able to output accurate recognition of surgical phases (91% accuracy, 90% F1 score) and steps (76% accuracy, 75% F1 score). CONCLUSIONS In this IDEAL stage 0 study, ML techniques have been developed to automatically analyze operative videos of eTSA pituitary surgery. This technology has previously been shown to be acceptable to neurosurgical teams and patients. ML-based surgical workflow analysis has numerous potential uses-such as education (e.g., automatic indexing of contemporary operative videos for teaching), improved operative efficiency (e.g., orchestrating the entire surgical team to a common workflow), and improved patient outcomes (e.g., comparison of surgical techniques or early detection of adverse events). Future directions include the real-time integration of Touch Surgery into the live operative environment as an IDEAL stage 1 (first-in-human) study, and further development of underpinning ML models using larger data sets.
Collapse
Affiliation(s)
- Danyal Z. Khan
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
| | - Imanol Luengo
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | | | - Carole Addis
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | - Lucy Culshaw
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | - Neil L. Dorward
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
| | - Pinja Haikka
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | - Abhiney Jain
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
| | - Karen Kerr
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | - Chan Hee Koh
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
| | - Hugo Layard Horsfall
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
| | - William Muirhead
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
| | - Paolo Palmisciano
- Department of Neurosurgery, Trauma Center, Gamma Knife Center, Cannizzaro Hospital, Catania, Italy; and
| | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, United Kingdom
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
- Digital Surgery Ltd., Medtronic, London, United Kingdom
| | - Hani J. Marcus
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London
| |
Collapse
|
98
|
Artificial Intelligence Algorithm in Classification and Recognition of Primary Hepatic Carcinoma Images under Magnetic Resonance Imaging. CONTRAST MEDIA & MOLECULAR IMAGING 2022; 2022:8950600. [PMID: 35800234 PMCID: PMC9197610 DOI: 10.1155/2022/8950600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/10/2022] [Accepted: 05/12/2022] [Indexed: 11/27/2022]
Abstract
This study aimed to discuss the application value of the bias field correction algorithm in magnetic resonance imaging (MRI) images of patients with primary hepatic carcinoma (PHC). In total, 52 patients with PHC were selected as the experimental group and divided into three subgroups: mild (15 cases), moderate (19 cases), and severe (18 cases) according to pathological grading. Another 52 patients with hepatic nodules in the same period were included in the control group. All the patients underwent dynamic contrast-enhanced (DCE) MRI examination, and the image qualities of MRI before and after bias field correction were compared. The DCE-MRI perfusion parameters were measured, including the transport constant Ktrans, reverse rate constant Kep, extravascular extracellular volume fraction (Ve), plasma volume (Vp), microvascular density (MVD), hepatic artery perfusion index (HPI), mean transit time of contrast agent (MTT), time to peak (TTP), blood volume (BV), hepatic arterial perfusion (HAP), full perfusion (FP), and portal venous perfusion (PVP). It was found that the sensitivity (93.63%), specificity (71.62%), positive predictive value (95.63%), negative predictive value (71.62%), and accuracy (90.01%) of MRI examination processed by the bias field correction algorithm were all significantly greater than those before processing (P < 0.05). The Ktrans, Kep, Ve, Vp, and MVD of patients in the experimental group were significantly larger than those of the control group, and severe group> moderate group> mild group (P < 0.05). HPI, MTT, TTP, BV, and HAP of patients in the experimental group were also significantly greater than those of the control group, which was shown as severe group > moderate group > mild group (P < 0.05). FP and PVP of the experimental group were significantly lower than those of the control group, and severe group < moderate group < mild group (P < 0.05). It was suggested that in MRI images of patients with PHC, the bias field correction algorithm could significantly improve the diagnosis rate. Each perfusion parameter was related to the pathological grading, which could be used to evaluate the prognosis of patients.
Collapse
|
99
|
Uyeda AM, Curtis JR, Engelberg RA, Brumback LC, Guo Y, Sibley J, Lober WB, Cohen T, Torrence J, Heywood J, Paul SR, Kross EK, Lee RY. Mixed-methods evaluation of three natural language processing modeling approaches for measuring documented goals-of-care discussions in the electronic health record. J Pain Symptom Manage 2022; 63:e713-e723. [PMID: 35182715 PMCID: PMC9124686 DOI: 10.1016/j.jpainsymman.2022.02.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Revised: 01/05/2022] [Accepted: 02/07/2022] [Indexed: 01/02/2023]
Abstract
CONTEXT Documented goals-of-care discussions are an important quality metric for patients with serious illness. Natural language processing (NLP) is a promising approach for identifying goals-of-care discussions in the electronic health record (EHR). OBJECTIVES To compare three NLP modeling approaches for identifying EHR documentation of goals-of-care discussions and generate hypotheses about differences in performance. METHODS We conducted a mixed-methods study to evaluate performance and misclassification for three NLP featurization approaches modeled with regularized logistic regression: bag-of-words (BOW), rule-based, and a hybrid approach. From a prospective cohort of 150 patients hospitalized with serious illness over 2018 to 2020, we collected 4391 inpatient EHR notes; 99 (2.3%) contained documented goals-of-care discussions. We used leave-one-out cross-validation to estimate performance by comparing pooled NLP predictions to human abstractors with receiver-operating-characteristic (ROC) and precision-recall (PR) analyses. We qualitatively examined a purposive sample of 70 NLP-misclassified notes using content analysis to identify linguistic features that allowed us to generate hypotheses underpinning misclassification. RESULTS All three modeling approaches discriminated between notes with and without goals-of-care discussions (AUCROC: BOW, 0.907; rule-based, 0.948; hybrid, 0.965). Precision and recall were only moderate (precision at 70% recall: BOW, 16.2%; rule-based, 50.4%; hybrid, 49.3%; AUCPR: BOW, 0.505; rule-based, 0.579; hybrid, 0.599). Qualitative analysis revealed patterns underlying performance differences between BOW and rule-based approaches. CONCLUSION NLP holds promise for identifying EHR-documented goals-of-care discussions. However, the rarity of goals-of-care content in EHR data limits performance. Our findings highlight opportunities to optimize NLP modeling approaches, and support further exploration of different NLP approaches to identify goals-of-care discussions.
Collapse
Affiliation(s)
- Alison M Uyeda
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA
| | - J Randall Curtis
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA; Department of Biobehavioral Nursing and Health Informatics (J.R.C., J.S., W.B.L.), University of Washington, Seattle, WA.
| | - Ruth A Engelberg
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| | - Lyndia C Brumback
- Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Department of Biostatistics (L.C.B.), University of Washington, Seattle, WA
| | - Yue Guo
- Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Department of Biomedical Informatics and Medical Education (Y.G., W.B.L., T.C.), University of Washington, Seattle, WA
| | - James Sibley
- Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Department of Biobehavioral Nursing and Health Informatics (J.R.C., J.S., W.B.L.), University of Washington, Seattle, WA
| | - William B Lober
- Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Department of Biomedical Informatics and Medical Education (Y.G., W.B.L., T.C.), University of Washington, Seattle, WA; Department of Biobehavioral Nursing and Health Informatics (J.R.C., J.S., W.B.L.), University of Washington, Seattle, WA
| | - Trevor Cohen
- Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Department of Biomedical Informatics and Medical Education (Y.G., W.B.L., T.C.), University of Washington, Seattle, WA
| | - Janaki Torrence
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| | - Joanna Heywood
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| | - Sudiptho R Paul
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| | - Erin K Kross
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| | - Robert Y Lee
- Department of Medicine (A.M.U., J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Cambia Palliative Care Center of Excellence at UW Medicine (A.M.U., J.R.C., R.A.E., L.C.B., Y.G., J.S., W.B.L., T.C., J.T., J.H., S.R.P., E.K.K., R.Y.L.), University of Washington, Seattle, WA; Division of Pulmonary, Critical Care, and Sleep Medicine (J.R.C., R.A.E., J.T., J.H., S.R.P., E.K.K., R.Y.L.), Department of Medicine, Harborview Medical Center, University of Washington, Seattle, WA
| |
Collapse
|
100
|
Kaplar A, Stošović M, Kaplar A, Brković V, Naumović R, Kovačević A. Evaluation of clinical named entity recognition methods for Serbian electronic health records. Int J Med Inform 2022; 164:104805. [PMID: 35653828 DOI: 10.1016/j.ijmedinf.2022.104805] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 05/06/2022] [Accepted: 05/22/2022] [Indexed: 11/25/2022]
Abstract
BACKGROUND AND OBJECTIVES The importance of clinical natural language processing (NLP) has increased with the adoption of electronic health records (EHRs). One of the critical tasks in clinical NLP is named entity recognition (NER). Clinical NER in the Serbian language is a severely under-researched area. The few approaches that have been proposed so far are based on rules or machine-learning models with hand-crafted features, while current state-of-the-art models have not been explored. The objective of this paper is to assess the performance of state-of-the-art NER methods on clinical narratives in the Serbian language. MATERIALS AND METHODS We designed an experimental setup for a comprehensive evaluation of state-of-the-art NER models. The gold standard corpus we used for the evaluation is comprised of discharge summaries from the Clinic for Nephrology at the University Clinical Center of Serbia. The following models were evaluated: conditional random fields (CRF), multilingual transformers (BERT Multilingual and XLM RoBERTa), and long short-term memory (LSTM) recurrent neural networks, and their ensembles. In addition, we investigated the necessity of the pretraining task of transformer based models and the use of pretrained word embeddings with LSTM model. RESULTS Our results show that individually CRF had the best precision, the pretrained BERT Multilingual model had the best recall values, and the LSTM model had the best F1 score. The best performance was achieved by combining the existing models in a majority voting ensemble with an F1 score of 0.892. The presented results are similar to the inter annotator agreement on our gold standard corpus and are comparable to existing state-of-the-art results for clinical NER reported in literature. CONCLUSION Existing state-of-the-art models can provide viable results for clinical named entity recognition when applied to languages with the complexity of the Serbian language without major modifications.
Collapse
Affiliation(s)
- Aleksandar Kaplar
- Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia
| | - Milan Stošović
- Clinic of Nephrology, University Clinical Center of Serbia, Belgrade, Serbia
| | - Aleksandra Kaplar
- Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia
| | - Voin Brković
- Clinic of Nephrology, University Clinical Center of Serbia, Belgrade, Serbia
| | - Radomir Naumović
- Clinic of Nephrology, University Clinical Center of Serbia, Belgrade, Serbia
| | | |
Collapse
|