1
|
Syyrilä T, Koskiniemi S, Manias E, Härkänen M. Taxonomy development methods regarding patient safety in health sciences - A systematic review. Int J Med Inform 2024; 187:105438. [PMID: 38579660 DOI: 10.1016/j.ijmedinf.2024.105438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Revised: 03/01/2024] [Accepted: 03/25/2024] [Indexed: 04/07/2024]
Abstract
BACKGROUND Taxonomies are needed for automated analysis of clinical data in healthcare. Few reviews of the taxonomy development methods used in health sciences are found. This systematic review aimed to describe the scope of the available taxonomies relative to patient safety, the methods used for taxonomy development, and the strengths and limitations of the methods. The purpose of this systematic review is to guide future taxonomy development projects. METHODS The CINAHL, PubMed, Scopus, and Web of Science databases were searched for studies from January 2012 to April 25, 2023. Two authors selected the studies using inclusion and exclusion criteria and critical appraisal checklists. The data were analysed inductively, and the results were reported narratively. RESULTS The studies (n = 13) across healthcare concerned mainly taxonomies of adverse events and medication safety but little for specialised fields and information technology. Critical appraisal indicated inadequate reporting of the used taxonomy development methods. Ten phases of taxonomy development were identified: (1) defining purpose and (2) the theory base for development, (3) relevant data sources' identification, (4) main terms' identification and definitions, (5) items' coding and pooling, (6) reliability and validity evaluation of coding and/or codes, (7) development of a hierarchical structure, (8) testing the structure, (9) piloting the taxonomy and (10) reporting application and validation of the final taxonomy. Seventeen statistical tests and seven software systems were utilised, but automated data extraction methods were used rarely. Multimethod and multi-stakeholder approach, code- and hierarchy testing and piloting were strengths and time consumption and small samples in testing limitations. CONCLUSION New taxonomies are needed on diverse specialities and information technology related to patient safety. Structured method is needed for taxonomy development, reporting and appraisal to strengthen taxonomies' quality. A new guide was proposed for taxonomy development, for which testing is required. Prospero registration number CRD42023411022.
Collapse
Affiliation(s)
- Tiina Syyrilä
- Department of Nursing Science, University of Eastern Finland, Finland.
| | - Saija Koskiniemi
- Department of Nursing Science, University of Eastern Finland, Finland
| | | | - Marja Härkänen
- Department of Nursing Science, University of Eastern Finland, Finland; Research Centre for Nursing Science and Social and Health Management, Kuopio University Hospital, Wellbeing Services County of North Savo, Finland
| |
Collapse
|
2
|
Abedian Kalkhoran H, Zwaveling J, van Hunsel F, Kant A. An innovative method to strengthen evidence for potential drug safety signals using Electronic Health Records. J Med Syst 2024; 48:51. [PMID: 38753223 PMCID: PMC11098892 DOI: 10.1007/s10916-024-02070-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Accepted: 04/25/2024] [Indexed: 05/19/2024]
Abstract
Reports from spontaneous reporting systems (SRS) are hypothesis generating. Additional evidence such as more reports is required to determine whether the generated drug-event associations are in fact safety signals. However, underreporting of adverse drug reactions (ADRs) delays signal detection. Through the use of natural language processing, different sources of real-world data can be used to proactively collect additional evidence for potential safety signals. This study aims to explore the feasibility of using Electronic Health Records (EHRs) to identify additional cases based on initial indications from spontaneous ADR reports, with the goal of strengthening the evidence base for potential safety signals. For two confirmed and two potential signals generated by the SRS of the Netherlands Pharmacovigilance Centre Lareb, targeted searches in the EHR of the Leiden University Medical Centre were performed using a text-mining based tool, CTcue. The search for additional cases was done by constructing and running queries in the structured and free-text fields of the EHRs. We identified at least five additional cases for the confirmed signals and one additional case for each potential safety signal. The majority of the identified cases for the confirmed signals were documented in the EHRs before signal detection by the Dutch Medicines Evaluation Board. The identified cases for the potential signals were reported to Lareb as further evidence for signal detection. Our findings highlight the feasibility of performing targeted searches in the EHR based on an underlying hypothesis to provide further evidence for signal generation.
Collapse
Affiliation(s)
- H Abedian Kalkhoran
- Department of Clinical Pharmacology and Toxicology, Leiden University Medical Centre, Leiden, the Netherlands.
- Department of Pharmacy, Haga Teaching Hospital, The Hague, the Netherlands.
| | - J Zwaveling
- Department of Clinical Pharmacology and Toxicology, Leiden University Medical Centre, Leiden, the Netherlands
| | - F van Hunsel
- The Netherlands Pharmacovigilance Centre Lareb, 's-Hertogenbosch, the Netherlands
| | - A Kant
- Department of Clinical Pharmacology and Toxicology, Leiden University Medical Centre, Leiden, the Netherlands
- The Netherlands Pharmacovigilance Centre Lareb, 's-Hertogenbosch, the Netherlands
| |
Collapse
|
3
|
Modi S, Kasmiran KA, Mohd Sharef N, Sharum MY. Extracting adverse drug events from clinical Notes: A systematic review of approaches used. J Biomed Inform 2024; 151:104603. [PMID: 38331081 DOI: 10.1016/j.jbi.2024.104603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/10/2024]
Abstract
BACKGROUND An adverse drug event (ADE) is any unfavorable effect that occurs due to the use of a drug. Extracting ADEs from unstructured clinical notes is essential to biomedical text extraction research because it helps with pharmacovigilance and patient medication studies. OBJECTIVE From the considerable amount of clinical narrative text, natural language processing (NLP) researchers have developed methods for extracting ADEs and their related attributes. This work presents a systematic review of current methods. METHODOLOGY Two biomedical databases have been searched from June 2022 until December 2023 for relevant publications regarding this review, namely the databases PubMed and Medline. Similarly, we searched the multi-disciplinary databases IEEE Xplore, Scopus, ScienceDirect, and the ACL Anthology. We adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement guidelines and recommendations for reporting systematic reviews in conducting this review. Initially, we obtained 5,537 articles from the search results from the various databases between 2015 and 2023. Based on predefined inclusion and exclusion criteria for article selection, 100 publications have undergone full-text review, of which we consider 82 for our analysis. RESULTS We determined the general pattern for extracting ADEs from clinical notes, with named entity recognition (NER) and relation extraction (RE) being the dual tasks considered. Researchers that tackled both NER and RE simultaneously have approached ADE extraction as a "pipeline extraction" problem (n = 22), as a "joint task extraction" problem (n = 7), and as a "multi-task learning" problem (n = 6), while others have tackled only NER (n = 27) or RE (n = 20). We further grouped the reviews based on the approaches for data extraction, namely rule-based (n = 8), machine learning (n = 11), deep learning (n = 32), comparison of two or more approaches (n = 11), hybrid (n = 12) and large language models (n = 8). The most used datasets are MADE 1.0, TAC 2017 and n2c2 2018. CONCLUSION Extracting ADEs is crucial, especially for pharmacovigilance studies and patient medications. This survey showcases advances in ADE extraction research, approaches, datasets, and state-of-the-art performance in them. Challenges and future research directions are highlighted. We hope this review will guide researchers in gaining background knowledge and developing more innovative ways to address the challenges.
Collapse
Affiliation(s)
- Salisu Modi
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia; Department of Computer Science, Sokoto State University, Sokoto, Nigeria.
| | - Khairul Azhar Kasmiran
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Nurfadhlina Mohd Sharef
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Mohd Yunus Sharum
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| |
Collapse
|
4
|
Deimazar G, Sheikhtaheri A. Machine learning models to detect and predict patient safety events using electronic health records: A systematic review. Int J Med Inform 2023; 180:105246. [PMID: 37837710 DOI: 10.1016/j.ijmedinf.2023.105246] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 10/02/2023] [Accepted: 10/08/2023] [Indexed: 10/16/2023]
Abstract
INTRODUCTION Identifying patient safety events using electronic health records (EHRs) and automated machine learning-based detection methods can help improve the efficiency and quality of healthcare service provision. OBJECTIVE This study aimed to systematically review machine learning-based methods and techniques, as well as their results for patient safety event management using EHRs. METHODS We reviewed the studies that focused on machine learning techniques, including automatic prediction and detection of patient safety events and medical errors through EHR analysis to manage patient safety events. The data were collected by searching Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE Xplore databases. RESULTS After screening, 41 papers were reviewed. Support vector machine (SVM), random forest, conditional random field (CRF), and bidirectional long short-term memory with conditional random field (BiLSTM-CRF) algorithms were mostly applied to predict, identify, and classify patient safety events using EHRs; however, they had different performances. BiLSTM-CRF was employed in most of the studies to extract and identify concepts, e.g., adverse drug events (ADEs) and adverse drug reactions (ADRs), as well as relationships between drug and severity, drug and ADEs, drug and ADRs. Recurrent neural networks (RNN) and BiLSTM-CRF had the best results in detecting ADEs compared to other patient safety events. Linear classifiers and Naive Bayes (NB) had the highest performance for ADR detection. Logistic regression had the best results in detecting surgical site infections. According to the findings, the quality of articles has non-significantly improved in recent years, but they had low average scores. CONCLUSIONS Machine learning can be useful in automatic detection and prediction of patient safety events. However, most of these algorithms have not yet been externally validated or prospectively tested. Therefore, further studies are required to improve the performance of these automated systems.
Collapse
Affiliation(s)
- Ghasem Deimazar
- Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
| | - Abbas Sheikhtaheri
- Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
5
|
Wang L, Song Y, Wang H, Zhang X, Wang M, He J, Li S, Zhang L, Li K, Cao L. Advances of Artificial Intelligence in Anti-Cancer Drug Design: A Review of the Past Decade. Pharmaceuticals (Basel) 2023; 16:253. [PMID: 37259400 PMCID: PMC9963982 DOI: 10.3390/ph16020253] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 01/25/2023] [Accepted: 02/06/2023] [Indexed: 10/13/2023] Open
Abstract
Anti-cancer drug design has been acknowledged as a complicated, expensive, time-consuming, and challenging task. How to reduce the research costs and speed up the development process of anti-cancer drug designs has become a challenging and urgent question for the pharmaceutical industry. Computer-aided drug design methods have played a major role in the development of cancer treatments for over three decades. Recently, artificial intelligence has emerged as a powerful and promising technology for faster, cheaper, and more effective anti-cancer drug designs. This study is a narrative review that reviews a wide range of applications of artificial intelligence-based methods in anti-cancer drug design. We further clarify the fundamental principles of these methods, along with their advantages and disadvantages. Furthermore, we collate a large number of databases, including the omics database, the epigenomics database, the chemical compound database, and drug databases. Other researchers can consider them and adapt them to their own requirements.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | - Kang Li
- Department of Biostatistics, School of Public Health, Harbin Medical University, Harbin 150081, China
| | - Lei Cao
- Department of Biostatistics, School of Public Health, Harbin Medical University, Harbin 150081, China
| |
Collapse
|
6
|
Adverse drug event detection using natural language processing: A scoping review of supervised learning methods. PLoS One 2023; 18:e0279842. [PMID: 36595517 PMCID: PMC9810201 DOI: 10.1371/journal.pone.0279842] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 12/15/2022] [Indexed: 01/04/2023] Open
Abstract
To reduce adverse drug events (ADEs), hospitals need a system to support them in monitoring ADE occurrence routinely, rapidly, and at scale. Natural language processing (NLP), a computerized approach to analyze text data, has shown promising results for the purpose of ADE detection in the context of pharmacovigilance. However, a detailed qualitative assessment and critical appraisal of NLP methods for ADE detection in the context of ADE monitoring in hospitals is lacking. Therefore, we have conducted a scoping review to close this knowledge gap, and to provide directions for future research and practice. We included articles where NLP was applied to detect ADEs in clinical narratives within electronic health records of inpatients. Quantitative and qualitative data items relating to NLP methods were extracted and critically appraised. Out of 1,065 articles screened for eligibility, 29 articles met the inclusion criteria. Most frequent tasks included named entity recognition (n = 17; 58.6%) and relation extraction/classification (n = 15; 51.7%). Clinical involvement was reported in nine studies (31%). Multiple NLP modelling approaches seem suitable, with Long Short Term Memory and Conditional Random Field methods most commonly used. Although reported overall performance of the systems was high, it provides an inflated impression given a steep drop in performance when predicting the ADE entity or ADE relation class. When annotating corpora, treating an ADE as a relation between a drug and non-drug entity seems the best practice. Future research should focus on semi-automated methods to reduce the manual annotation effort, and examine implementation of the NLP methods in practice.
Collapse
|
7
|
Timilsina M, Tandan M, Nováček V. Machine learning approaches for predicting the onset time of the adverse drug events in oncology. MACHINE LEARNING WITH APPLICATIONS 2022. [DOI: 10.1016/j.mlwa.2022.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
|
8
|
Guo S, Yang W, Han L, Song X, Wang G. A multi-layer soft lattice based model for Chinese clinical named entity recognition. BMC Med Inform Decis Mak 2022; 22:201. [PMID: 35908055 PMCID: PMC9338545 DOI: 10.1186/s12911-022-01924-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Objective Named entity recognition (NER) is a key and fundamental part of many medical and clinical tasks, including the establishment of a medical knowledge graph, decision-making support, and question answering systems. When extracting entities from electronic health records (EHRs), NER models mostly apply long short-term memory (LSTM) and have surprising performance in clinical NER. However, increasing the depth of the network is often required by these LSTM-based models to capture long-distance dependencies. Therefore, these LSTM-based models that have achieved high accuracy generally require long training times and extensive training data, which has obstructed the adoption of LSTM-based models in clinical scenarios with limited training time. Method Inspired by Transformer, we combine Transformer with Soft Term Position Lattice to form soft lattice structure Transformer, which models long-distance dependencies similarly to LSTM. Our model consists of four components: the WordPiece module, the BERT module, the soft lattice structure Transformer module, and the CRF module. Result Our experiments demonstrated that this approach increased the F1 by 1–5% in the CCKS NER task compared to other models based on LSTM with CRF and consumed less training time. Additional evaluations showed that lattice structure transformer shows good performance for recognizing long medical terms, abbreviations, and numbers. The proposed model achieve 91.6% f-measure in recognizing long medical terms and 90.36% f-measure in abbreviations, and numbers. Conclusions By using soft lattice structure Transformer, the method proposed in this paper captured Chinese words to lattice information, making our model suitable for Chinese clinical medical records. Transformers with Mutilayer soft lattice Chinese word construction can capture potential interactions between Chinese characters and words.
Collapse
Affiliation(s)
- Shuli Guo
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Wentao Yang
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Lina Han
- Department of Cardiology, The Second Medical Center, National Clinical Research Center for Geriatric Diseases, Chinese PLA General Hospital, Beijing, China.
| | - Xiaowei Song
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Guowei Wang
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
9
|
Utilizing Deep Learning for Detecting Adverse Drug Events in Structured and Unstructured Regulatory Drug Data Sets. Pharmaceut Med 2022; 36:307-317. [DOI: 10.1007/s40290-022-00434-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/01/2022] [Indexed: 10/16/2022]
|
10
|
Kim HR, Sung M, Park JA, Jeong K, Kim HH, Lee S, Park YR. Analyzing adverse drug reaction using statistical and machine learning methods: A systematic review. Medicine (Baltimore) 2022; 101:e29387. [PMID: 35758373 PMCID: PMC9276413 DOI: 10.1097/md.0000000000029387] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Accepted: 04/12/2022] [Indexed: 11/26/2022] Open
Abstract
BACKGROUND Adverse drug reactions (ADRs) are unintended negative drug-induced responses. Determining the association between drugs and ADRs is crucial, and several methods have been proposed to demonstrate this association. This systematic review aimed to examine the analytical tools by considering original articles that utilized statistical and machine learning methods for detecting ADRs. METHODS A systematic literature review was conducted based on articles published between 2015 and 2020. The keywords used were statistical, machine learning, and deep learning methods for detecting ADR signals. The study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA) guidelines. RESULTS We reviewed 72 articles, of which 51 and 21 addressed statistical and machine learning methods, respectively. Electronic medical record (EMR) data were exclusively analyzed using the regression method. For FDA Adverse Event Reporting System (FAERS) data, components of the disproportionality method were preferable. DrugBank was the most used database for machine learning. Other methods accounted for the highest and supervised methods accounted for the second highest. CONCLUSIONS Using the 72 main articles, this review provides guidelines on which databases are frequently utilized and which analysis methods can be connected. For statistical analysis, >90% of the cases were analyzed by disproportionate or regression analysis with each spontaneous reporting system (SRS) data or electronic medical record (EMR) data; for machine learning research, however, there was a strong tendency to analyze various data combinations. Only half of the DrugBank database was occupied, and the k-nearest neighbor method accounted for the greatest proportion.
Collapse
Affiliation(s)
- Hae Reong Kim
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| | - MinDong Sung
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| | - Ji Ae Park
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| | - Kyeongseob Jeong
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| | - Ho Heon Kim
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| | - Suehyun Lee
- Department of Biomedical Informatics, Konyang University College of Medicine, Daejeon, South Korea
| | - Yu Rang Park
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea
| |
Collapse
|
11
|
Mueller B, Kinoshita T, Peebles A, Graber MA, Lee S. Artificial intelligence and machine learning in emergency medicine: a narrative review. Acute Med Surg 2022; 9:e740. [PMID: 35251669 PMCID: PMC8887797 DOI: 10.1002/ams2.740] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Revised: 01/26/2022] [Accepted: 02/06/2022] [Indexed: 12/20/2022] Open
Abstract
AIM The emergence and evolution of artificial intelligence (AI) has generated increasing interest in machine learning applications for health care. Specifically, researchers are grasping the potential of machine learning solutions to enhance the quality of care in emergency medicine. METHODS We undertook a narrative review of published works on machine learning applications in emergency medicine and provide a synopsis of recent developments. RESULTS This review describes fundamental concepts of machine learning and presents clinical applications for triage, risk stratification specific to disease, medical imaging, and emergency department operations. Additionally, we consider how machine learning models could contribute to the improvement of causal inference in medicine, and to conclude, we discuss barriers to safe implementation of AI. CONCLUSION We intend that this review serves as an introduction to AI and machine learning in emergency medicine.
Collapse
Affiliation(s)
- Brianna Mueller
- Department of Business Analytics The University of Iowa Tippie College of Business Iowa City Iowa USA
| | | | - Alexander Peebles
- Department of Emergency Medicine The University of Iowa Carver College of Medicine Iowa City Iowa USA
| | - Mark A Graber
- Department of Emergency Medicine The University of Iowa Carver College of Medicine Iowa City Iowa USA
| | - Sangil Lee
- Department of Emergency Medicine The University of Iowa Carver College of Medicine Iowa City Iowa USA
| |
Collapse
|
12
|
Liang L, Hu J, Sun G, Hong N, Wu G, He Y, Li Y, Hao T, Liu L, Gong M. Artificial Intelligence-Based Pharmacovigilance in the Setting of Limited Resources. Drug Saf 2022; 45:511-519. [PMID: 35579814 PMCID: PMC9112260 DOI: 10.1007/s40264-022-01170-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/27/2022] [Indexed: 01/28/2023]
Abstract
With the rapid development of artificial intelligence (AI) technologies, and the large amount of pharmacovigilance-related data stored in an electronic manner, data-driven automatic methods need to be urgently applied to all aspects of pharmacovigilance to assist healthcare professionals. However, the quantity and quality of data directly affect the performance of AI, and there are particular challenges to implementing AI in limited-resource settings. Analyzing challenges and solutions for AI-based pharmacovigilance in resource-limited settings can improve pharmacovigilance frameworks and capabilities in these settings. In this review, we summarize the challenges into four categories: establishing a database for an AI-based pharmacovigilance system, lack of human resources, weak AI technology and insufficient government support. This study also discusses possible solutions and future perspectives on AI-based pharmacovigilance in resource-limited settings.
Collapse
Affiliation(s)
- Likeng Liang
- School of Computer Science, South China Normal University, Guangzhou, China
| | - Jifa Hu
- The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Gang Sun
- Key Laboratory of Oncology of Xinjiang Uyghur Autonomous Region, The Affiliated Cancer Hospital of Xinjiang Medical University, Ürümqi, China
| | - Na Hong
- Digital Health China Technologies Co., Ltd., Beijing, China
| | - Ge Wu
- Digital Health China Technologies Co., Ltd., Beijing, China
| | - Yuejun He
- Digital Health China Technologies Co., Ltd., Beijing, China
| | - Yong Li
- School of Computer Science, South China Normal University, Guangzhou, China
| | - Tianyong Hao
- School of Computer Science, South China Normal University, Guangzhou, China
| | - Li Liu
- Institute of Health Management, Southern Medical University, Guangzhou, China
| | - Mengchun Gong
- Institute of Health Management, Southern Medical University, Guangzhou, China
| |
Collapse
|
13
|
Edrees H, Song W, Syrowatka A, Simona A, Amato MG, Bates DW. Intelligent Telehealth in Pharmacovigilance: A Future Perspective. Drug Saf 2022; 45:449-458. [PMID: 35579810 PMCID: PMC9112241 DOI: 10.1007/s40264-022-01172-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/02/2022] [Indexed: 01/28/2023]
Abstract
Pharmacovigilance improves patient safety by detecting and preventing adverse drug events. However, challenges exist that limit adverse drug event detection, resulting in many adverse drug events being underreported or inaccurately reported. One challenge includes having access to large data sets from various sources including electronic health records and wearable medical devices. Artificial intelligence, including machine learning methods, such as natural language processing and deep learning, can detect and extract information about adverse drug events, thus automating the pharmacovigilance process and improving the surveillance of known and documented adverse drug events. In addition, with the increased demand for telehealth services, for managing both acute and chronic diseases, artificial intelligence methods can play a role in detecting and preventing adverse drug events. In this review, we discuss two use cases of how artificial intelligence methods may be useful to improve the quality of pharmacovigilance and the role of artificial intelligence in telehealth practices.
Collapse
Affiliation(s)
- Heba Edrees
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA ,Department of Pharmacy Practice, MCPHS University, Boston, MA USA ,Harvard Medical School, 1620 Tremont St., 3rd Floor, Boston, MA 02120 USA
| | - Wenyu Song
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA ,Harvard Medical School, 1620 Tremont St., 3rd Floor, Boston, MA 02120 USA
| | - Ania Syrowatka
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA ,Harvard Medical School, 1620 Tremont St., 3rd Floor, Boston, MA 02120 USA
| | - Aurélien Simona
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA ,Harvard Medical School, 1620 Tremont St., 3rd Floor, Boston, MA 02120 USA
| | - Mary G. Amato
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA
| | - David W. Bates
- Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA USA ,Harvard Medical School, 1620 Tremont St., 3rd Floor, Boston, MA 02120 USA ,Department of Health Policy and Management, Harvard School of Public Health, Boston, MA USA
| |
Collapse
|
14
|
de Oliveira JM, da Costa CA, Antunes RS. Data structuring of electronic health records: a systematic review. HEALTH AND TECHNOLOGY 2021. [DOI: 10.1007/s12553-021-00607-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
|
15
|
Bright RA, Rankin SK, Dowdy K, Blok SV, Bright SJ, Palmer LAM. Finding Potential Adverse Events in the Unstructured Text of Electronic Health Care Records: Development of the Shakespeare Method. JMIRX MED 2021; 2:e27017. [PMID: 37725533 PMCID: PMC10414364 DOI: 10.2196/27017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/07/2021] [Revised: 04/03/2021] [Accepted: 05/01/2021] [Indexed: 09/21/2023]
Abstract
BACKGROUND Big data tools provide opportunities to monitor adverse events (patient harm associated with medical care) (AEs) in the unstructured text of electronic health care records (EHRs). Writers may explicitly state an apparent association between treatment and adverse outcome ("attributed") or state the simple treatment and outcome without an association ("unattributed"). Many methods for finding AEs in text rely on predefining possible AEs before searching for prespecified words and phrases or manual labeling (standardization) by investigators. We developed a method to identify possible AEs, even if unknown or unattributed, without any prespecifications or standardization of notes. Our method was inspired by word-frequency analysis methods used to uncover the true authorship of disputed works credited to William Shakespeare. We chose two use cases, "transfusion" and "time-based." Transfusion was chosen because new transfusion AE types were becoming recognized during the study data period; therefore, we anticipated an opportunity to find unattributed potential AEs (PAEs) in the notes. With the time-based case, we wanted to simulate near real-time surveillance. We chose time periods in the hope of detecting PAEs due to contaminated heparin from mid-2007 to mid-2008 that were announced in early 2008. We hypothesized that the prevalence of contaminated heparin may have been widespread enough to manifest in EHRs through symptoms related to heparin AEs, independent of clinicians' documentation of attributed AEs. OBJECTIVE We aimed to develop a new method to identify attributed and unattributed PAEs using the unstructured text of EHRs. METHODS We used EHRs for adult critical care admissions at a major teaching hospital (2001-2012). For each case, we formed a group of interest and a comparison group. We concatenated the text notes for each admission into one document sorted by date, and deleted replicate sentences and lists. We identified statistically significant words in the group of interest versus the comparison group. Documents in the group of interest were filtered to those words, followed by topic modeling on the filtered documents to produce topics. For each topic, the three documents with the maximum topic scores were manually reviewed to identify PAEs. RESULTS Topics centered around medical conditions that were unique to or more common in the group of interest, including PAEs. In each use case, most PAEs were unattributed in the notes. Among the transfusion PAEs was unattributed evidence of transfusion-associated cardiac overload and transfusion-related acute lung injury. Some of the PAEs from mid-2007 to mid-2008 were increased unattributed events consistent with AEs related to heparin contamination. CONCLUSIONS The Shakespeare method could be a useful supplement to AE reporting and surveillance of structured EHR data. Future improvements should include automation of the manual review process.
Collapse
Affiliation(s)
- Roselie A Bright
- US Food and Drug Administration, Silver Spring, MD, United States
| | | | | | | | - Susan J Bright
- US Food and Drug Administration, Rockville, MD, United States
| | | |
Collapse
|
16
|
Cooper A, Horrocks J, Goodday S, Keown-Stoneman C, Duffy A. Predicting the risk and timing of major mood disorder in offspring of bipolar parents: exploring the utility of a neural network approach. Int J Bipolar Disord 2021; 9:22. [PMID: 34195908 PMCID: PMC8245610 DOI: 10.1186/s40345-021-00228-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 05/27/2021] [Indexed: 12/23/2022] Open
Abstract
Background Bipolar disorder onset peaks over early adulthood and confirmed family history is a robust risk factor. However, penetrance within families varies and most children of bipolar parents will not develop the illness. Individualized risk prediction would be helpful for identifying those young people most at risk and to inform targeted intervention. Using prospectively collected data from the Canadian Flourish High-risk Offspring cohort study available in routine practice, we explored the use of a neural network, known as the Partial Logistic Artificial Neural Network (PLANN) to predict the time to diagnosis of major mood disorders in 1, 3 and 5-year intervals. Results Overall, for predictive performance, PLANN outperformed the more traditional discrete survival model for 3-year and 5-year predictions. PLANN was better able to discriminate or rank individuals based on their risk of developing a major mood disorder, better able to predict the probability of developing a major mood disorder and better able to identify individuals who would be diagnosed in future time intervals. The average AUC achieved by PLANN for 5-year prediction was 0.74, which indicates good discrimination. Conclusions This evaluation of PLANN is a useful step in the investigation of using neural networks as tools in the prediction of mood disorders in at-risk individuals and the potential that neural networks have in this field. Future research is needed to replicate these findings in a separate high-risk offspring sample. Supplementary Information The online version contains supplementary material available at 10.1186/s40345-021-00228-2.
Collapse
Affiliation(s)
- Alysha Cooper
- Department of Mathematics and Statistics, University of Guelph, Guelph, ON, Canada
| | - Julie Horrocks
- Department of Mathematics and Statistics, University of Guelph, Guelph, ON, Canada
| | - Sarah Goodday
- Department of Psychiatry, University of Oxford, Oxford, UK.,4YouandMe, Seattle, USA
| | | | - Anne Duffy
- Department of Psychiatry, University of Oxford, Oxford, UK. .,Department of Psychiatry, Queen's University, Kingston, ON, Canada.
| |
Collapse
|
17
|
Yang X, Bian J, Fang R, Bjarnadottir RI, Hogan WR, Wu Y. Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting. J Am Med Inform Assoc 2021; 27:65-72. [PMID: 31504605 DOI: 10.1093/jamia/ocz144] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2019] [Revised: 05/30/2019] [Accepted: 07/22/2019] [Indexed: 01/19/2023] Open
Abstract
OBJECTIVE To develop a natural language processing system that identifies relations of medications with adverse drug events from clinical narratives. This project is part of the 2018 n2c2 challenge. MATERIALS AND METHODS We developed a novel clinical named entity recognition method based on an recurrent convolutional neural network and compared it to a recurrent neural network implemented using the long-short term memory architecture, explored methods to integrate medical knowledge as embedding layers in neural networks, and investigated 3 machine learning models, including support vector machines, random forests and gradient boosting for relation classification. The performance of our system was evaluated using annotated data and scripts provided by the 2018 n2c2 organizers. RESULTS Our system was among the top ranked. Our best model submitted during this challenge (based on recurrent neural networks and support vector machines) achieved lenient F1 scores of 0.9287 for concept extraction (ranked third), 0.9459 for relation classification (ranked fourth), and 0.8778 for the end-to-end relation extraction (ranked second). We developed a novel named entity recognition model based on a recurrent convolutional neural network and further investigated gradient boosting for relation classification. The new methods improved the lenient F1 scores of the 3 subtasks to 0.9292, 0.9633, and 0.8880, respectively, which are comparable to the best performance reported in this challenge. CONCLUSION This study demonstrated the feasibility of using machine learning methods to extract the relations of medications with adverse drug events from clinical narratives.
Collapse
Affiliation(s)
- Xi Yang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Ruogu Fang
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, Florida, USA
| | - Ragnhildur I Bjarnadottir
- Department of Family, Community and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, USA
| | - William R Hogan
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Yonghui Wu
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| |
Collapse
|
18
|
Wu S, Roberts K, Datta S, Du J, Ji Z, Si Y, Soni S, Wang Q, Wei Q, Xiang Y, Zhao B, Xu H. Deep learning in clinical natural language processing: a methodical review. J Am Med Inform Assoc 2021; 27:457-470. [PMID: 31794016 DOI: 10.1093/jamia/ocz200] [Citation(s) in RCA: 167] [Impact Index Per Article: 55.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Revised: 10/15/2019] [Accepted: 11/09/2019] [Indexed: 02/07/2023] Open
Abstract
OBJECTIVE This article methodically reviews the literature on deep learning (DL) for natural language processing (NLP) in the clinical domain, providing quantitative analysis to answer 3 research questions concerning methods, scope, and context of current research. MATERIALS AND METHODS We searched MEDLINE, EMBASE, Scopus, the Association for Computing Machinery Digital Library, and the Association for Computational Linguistics Anthology for articles using DL-based approaches to NLP problems in electronic health records. After screening 1,737 articles, we collected data on 25 variables across 212 papers. RESULTS DL in clinical NLP publications more than doubled each year, through 2018. Recurrent neural networks (60.8%) and word2vec embeddings (74.1%) were the most popular methods; the information extraction tasks of text classification, named entity recognition, and relation extraction were dominant (89.2%). However, there was a "long tail" of other methods and specific tasks. Most contributions were methodological variants or applications, but 20.8% were new methods of some kind. The earliest adopters were in the NLP community, but the medical informatics community was the most prolific. DISCUSSION Our analysis shows growing acceptance of deep learning as a baseline for NLP research, and of DL-based NLP in the medical community. A number of common associations were substantiated (eg, the preference of recurrent neural networks for sequence-labeling named entity recognition), while others were surprisingly nuanced (eg, the scarcity of French language clinical NLP with deep learning). CONCLUSION Deep learning has not yet fully penetrated clinical NLP and is growing rapidly. This review highlighted both the popular and unique trends in this active field.
Collapse
Affiliation(s)
- Stephen Wu
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Kirk Roberts
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Surabhi Datta
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Jingcheng Du
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Zongcheng Ji
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Yuqi Si
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Sarvesh Soni
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Qiong Wang
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Qiang Wei
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Yang Xiang
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Bo Zhao
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Hua Xu
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| |
Collapse
|
19
|
Space Physical Sensor Protection and Control System Based on Neural Network Prediction: Application in Princess Elizabeth Area of Antarctica. SENSORS 2020; 20:s20174662. [PMID: 32824950 PMCID: PMC7506621 DOI: 10.3390/s20174662] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/30/2020] [Revised: 08/07/2020] [Accepted: 08/11/2020] [Indexed: 11/17/2022]
Abstract
In the inland areas of Antarctica, the establishment of an unmanned automatic observation support system is an urgent problem and challenge. This article introduces the development and application of an unmanned control system suitable for inland Antarctica. The system is called RIOD (Remote Control, Image Acquisition, Operation Maintenance, and Document Management System) for short. At the beginning of this research project, a mathematical model of heat conduction in the surface observation chamber was established, and the control strategy was determined through mathematical relationships and field experiments. Based on the analysis of local meteorological data, various neural network models are compared, and the training model with the smallest error is used to predict the future ambient temperature. Moreover, the future temperature is substituted into the mathematical model of thermal conductivity to obtain the input value of the next input power, to formulate the operation strategy for the system. This method maintains the regular operation of the sensor while reducing energy consumption. The RIOD system has been deployed in the Tai-Shan camp in China’s Antarctic inland inspection route. The application results 4.5 months after deployment show that the RIOD system can maintain stable operation at lower temperatures. This technology solves the demand for unmanned high-altitude physical observation or astronomical observation stations in inland areas.
Collapse
|
20
|
Abstract
OBJECTIVES We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes-diseases and drugs (or medications)-and relations between them. METHODS For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. RESULTS In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. CONCLUSIONS The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.
Collapse
Affiliation(s)
- Udo Hahn
- Jena University Language & Information Engineering (JULIE) Lab, Friedrich-Schiller-Universität Jena, Jena, Germany
| | - Michel Oleynik
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria
| |
Collapse
|
21
|
Imai S, Takekuma Y, Kashiwagi H, Miyai T, Kobayashi M, Iseki K, Sugawara M. Validation of the usefulness of artificial neural networks for risk prediction of adverse drug reactions used for individual patients in clinical practice. PLoS One 2020; 15:e0236789. [PMID: 32726360 PMCID: PMC7390378 DOI: 10.1371/journal.pone.0236789] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2019] [Accepted: 07/14/2020] [Indexed: 11/18/2022] Open
Abstract
Artificial neural networks are the main tools for data mining and were inspired by the human brain and nervous system. Studies have demonstrated their usefulness in medicine. However, no studies have used artificial neural networks for the prediction of adverse drug reactions. We aimed to validate the usefulness of artificial neural networks for the prediction of adverse drug reactions and focused on vancomycin -induced nephrotoxicity. For constructing an artificial neural network, a multilayer perceptron algorithm was employed. A 10-fold cross validation method was adopted for evaluating the resultant artificial neural network. In total, 1141 patients who received vancomycin at Hokkaido University Hospital from November 2011 to February 2019 were enrolled. Among these patients, 179 (15.7%) developed vancomycin -induced nephrotoxicity. The top three risk factors of vancomycin -induced nephrotoxicity which are relatively important in the artificial neural networks were average vancomycin trough concentration ≥ 13.0 mg/L and concomitant use of piperacillin–tazobactam and vasopressor drugs. The predictive accuracy of the artificial neural network was 86.3% and that of the multiple logistic regression model (conventional statistical method) was 85.1%. Moreover, area under the receiver operating characteristic curve (AUROC) of the artificial neural network was 0.83. In the 10-fold cross-validation, the accuracy obtained was 86.0% and AUROC was 0.82. The artificial neural network model predicting the vancomycin -induced nephrotoxicity showed good predictive performance. This appears to be the first report of the usefulness of artificial neural networks for an adverse drug reactions risk prediction model.
Collapse
Affiliation(s)
- Shungo Imai
- Faculty of Pharmaceutical Sciences, Hokkaido University, Sapporo, Japan
- * E-mail:
| | - Yoh Takekuma
- Department of Pharmacy, Hokkaido University Hospital, Sapporo, Japan
| | - Hitoshi Kashiwagi
- Faculty of Pharmaceutical Sciences, Hokkaido University, Sapporo, Japan
| | - Takayuki Miyai
- Graduate School of Life Science, Hokkaido University, Sapporo, Japan
| | - Masaki Kobayashi
- Faculty of Pharmaceutical Sciences, Hokkaido University, Sapporo, Japan
| | - Ken Iseki
- Faculty of Pharmaceutical Sciences, Hokkaido University, Sapporo, Japan
| | - Mitsuru Sugawara
- Faculty of Pharmaceutical Sciences, Hokkaido University, Sapporo, Japan
- Department of Pharmacy, Hokkaido University Hospital, Sapporo, Japan
| |
Collapse
|
22
|
Dandala B, Joopudi V, Tsou CH, Liang JJ, Suryanarayanan P. Extraction of Information Related to Drug Safety Surveillance From Electronic Health Record Notes: Joint Modeling of Entities and Relations Using Knowledge-Aware Neural Attentive Models. JMIR Med Inform 2020; 8:e18417. [PMID: 32459650 PMCID: PMC7382020 DOI: 10.2196/18417] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 05/12/2020] [Accepted: 05/13/2020] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND An adverse drug event (ADE) is commonly defined as "an injury resulting from medical intervention related to a drug." Providing information related to ADEs and alerting caregivers at the point of care can reduce the risk of prescription and diagnostic errors and improve health outcomes. ADEs captured in structured data in electronic health records (EHRs) as either coded problems or allergies are often incomplete, leading to underreporting. Therefore, it is important to develop capabilities to process unstructured EHR data in the form of clinical notes, which contain a richer documentation of a patient's ADE. Several natural language processing (NLP) systems have been proposed to automatically extract information related to ADEs. However, the results from these systems showed that significant improvement is still required for the automatic extraction of ADEs from clinical notes. OBJECTIVE This study aims to improve the automatic extraction of ADEs and related information such as drugs, their attributes, and reason for administration from the clinical notes of patients. METHODS This research was conducted using discharge summaries from the Medical Information Mart for Intensive Care III (MIMIC-III) database obtained through the 2018 National NLP Clinical Challenges (n2c2) annotated with drugs, drug attributes (ie, strength, form, frequency, route, dosage, duration), ADEs, reasons, and relations between drugs and other entities. We developed a deep learning-based system for extracting these drug-centric concepts and relations simultaneously using a joint method enhanced with contextualized embeddings, a position-attention mechanism, and knowledge representations. The joint method generated different sentence representations for each drug, which were then used to extract related concepts and relations simultaneously. Contextualized representations trained on the MIMIC-III database were used to capture context-sensitive meanings of words. The position-attention mechanism amplified the benefits of the joint method by generating sentence representations that capture long-distance relations. Knowledge representations were obtained from graph embeddings created using the US Food and Drug Administration Adverse Event Reporting System database to improve relation extraction, especially when contextual clues were insufficient. RESULTS Our system achieved new state-of-the-art results on the n2c2 data set, with significant improvements in recognizing crucial drug-reason (F1=0.650 versus F1=0.579) and drug-ADE (F1=0.490 versus F1=0.476) relations. CONCLUSIONS This study presents a system for extracting drug-centric concepts and relations that outperformed current state-of-the-art results and shows that contextualized embeddings, position-attention mechanisms, and knowledge graph embeddings effectively improve deep learning-based concepts and relation extraction. This study demonstrates the potential for deep learning-based methods to help extract real-world evidence from unstructured patient data for drug safety surveillance.
Collapse
|
23
|
Nguyen VH, Sugiyama K, Kan MY, Halder K. Neural side effect discovery from user credibility and experience-assessed online health discussions. J Biomed Semantics 2020; 11:5. [PMID: 32641159 PMCID: PMC7341623 DOI: 10.1186/s13326-020-00221-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2019] [Accepted: 06/07/2020] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Health 2.0 allows patients and caregivers to conveniently seek medical information and advice via e-portals and online discussion forums, especially regarding potential drug side effects. Although online health communities are helpful platforms for obtaining non-professional opinions, they pose risks in communicating unreliable and insufficient information in terms of quality and quantity. Existing methods in extracting user-reported adverse drug reactions (ADRs) in online health forums are not only insufficiently accurate as they disregard user credibility and drug experience, but are also expensive as they rely on supervised ground truth annotation of individual statement. We propose a NEural ArchiTecture for Drug side effect prediction (NEAT), which is optimized on the task of drug side effect discovery based on a complete discussion while being attentive to user credibility and experience, thus, addressing the mentioned shortcomings. We train our neural model in a self-supervised fashion using ground truth drug side effects from mayoclinic.org. NEAT learns to assign each user a score that is descriptive of their credibility and highlights the critical textual segments of their post. RESULTS Experiments show that NEAT improves drug side effect discovery from online health discussion by 3.04% from user-credibility agnostic baselines, and by 9.94% from non-neural baselines in term of F1. Additionally, the latent credibility scores learned by the model correlate well with trustworthiness signals, such as the number of "thanks" received by other forum members, and improve credibility heuristics such as number of posts by 0.113 in term of Spearman's rank correlation coefficient. Experience-based self-supervised attention highlights critical phrases such as mentioned side effects, and enhances fully supervised ADR extraction models based on sequence labelling by 5.502% in terms of precision. CONCLUSIONS NEAT considers both user credibility and experience in online health forums, making feasible a self-supervised approach to side effect prediction for mentioned drugs. The derived user credibility and attention mechanism are transferable and improve downstream ADR extraction models. Our approach enhances automatic drug side effect discovery and fosters research in several domains including pharmacovigilance and clinical studies.
Collapse
Affiliation(s)
- Van-Hoang Nguyen
- School of Computing, National University of Singapore, 13 Computing Drive, Singapore, 117417 Singapore
| | - Kazunari Sugiyama
- School of Computing, National University of Singapore, 13 Computing Drive, Singapore, 117417 Singapore
| | - Min-Yen Kan
- School of Computing, National University of Singapore, 13 Computing Drive, Singapore, 117417 Singapore
| | - Kishaloy Halder
- School of Computing, National University of Singapore, 13 Computing Drive, Singapore, 117417 Singapore
| |
Collapse
|
24
|
Stergiopoulos S, Fehrle M, Caubel P, Tan L, Jebson L. Adverse Drug Reaction Case Safety Practices in Large Biopharmaceutical Organizations from 2007 to 2017: An Industry Survey. Pharmaceut Med 2020; 33:499-510. [PMID: 31933240 DOI: 10.1007/s40290-019-00307-x] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
INTRODUCTION Drug safety remains a top global public health concern. An increase in the number of data sources available has increased the complexity of pharmacovigilance operations, so the US FDA has created draft guidance focusing on optimizing drug safety data for well-characterized medicines. However, to date, no data demonstrating changes in reports have been presented. OBJECTIVES This study provided data assessing changes in individual case safety reports (ICSRs) and aggregate reports (ARs) for large biopharmaceutical companies from 2007 to 2017. This study also evaluated current trends on the use of advanced machine and deep learning in order to process all data captured for ICSRs as well as opinions from industry thought leaders on creating a sustainable case-processing operation. METHODOLOGY Using data captured from Navitas Life Science's annual pvnet® benchmark, we calculated workload indicators characterizing pharmacovigilance operations for large biopharmaceutical organizations. Workload indicators included the number of ICSRs by organization, the number of ARs, and the number and types of data sources used. We also conducted structured in-depth interviews with seven biopharmaceutical executives to discover the reasons for changes in workload indicators across time as well as current strategies for increasing efficiencies in drug safety reporting. RESULTS The median number of ICSRs increased from 84,960 cases in 2007 to over 200,000 cases in 2017; this increase was largely attributable to an increase in both nonserious cases and follow-up cases. Member companies reported using 12 ± 3 data sources for case identification. The number of ARs also increased from a median of 70 reports in 2007 to 258 reports in 2017. To address these increases, 61% of the biopharmaceutical organizations we surveyed planned to adopt machine learning for full ICSR processing; however, as of 2018, none of the organizations surveyed had mechanisms in place. CONCLUSION This study demonstrated that pharmacovigilance departments are currently burdened by ever-increasing case volumes. With increased guidance from regulatory agencies, as well as improvements in artificial intelligence and natural language processing, biopharmaceutical organizations must determine the most resource-efficient and sustainable methods to process the growing volume of cases.
Collapse
Affiliation(s)
- Stella Stergiopoulos
- Tufts Center for the Study of Drug Development, Tufts University School of Medicine, 75 Kneeland Street, Ste 1100, Boston, MA, 02111, USA.
| | | | | | - Louise Tan
- Pvnet®, Navitas Life Sciences GmbH, 60528, Frankfurt, Germany
| | - Louise Jebson
- Pvnet®, Navitas Life Sciences GmbH, 60528, Frankfurt, Germany
| |
Collapse
|
25
|
AlSaad R, Malluhi Q, Janahi I, Boughorbel S. Interpreting patient-Specific risk prediction using contextual decomposition of BiLSTMs: application to children with asthma. BMC Med Inform Decis Mak 2019; 19:214. [PMID: 31703676 PMCID: PMC6842261 DOI: 10.1186/s12911-019-0951-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2019] [Accepted: 10/28/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Predictive modeling with longitudinal electronic health record (EHR) data offers great promise for accelerating personalized medicine and better informs clinical decision-making. Recently, deep learning models have achieved state-of-the-art performance for many healthcare prediction tasks. However, deep models lack interpretability, which is integral to successful decision-making and can lead to better patient care. In this paper, we build upon the contextual decomposition (CD) method, an algorithm for producing importance scores from long short-term memory networks (LSTMs). We extend the method to bidirectional LSTMs (BiLSTMs) and use it in the context of predicting future clinical outcomes using patients' EHR historical visits. METHODS We use a real EHR dataset comprising 11071 patients, to evaluate and compare CD interpretations from LSTM and BiLSTM models. First, we train LSTM and BiLSTM models for the task of predicting which pre-school children with respiratory system-related complications will have asthma at school-age. After that, we conduct quantitative and qualitative analysis to evaluate the CD interpretations produced by the contextual decomposition of the trained models. In addition, we develop an interactive visualization to demonstrate the utility of CD scores in explaining predicted outcomes. RESULTS Our experimental evaluation demonstrate that whenever a clear visit-level pattern exists, the models learn that pattern and the contextual decomposition can appropriately attribute the prediction to the correct pattern. In addition, the results confirm that the CD scores agree to a large extent with the importance scores generated using logistic regression coefficients. Our main insight was that rather than interpreting the attribution of individual visits to the predicted outcome, we could instead attribute a model's prediction to a group of visits. CONCLUSION We presented a quantitative and qualitative evidence that CD interpretations can explain patient-specific predictions using CD attributions of individual visits or a group of visits.
Collapse
Affiliation(s)
- Rawan AlSaad
- Machine Learning Group, Sidra Medicine, Doha, Qatar
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Qutaibah Malluhi
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | - Ibrahim Janahi
- Division of Pediatric Pulmonology, Sidra Medicine, Doha, Qatar
| | | |
Collapse
|
26
|
Liu F, Jagannatha A, Yu H. Towards Drug Safety Surveillance and Pharmacovigilance: Current Progress in Detecting Medication and Adverse Drug Events from Electronic Health Records. Drug Saf 2019; 42:95-97. [PMID: 30649734 DOI: 10.1007/s40264-018-0766-8] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
- Feifan Liu
- Department of Quantitative Health Sciences and Radiology, University of Massachusetts Medical School, Worcester, MA, USA
| | - Abhyuday Jagannatha
- College of Information and Computer Sciences, University of Massachusetts, Amherst, MA, USA
| | - Hong Yu
- College of Information and Computer Sciences, University of Massachusetts, Amherst, MA, USA. .,Department of Computer Science, University of Massachusetts, 220 Pawtucket St, Lowell, MA, 01854-2874, USA. .,Department of Medicine, University of Massachusetts Medical School, Worcester, MA, USA. .,Bedford Veterans Affairs Medical Center, Bedford, MA, USA.
| |
Collapse
|