1
|
Dong W, Da Roza CC, Cheng D, Zhang D, Xiang Y, Seto WK, Wong WCW. Development and validation of HBV surveillance models using big data and machine learning. Ann Med 2024; 56:2314237. [PMID: 38340309 PMCID: PMC10860422 DOI: 10.1080/07853890.2024.2314237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 01/30/2024] [Indexed: 02/12/2024] Open
Abstract
BACKGROUND The construction of a robust healthcare information system is fundamental to enhancing countries' capabilities in the surveillance and control of hepatitis B virus (HBV). Making use of China's rapidly expanding primary healthcare system, this innovative approach using big data and machine learning (ML) could help towards the World Health Organization's (WHO) HBV infection elimination goals of reaching 90% diagnosis and treatment rates by 2030. We aimed to develop and validate HBV detection models using routine clinical data to improve the detection of HBV and support the development of effective interventions to mitigate the impact of this disease in China. METHODS Relevant data records extracted from the Family Medicine Clinic of the University of Hong Kong-Shenzhen Hospital's Hospital Information System were structuralized using state-of-the-art Natural Language Processing techniques. Several ML models have been used to develop HBV risk assessment models. The performance of the ML model was then interpreted using the Shapley value (SHAP) and validated using cohort data randomly divided at a ratio of 2:1 using a five-fold cross-validation framework. RESULTS The patterns of physical complaints of patients with and without HBV infection were identified by processing 158,988 clinic attendance records. After removing cases without any clinical parameters from the derivation sample (n = 105,992), 27,392 cases were analysed using six modelling methods. A simplified model for HBV using patients' physical complaints and parameters was developed with good discrimination (AUC = 0.78) and calibration (goodness of fit test p-value >0.05). CONCLUSIONS Suspected case detection models of HBV, showing potential for clinical deployment, have been developed to improve HBV surveillance in primary care setting in China. (Word count: 264).
Collapse
Affiliation(s)
- Weinan Dong
- Department of Family Medicine and Primary Care, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| | - Cecilia Clara Da Roza
- Department of Family Medicine and Primary Care, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| | - Dandan Cheng
- Department of Family Medicine and Primary Care, The University of Hong Kong-Shenzhen Hospital, Shenzhen, Guangdong, China
| | - Dahao Zhang
- Department of Family Medicine and Primary Care, The University of Hong Kong-Shenzhen Hospital, Shenzhen, Guangdong, China
| | - Yuling Xiang
- Department of Family Medicine and Primary Care, The University of Hong Kong-Shenzhen Hospital, Shenzhen, Guangdong, China
| | - Wai Kay Seto
- Department of Medicine and State Key Laboratory of Liver Research, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
- Department of Medicine, The University of Hong Kong-Shenzhen Hospital, Shenzhen, Guangdong, China
| | - William C. W. Wong
- Department of Family Medicine and Primary Care, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
- Department of Family Medicine and Primary Care, The University of Hong Kong-Shenzhen Hospital, Shenzhen, Guangdong, China
| |
Collapse
|
2
|
van de Burgt BWM, Wasylewicz ATM, Dullemond B, Jessurun NT, Grouls RJE, Bouwman RA, Korsten EHM, Egberts TCG. Development of a text mining algorithm for identifying adverse drug reactions in electronic health records. JAMIA Open 2024; 7:ooae070. [PMID: 39156048 PMCID: PMC11328534 DOI: 10.1093/jamiaopen/ooae070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 07/03/2024] [Accepted: 08/13/2024] [Indexed: 08/20/2024] Open
Abstract
Objective Adverse drug reactions (ADRs) are a significant healthcare concern. They are often documented as free text in electronic health records (EHRs), making them challenging to use in clinical decision support systems (CDSS). The study aimed to develop a text mining algorithm to identify ADRs in free text of Dutch EHRs. Materials and Methods In Phase I, our previously developed CDSS algorithm was recoded and improved upon with the same relatively large dataset of 35 000 notes (Step A), using R to identify possible ADRs with Medical Dictionary for Regulatory Activities (MedDRA) terms and the related Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) (Step B). In Phase II, 6 existing text-mining R-scripts were used to detect and present unique ADRs, and positive predictive value (PPV) and sensitivity were observed. Results In Phase IA, the recoded algorithm performed better than the previously developed CDSS algorithm, resulting in a PPV of 13% and a sensitivity of 93%. For The sensitivity for serious ADRs was 95%. The algorithm identified 58 additional possible ADRs. In Phase IB, the algorithm achieved a PPV of 10%, a sensitivity of 86%, and an F-measure of 0.18. In Phase II, four R-scripts enhanced the sensitivity and PPV of the algorithm, resulting in a PPV of 70%, a sensitivity of 73%, an F-measure of 0.71, and a 63% sensitivity for serious ADRs. Discussion and Conclusion The recoded Dutch algorithm effectively identifies ADRs from free-text Dutch EHRs using R-scripts and MedDRA/SNOMED-CT. The study details its limitations, highlighting the algorithm's potential and significant improvements.
Collapse
Affiliation(s)
- Britt W M van de Burgt
- Division of Clinical Pharmacy, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
- Division Healthcare Intelligence, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
- Department of Electrical Engineering, Signal Processing Group, Technical University Eindhoven, 5612 AP Eindhoven, The Netherlands
| | - Arthur T M Wasylewicz
- Division Healthcare Intelligence, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Bjorn Dullemond
- Department of Mathematics and Computer Science, Technical University Eindhoven, 5612 AP Eindhoven, The Netherlands
| | - Naomi T Jessurun
- Netherlands Pharmacovigilance Centre LAREB, 5237 MH 's-Hertogenbosch, The Netherlands
| | - Rene J E Grouls
- Division of Clinical Pharmacy, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - R Arthur Bouwman
- Department of Electrical Engineering, Signal Processing Group, Technical University Eindhoven, 5612 AP Eindhoven, The Netherlands
- Department of Anesthesiology, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Erik H M Korsten
- Division Healthcare Intelligence, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
- Department of Electrical Engineering, Signal Processing Group, Technical University Eindhoven, 5612 AP Eindhoven, The Netherlands
| | - Toine C G Egberts
- Department of Clinical Pharmacy, University Medical Centre Utrecht, 3584 CX Utrecht, The Netherlands
- Department of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Faculty of Science, Utrecht University, 3584 CX Utrecht, The Netherlands
| |
Collapse
|
3
|
AlShehri Y, Sidhu A, Lakshmanan LVS, Lefaivre KA. Applications of Natural Language Processing for Automated Clinical Data Analysis in Orthopaedics. J Am Acad Orthop Surg 2024; 32:439-446. [PMID: 38626429 DOI: 10.5435/jaaos-d-23-00839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 02/20/2024] [Indexed: 04/18/2024] Open
Abstract
Natural language processing is an exciting and emerging field in health care that can transform the field of orthopaedics. It can aid in the process of automated clinical data analysis, changing the way we extract data for various purposes including research and registry formation, diagnosis, and medical billing. This scoping review will look at the various applications of NLP in orthopaedics. Specific examples of NLP applications include identification of essential data elements from surgical and imaging reports, patient feedback analysis, and use of AI conversational agents for patient engagement. We will demonstrate how NLP has proven itself to be a powerful and valuable tool. Despite these potential advantages, there are drawbacks we must consider. Concerns with data quality, bias, privacy, and accessibility may stand as barriers in the way of widespread implementation of NLP technology. As natural language processing technology continues to develop, it has the potential to revolutionize orthopaedic research and clinical practices and enhance patient outcomes.
Collapse
Affiliation(s)
- Yasir AlShehri
- From the Department of Orthopedics, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia (AlShehri), the Department of Orthopaedics, Faculty of Medicine, The University of British Columbia, Vancouver, BC, Canada (Sidhu and Lefaivre), and the Department of Computer Science, The University of British Columbia, Vancouver, BC, Canada (Lakshmanan)
| | | | | | | |
Collapse
|
4
|
Gallifant J, Celi LA, Sharon E, Bitterman DS. Navigating the Complexities of Artificial Intelligence-Enabled Real-World Data Collection for Oncology Pharmacovigilance. JCO Clin Cancer Inform 2024; 8:e2400051. [PMID: 38713889 DOI: 10.1200/cci.24.00051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 04/03/2024] [Indexed: 05/09/2024] Open
Abstract
This new editorial discusses the promise and challenges of successful integration of natural language processing methods into electronic health records for timely, robust, and fair oncology pharmacovigilance.
Collapse
Affiliation(s)
- Jack Gallifant
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA
- Department of Critical Care, Guy's & St Thomas' NHS Trust, London, United Kingdom
| | - Leo Anthony Celi
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA
- Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA
| | - Elad Sharon
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA
| | - Danielle S Bitterman
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA
- Department of Radiation Oncology, Brigham and Women's Hospital/Dana-Farber Cancer Institute, Boston, MA
| |
Collapse
|
5
|
Li L, Badgery-Parker T, Merchant A, Fitzpatrick E, Raban MZ, Mumford V, Metri NJ, Hibbert PD, Mccullagh C, Dickinson M, Westbrook JI. Paediatric medication incident reporting: a multicentre comparison study of medication errors identified at audit, detected by staff and reported to an incident system. BMJ Qual Saf 2024:bmjqs-2023-016711. [PMID: 38621921 DOI: 10.1136/bmjqs-2023-016711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 04/01/2024] [Indexed: 04/17/2024]
Abstract
OBJECTIVES To compare medication errors identified at audit and via direct observation with medication errors reported to an incident reporting system at paediatric hospitals and to investigate differences in types and severity of errors detected and reported by staff. METHODS This is a comparison study at two tertiary referral paediatric hospitals between 2016 and 2020 in Australia. Prescribing errors were identified from a medication chart audit of 7785 patient records. Medication administration errors were identified from a prospective direct observational study of 5137 medication administration doses to 1530 patients. Medication errors reported to the hospitals' incident reporting system were identified and matched with errors identified at audit and observation. RESULTS Of 11 302 clinical prescribing errors identified at audit, 3.2 per 1000 errors (95% CI 2.3 to 4.4, n=36) had an incident report. Of 2224 potentially serious prescribing errors from audit, 26.1% (95% CI 24.3 to 27.9, n=580) were detected by staff and 11.2 per 1000 errors (95% CI 7.6 to 16.5, n=25) were reported to the incident system. Although the prescribing error detection rates varied between the two hospitals, there was no difference in incident reporting rates regardless of error severity. Of 40 errors associated with actual patient harm, only 7 (17.5%; 95% CI 8.7% to 31.9%) were detected by staff and 4 (10.0%; 95% CI 4.0% to 23.1%) had an incident report. None of the 2883 clinical medication administration errors observed, including 903 potentially serious errors and 144 errors associated with actual patient harm, had incident reports. CONCLUSION Incident reporting data do not provide an accurate reflection of medication errors and related harm to children in hospitals. Failure to detect medication errors is likely to be a significant contributor to low error reporting rates. In an era of electronic health records, new automated approaches to monitor medication safety should be pursued to provide real-time monitoring.
Collapse
Affiliation(s)
- Ling Li
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Tim Badgery-Parker
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Alison Merchant
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Erin Fitzpatrick
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Magdalena Z Raban
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Virginia Mumford
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Najwa-Joelle Metri
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Peter Damian Hibbert
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| | - Cheryl Mccullagh
- Executive, Beamtree, Redfern, New South Wales, Australia
- The Sydney Children's Hospitals Network Randwick and Westmead, Sydney, New South Wales, Australia
| | - Michael Dickinson
- The Sydney Children's Hospitals Network Randwick and Westmead, Sydney, New South Wales, Australia
| | - Johanna I Westbrook
- Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
| |
Collapse
|
6
|
Reniewicz J, Suryaprakash V, Kowalczyk J, Blacha A, Kostello G, Tan H, Wang Y, Reineke P, Manissero D. Artificial intelligence / machine-learning tool for post-market surveillance of in vitro diagnostic assays. N Biotechnol 2024; 79:82-90. [PMID: 38040287 DOI: 10.1016/j.nbt.2023.11.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Revised: 11/17/2023] [Accepted: 11/28/2023] [Indexed: 12/03/2023]
Abstract
The study compares an artificial intelligence technology with traditional manual search of literature databases to assess the accuracy and efficiency of retrieving relevant articles for post-market surveillance of in vitro diagnostic and medical devices under the Medical Device Regulation and In Vitro Diagnostic Medical Device Regulation. Over a 3-year period, literature searches and technical assessment searches were performed manually or using the Huma.AI platform to retrieve relevant articles related to the safety and performance of selected in vitro diagnostic and medical devices. The manual search involved refined keyword searches, screening of titles/abstracts / full text, and extraction of relevant information. The Huma.AI search utilized advanced caching techniques and a natural language processing system to identify relevant reports. Searches were conducted on PubMed and PubMed Central. The number of identified relevant reports, precision rates, and time requirements for each approach were analyzed. The Huma.AI system outperformed the manual search in terms of the number of identified relevant articles in almost all cases. The average precision rates per year were significantly higher and more consistent with the Huma.AI search compared with the manual search. The Huma.AI system also took significantly less time to perform the searches and analyze the outputs than the manual search. The study demonstrated that the Huma.AI platform was more effective and efficient in identifying relevant articles compared with the manual approach.
Collapse
Affiliation(s)
- Joanna Reniewicz
- QIAGEN Wrocław, ul. Powstańców Śląskich 95, 53-332 Wrocław, Poland
| | | | | | - Anna Blacha
- QIAGEN Manchester Ltd, CityLabs, 2.0 Hathersage Rd, M13 0BH Manchester, UK.
| | - Greg Kostello
- Huma.AI, 3000 El Camino Real, Building 4, Suite 200-69, Palo Alto, CA 94306, USA
| | - Haiming Tan
- Huma.AI, 3000 El Camino Real, Building 4, Suite 200-69, Palo Alto, CA 94306, USA
| | - Yan Wang
- Huma.AI, 3000 El Camino Real, Building 4, Suite 200-69, Palo Alto, CA 94306, USA
| | - Patrick Reineke
- Huma.AI, 3000 El Camino Real, Building 4, Suite 200-69, Palo Alto, CA 94306, USA
| | - Davide Manissero
- QIAGEN Manchester Ltd, CityLabs, 2.0 Hathersage Rd, M13 0BH Manchester, UK
| |
Collapse
|
7
|
Wong ZSY, Waters N, Liu J, Ushiro S. A large dataset of annotated incident reports on medication errors. Sci Data 2024; 11:260. [PMID: 38424103 PMCID: PMC10904777 DOI: 10.1038/s41597-024-03036-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 02/01/2024] [Indexed: 03/02/2024] Open
Abstract
Incident reports of medication errors are valuable learning resources for improving patient safety. However, pertinent information is often contained within unstructured free text, which prevents automated analysis and limits the usefulness of these data. Natural language processing can structure this free text automatically and retrieve relevant past incidents and learning materials, but to be able to do so requires a large, fully annotated and validated corpus of incident reports. We present a corpus of 58,658 machine-annotated incident reports of medication errors that can be used to advance the development of information extraction models and subsequent incident learning. We report the best F1-scores for the annotated dataset: 0.97 and 0.76 for named entity recognition and intention/factuality analysis, respectively, for the cross-validation exercise. Our dataset contains 478,175 named entities and differentiates between incident types by recognising discrepancies between what was intended and what actually occurred. We explain our annotation workflow and technical validation and provide access to the validation datasets and machine annotator for labelling future incident reports of medication errors.
Collapse
Affiliation(s)
- Zoie S Y Wong
- Graduate School of Public Health, St. Luke's International University, 3-6-2 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.
- School of Medical Sciences, The University of Sydney, Camperdown, NSW, 2006, Australia.
| | - Neil Waters
- Graduate School of Public Health, St. Luke's International University, 3-6-2 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Jiaxing Liu
- School of Statistics and Mathematics, Zhongnan University of Economics and Law, Nanhu Blvd, Wuhan, Hubei, 430073, China
| | - Shin Ushiro
- Division of Patient Safety, Kyushu University Hospital, 3-1-1 Maidashi, Higashi-ku, Fukuoka, 812-8582, Japan
- Japan Council for Quality Health Care (JQ), 1-4-17, Toyo Bldg., Kandamisaki-cho, Chiyoda-ku, Tokyo, 101-0061, Japan
| |
Collapse
|
8
|
Xu Y, Zheng X, Li Y, Ye X, Cheng H, Wang H, Lyu J. Exploring patient medication adherence and data mining methods in clinical big data: A contemporary review. J Evid Based Med 2023; 16:342-375. [PMID: 37718729 DOI: 10.1111/jebm.12548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 08/30/2023] [Indexed: 09/19/2023]
Abstract
BACKGROUND Increasingly, patient medication adherence data are being consolidated from claims databases and electronic health records (EHRs). Such databases offer an indirect avenue to gauge medication adherence in our data-rich healthcare milieu. The surge in data accessibility, coupled with the pressing need for its conversion to actionable insights, has spotlighted data mining, with machine learning (ML) emerging as a pivotal technique. Nonadherence poses heightened health risks and escalates medical costs. This paper elucidates the synergistic interaction between medical database mining for medication adherence and the role of ML in fostering knowledge discovery. METHODS We conducted a comprehensive review of EHR applications in the realm of medication adherence, leveraging ML techniques. We expounded on the evolution and structure of medical databases pertinent to medication adherence and harnessed both supervised and unsupervised ML paradigms to delve into adherence and its ramifications. RESULTS Our study underscores the applications of medical databases and ML, encompassing both supervised and unsupervised learning, for medication adherence in clinical big data. Databases like SEER and NHANES, often underutilized due to their intricacies, have gained prominence. Employing ML to excavate patient medication logs from these databases facilitates adherence analysis. Such findings are pivotal for clinical decision-making, risk stratification, and scholarly pursuits, aiming to elevate healthcare quality. CONCLUSION Advanced data mining in the era of big data has revolutionized medication adherence research, thereby enhancing patient care. Emphasizing bespoke interventions and research could herald transformative shifts in therapeutic modalities.
Collapse
Affiliation(s)
- Yixian Xu
- Department of Anesthesiology, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Xinkai Zheng
- Department of Dermatology, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Yuanjie Li
- Planning & Discipline Construction Office, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Xinmiao Ye
- Department of Anesthesiology, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Hongtao Cheng
- School of Nursing, Jinan University, Guangzhou, China
| | - Hao Wang
- Department of Anesthesiology, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Jun Lyu
- Department of Clinical Research, The First Affiliated Hospital of Jinan University, Guangzhou, China
- Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Informatization, Guangzhou, China
| |
Collapse
|
9
|
Davis SE, Zabotka L, Desai RJ, Wang SV, Maro JC, Coughlin K, Hernández-Muñoz JJ, Stojanovic D, Shah NH, Smith JC. Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review. Drug Saf 2023; 46:725-742. [PMID: 37340238 DOI: 10.1007/s40264-023-01325-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/31/2023] [Indexed: 06/22/2023]
Abstract
INTRODUCTION Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance. METHODS To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices. RESULTS We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations. CONCLUSION Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.
Collapse
Affiliation(s)
- Sharon E Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN, 37203, USA
- Vanderbilt University School of Medicine, Nashville, TN, USA
| | | | - Rishi J Desai
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Shirley V Wang
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Judith C Maro
- Harvard Medical School, Boston, MA, USA
- Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | | | | | | | - Nigam H Shah
- School of Medicine, Stanford University, Stanford, CA, USA
- Stanford Health Care, Palo Alto, CA, USA
| | - Joshua C Smith
- Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN, 37203, USA.
- Vanderbilt University School of Medicine, Nashville, TN, USA.
| |
Collapse
|
10
|
Allen KS, Hood DR, Cummins J, Kasturi S, Mendonca EA, Vest JR. Natural language processing-driven state machines to extract social factors from unstructured clinical documentation. JAMIA Open 2023; 6:ooad024. [PMID: 37081945 PMCID: PMC10112959 DOI: 10.1093/jamiaopen/ooad024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 03/08/2023] [Accepted: 03/28/2023] [Indexed: 04/22/2023] Open
Abstract
Objective This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability. Materials and Methods Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity. Results PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Discussion We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics. Conclusion The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.
Collapse
Affiliation(s)
- Katie S Allen
- Corresponding Author: Katie S. Allen, BS, Center for Biomedical Informatics, Regenstrief Institute, Inc., 1101 W. 10th Street, Indianapolis, IN 46202, USA;
| | - Dan R Hood
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Jonathan Cummins
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Suranga Kasturi
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Eneida A Mendonca
- Division of Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA
- Department of Pediatrics, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Joshua R Vest
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
- Department of Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, USA
| |
Collapse
|
11
|
Botsis T, Kreimeyer K. Improving drug safety with adverse event detection using natural language processing. Expert Opin Drug Saf 2023; 22:659-668. [PMID: 37339273 DOI: 10.1080/14740338.2023.2228197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 06/19/2023] [Indexed: 06/22/2023]
Abstract
INTRODUCTION Pharmacovigilance (PV) involves monitoring and aggregating adverse event information from a variety of data sources, including health records, biomedical literature, spontaneous adverse event reports, product labels, and patient-generated content like social media posts, but the most pertinent details in these sources are typically available in narrative free-text formats. Natural language processing (NLP) techniques can be used to extract clinically relevant information from PV texts to inform decision-making. AREAS COVERED We conducted a non-systematic literature review by querying the PubMed database to examine the uses of NLP in drug safety and distilled the findings to present our expert opinion on the topic. EXPERT OPINION New NLP techniques and approaches continue to be applied for drug safety use cases; however, systems that are fully deployed and in use in a clinical environment remain vanishingly rare. To see high-performing NLP techniques implemented in the real setting will require long-term engagement with end users and other stakeholders and revised workflows in fully formulated business plans for the targeted use cases. Additionally, we found little to no evidence of extracted information placed into standardized data models, which should be a way to make implementations more portable and adaptable.
Collapse
Affiliation(s)
- Taxiarchis Botsis
- Department of Oncology, the Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Kory Kreimeyer
- Department of Oncology, the Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
12
|
Greenberg ZF, Graim KS, He M. Towards artificial intelligence-enabled extracellular vesicle precision drug delivery. Adv Drug Deliv Rev 2023:114974. [PMID: 37356623 DOI: 10.1016/j.addr.2023.114974] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 06/21/2023] [Accepted: 06/22/2023] [Indexed: 06/27/2023]
Abstract
Extracellular Vesicles (EVs), particularly exosomes, recently exploded into nanomedicine as an emerging drug delivery approach due to their superior biocompatibility, circulating stability, and bioavailability in vivo. However, EV heterogeneity makes molecular targeting precision a critical challenge. Deciphering key molecular drivers for controlling EV tissue targeting specificity is in great need. Artificial intelligence (AI) brings powerful prediction ability for guiding the rational design of engineered EVs in precision control for drug delivery. This review focuses on cutting-edge nano-delivery via integrating large-scale EV data with AI to develop AI-directed EV therapies and illuminate the clinical translation potential. We briefly review the current status of EVs in drug delivery, including the current frontier, limitations, and considerations to advance the field. Subsequently, we detail the future of AI in drug delivery and its impact on precision EV delivery. Our review discusses the current universal challenge of standardization and critical considerations when using AI combined with EVs for precision drug delivery. Finally, we will conclude this review with a perspective on future clinical translation led by a combined effort of AI and EV research.
Collapse
Affiliation(s)
- Zachary F Greenberg
- Department of Pharmaceutics, College of Pharmacy, University of Florida, Gainesville, Florida, 32610, USA
| | - Kiley S Graim
- Department of Computer & Information Science & Engineering, Herbert Wertheim College of Engineering, University of Florida, Gainesville, Florida, 32610, USA
| | - Mei He
- Department of Pharmaceutics, College of Pharmacy, University of Florida, Gainesville, Florida, 32610, USA.
| |
Collapse
|
13
|
Bikdeli B, Lo YC, Khairani CD, Bejjani A, Jimenez D, Barco S, Mahajan S, Caraballo C, Secemsky EA, Klok FA, Hunsaker AR, Aghayev A, Muriel A, Wang Y, Hussain MA, Appah-Sampong A, Lu Y, Lin Z, Aneja S, Khera R, Goldhaber SZ, Zhou L, Monreal M, Krumholz HM, Piazza G. Developing Validated Tools to Identify Pulmonary Embolism in Electronic Databases: Rationale and Design of the PE-EHR+ Study. Thromb Haemost 2023; 123:649-662. [PMID: 36809777 PMCID: PMC11200175 DOI: 10.1055/a-2039-3222] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
BACKGROUND Contemporary pulmonary embolism (PE) research, in many cases, relies on data from electronic health records (EHRs) and administrative databases that use International Classification of Diseases (ICD) codes. Natural language processing (NLP) tools can be used for automated chart review and patient identification. However, there remains uncertainty with the validity of ICD-10 codes or NLP algorithms for patient identification. METHODS The PE-EHR+ study has been designed to validate ICD-10 codes as Principal Discharge Diagnosis, or Secondary Discharge Diagnoses, as well as NLP tools set out in prior studies to identify patients with PE within EHRs. Manual chart review by two independent abstractors by predefined criteria will be the reference standard. Sensitivity, specificity, and positive and negative predictive values will be determined. We will assess the discriminatory function of code subgroups for intermediate- and high-risk PE. In addition, accuracy of NLP algorithms to identify PE from radiology reports will be assessed. RESULTS A total of 1,734 patients from the Mass General Brigham health system have been identified. These include 578 with ICD-10 Principal Discharge Diagnosis codes for PE, 578 with codes in the secondary position, and 578 without PE codes during the index hospitalization. Patients within each group were selected randomly from the entire pool of patients at the Mass General Brigham health system. A smaller subset of patients will also be identified from the Yale-New Haven Health System. Data validation and analyses will be forthcoming. CONCLUSIONS The PE-EHR+ study will help validate efficient tools for identification of patients with PE in EHRs, improving the reliability of efficient observational studies or randomized trials of patients with PE using electronic databases.
Collapse
Affiliation(s)
- Behnood Bikdeli
- Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
- Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
- Cardiovascular Research Foundation (CRF), New York, New York, United States
| | - Ying-Chih Lo
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Candrika D Khairani
- Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Antoine Bejjani
- Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - David Jimenez
- Respiratory Department, Hospital Ramón y Cajal and Medicine Department, Universidad de Alcalá (Instituto de Ramón y Cajal de Investigación Sanitaria), Centro de Investigación Biomédica en Red de Enfermedades Respiratorias, Madrid, Spain
| | - Stefano Barco
- Department of Angiology, University Hospital Zurich, Zurich, Switzerland
- Center for Thrombosis and Hemostasis, Johannes Gutenberg University Mainz, Mainz, Germany
| | - Shiwani Mahajan
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
- Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States
| | - César Caraballo
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
| | - Eric A Secemsky
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
- Division of Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States
| | - Frederikus A Klok
- Department of Medicine - Thrombosis and Hemostasis, Leiden University Medical Centre, Leiden, The Netherlands
| | - Andetta R Hunsaker
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Ayaz Aghayev
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Alfonso Muriel
- Clinical Biostatistics Unit. Hospital Universitario Ramón y Cajal. IRYCIS, CIBERESP: Universidad de Alcalá. Madrid, Spain
| | - Yun Wang
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
- Centre for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Abena Appah-Sampong
- Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Yuan Lu
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
| | - Zhenqiu Lin
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
| | - Sanjay Aneja
- Department of Therapeutic Radiology, Yale School of Medicine, New Haven, Connecticut, United States
| | - Rohan Khera
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
- Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States
| | - Samuel Z Goldhaber
- Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
- Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Li Zhou
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Manuel Monreal
- Cátedra de Enfermedad Tromboembólica, Universidad Católica de Murcia, Murcia, Spain
| | - Harlan M Krumholz
- YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States
- Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States
- Department of Health Policy and Management, Yale School of Public Health, New Haven, Connecticut, United States
| | - Gregory Piazza
- Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
- Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| |
Collapse
|
14
|
Duffy CC, Bass GA, Yura C, Dymek M, Lorenzi C, Kaplan LJ, Clapp JT, Atkins JH. Thematic mapping of perioperative incident reporting data to relational coordination domains. J Interprof Care 2023; 37:245-253. [PMID: 36739556 DOI: 10.1080/13561820.2022.2057454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Communication failure is a common root cause of adverse clinical events. Problematic communication domains are difficult to decipher, and communication improvement strategies are scarce. This study compared perioperative incident reports (IR) identifying potential communication failures with the results of a contemporaneous peri-operative Relational Coordination (RC) survey. We hypothesised that IR-prevalent themes would map to areas-of-weakness identified in the RC survey. Perioperative IRs filed between 2018 and 2020 (n = 6,236) were manually reviewed to identify communication failures (n = 1049). The IRs were disaggregated into seven RC theory domains and compared with the RC survey. Report disaggregation ratings demonstrated a three-way inter-rater agreement of 91.2%. Of the 1,049 communication failure-related IRs, shared knowledge deficits (n = 479, 46%) or accurate communication (n = 465, 44%) were most frequently identified. Communication frequency failures (n = 3, 0.3%) were rarely coded. Comparatively, shared knowledge was the weakest domain in the RC survey, while communication frequency was the strongest, correlating well with our IR data. Linking IR with RC domains offers a novel approach to assessing the specific elements of communication failures with an acute care facility. This approach provides a deployable mechanism to trend intra- and inter-domain progress in communication success, and develop targeted interventions to mitigate against communication failure-related adverse events.
Collapse
Affiliation(s)
- Caoimhe C Duffy
- Department of Anesthesiology & Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA.,Division of Perioperative & Procedural Services, Hospital of the University of Pennsylvania, Philadelphia, PA, USA.,Leonard Davis Institute of Health Economics at the University of Pennsylvania, Philadelphia, PA, USA
| | - Gary A Bass
- Leonard Davis Institute of Health Economics at the University of Pennsylvania, Philadelphia, PA, USA.,Division of Trauma, Surgical Critical Care and Emergency Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
| | - Chris Yura
- Division of Perioperative & Procedural Services, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Malwina Dymek
- Department of Anesthesiology & Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
| | - Cara Lorenzi
- Division of Perioperative & Procedural Services, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Lewis J Kaplan
- Division of Trauma, Surgical Critical Care and Emergency Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA.,Section of Surgical Critical Care, Corporal Michael Crescencz Veterans Affairs Medical Center, Philadelphia, PA, USA
| | - Justin T Clapp
- Department of Anesthesiology & Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA.,Leonard Davis Institute of Health Economics at the University of Pennsylvania, Philadelphia, PA, USA
| | - Joshua H Atkins
- Department of Anesthesiology & Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
15
|
Zhang C, Cao X. Biological gene extraction path based on knowledge graph and natural language processing. Front Genet 2023; 13:1086379. [PMID: 36712855 PMCID: PMC9880067 DOI: 10.3389/fgene.2022.1086379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 12/09/2022] [Indexed: 01/15/2023] Open
Abstract
The continuous progress of society and the vigorous development of science and technology have brought people the dawn of maintaining health and preventing and controlling diseases. At the same time, with the update and iteration of bioinformatics technology, the current biological gene research has also undergone revolutionary changes. However, a long-standing problem in genetic research has always plagued researchers, that is, how to find the most needed sample genes from a large number of sample genes, so as to reduce unnecessary research and reduce research costs. By studying the extraction path of biological genes, it can help researchers to extract the most valuable research genes and avoid wasting time and energy. In order to solve the above problems, this paper used the Bhattacharyya distance index and the Gini index to screen the sample genes when extracting the characteristic genes of breast cancer. In the selected 49 public genes, 6 principal components were extracted by principal component analysis (PCA), and finally the experimental results were tested. It was found that when the optimal number of characteristic genes was selected as 5, the recognition rate of genes reached the highest 90.31%, which met the experimental requirements. In addition, the experiment also proved that the characteristic gene extraction method designed in this paper had a removal rate of 99.75% of redundant genes, which can greatly reduce the time and money cost of research.
Collapse
Affiliation(s)
- Canlin Zhang
- Sorenson Communications, Salt Lake City, UT, United States
| | - Xiaopei Cao
- College of Creative Culture and Communication, Zhejiang Normal University, Jinhua, Zhejiang, China,*Correspondence: Xiaopei Cao,
| |
Collapse
|
16
|
Adverse drug event detection using natural language processing: A scoping review of supervised learning methods. PLoS One 2023; 18:e0279842. [PMID: 36595517 PMCID: PMC9810201 DOI: 10.1371/journal.pone.0279842] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 12/15/2022] [Indexed: 01/04/2023] Open
Abstract
To reduce adverse drug events (ADEs), hospitals need a system to support them in monitoring ADE occurrence routinely, rapidly, and at scale. Natural language processing (NLP), a computerized approach to analyze text data, has shown promising results for the purpose of ADE detection in the context of pharmacovigilance. However, a detailed qualitative assessment and critical appraisal of NLP methods for ADE detection in the context of ADE monitoring in hospitals is lacking. Therefore, we have conducted a scoping review to close this knowledge gap, and to provide directions for future research and practice. We included articles where NLP was applied to detect ADEs in clinical narratives within electronic health records of inpatients. Quantitative and qualitative data items relating to NLP methods were extracted and critically appraised. Out of 1,065 articles screened for eligibility, 29 articles met the inclusion criteria. Most frequent tasks included named entity recognition (n = 17; 58.6%) and relation extraction/classification (n = 15; 51.7%). Clinical involvement was reported in nine studies (31%). Multiple NLP modelling approaches seem suitable, with Long Short Term Memory and Conditional Random Field methods most commonly used. Although reported overall performance of the systems was high, it provides an inflated impression given a steep drop in performance when predicting the ADE entity or ADE relation class. When annotating corpora, treating an ADE as a relation between a drug and non-drug entity seems the best practice. Future research should focus on semi-automated methods to reduce the manual annotation effort, and examine implementation of the NLP methods in practice.
Collapse
|
17
|
Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. MULTIMEDIA TOOLS AND APPLICATIONS 2023; 82:3713-3744. [PMID: 35855771 PMCID: PMC9281254 DOI: 10.1007/s11042-022-13428-4] [Citation(s) in RCA: 82] [Impact Index Per Article: 82.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2021] [Revised: 03/23/2022] [Accepted: 07/02/2022] [Indexed: 05/17/2023]
Abstract
Natural language processing (NLP) has recently gained much attention for representing and analyzing human language computationally. It has spread its applications in various fields such as machine translation, email spam detection, information extraction, summarization, medical, and question answering etc. In this paper, we first distinguish four phases by discussing different levels of NLP and components of Natural Language Generation followed by presenting the history and evolution of NLP. We then discuss in detail the state of the art presenting the various applications of NLP, current trends, and challenges. Finally, we present a discussion on some available datasets, models, and evaluation metrics in NLP.
Collapse
Affiliation(s)
- Diksha Khurana
- Department of Computer Science, Manav Rachna International Institute of Research and Studies, Faridabad, India
| | - Aditya Koli
- Department of Computer Science, Manav Rachna International Institute of Research and Studies, Faridabad, India
| | - Kiran Khatter
- Department of Computer Science, BML Munjal University, Gurgaon, India
| | - Sukhdev Singh
- Department of Statistics, Amity University Punjab, Mohali, India
| |
Collapse
|
18
|
Chapman AB, Peterson KS, Rutter E, Nevers M, Zhang M, Ying J, Jones M, Classen D, Jones B. Development and evaluation of an interoperable natural language processing system for identifying pneumonia across clinical settings of care and institutions. JAMIA Open 2022; 5:ooac114. [PMID: 36601365 PMCID: PMC9801965 DOI: 10.1093/jamiaopen/ooac114] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Revised: 11/26/2022] [Accepted: 12/22/2022] [Indexed: 12/31/2022] Open
Abstract
Objective To evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system that extracts diagnostic assertions of pneumonia in different clinical notes and institutions. Materials and Methods A rule-based NLP system was designed to identify assertions of pneumonia in 3 types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs (VA). Interoperability was assessed using data from the University of Utah (UU). Results The NLP system was comprised of 782 rules and achieved moderate-to-high performance in all 3 note types in VA (precision/recall/f1: emergency = 88.1/86.0/87.1; radiology = 71.4/96.2/82.0; discharge = 88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency = 84.7/94.3/89.3; radiology = 79.7/100.0/87.9; discharge = 65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency = 89.3/94.3/91.7; radiology = 87.0/100.0/93.1; discharge = 75.0/95.1/83.4). Conclusion NLP can be used to accurately identify the diagnosis of pneumonia across different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.
Collapse
Affiliation(s)
- Alec B Chapman
- Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Center, Veterans Affairs (VA) Salt Lake City Health Care System, Salt Lake City, Utah, USA,Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah, USA,Department of Population Health Sciences, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Kelly S Peterson
- Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Center, Veterans Affairs (VA) Salt Lake City Health Care System, Salt Lake City, Utah, USA,Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah, USA,Veterans Health Administration Office of Analytics and Performance Integration, Washington, District of Columbia, USA
| | - Elizabeth Rutter
- George E. Wahlen Veterans Affairs (VA) Medical Center, Salt Lake City, Utah, USA,Emergency Physicians Integrated Care (EPIC, LLC), Salt Lake City, Utah, USA
| | - Mckenna Nevers
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Mingyuan Zhang
- Department of Population Health Sciences, University of Utah School of Medicine, Salt Lake City, Utah, USA,Data Science Service, University of Utah, Salt Lake City, Utah, USA
| | - Jian Ying
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Makoto Jones
- Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Center, Veterans Affairs (VA) Salt Lake City Health Care System, Salt Lake City, Utah, USA,Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - David Classen
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Barbara Jones
- Corresponding Author: Barbara Jones, MD, MS, Division of Pulmonary & Critical Care Medicine, University of Utah School of Medicine, 295 Chipeta Way, Salt Lake City, UT 84108, USA;
| |
Collapse
|
19
|
Uematsu H, Uemura M, Kurihara M, Umemura T, Hiramatsu M, Kitano F, Fukami T, Nagao Y. Development of a Novel Scoring System to Quantify the Severity of Incident Reports: An Exploratory Research Study. J Med Syst 2022; 46:106. [PMID: 36503962 DOI: 10.1007/s10916-022-01893-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2022] [Accepted: 11/18/2022] [Indexed: 12/14/2022]
Abstract
Incident reporting systems have been widely adopted to collect information about patient safety incidents. Much of the value of incident reports lies in the free-text section. Computer processing of semantic information may be helpful to analyze this. We developed a novel scoring system for decision making to assess the severity of incidents using the semantic characteristics of the text in incident reports, and compared its results with experts' opinions. We retrospectively analyzed free-text data from incident reports from January 2012 to September 2021 at Nagoya University Hospital, Aichi, Japan. The sample was allocated to training and validation datasets using the hold-out method. Morphological analysis was used to segment terms in the training dataset. We calculated a severity term score, a severity report score and severity group score, by report volume size, and compared these with conventional severity classifications by patient safety experts and reporters. We allocated 96,082 incident reports into two groups. We calculated 1,802 severity term scores from the 48,041 reports in the training dataset. There was a significant difference in severity report score between reports categorized as severe and not severe by experts (95% confidence interval [CI] -0.83 to -0.80, p < 0.001, d = 0.81). Severity group scores were positively associated with severity ratings from experts and reporters (correlation coefficients 0.73 [95% CI 0.63-0.80, p < 0.001] and 0.79 [95% CI 0.71-0.85, p < 0.001]) for all departments. Our severity scoring system could therefore contribute to better organizational patient safety.
Collapse
Affiliation(s)
- Haruhiro Uematsu
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan.
| | - Masakazu Uemura
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| | - Masaru Kurihara
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| | - Tomomi Umemura
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| | - Mariko Hiramatsu
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| | - Fumimasa Kitano
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| | - Tatsuya Fukami
- Department of Patient Safety, Shimane University Hospital, Izumo, Japan
| | - Yoshimasa Nagao
- Department of Patient Safety, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, 466-8560, Nagoya, Japan
| |
Collapse
|
20
|
Zyryanov SK, Zatolochina KE, Kazakov AS. Current patient safety issues: the role of pharmacovigilance. Public Health 2022. [DOI: 10.21045/2782-1676-2021-2-3-25-34] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In the current conditions of the pandemic, the burden on the healthcare system, including the pharmacovigilance system monitoring the safety of pharmacotherapy, has significantly increased in all countries. An integral component in ensuring the safety of pharmacotherapy is the identification and prevention of the development of adverse drug reactions (ADR), which are a serious health problem worldwide. One of the modern problems of healthcare, including pharmacovigilance, was the lack of vaccines and drugs for the treatment and prevention of COVID-19 in the first waves of the pandemic, which led to the use of off-label a large number of drugs (hydroxychloroquine, azithromycin, ivermectin) for the treatment of patients with COVID-19 despite the fact that scientific data their benefits were of poor quality and based on in vitro studies. The accelerated approval of drugs and vaccines to combat the COVID-19 pandemic also highlighted the need for rapid data on the safety of drugs in the post-marketing period. However, despite the fact that pharmacovigilance is developing, it still lags behind the impressive scientific and technological achievements achieved in other areas of medicine. Unfortunately, spontaneous reporting does not assess the true prevalence of ADR well, since reporting indicators can vary significantly depending on the motivation, availability of time, qualifications, fear of punishment and similar factors of the sender. Given these known limitations of the spontaneous messaging method, additional strategies for detecting ADR are often used, including trigger tools, manual viewing of medical records and automated monitoring.
Collapse
Affiliation(s)
- S. K. Zyryanov
- People’s Friendship University of Russia (RUDN University)
| | | | - A. S. Kazakov
- People’s Friendship University of Russia (RUDN University)
| |
Collapse
|
21
|
Li R, Curtis K, Zaidi ST, Van C, Castelino R. A new paradigm in adverse drug reaction reporting: consolidating the evidence for an intervention to improve reporting. Expert Opin Drug Saf 2022; 21:1193-1204. [DOI: 10.1080/14740338.2022.2118712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Affiliation(s)
- Raymond Li
- Faculty of Medicine and Health, University of Sydney, Parramatta Road, Camperdown NSW 2006
| | - Kate Curtis
- Faculty of Medicine and Health, University of Sydney, Parramatta Road, Camperdown NSW 2006
| | | | - Connie Van
- Faculty of Medicine and Health, University of Sydney, Parramatta Road, Camperdown NSW 2006
| | - Ronald Castelino
- Faculty of Medicine and Health, University of Sydney, Parramatta Road, Camperdown NSW 2006
| |
Collapse
|
22
|
Walsh J, Dwumfour C, Cave J, Griffiths F. Spontaneously generated online patient experience data - how and why is it being used in health research: an umbrella scoping review. BMC Med Res Methodol 2022; 22:139. [PMID: 35562661 PMCID: PMC9106384 DOI: 10.1186/s12874-022-01610-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 04/13/2022] [Indexed: 11/10/2022] Open
Abstract
PURPOSE Social media has led to fundamental changes in the way that people look for and share health related information. There is increasing interest in using this spontaneously generated patient experience data as a data source for health research. The aim was to summarise the state of the art regarding how and why SGOPE data has been used in health research. We determined the sites and platforms used as data sources, the purposes of the studies, the tools and methods being used, and any identified research gaps. METHODS A scoping umbrella review was conducted looking at review papers from 2015 to Jan 2021 that studied the use of SGOPE data for health research. Using keyword searches we identified 1759 papers from which we included 58 relevant studies in our review. RESULTS Data was used from many individual general or health specific platforms, although Twitter was the most widely used data source. The most frequent purposes were surveillance based, tracking infectious disease, adverse event identification and mental health triaging. Despite the developments in machine learning the reviews included lots of small qualitative studies. Most NLP used supervised methods for sentiment analysis and classification. Very early days, methods need development. Methods not being explained. Disciplinary differences - accuracy tweaks vs application. There is little evidence of any work that either compares the results in both methods on the same data set or brings the ideas together. CONCLUSION Tools, methods, and techniques are still at an early stage of development, but strong consensus exists that this data source will become very important to patient centred health research.
Collapse
Affiliation(s)
- Julia Walsh
- Warwick Medical School, University of Warwick, Coventry, UK.
| | | | - Jonathan Cave
- Department of Economics, University of Warwick, Coventry, UK
| | - Frances Griffiths
- Warwick Medical School, University of Warwick, Coventry, UK.,Centre for Health Policy, University of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
23
|
Christy SM, Reich RR, Rathwell JA, Vadaparampil ST, Isaacs-Soriano KA, Friedman MS, Roetzheim RG, Giuliano AR. Using the Electronic Health Record to Characterize the Hepatitis C Virus Care Cascade. Public Health Rep 2022; 137:498-505. [PMID: 33831316 PMCID: PMC9109542 DOI: 10.1177/00333549211005812] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023] Open
Abstract
OBJECTIVES Chronic hepatitis C virus (HCV) infection is one of the main causes of hepatocellular carcinoma. Before initiating a multilevel HCV screening intervention, we sought to (1) describe concordance between the electronic health record (EHR) data warehouse and manual medical record review in recording aspects of HCV testing and treatment and (2) estimate the percentage of patients with chronic HCV infection who initiated and completed HCV treatment using manual medical record review. METHODS We examined the medical records for 177 patients (100 randomly selected patients born during 1945-1965 without evidence of HCV testing and 77 adult patients of any birth cohort who had completed HCV testing) with a primary care or relevant specialist visit at an academic health care system in Tampa, Florida, from 2015 through 2018. We used the Cohen κ coefficient to examine the degree of concordance between the searchable data warehouse and the medical record review abstractions. Descriptive statistics characterized referral to and receipt of treatment among patients with chronic HCV infection from medical record review. RESULTS We found generally good concordance between the data warehouse abstraction and medical record review for HCV testing data (κ ranged from 0.66 to 0.87). However, the data warehouse failed to capture data on HCV treatment variables. According to medical record review, 28 patients had chronic HCV infection; 16 patients were prescribed treatment, 14 initiated treatment, and 9 achieved and had a reported posttreatment undetected HCV viral load. CONCLUSIONS Using data warehouse data provides generally reliable HCV testing information. However, without the use of natural language processing and purposeful EHR design, manual medical record reviews will likely be required to characterize treatment initiation and completion.
Collapse
Affiliation(s)
- Shannon M. Christy
- Department of Health Outcomes and Behavior, Division of
Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL,
USA
- Department of Gastrointestinal Oncology, H. Lee Moffitt Cancer
Center and Research Institute, Tampa, FL, USA
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Oncologic Sciences, Morsani College of Medicine,
University of South Florida, Tampa, FL, USA
| | - Richard R. Reich
- Biostatistics and Bioinformatics Shared Resource, H. Lee Moffitt
Cancer Center and Research Institute, Tampa, FL, USA
| | - Julie A. Rathwell
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Cancer Epidemiology, Division of Population Science,
H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | - Susan T. Vadaparampil
- Department of Health Outcomes and Behavior, Division of
Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL,
USA
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Oncologic Sciences, Morsani College of Medicine,
University of South Florida, Tampa, FL, USA
| | - Kimberly A. Isaacs-Soriano
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Cancer Epidemiology, Division of Population Science,
H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | - Mark S. Friedman
- Department of Gastrointestinal Oncology, H. Lee Moffitt Cancer
Center and Research Institute, Tampa, FL, USA
- Department of Oncologic Sciences, Morsani College of Medicine,
University of South Florida, Tampa, FL, USA
| | - Richard G. Roetzheim
- Department of Health Outcomes and Behavior, Division of
Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL,
USA
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Family Medicine, Morsani College of Medicine,
University of South Florida, Tampa, FL, USA
| | - Anna R. Giuliano
- Center for Immunization and Infection Research in Cancer, H. Lee
Moffitt Cancer Center and Research Institute, Tampa, FL, USA
- Department of Oncologic Sciences, Morsani College of Medicine,
University of South Florida, Tampa, FL, USA
- Department of Cancer Epidemiology, Division of Population Science,
H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| |
Collapse
|
24
|
Vunikili R, Glicksberg BS, Johnson KW, Dudley JT, Subramanian L, Shameer K. Predictive Modelling of Susceptibility to Substance Abuse, Mortality and Drug-Drug Interactions in Opioid Patients. Front Artif Intell 2021; 4:742723. [PMID: 34957391 PMCID: PMC8702828 DOI: 10.3389/frai.2021.742723] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 10/25/2021] [Indexed: 01/16/2023] Open
Abstract
Objective: Opioids are a class of drugs that are known for their use as pain relievers. They bind to opioid receptors on nerve cells in the brain and the nervous system to mitigate pain. Addiction is one of the chronic and primary adverse events of prolonged usage of opioids. They may also cause psychological disorders, muscle pain, depression, anxiety attacks etc. In this study, we present a collection of predictive models to identify patients at risk of opioid abuse and mortality by using their prescription histories. Also, we discover particularly threatening drug-drug interactions in the context of opioid usage. Methods and Materials: Using a publicly available dataset from MIMIC-III, two models were trained, Logistic Regression with L2 regularization (baseline) and Extreme Gradient Boosting (enhanced model), to classify the patients of interest into two categories based on their susceptibility to opioid abuse. We’ve also used K-Means clustering, an unsupervised algorithm, to explore drug-drug interactions that might be of concern. Results: The baseline model for classifying patients susceptible to opioid abuse has an F1 score of 76.64% (accuracy 77.16%) while the enhanced model has an F1 score of 94.45% (accuracy 94.35%). These models can be used as a preliminary step towards inferring the causal effect of opioid usage and can help monitor the prescription practices to minimize the opioid abuse. Discussion and Conclusion: Results suggest that the enhanced model provides a promising approach in preemptive identification of patients at risk for opioid abuse. By discovering and correlating the patterns contributing to opioid overdose or abuse among a variety of patients, machine learning models can be used as an efficient tool to help uncover the existing gaps and/or fraudulent practices in prescription writing. To quote an example of one such incidental finding, our study discovered that insulin might possibly be interacting with opioids in an unfavourable way leading to complications in diabetic patients. This indicates that diabetic patients under long term opioid usage might need to take increased amounts of insulin to make it more effective. This observation backs up prior research studies done on a similar aspect. To increase the translational value of our work, the predictive models and the associated software code are made available under the MIT License.
Collapse
Affiliation(s)
- Ramya Vunikili
- Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York, NY, United States.,Department of Information Services, Center for Research Informatics and Innovation, Northwell Health, New York, NY, United States
| | - Benjamin S Glicksberg
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, United States
| | - Kipp W Johnson
- Institute for Next Generation Healthcare, Mount Sinai Health System, New York, NY, United States
| | - Joel T Dudley
- Institute for Next Generation Healthcare, Mount Sinai Health System, New York, NY, United States
| | - Lakshminarayanan Subramanian
- Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York, NY, United States
| | - Khader Shameer
- Department of Information Services, Center for Research Informatics and Innovation, Northwell Health, New York, NY, United States.,Institute for Next Generation Healthcare, Mount Sinai Health System, New York, NY, United States
| |
Collapse
|
25
|
Chopard D, Treder MS, Corcoran P, Ahmed N, Johnson C, Busse M, Spasic I. Text Mining of Adverse Events in Clinical Trials: Deep Learning Approach. JMIR Med Inform 2021; 9:e28632. [PMID: 34951601 PMCID: PMC8742206 DOI: 10.2196/28632] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Revised: 08/01/2021] [Accepted: 11/14/2021] [Indexed: 11/28/2022] Open
Abstract
BACKGROUND Pharmacovigilance and safety reporting, which involve processes for monitoring the use of medicines in clinical trials, play a critical role in the identification of previously unrecognized adverse events or changes in the patterns of adverse events. OBJECTIVE This study aims to demonstrate the feasibility of automating the coding of adverse events described in the narrative section of the serious adverse event report forms to enable statistical analysis of the aforementioned patterns. METHODS We used the Unified Medical Language System (UMLS) as the coding scheme, which integrates 217 source vocabularies, thus enabling coding against other relevant terminologies such as the International Classification of Diseases-10th Revision, Medical Dictionary for Regulatory Activities, and Systematized Nomenclature of Medicine). We used MetaMap, a highly configurable dictionary lookup software, to identify the mentions of the UMLS concepts. We trained a binary classifier using Bidirectional Encoder Representations from Transformers (BERT), a transformer-based language model that captures contextual relationships, to differentiate between mentions of the UMLS concepts that represented adverse events and those that did not. RESULTS The model achieved a high F1 score of 0.8080, despite the class imbalance. This is 10.15 percent points lower than human-like performance but also 17.45 percent points higher than that of the baseline approach. CONCLUSIONS These results confirmed that automated coding of adverse events described in the narrative section of serious adverse event reports is feasible. Once coded, adverse events can be statistically analyzed so that any correlations with the trialed medicines can be estimated in a timely fashion.
Collapse
Affiliation(s)
- Daphne Chopard
- School of Computer Science & Informatics, Cardiff University, Cardiff, United Kingdom
| | - Matthias S Treder
- School of Computer Science & Informatics, Cardiff University, Cardiff, United Kingdom
| | - Padraig Corcoran
- School of Computer Science & Informatics, Cardiff University, Cardiff, United Kingdom
| | - Nagheen Ahmed
- Centre for Trials Research, Cardiff University, Cardiff, United Kingdom
| | - Claire Johnson
- Centre for Trials Research, Cardiff University, Cardiff, United Kingdom
| | - Monica Busse
- Centre for Trials Research, Cardiff University, Cardiff, United Kingdom
| | - Irena Spasic
- School of Computer Science & Informatics, Cardiff University, Cardiff, United Kingdom
| |
Collapse
|
26
|
Pham P, Cheng C, Wu E, Kim I, Zhang R, Ma Y, Kortepeter CM, Muñoz MA. Leveraging Case Narratives to Enhance Patient Age Ascertainment from Adverse Event Reports. Pharmaceut Med 2021; 35:307-316. [PMID: 34476768 DOI: 10.1007/s40290-021-00398-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/05/2021] [Indexed: 12/17/2022]
Abstract
INTRODUCTION Missing age presents a significant challenge when evaluating individual case safety reports (ICSRs) in the FDA Adverse Event Reporting System (FAERS). When age is missing in an ICSR's structured field, it may be in the report's free-text narrative. OBJECTIVES This study aimed to evaluate the performance and assess the potential impact of a rule-based natural language processing (NLP) tool that utilizes a text string search to identify patients' numerical age from unstructured narratives. METHODS Using FAERS ICSRs from 2002 to 2018, we evaluated the annual proportion of ICSRs with age missing in the structured field before and after NLP application. Reviewers manually identified patients' age from ICSR narratives (gold standard) from a random sample of 1500 ICSRs. The gold standard was compared to the NLP-identified age. RESULTS During the study period, the percentage of ICSRs missing age in the structured field increased from 21.9 to 43.8%. The NLP tool performed well among the random sample: sensitivity 98.5%, specificity 92.9%, positive predictive value (PPV) 94.9%, and F-measure 96.7%. It also performed well for the subset of ICSRs missing age in the structured field; when applied to these cases, NLP identified age for an additional one million ICSRs (10% of the total number of ICSRs from 2002 to 2018) and decreased the percentage of ICSRs missing age to 27% overall. CONCLUSIONS NLP has potential utility to extract patients' age from ICSR narratives. Use of this tool would enhance pharmacovigilance and research using FAERS data.
Collapse
Affiliation(s)
- Phuong Pham
- Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL, USA.,Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Carmen Cheng
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA.
| | - Eileen Wu
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Ivone Kim
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Rongmei Zhang
- Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Yong Ma
- Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Cindy M Kortepeter
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| | - Monica A Muñoz
- Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL, USA.,Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA
| |
Collapse
|
27
|
Yang LWY, Ng WY, Foo LL, Liu Y, Yan M, Lei X, Zhang X, Ting DSW. Deep learning-based natural language processing in ophthalmology: applications, challenges and future directions. Curr Opin Ophthalmol 2021; 32:397-405. [PMID: 34324453 DOI: 10.1097/icu.0000000000000789] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) is the fourth industrial revolution in mankind's history. Natural language processing (NLP) is a type of AI that transforms human language, to one that computers can interpret and process. NLP is still in the formative stages of development in healthcare, with promising applications and potential challenges in its applications. This review provides an overview of AI-based NLP, its applications in healthcare and ophthalmology, next-generation use case, as well as potential challenges in deployment. RECENT FINDINGS The integration of AI-based NLP systems into existing clinical care shows considerable promise in disease screening, risk stratification, and treatment monitoring, amongst others. Stakeholder collaboration, greater public acceptance, and advancing technologies will continue to shape the NLP landscape in healthcare and ophthalmology. SUMMARY Healthcare has always endeavored to be patient centric and personalized. For AI-based NLP systems to become an eventual reality in larger-scale applications, it is pertinent for key stakeholders to collaborate and address potential challenges in application. Ultimately, these would enable more equitable and generalizable use of NLP systems for the betterment of healthcare and society.
Collapse
Affiliation(s)
| | - Wei Yan Ng
- Singapore National Eye Centre, Singapore Eye Research Institute
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| | - Li Lian Foo
- Singapore National Eye Centre, Singapore Eye Research Institute
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| | - Yong Liu
- Institute of High Performance Computing, A STAR
| | - Ming Yan
- Institute of High Performance Computing, A STAR
| | | | | | - Daniel Shu Wei Ting
- Singapore National Eye Centre, Singapore Eye Research Institute
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| |
Collapse
|
28
|
Romanelli RJ, Schwartz NRM, Dixon WG, Rodriguez-Watson C, Sauer BC, Albright D, Marcum ZA. The use of narrative electronic prescribing instructions in pharmacoepidemiology: A scoping review for the International Society for Pharmacoepidemiology. Pharmacoepidemiol Drug Saf 2021; 30:1281-1292. [PMID: 34278660 PMCID: PMC8419095 DOI: 10.1002/pds.5331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 07/10/2021] [Accepted: 07/12/2021] [Indexed: 11/22/2022]
Abstract
Narrative electronic prescribing instructions (NEPIs) are text that convey information on the administration or co‐administration of a drug as directed by a prescriber. For researchers, NEPIs have the potential to advance our understanding of the risks and benefits of medications in populations; however, due to their unstructured nature, they are not often utilized. The goal of this scoping review was to evaluate how NEPIs are currently employed in research, identify opportunities and challenges for their broader application, and provide recommendations on their future use. The scoping review comprised a comprehensive literature review and a survey of key stakeholders. From the literature review, we identified 33 primary articles that described the use of NEPIs. The majority of articles (n = 19) identified issues with the quality of information in NEPIs compared with structured prescribing information; nine articles described the development of novel algorithms that performed well in extracting information from NEPIs, and five described the used of manual or simpler algorithms to extract prescribing information from NEPIs. A survey of 19 stakeholders indicated concerns for the quality of information in NEPIs and called for standardization of NEPIs to reduce data variability/errors. Nevertheless, stakeholders believed NEPIs present an opportunity to identify prescriber's intent for the prescription and to study temporal treatment patterns. In summary, NEPIs hold much promise for advancing the field of pharmacoepidemiology. Researchers should take advantage of addressing important questions that can be uniquely answered with NEPIs, but exercise caution when using this information and carefully consider the quality of the data.
Collapse
Affiliation(s)
- Robert J Romanelli
- Center for Health Systems Research, Sutter Health, Walnut Creek, California, USA
| | - Naomi R M Schwartz
- The Comparative Health Outcomes Policy and Economics Institute, School of Pharmacy, University of Washington, Seattle, Washington, USA
| | - William G Dixon
- Center for Epidemiology Versus Arthritis, University of Manchester, Manchester, UK
| | - Carla Rodriguez-Watson
- Innovation in Medical Evidence Development and Surveillance (IMEDS), Reagan-Udall Foundation for the Food and Drug Administration, Washington, DC, USA
| | - Brian C Sauer
- Division of Epidemiology, University of Utah, Salt Lake City, Utah, USA
| | | | - Zachary A Marcum
- The Comparative Health Outcomes Policy and Economics Institute, School of Pharmacy, University of Washington, Seattle, Washington, USA
| |
Collapse
|
29
|
杨 羽, 王 胜, 詹 思. [Utilizing social media data in post-market safety surveillance]. BEIJING DA XUE XUE BAO. YI XUE BAN = JOURNAL OF PEKING UNIVERSITY. HEALTH SCIENCES 2021; 53:623-627. [PMID: 34145872 PMCID: PMC8220064 DOI: 10.19723/j.issn.1671-167x.2021.03.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Indexed: 06/12/2023]
Abstract
Post-marketing surveillance is the principal means to ensure drug use safety. The spontaneous report is the essential method of post-marketing surveillance for drug safety. Often, most spontaneous reports come from medical staff and sometimes come from patients who use the drug. The posts published by individuals on social media platforms that contain drugs and related adverse reaction content have gradually been seen as a new data source similar to spontaneous reports from drug users in recent years. Those user-generated posts potentially provide researchers and regulators with new opportunities to conduct post-marketing surveillance for drug safety from patients' perspectives mostly rather than medical professionals and can afford the possibility theoretically to discover drug-related safety issues earlier than traditional methods. Social media data as a new data source for safety signal detection and signal reinforcement have the unique advantages, such as population coverage, type of drugs, type of adverse reactions, data timeliness and quantity. Most of the social media data used in post-marketing surveillance research for drug safety are still text data in English, and even multiple languages are used by different people worldwide on several social media platforms. Unfortunately, there is still a controversy in the academic circles whether social media data can be used as reliable data sources for routine post-marketing surveillance for drug safety. A couple of obstacles of data, methods and ethics must be overcome before leveraging social media data for post-marketing surveillance. The number of Chinese social media users is large, and the social media data in the Chinese language is rapidly snowballing, which can be employed as the potential data source for post-marketing surveillance for drug safety. However, due to the Chinese language's specific characteristics, the text's diversity is different from the English text, and there is not enough accepted corpus in medical scenarios. Besides, the lack of domestic laws and regulations on privacy and security protection of social media data poses more challenges for applying Chinese social media data for post-market surveillance. The significance of social media data to post-marketing surveillance for drug safety is undoubtedly significant. It will be an essential development direction for future research to overcome the challenges of using social media data by developing new technologies and establishing new mechanisms.
Collapse
Affiliation(s)
- 羽 杨
- 北京大学健康医疗大数据国家研究院, 北京 100191National Institute of Health Data Science, Peking University, Beijing 100191, China
| | - 胜锋 王
- 北京大学公共卫生学院流行病学与卫生统计学系, 北京 100191Department of Epidemiology and Biostatistics, Peking University School of Public Health, Beijing 100191, Chian
| | - 思延 詹
- 北京大学公共卫生学院流行病学与卫生统计学系, 北京 100191Department of Epidemiology and Biostatistics, Peking University School of Public Health, Beijing 100191, Chian
| |
Collapse
|
30
|
Siddiqui Z, Wang Y, Patel J, Thyvalikakath T. Differences in medication usage of dental patients by age, gender, race/ethnicity and insurance status. Technol Health Care 2021; 29:1099-1108. [PMID: 33896855 DOI: 10.3233/thc-202171] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
BACKGROUND Limited studies have investigated the medication profile of young adult dental patients despite the high prevalence of prescription opioid abuse in this population. OBJECTIVE This study investigated the extent and differences in medication usage of dental patients older than 18 years by age, race/ethnicity, gender, insurance status and mechanism of action in an academic dental clinic setting. METHODS Using an automated approach, medication names in the Electronic Dental Record were retrieved and classified according to the National Drug Code directory. Descriptive statistics, multivariable ANOVA and Post hoc tests were performed to detect differences in the number of medications by patient demographics. RESULTS Of the 11,220 adult patients, 53 percent reported taking at least one medication with significant differences in medication usage by demographics. Hydroxymethylglutaryl-coenzyme A reductase inhibitors (21-36%), and angiotensin-converting enzyme inhibitors (19-23%) ranked the top two medication classes among patients 55 years and older. Opioid agonists (7-14%), and Selective Serotonin Reuptake Inhibitors (SSRIs) (5-12%) ranked the top two medication classes among patients aged 18-54 years. CONCLUSIONS The results underscore the importance of dental providers to review medical and medication histories of patients regardless of their age to avoid adverse events and to determine patient's risk for opioid abuse.
Collapse
Affiliation(s)
- Zasim Siddiqui
- Department of Cariology, Operative Dentistry and Dental Public Health, Indiana University School of Dentistry, IUPUI Indianapolis, IN, USA
| | - Yue Wang
- Department of Cariology, Operative Dentistry and Dental Public Health, Indiana University School of Dentistry, IUPUI Indianapolis, IN, USA
| | - Jay Patel
- Department of Cariology, Operative Dentistry and Dental Public Health, Indiana University School of Dentistry, IUPUI Indianapolis, IN, USA
| | - Thankam Thyvalikakath
- Department of Cariology, Operative Dentistry and Dental Public Health, Indiana University School of Dentistry, IUPUI Indianapolis, IN, USA.,Research Scientist, Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, IN, USA
| |
Collapse
|
31
|
McLenon M, Okuhn S, Lancaster EM, Hull MM, Adams JL, McGlynn E, Avins AL, Chang RW. Validation of natural language processing to determine the presence and size of abdominal aortic aneurysms in a large integrated health system. J Vasc Surg 2021; 74:459-466.e3. [PMID: 33548429 DOI: 10.1016/j.jvs.2020.12.090] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Accepted: 12/23/2020] [Indexed: 01/31/2023]
Abstract
OBJECTIVE Previous studies of the natural history of abdominal aortic aneurysms (AAAs) have been limited by small cohort sizes or heterogeneous analyses of pooled data. By quickly and efficiently extracting imaging data from the health records, natural language processing (NLP) has the potential to substantially improve how we study and care for patients with AAAs. The aim of the present study was to test the ability of an NLP tool to accurately identify the presence or absence of AAAs and detect the maximal abdominal aortic diameter in a large dataset of imaging study reports. METHODS Relevant imaging study reports (n = 230,660) from 2003 to 2017 were obtained for 32,778 patients followed up in a prospective aneurysm surveillance registry within a large, diverse, integrated healthcare system. A commercially available NLP algorithm was used to assess the presence of AAAs, confirm the absence of AAAs, and extract the maximal diameter of the abdominal aorta, if stated. A blinded expert manual review of 18,000 randomly selected imaging reports was used as the reference standard. The positive predictive value (PPV or precision), sensitivity (recall), and the kappa statistics were calculated. RESULTS Of the randomly selected 18,000 studies that underwent expert manual review, 48.7% were positive for AAAs. In confirming the presence of an AAA, the interrater reliability of the NLP compared with the expert review showed a kappa value of 0.84 (95% confidence interval [CI], 0.83-0.85), with a PPV of 95% and sensitivity of 88.5%. The NLP algorithm showed similar results for confirming the absence of an AAA, with a kappa of 0.79 (95% CI, 0.799-0.80), PPV of 77.7%, and sensitivity of 91.9%. The kappa, PPV, and sensitivity of the NLP for correctly identifying the maximal aortic diameter was 0.88 (95% CI, 0.87-0.89), 88.8%, and 88.2% respectively. CONCLUSIONS The use of NLP software can accurately analyze large volumes of radiology report data to detect AAA disease and assemble a contemporary aortic diameter-based cohort of patients for longitudinal analysis to guide surveillance, medical management, and operative decision making. It can also potentially be used to identify from the electronic medical records pre- and postoperative AAA patients "lost to follow-up," leverage human resources engaged in the ongoing surveillance of patients with AAAs, and facilitate the construction and implementation of AAA screening programs.
Collapse
Affiliation(s)
| | - Steven Okuhn
- Division of Vascular Surgery, Department of Surgery, Veterans Affairs San Francisco Healthcare System, San Francisco, Calif; Division of Vascular Surgery, Department of Surgery, University of California, San Francisco, San Francisco, Calif
| | - Elizabeth M Lancaster
- Division of Vascular Surgery, Department of Surgery, University of California, San Francisco, San Francisco, Calif
| | - Michaela M Hull
- Kaiser Permanente Center for Effectiveness and Safety Research, Pasadena, Calif
| | - John L Adams
- Kaiser Permanente Center for Effectiveness and Safety Research, Pasadena, Calif
| | - Elizabeth McGlynn
- Kaiser Permanente Center for Effectiveness and Safety Research, Pasadena, Calif
| | - Andrew L Avins
- Division of Research, Kaiser Permanente Northern California, Oakland, Calif; Department of Medicine, University of California, San Francisco, San Francisco, Calif; Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, Calif
| | - Robert W Chang
- Division of Research, Kaiser Permanente Northern California, Oakland, Calif; Division of Vascular Surgery, Department of Surgery, The Permanente Medical Group, South San Francisco, Calif.
| |
Collapse
|
32
|
Bacchi S, Gluck S, Tan Y, Chim I, Cheng J, Gilbert T, Menon DK, Jannes J, Kleinig T, Koblar S. Prediction of general medical admission length of stay with natural language processing and deep learning: a pilot study. Intern Emerg Med 2020; 15:989-995. [PMID: 31898204 DOI: 10.1007/s11739-019-02265-3] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Accepted: 12/17/2019] [Indexed: 12/25/2022]
Abstract
Length of stay (LOS) and discharge destination predictions are key parts of the discharge planning process for general medical hospital inpatients. It is possible that machine learning, using natural language processing, may be able to assist with accurate LOS and discharge destination prediction for this patient group. Emergency department triage and doctor notes were retrospectively collected on consecutive general medical and acute medical unit admissions to a single tertiary hospital from a 2-month period in 2019. These data were used to assess the feasibility of predicting LOS and discharge destination using natural language processing and a variety of machine learning models. 313 patients were included in the study. The artificial neural network achieved the highest accuracy on the primary outcome of predicting whether a patient would remain in hospital for > 2 days (accuracy 0.82, area under the received operator curve 0.75, sensitivity 0.47 and specificity 0.97). When predicting LOS as an exact number of days, the artificial neural network achieved a mean absolute error of 2.9 and a mean squared error of 16.8 on the test set. For the prediction of home as a discharge destination (vs any non-home alternative), all models performed similarly with an accuracy of approximately 0.74. This study supports the feasibility of using natural language processing to predict general medical inpatient LOS and discharge destination. Further research is indicated with larger, more detailed, datasets from multiple centres to optimise and examine the accuracy that may be achieved with such predictions.
Collapse
Affiliation(s)
- Stephen Bacchi
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia.
- University of Adelaide, Adelaide, SA, 5005, Australia.
| | - Samuel Gluck
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| | - Yiran Tan
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| | - Ivana Chim
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
| | - Joy Cheng
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
| | - Toby Gilbert
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| | - David K Menon
- Division of Anaesthesia, University of Cambridge, Cambridge, CB2 0QQ, UK
| | - Jim Jannes
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| | - Timothy Kleinig
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| | - Simon Koblar
- Neurology Department, Royal Adelaide Hospital, Port Road, Adelaide, SA, 5000, Australia
- University of Adelaide, Adelaide, SA, 5005, Australia
| |
Collapse
|
33
|
The present and future role of artificial intelligence and machine learning in anesthesiology. Int Anesthesiol Clin 2020; 58:7-16. [PMID: 32841964 DOI: 10.1097/aia.0000000000000294] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
34
|
Straub L, Gagne JJ, Maro JC, Nguyen MD, Beaulieu N, Brown JS, Kennedy A, Johnson M, Wright A, Zhou L, Wang SV. Evaluation of Use of Technologies to Facilitate Medical Chart Review. Drug Saf 2020; 42:1071-1080. [PMID: 31111340 DOI: 10.1007/s40264-019-00838-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
INTRODUCTION While medical chart review remains the gold standard to validate health conditions or events identified in administrative claims and electronic health record databases, it is time consuming, expensive and can involve subjective decisions. AIM The aim of this study was to describe the landscape of technology-enhanced approaches that could be used to facilitate medical chart review within and across distributed data networks. METHOD We conducted a semi-structured survey regarding processes for medical chart review with organizations that either routinely do medical chart review or use technologies that could facilitate chart review. RESULTS Fifteen out of 17 interviewed organizations used optical character recognition (OCR) or natural language processing (NLP) in their chart review process. None used handwriting recognition software. While these organizations found OCR and NLP to be useful for expediting extraction of useful information from medical charts, they also mentioned several challenges. Quality of medical scans can be variable, interfering with the accuracy of OCR. Additionally, linguistic complexity in medical notes and heterogeneity in reporting templates used by different healthcare systems can reduce the transportability of NLP-based algorithms to diverse healthcare settings. CONCLUSION New technologies including OCR and NLP are currently in use by various organizations involved in medical chart review. While technology-enhanced approaches could scale up capacity to validate key variables and make information about important clinical variables from medical records more generally available for research purposes, they often require considerable customization when employed in a distributed data environment with multiple, diverse healthcare settings.
Collapse
Affiliation(s)
- Loreen Straub
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont Street, Suite 3030, Boston, MA, 02120, USA.
| | - Joshua J Gagne
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont Street, Suite 3030, Boston, MA, 02120, USA
| | - Judith C Maro
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA, USA
| | - Michael D Nguyen
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Nicolas Beaulieu
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA, USA
| | - Jeffrey S Brown
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA, USA
| | - Adee Kennedy
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA, USA
| | - Margaret Johnson
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, MA, USA
| | - Adam Wright
- Division of General Internal Medicine, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Li Zhou
- Division of General Internal Medicine, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Shirley V Wang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont Street, Suite 3030, Boston, MA, 02120, USA
| |
Collapse
|
35
|
Lewis DJ, McCallum JF. Utilizing Advanced Technologies to Augment Pharmacovigilance Systems: Challenges and Opportunities. Ther Innov Regul Sci 2019; 54:888-899. [PMID: 32557311 PMCID: PMC7362887 DOI: 10.1007/s43441-019-00023-3] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2019] [Accepted: 11/04/2019] [Indexed: 01/01/2023]
Abstract
There are significant challenges and opportunities in deploying and utilizing advanced information technology (IT) within pharmacovigilance (PV) systems and across the pharmaceutical industry. Various aspects of PV will benefit from automation (e.g., by improving standardization or increasing data quality). Several themes are developed, highlighting the challenges faced, exploring solutions, and assessing the potential for further research. Automation of the workflow for processing of individual case safety reports (ICSRs) is adopted as a use case. This involves a logical progression through a series of steps that when linked together comprise the complete work process required for the effective management of ICSRs. We recognize that the rapid development of new technologies will invariably outpace the regulations applicable to PV systems. Nevertheless, we believe that such systems may be improved by intelligent automation. It is incumbent on the owners of these systems to explore opportunities presented by new technologies with regulators in order to evaluate the applicability, design, deployment, performance, validation and maintenance of advanced technologies to ensure that the PV system continues to be fit for purpose. Proposed approaches to the validation of automated PV systems are presented. A series of definitions and a critical appraisal of important considerations are provided in the form of use cases. We summarize progress made and opportunities for the development of automation of future systems. The overall goal of automation is to provide high quality safety data in the correct format, in context, more quickly, and with less manual effort. This will improve the evidence available for scientific assessment and helps to inform and expedite decisions about the minimization of risks associated with medicines.
Collapse
Affiliation(s)
- David John Lewis
- Novartis Global Drug Development, Novartis Pharma GmbH, Oeflinger Strasse 44, D-79664, Wehr, Germany. .,Department of Pharmacy, Pharmacology and Postgraduate Medicine, University of Hertfordshire, Hatfield, Hertfordshire, AL10 9AB, UK.
| | - John Fraser McCallum
- Product Development Safety Risk Management, Roche Products Limited, 6 Falcon Way, Shire Park, Welwyn Garden City, Hertfordshire, AL7 1TW, UK
| |
Collapse
|
36
|
Natural Language Processing Combined with ICD-9-CM Codes as a Novel Method to Study the Epidemiology of Allergic Drug Reactions. THE JOURNAL OF ALLERGY AND CLINICAL IMMUNOLOGY-IN PRACTICE 2019; 8:1032-1038.e1. [PMID: 31857264 DOI: 10.1016/j.jaip.2019.12.007] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/24/2019] [Revised: 11/25/2019] [Accepted: 12/02/2019] [Indexed: 11/20/2022]
Abstract
BACKGROUND Allergic drug reaction epidemiologic data are sparse because it remains difficult to identify true cases in large data sets using manual chart review. OBJECTIVE To develop and validate a novel informatics method based on natural language processing (NLP) in combination with International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes that identifies allergic drug reactions in the electronic health record. METHODS Previously studied and high-yield ICD-9-CM codes were used to screen for possible allergic drug reactions among all inpatients admitted in 2007 and 2008. A random sample was selected for manual chart review to identify true cases of allergic drug reactions. A rule-based NLP algorithm was then developed to identify allergic drug reactions using free-text clinical notes and discharge summaries from the filtered cases. The performance of using manual chart review of ICD-9-CM codes alone was compared with ICD-9-CM codes in combination with NLP. RESULTS Of 3907 cases identified by ICD-9-CM codes, 725 (19%) were randomly selected for manual chart review; 335 were confirmed as allergic drug reactions, resulting in a positive predictive value (PPV) of 46% (range: 18%-79%) when using ICD-9-CM codes alone. Our NLP algorithm in combination with ICD-9-CM codes achieved a PPV of 86% (range: 69%-100%). Among the 335 confirmed positive cases, NLP identified 259 true cases, resulting in a recall/sensitivity of 77% (range: 26%-100%). Among the 390 negative cases, NLP achieved a specificity of 89% (range: 69%-100%). CONCLUSION Using NLP with ICD-9-CM codes improved identification of allergic drug reactions. The resulting decrease in manual chart review effort will facilitate large epidemiology studies of this understudied area.
Collapse
|
37
|
Li R, Zaidi STR, Chen T, Castelino R. Effectiveness of interventions to improve adverse drug reaction reporting by healthcare professionals over the last decade: A systematic review. Pharmacoepidemiol Drug Saf 2019; 29:1-8. [DOI: 10.1002/pds.4906] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Revised: 08/27/2019] [Accepted: 09/11/2019] [Indexed: 12/17/2022]
Affiliation(s)
- Raymond Li
- Faculty of Medicine and HealthUniversity of Sydney Sydney Australia
| | - Syed Tabish Razi Zaidi
- Faculty of Medicine and HealthUniversity of Leeds Leeds England
- National Institute for Health Research (NIHR) Yorkshire and Humber Patient Safety Translational Research Centre (NIHR Yorkshire and Humber PSTRC) West Yorkshire England
| | - Timothy Chen
- Faculty of Medicine and HealthUniversity of Sydney Sydney Australia
| | - Ronald Castelino
- Faculty of Medicine and HealthUniversity of Sydney Sydney Australia
| |
Collapse
|
38
|
Shull JG. Digital Health and the State of Interoperable Electronic Health Records. JMIR Med Inform 2019; 7:e12712. [PMID: 31682583 PMCID: PMC6913749 DOI: 10.2196/12712] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Revised: 05/29/2019] [Accepted: 09/02/2019] [Indexed: 12/16/2022] Open
Abstract
Digital health systems and innovative care delivery within these systems have great potential to improve national health care and positively impact the health outcomes of patients. However, currently, very few countries have systems that can implement digital interventions at scale. This is partly because of the lack of interoperable electronic health records (EHRs). It is difficult to make decisions for an individual or population when the data on that person or population are dispersed over multiple incompatible systems.
This viewpoint paper has highlighted some key obstacles of current EHRs and some promising successes, with the goal of promoting EHR evolution and advocating for frameworks that develop digital health systems that serve populations—a critical goal as we move further into this data-rich century with an ever-increasing number of patients who live longer and depend on health care services where resources may already be strained.
This paper aimed to analyze the evolution, obstacles, and current landscape of EHRs and identify fundamental areas of hindrance for interoperability. It also aimed to highlight countries where advances have been made and extract best practices from these examples.
The obstacles to EHR interoperability are not easily solved, but improving the current situation in countries where a national policy is not in place will require a focused inquiry into solutions from various sources in the public and private sector.
Effort must be made on a national scale to seek solutions for optimally interoperable EHRs beyond status quo solutions. A list of considerations for best practices is suggested.
Collapse
Affiliation(s)
- Jessica Germaine Shull
- PhD Program, Department of Biomedicine, University of Barcelona, Barcelona, Spain.,Institute for Biomedical Research Bellvitge, Hospital Bellvitge, Barcelona, Spain
| |
Collapse
|
39
|
Executive summary of the 2019 ASHP Commission on Goals: Impact of artificial intelligence on healthcare and pharmacy practice. Am J Health Syst Pharm 2019; 76:2087-2092. [DOI: 10.1093/ajhp/zxz205] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
|
40
|
Wang SV, Patterson OV, Gagne JJ, Brown JS, Ball R, Jonsson P, Wright A, Zhou L, Goettsch W, Bate A. Transparent Reporting on Research Using Unstructured Electronic Health Record Data to Generate ‘Real World’ Evidence of Comparative Effectiveness and Safety. Drug Saf 2019; 42:1297-1309. [DOI: 10.1007/s40264-019-00851-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|
41
|
Cortes D, Leung J, Ryl A, Lieu J. Pharmacy Informatics: Where Medication Use and Technology Meet. Can J Hosp Pharm 2019; 72:320-326. [PMID: 31452544 PMCID: PMC6699873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Affiliation(s)
- Daniel Cortes
- BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, ACPR, RPh, is with Clinical Informatics, St Joseph's Health Centre, Unity Health Toronto, Toronto, Ontario
| | - Jodie Leung
- BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, ACPR, RPh, is with Clinical Informatics, St Joseph's Health Centre, Unity Health Toronto, Toronto, Ontario
| | - Andrea Ryl
- BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, ACPR, RPh, is with Clinical Informatics, St Joseph's Health Centre, Unity Health Toronto, Toronto, Ontario
| | - Jenny Lieu
- BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department and Clinical Informatics, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, RPh, is with the Pharmacy Department, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario BScPhm, ACPR, RPh, is with Clinical Informatics, St Joseph's Health Centre, Unity Health Toronto, Toronto, Ontario
| |
Collapse
|
42
|
Blumenthal KG, Topaz M, Zhou L, Harkness T, Sa'adon R, Bar-Bachar O, Long AA. Mining social media data to assess the risk of skin and soft tissue infections from allergen immunotherapy. J Allergy Clin Immunol 2019; 144:129-134. [PMID: 30721764 PMCID: PMC6612292 DOI: 10.1016/j.jaci.2019.01.029] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2018] [Revised: 01/02/2019] [Accepted: 01/23/2019] [Indexed: 11/18/2022]
Abstract
BACKGROUND Allergen immunotherapy (AIT) treatment for allergic rhinitis and asthma is used by 2.6 million Americans annually. Clinical and sterility testing studies identify no risk of contamination or infection from extracts prepared using recommended aseptic techniques, but regulatory concerns persist. Social media can be used to investigate rare adverse effects not captured by traditional studies. OBJECTIVE We sought to investigate large social media databases for suggestion of AIT skin and soft tissue infection (SSTI) risk and compare this risk to a comparator procedure with a sterile pharmaceutical. METHODS We analyzed US-restricted data from more than 10 common text-based social media platforms including Facebook, Twitter, and Reddit between 2012 and 2016. We used natural language processing (NLP) to identify posts related to AIT and, separately, influenza vaccination (comparator procedure). NLP was followed by manual review to identify posts suggesting a possible SSTI associated with either AIT or influenza vaccination. SSTI frequencies with 95% CIs were compared. RESULTS We identified 25,126 AIT posts, which were matched by social media platform to 25,126 influenza vaccination-related posts. NLP identified 4088 (16.3%) AIT posts that required manual review, with 6 posts (0.02%; 95% CI, 0.005%-0.043%) indicative of possible AIT-related SSTI. NLP identified 2689 (10.7%) influenza posts that required manual review, with 7 posts (0.03%; 95% CI, 0.007%-0.048%) indicative of possible influenza vaccination-related SSTI. CONCLUSIONS Social media data suggest that SSTI from AIT and influenza vaccination are equally rare events. Given that AIT's SSTI risk appears comparable to the risk using a sterile pharmaceutical based on social media data, current aseptic technique procedures seem safe.
Collapse
Affiliation(s)
- Kimberly G Blumenthal
- Division of Rheumatology, Allergy and Immunology, Massachusetts General Hospital, Boston, Mass; Harvard Medical School, Boston, Mass
| | - Maxim Topaz
- Cheryl Spencer Department of Nursing, University of Haifa, Haifa, Israel; Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Mass
| | - Li Zhou
- Harvard Medical School, Boston, Mass; Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Mass
| | - Tyler Harkness
- Division of Rheumatology, Allergy and Immunology, Massachusetts General Hospital, Boston, Mass
| | | | - Ofrit Bar-Bachar
- Cheryl Spencer Department of Nursing, University of Haifa, Haifa, Israel
| | - Aidan A Long
- Division of Rheumatology, Allergy and Immunology, Massachusetts General Hospital, Boston, Mass; Harvard Medical School, Boston, Mass.
| |
Collapse
|
43
|
Beck EM, Hatton ND, Ryan JJ. Novel techniques for advancing our understanding of pulmonary arterial hypertension. Eur Respir J 2019; 53:53/5/1900556. [DOI: 10.1183/13993003.00556-2019] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2019] [Accepted: 03/20/2019] [Indexed: 01/18/2023]
|
44
|
Arnoux-Guenegou A, Girardeau Y, Chen X, Deldossi M, Aboukhamis R, Faviez C, Dahamna B, Karapetiantz P, Guillemin-Lanne S, Lillo-Le Louët A, Texier N, Burgun A, Katsahian S. The Adverse Drug Reactions From Patient Reports in Social Media Project: Protocol for an Evaluation Against a Gold Standard. JMIR Res Protoc 2019; 8:e11448. [PMID: 31066711 PMCID: PMC6528435 DOI: 10.2196/11448] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2018] [Revised: 11/16/2018] [Accepted: 12/21/2018] [Indexed: 12/30/2022] Open
Abstract
Background Social media is a potential source of information on postmarketing drug safety surveillance that still remains unexploited nowadays. Information technology solutions aiming at extracting adverse reactions (ADRs) from posts on health forums require a rigorous evaluation methodology if their results are to be used to make decisions. First, a gold standard, consisting of manual annotations of the ADR by human experts from the corpus extracted from social media, must be implemented and its quality must be assessed. Second, as for clinical research protocols, the sample size must rely on statistical arguments. Finally, the extraction methods must target the relation between the drug and the disease (which might be either treated or caused by the drug) rather than simple co-occurrences in the posts. Objective We propose a standardized protocol for the evaluation of a software extracting ADRs from the messages on health forums. The study is conducted as part of the Adverse Drug Reactions from Patient Reports in Social Media project. Methods Messages from French health forums were extracted. Entity recognition was based on Racine Pharma lexicon for drugs and Medical Dictionary for Regulatory Activities terminology for potential adverse events (AEs). Natural language processing–based techniques automated the ADR information extraction (relation between the drug and AE entities). The corpus of evaluation was a random sample of the messages containing drugs and/or AE concepts corresponding to recent pharmacovigilance alerts. A total of 2 persons experienced in medical terminology manually annotated the corpus, thus creating the gold standard, according to an annotator guideline. We will evaluate our tool against the gold standard with recall, precision, and f-measure. Interannotator agreement, reflecting gold standard quality, will be evaluated with hierarchical kappa. Granularities in the terminologies will be further explored. Results Necessary and sufficient sample size was calculated to ensure statistical confidence in the assessed results. As we expected a global recall of 0.5, we needed at least 384 identified ADR concepts to obtain a 95% CI with a total width of 0.10 around 0.5. The automated ADR information extraction in the corpus for evaluation is already finished. The 2 annotators already completed the annotation process. The analysis of the performance of the ADR information extraction module as compared with gold standard is ongoing. Conclusions This protocol is based on the standardized statistical methods from clinical research to create the corpus, thus ensuring the necessary statistical power of the assessed results. Such evaluation methodology is required to make the ADR information extraction software useful for postmarketing drug safety surveillance. International Registered Report Identifier (IRRID) RR1-10.2196/11448
Collapse
Affiliation(s)
- Armelle Arnoux-Guenegou
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France
| | - Yannick Girardeau
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France.,Département d'Informatique Médicale, Hôpital Européen Georges-Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France
| | - Xiaoyi Chen
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France
| | | | - Rim Aboukhamis
- Centre Régional de Pharmacovigilance, Hôpital Européen Georges-Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France
| | | | - Badisse Dahamna
- Service d'Informatique Biomédicale, D2IM, Centre Hospitalier Universitaire de Rouen, Rouen, France
| | - Pierre Karapetiantz
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France
| | | | - Agnès Lillo-Le Louët
- Centre Régional de Pharmacovigilance, Hôpital Européen Georges-Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France
| | | | - Anita Burgun
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France.,Département d'Informatique Médicale, Hôpital Européen Georges-Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France.,INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Paris Descartes University, Sorbonne Paris Cité, Paris, France
| | - Sandrine Katsahian
- INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Centre de Recherche des Cordeliers, Paris, France.,INSERM U1138 - Team 22, Information Sciences to Support Personalized Medicine, Paris Descartes University, Sorbonne Paris Cité, Paris, France.,Clinical Research Unit Hôpitaux Universitaires Paris Ouest, Hôpital Européen Georges-Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France.,INSERM CIC1418, Clinical Epidemiology, Hôpital Européen Georges-Pompidou, Paris, France
| |
Collapse
|
45
|
Pendergrass SA, Crawford DC. Using Electronic Health Records To Generate Phenotypes For Research. CURRENT PROTOCOLS IN HUMAN GENETICS 2019; 100:e80. [PMID: 30516347 PMCID: PMC6318047 DOI: 10.1002/cphg.80] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Electronic health records contain patient-level data collected during and for clinical care. Data within the electronic health record include diagnostic billing codes, procedure codes, vital signs, laboratory test results, clinical imaging, and physician notes. With repeated clinic visits, these data are longitudinal, providing important information on disease development, progression, and response to treatment or intervention strategies. The near universal adoption of electronic health records nationally has the potential to provide population-scale real-world clinical data accessible for biomedical research, including genetic association studies. For this research potential to be realized, high-quality research-grade variables must be extracted from these clinical data warehouses. We describe here common and emerging electronic phenotyping approaches applied to electronic health records, as well as current limitations of both the approaches and the biases associated with these clinically collected data that impact their use in research. © 2018 by John Wiley & Sons, Inc.
Collapse
Affiliation(s)
- Sarah A. Pendergrass
- Biomedical and Translational Informatics Institute,
Geisinger Research, Rockville MD
| | - Dana C. Crawford
- Institute for Computational Biology, Department of
Population and Quantitative Health Sciences, Case Western Reserve University,
Cleveland, OH
| |
Collapse
|
46
|
Usui M, Aramaki E, Iwao T, Wakamiya S, Sakamoto T, Mochizuki M. Extraction and Standardization of Patient Complaints from Electronic Medication Histories for Pharmacovigilance: Natural Language Processing Analysis in Japanese. JMIR Med Inform 2018; 6:e11021. [PMID: 30262450 PMCID: PMC6231790 DOI: 10.2196/11021] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2018] [Revised: 08/07/2018] [Accepted: 08/25/2018] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Despite the growing number of studies using natural language processing for pharmacovigilance, there are few reports on manipulating free text patient information in Japanese. OBJECTIVE This study aimed to establish a method of extracting and standardizing patient complaints from electronic medication histories accumulated in a Japanese community pharmacy for the detection of possible adverse drug event (ADE) signals. METHODS Subjective information included in electronic medication history data provided by a Japanese pharmacy operating in Hiroshima, Japan from September 1, 2015 to August 31, 2016, was used as patients' complaints. We formulated search rules based on morphological analysis and daily (nonmedical) speech and developed a system that automatically executes the search rules and annotates free text data with International Classification of Diseases, Tenth Revision (ICD-10) codes. The performance of the system was evaluated through comparisons with data manually annotated by health care workers for a data set of 5000 complaints. RESULTS Of 5000 complaints, the system annotated 2236 complaints with ICD-10 codes, whereas health care workers annotated 2348 statements. There was a match in the annotation of 1480 complaints between the system and manual work. System performance was .66 regarding precision, .63 in recall, and .65 for the F-measure. CONCLUSIONS Our results suggest that the system may be helpful in extracting and standardizing patients' speech related to symptoms from massive amounts of free text data, replacing manual work. After improving the extraction accuracy, we expect to utilize this system to detect signals of possible ADEs from patients' complaints in the future.
Collapse
Affiliation(s)
- Misa Usui
- Division of Hospital Pharmacy Science, Graduate School of Pharmaceutical Sciences, Keio University, Tokyo, Japan
| | - Eiji Aramaki
- Social Computing Lab, Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan
| | - Tomohide Iwao
- Social Computing Lab, Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan
| | - Shoko Wakamiya
- Social Computing Lab, Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan
| | | | - Mayumi Mochizuki
- Division of Hospital Pharmacy Science, Faculty of Pharmacy, Keio University, Tokyo, Japan.,Department of Pharmacy, Keio University Hospital, Tokyo, Japan
| |
Collapse
|
47
|
Kane-Gill SL. Innovations in Medication Safety: Services and Technologies to Enhance the Understanding and Prevention of Adverse Drug Reactions. Pharmacotherapy 2018; 38:782-784. [PMID: 30033608 DOI: 10.1002/phar.2154] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Affiliation(s)
- Sandra L Kane-Gill
- Department of Pharmacy and Therapeutics, University of Pittsburgh School of Pharmacy, Pittsburgh, PA, USA.,Department of Pharmacy, UPMC Presbyterian Shadyside, Pittsburgh, PA, USA
| |
Collapse
|