1
|
Saikali M, Békarian G, Khabouth J, Mourad C, Saab A. Automated Detection of Patient Harm: Implementation and Prospective Evaluation of a Real-Time Broad-Spectrum Surveillance Application in a Hospital With Limited Resources. J Patient Saf 2023; 19:128-136. [PMID: 36622740 DOI: 10.1097/pts.0000000000001096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
OBJECTIVES This study aimed to prospectively validate an application that automates the detection of broad categories of hospital adverse events (AEs) extracted from a basic hospital information system, and to efficiently mobilize resources to reduce the level of acquired patient harm. METHODS Data were collected from an internally designed software, extracting results from 14 triggers indicative of patient harm, querying clinical and administrative databases including all inpatient admissions (n = 8760) from October 2019 to June 2020. Representative samples of the triggered cases were clinically validated using chart review by a consensus expert panel. The positive predictive value (PPV) of each trigger was evaluated, and the detection sensitivity of the surveillance system was estimated relative to incidence ranges in the literature. RESULTS The system identified 394 AEs among 946 triggered cases, associated with 291 patients, yielding an overall PPV of 42%. Variability was observed among the trigger PPVs and among the estimated detection sensitivities across the harm categories, the highest being for the healthcare-associated infections. The median length of stay of patients with an AE showed to be significantly higher than the median for the overall patient population. CONCLUSIONS This application was able to identify AEs across a broad spectrum of harm categories, in a real-time manner, while reducing the use of resources required by other harm detection methods. Such a system could serve as a promising patient safety tool for AE surveillance, allowing for timely, targeted, and resource-efficient interventions, even for hospitals with limited resources.
Collapse
Affiliation(s)
- Melody Saikali
- From the Quality and Patient Safety Department, Lebanese Hospital Geitaoui-University Medical Center
| | - Gariné Békarian
- From the Quality and Patient Safety Department, Lebanese Hospital Geitaoui-University Medical Center
| | - José Khabouth
- Department of Internal Medicine, Faculty of Medicine, Lebanese University, Beirut, Lebanon
| | - Charbel Mourad
- Department of Medical Imaging, Faculty of Medicine, Lebanese University, Beirut, Lebanon
| | | |
Collapse
|
2
|
Weeks HL, Beck C, McNeer E, Williams ML, Bejan CA, Denny JC, Choi L. medExtractR: A targeted, customizable approach to medication extraction from electronic health records. J Am Med Inform Assoc 2021; 27:407-418. [PMID: 31943012 DOI: 10.1093/jamia/ocz207] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2019] [Revised: 11/08/2019] [Accepted: 11/22/2019] [Indexed: 12/11/2022] Open
Abstract
OBJECTIVE We developed medExtractR, a natural language processing system to extract medication information from clinical notes. Using a targeted approach, medExtractR focuses on individual drugs to facilitate creation of medication-specific research datasets from electronic health records. MATERIALS AND METHODS Written using the R programming language, medExtractR combines lexicon dictionaries and regular expressions to identify relevant medication entities (eg, drug name, strength, frequency). MedExtractR was developed on notes from Vanderbilt University Medical Center, using medications prescribed with varying complexity. We evaluated medExtractR and compared it with 3 existing systems: MedEx, MedXN, and CLAMP (Clinical Language Annotation, Modeling, and Processing). We also demonstrated how medExtractR can be easily tuned for better performance on an outside dataset using the MIMIC-III (Medical Information Mart for Intensive Care III) database. RESULTS On 50 test notes per development drug and 110 test notes for an additional drug, medExtractR achieved high overall performance (F-measures >0.95), exceeding performance of the 3 existing systems across all drugs. MedExtractR achieved the highest F-measure for each individual entity, except drug name and dose amount for allopurinol. With tuning and customization, medExtractR achieved F-measures >0.90 in the MIMIC-III dataset. DISCUSSION The medExtractR system successfully extracted entities for medications of interest. High performance in entity-level extraction provides a strong foundation for developing robust research datasets for pharmacological research. When working with new datasets, medExtractR should be tuned on a small sample of notes before being broadly applied. CONCLUSIONS The medExtractR system achieved high performance extracting specific medications from clinical text, leading to higher-quality research datasets for drug-related studies than some existing general-purpose medication extraction tools.
Collapse
Affiliation(s)
- Hannah L Weeks
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Cole Beck
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Elizabeth McNeer
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Michael L Williams
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Cosmin A Bejan
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Joshua C Denny
- Department of Biomedical Informatics, Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Leena Choi
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
3
|
Information Extraction from Electronic Medical Records Using Multitask Recurrent Neural Network with Contextual Word Embedding. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9183658] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Clinical named entity recognition is an essential task for humans to analyze large-scale electronic medical records efficiently. Traditional rule-based solutions need considerable human effort to build rules and dictionaries; machine learning-based solutions need laborious feature engineering. For the moment, deep learning solutions like Long Short-term Memory with Conditional Random Field (LSTM–CRF) achieved considerable performance in many datasets. In this paper, we developed a multitask attention-based bidirectional LSTM–CRF (Att-biLSTM–CRF) model with pretrained Embeddings from Language Models (ELMo) in order to achieve better performance. In the multitask system, an additional task named entity discovery was designed to enhance the model’s perception of unknown entities. Experiments were conducted on the 2010 Informatics for Integrating Biology & the Bedside/Veterans Affairs (I2B2/VA) dataset. Experimental results show that our model outperforms the state-of-the-art solution both on the single model and ensemble model. Our work proposes an approach to improve the recall in the clinical named entity recognition task based on the multitask mechanism.
Collapse
|
4
|
Nakayama JY, Hertzberg V, Ho JC. Making sense of abbreviations in nursing notes: A case study on mortality prediction. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2019; 2019:275-284. [PMID: 31258980 PMCID: PMC6568120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Unstructured data from electronic health records hold potential for improving predictive models for health outcomes. Efforts to extract structured information from the unstructured data used text mining methodologies, such as topic modeling and sentiment analysis. However, such methods do not account for abbreviations. Nursing notes have valuable information about nurses' assessments and interventions, and the abbreviation use is common. Thus, abbreviation disambiguation may add more insight when using unstructured text for predictive modeling. We present a new process to extract structured information from nursing notes through abbreviation normalization, lemmatization, and stop word removal. Our study found that abbreviation disambiguation in nursing notes for subsequent topic modeling and sentiment analysis improved prediction of in-hospital and 30-day mortality while controlling for comorbidity.
Collapse
Affiliation(s)
| | - Vicki Hertzberg
- Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, GA
- Department of Computer Science, Emory University, Atlanta, GA
| | - Joyce C Ho
- Department of Computer Science, Emory University, Atlanta, GA
| |
Collapse
|
5
|
Affiliation(s)
- Yoav Mintz
- Department of General Surgery, Hadassah Hebrew-University Medical Center, Jerusalem, Israel
| | - Ronit Brodie
- Department of General Surgery, Hadassah Hebrew-University Medical Center, Jerusalem, Israel
| |
Collapse
|
6
|
Huang Y, Fried LF, Kyriakides TC, Johnson GR, Chiu S, Mcdonald L, Zhang JH. Automated safety event monitoring using electronic medical records in a clinical trial setting: Validation study using the VA NEPHRON-D trial. Clin Trials 2018; 16:81-89. [PMID: 30445841 DOI: 10.1177/1740774518813121] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background/Aims: Electronic medical records are now frequently used for capturing patient-level data in clinical trials. Within the Veterans Affairs health care system, electronic medical record data have been widely used in clinical trials to assess eligibility, facilitate referrals for recruitment, and conduct follow-up and safety monitoring. Despite the potential for increased efficiency in using electronic medical records to capture safety data via a centralized algorithm, it is important to evaluate the integrity and accuracy of electronic medical record–captured data. To this end, this investigation assesses data collection, both for general and study-specific safety endpoints, by comparing electronic medical record–based safety monitoring versus safety data collected during the course of the Veterans Affairs Nephropathy in Diabetes (VA NEPHRON-D) clinical trial. Methods: The VA NEPHRON-D study was a multicenter, double-blind, randomized clinical trial designed to compare the effect of combination therapy (losartan plus lisinopril) versus monotherapy (losartan) on the progression of kidney disease in individuals with diabetes and proteinuria. The trial’s safety outcomes included serious adverse events, hyperkalemia, and acute kidney injury. A subset of the participants (~62%, n = 895) enrolled in the trial’s long-term follow-up sub-study and consented to electronic medical record data collection. We applied an automated algorithm to search and capture safety data using the VA Corporate Data Warehouse which houses electronic medical record data. Using study safety data reported during the trial as the gold standard, we evaluated the sensitivity and precision of electronic medical record–based safety data and related treatment effects. Results: The sensitivity of the electronic medical record–based safety for hospitalizations was 65.3% without non-VA hospitalization events and 92.3% with the non-VA hospitalization events included. The sensitivity was only 54.3% for acute kidney injury and 87.3% for hyperkalemia. The precision of electronic medical record–based safety data was 89.4%, 38%, and 63.2% for hospitalization, acute kidney injury, and hyperkalemia, respectively. Relative treatment differences under the study and electronic medical record settings were 15% and 3% for hospitalization, 123% and 29% for acute kidney injury, and 238% and 140% for hyperkalemia, respectively. Conclusion: The accuracy of using automated electronic medical record safety data depends on the events of interest. Identification of all-cause hospitalizations would be reliable if search methods could, in addition to VA hospitalizations, also capture non-VA hospitalizations. However, hospitalization is different from a cause-specific serious adverse event that could be more sensitive to treatment effects. In addition, some study-specific safety events were not easily identified using the electronic medical records. This limits the effectiveness of the automated central database search for purposes of safety monitoring. Hence, this data captured approach should be carefully considered when implementing endpoint data collection in future pragmatic trials.
Collapse
Affiliation(s)
- Yuan Huang
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
- Department of Biostatistics, The University of Iowa, Iowa City, IA, USA
| | - Linda F Fried
- Renal Section, VA Pittsburgh Healthcare System and Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | - Tassos C Kyriakides
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
| | - Gary R Johnson
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
| | - Susannah Chiu
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
| | - Linda Mcdonald
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
| | - Jane H Zhang
- Cooperative Studies Program Coordinating Center (CSPCC), VA Connecticut Healthcare System, West Haven, CT, USA
| |
Collapse
|
7
|
Sundberg M, Perron CO, Kimia A, Landschaft A, Nigrovic LE, Nelson KA, Fine AM, Eisenberg M, Baskin MN, Neuman MI, Stack AM. A method to identify pediatric high-risk diagnoses missed in the emergency department. Diagnosis (Berl) 2018; 5:63-69. [PMID: 29858901 DOI: 10.1515/dx-2018-0005] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2018] [Accepted: 05/16/2018] [Indexed: 11/15/2022]
Abstract
BACKGROUND Diagnostic error can lead to increased morbidity, mortality, healthcare utilization and cost. The 2015 National Academy of Medicine report "Improving Diagnosis in Healthcare" called for improving diagnostic accuracy by developing innovative electronic approaches to reduce medical errors, including missed or delayed diagnosis. The objective of this article was to develop a process to detect potential diagnostic discrepancy between pediatric emergency and inpatient discharge diagnosis using a computer-based tool facilitating expert review. METHODS Using a literature search and expert opinion, we identified 10 pediatric diagnoses with potential for serious consequences if missed or delayed. We then developed and applied a computerized tool to identify linked emergency department (ED) encounters and hospitalizations with these discharge diagnoses. The tool identified discordance between ED and hospital discharge diagnoses. Cases identified as discordant were manually reviewed by pediatric emergency medicine experts to confirm discordance. RESULTS Our computerized tool identified 55,233 ED encounters for hospitalized children over a 5-year period, of which 2161 (3.9%) had one of the 10 selected high-risk diagnoses. After expert record review, we identified 67 (3.1%) cases with discordance between ED and hospital discharge diagnoses. The most common discordant diagnoses were Kawasaki disease and pancreatitis. CONCLUSIONS We successfully developed and applied a semi-automated process to screen a large volume of hospital encounters to identify discordant diagnoses for selected pediatric medical conditions. This process may be valuable for informing and improving ED diagnostic accuracy.
Collapse
Affiliation(s)
- Melissa Sundberg
- Boston Children's Hospital, Division of Emergency Medicine, 300 Longwood Ave, Boston, MA 02115, USA
| | - Catherine O Perron
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Amir Kimia
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | | | - Lise E Nigrovic
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Kyle A Nelson
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Andrew M Fine
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Matthew Eisenberg
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Marc N Baskin
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Mark I Neuman
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| | - Anne M Stack
- Boston Children's Hospital, Division of Emergency Medicine, Boston, MA, USA
| |
Collapse
|
8
|
Weber GM. Using Artificial Intelligence in an Intelligent Way to Improve Efficiency of a Heart Failure Care Team. J Card Fail 2018; 24:363-364. [PMID: 29679716 DOI: 10.1016/j.cardfail.2018.04.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2018] [Revised: 04/11/2018] [Accepted: 04/11/2018] [Indexed: 10/17/2022]
Affiliation(s)
- Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts; Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts.
| |
Collapse
|
9
|
Audet LA, Bourgault P, Rochefort CM. Associations between nurse education and experience and the risk of mortality and adverse events in acute care hospitals: A systematic review of observational studies. Int J Nurs Stud 2018; 80:128-146. [PMID: 29407346 DOI: 10.1016/j.ijnurstu.2018.01.007] [Citation(s) in RCA: 85] [Impact Index Per Article: 14.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2017] [Revised: 12/29/2017] [Accepted: 01/16/2018] [Indexed: 12/17/2022]
Abstract
OBJECTIVES To provide knowledge from the summarization of the evidence on the: a) associations between nurse education and experience and the occurrence of mortality and adverse events in acute care hospitals, and; b) benefits to patients and organizations of the recent Institute of Medicine's recommendation that 80% of registered nurses should be educated at the baccalaureate degree by 2020. DATA SOURCES A systematic search of English and French literature was conducted in six electronic databases: 1) Medline, 2) PubMed, 3) CINAHL, 4) Scopus, 5) Campbell, and 6) Cochrane databases. Additional studies were identified by searching bibliographies, prior reviews, and by contacting authors. REVIEW METHOD Studies were included if they: a) were published between January 1996 and August 2017; b) were based on a quantitative research design; c) examined the associations between registered nurse education or experience and at least one independently measured adverse event, and; d) were conducted in an adult acute care setting. Data were independently extracted, analysed, and synthesized by two authors and discrepancies were resolved by consensus. The methodological heterogeneity of the reviewed studies precluded the use of meta-analysis techniques. However, the methodological quality of each study was assessed using the STROBE criteria. FINDINGS Among 2109 retrieved articles, 27 studies (24 cross-sectional and three longitudinal studies) met our inclusion criteria. These studies examined 18 distinct adverse events, with mortality and failure to rescue being the most frequently investigated events. Overall, higher levels of education were associated with lower risks of failure to rescue and mortality in 75% and 61.1% of the reviewed studies pertaining to these adverse events, respectively. Nurse education was inconsistently related to the occurrence of the other events, which were the focus of only a small number of studies. Only one study examined the 80% threshold proposed by the Institute of Medicine and found evidence that it is associated with lower odds of hospital readmission and shorter lengths of stay, but unrelated to mortality. Nurse experience was inconsistently related to adverse event occurrence. CONCLUSION While evidence suggests that higher nurse education is associated with lower risks of mortality and failure to rescue, longitudinal studies are needed to better ascertain these associations and determine the specific thresholds that minimize risks. Further studies are needed to better document the association of nurse education and experience with other nursing-sensitive adverse events, as well as the benefits to patients and organizations of the Institute of Medicine's recommendation.
Collapse
Affiliation(s)
- Li-Anne Audet
- University of Sherbrooke, School of Nursing, Faculty of Medicine and Health Sciences, Sherbrooke, Quebec, Canada; Centre de recherche de l'Hôpital Charles-Le Moyne, Longueuil, Québec, Canada; Centre de recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Québec, Canada
| | - Patricia Bourgault
- University of Sherbrooke, School of Nursing, Faculty of Medicine and Health Sciences, Sherbrooke, Quebec, Canada; Centre de recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Québec, Canada
| | - Christian M Rochefort
- University of Sherbrooke, School of Nursing, Faculty of Medicine and Health Sciences, Sherbrooke, Quebec, Canada; Centre de recherche de l'Hôpital Charles-Le Moyne, Longueuil, Québec, Canada; Centre de recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Québec, Canada.
| |
Collapse
|
10
|
Electronic surveillance and using administrative data to identify healthcare associated infections. Curr Opin Infect Dis 2018; 29:394-9. [PMID: 27257794 DOI: 10.1097/qco.0000000000000282] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
PURPOSE OF REVIEW Traditional surveillance of healthcare associated infections (HCAI) is time consuming and error-prone. We have analysed literature of the past year to look at new developments in this field. It is divided into three parts: new algorithms for electronic surveillance, the use of administrative data for surveillance of HCAI, and the definition of new endpoints of surveillance, in accordance with an automatic surveillance approach. RECENT FINDINGS Most studies investigating electronic surveillance of HCAI have concentrated on bloodstream infection or surgical site infection. However, the lack of important parameters in hospital databases can lead to misleading results. The accuracy of administrative coding data was poor at identifying HCAI. New endpoints should be defined for automatic detection, with the most crucial step being to win clinicians' acceptance. SUMMARY Electronic surveillance with conventional endpoints is a successful method when hospital information systems implemented key changes and enhancements. One requirement is the access to systems for hospital administration and clinical databases.Although the primary source of data for HCAI surveillance is not administrative coding data, these are important components of a hospital-wide programme of automated surveillance. The implementation of new endpoints for surveillance is an approach which needs to be discussed further.
Collapse
|
11
|
Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, Liu S, Zeng Y, Mehrabi S, Sohn S, Liu H. Clinical information extraction applications: A literature review. J Biomed Inform 2018; 77:34-49. [PMID: 29162496 PMCID: PMC5771858 DOI: 10.1016/j.jbi.2017.11.011] [Citation(s) in RCA: 316] [Impact Index Per Article: 52.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2017] [Revised: 11/01/2017] [Accepted: 11/17/2017] [Indexed: 12/24/2022]
Abstract
BACKGROUND With the rapid adoption of electronic health records (EHRs), it is desirable to harvest information and knowledge from EHRs to support automated systems at the point of care and to enable secondary use of EHRs for clinical and translational research. One critical component used to facilitate the secondary use of EHR data is the information extraction (IE) task, which automatically extracts and encodes clinical information from text. OBJECTIVES In this literature review, we present a review of recent published research on clinical information extraction (IE) applications. METHODS A literature search was conducted for articles published from January 2009 to September 2016 based on Ovid MEDLINE In-Process & Other Non-Indexed Citations, Ovid MEDLINE, Ovid EMBASE, Scopus, Web of Science, and ACM Digital Library. RESULTS A total of 1917 publications were identified for title and abstract screening. Of these publications, 263 articles were selected and discussed in this review in terms of publication venues and data sources, clinical IE tools, methods, and applications in the areas of disease- and drug-related studies, and clinical workflow optimizations. CONCLUSIONS Clinical IE has been used for a wide range of applications, however, there is a considerable gap between clinical studies using EHR data and studies using clinical IE. This study enabled us to gain a more concrete understanding of the gap and to provide potential solutions to bridge this gap.
Collapse
Affiliation(s)
- Yanshan Wang
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Liwei Wang
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Majid Rastegar-Mojarad
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Sungrim Moon
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Feichen Shen
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Naveed Afzal
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Sijia Liu
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Yuqun Zeng
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Saeed Mehrabi
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Sunghwan Sohn
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States
| | - Hongfang Liu
- Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, United States.
| |
Collapse
|
12
|
Boyce RD, Jao J, Miller T, Kane-Gill SL. Automated Screening of Emergency Department Notes for Drug-Associated Bleeding Adverse Events Occurring in Older Adults. Appl Clin Inform 2017; 8:1022-1030. [PMID: 29241242 DOI: 10.4338/aci-2017-02-ra-0036] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
Objective To conduct research to show the value of text mining for automatically identifying suspected bleeding adverse drug events (ADEs) in the emergency department (ED).
Methods A corpus of ED admission notes was manually annotated for bleeding ADEs. The notes were taken for patients ≥ 65 years of age who had an ICD-9 code for bleeding, the presence of hemoglobin value ≤ 8 g/dL, or were transfused > 2 units of packed red blood cells. This training corpus was used to develop bleeding ADE algorithms using Random Forest and Classification and Regression Tree (CART). A completely separate set of notes was annotated and used to test the classification performance of the final models using the area under the ROC curve (AUROC).
Results The best performing CART resulted in an AUROC on the training set of 0.882. The model's AUROC on the test set was 0.827. At a sensitivity of 0.679, the model had a specificity of 0.908 and a positive predictive value (PPV) of 0.814. It had a relatively simple and intuitive structure consisting of 13 decision nodes and 14 leaf nodes. Decision path probabilities ranged from 0.041 to 1.0. The AUROC for the best performing Random Forest method on the training set was 0.917. On the test set, the model's AUROC was 0.859. At a sensitivity of 0.274, the model had a specificity of 0.986 and a PPV of 0.92.
Conclusion Both models accurately identify bleeding ADEs using the presence or absence of certain clinical concepts in ED admission notes for older adult patients. The CART model is particularly noteworthy because it does not require significant technical overhead to implement. Future work should seek to replicate the results on a larger test set pulled from another institution.
Collapse
Affiliation(s)
- Richard D Boyce
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Jeremy Jao
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Taylor Miller
- Department of Pharmacy, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, United States
| | - Sandra L Kane-Gill
- School of Pharmacy, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
13
|
|
14
|
Rochefort CM, Buckeridge DL, Tanguay A, Biron A, D'Aragon F, Wang S, Gallix B, Valiquette L, Audet LA, Lee TC, Jayaraman D, Petrucci B, Lefebvre P. Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol. BMC Health Serv Res 2017; 17:147. [PMID: 28209197 PMCID: PMC5314632 DOI: 10.1186/s12913-017-2069-7] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2016] [Accepted: 02/02/2017] [Indexed: 12/31/2022] Open
Abstract
Background Adverse events (AEs) in acute care hospitals are frequent and associated with significant morbidity, mortality, and costs. Measuring AEs is necessary for quality improvement and benchmarking purposes, but current detection methods lack in accuracy, efficiency, and generalizability. The growing availability of electronic health records (EHR) and the development of natural language processing techniques for encoding narrative data offer an opportunity to develop potentially better methods. The purpose of this study is to determine the accuracy and generalizability of using automated methods for detecting three high-incidence and high-impact AEs from EHR data: a) hospital-acquired pneumonia, b) ventilator-associated event and, c) central line-associated bloodstream infection. Methods This validation study will be conducted among medical, surgical and ICU patients admitted between 2013 and 2016 to the Centre hospitalier universitaire de Sherbrooke (CHUS) and the McGill University Health Centre (MUHC), which has both French and English sites. A random 60% sample of CHUS patients will be used for model development purposes (cohort 1, development set). Using a random sample of these patients, a reference standard assessment of their medical chart will be performed. Multivariate logistic regression and the area under the curve (AUC) will be employed to iteratively develop and optimize three automated AE detection models (i.e., one per AE of interest) using EHR data from the CHUS. These models will then be validated on a random sample of the remaining 40% of CHUS patients (cohort 1, internal validation set) using chart review to assess accuracy. The most accurate models developed and validated at the CHUS will then be applied to EHR data from a random sample of patients admitted to the MUHC French site (cohort 2) and English site (cohort 3)—a critical requirement given the use of narrative data –, and accuracy will be assessed using chart review. Generalizability will be determined by comparing AUCs from cohorts 2 and 3 to those from cohort 1. Discussion This study will likely produce more accurate and efficient measures of AEs. These measures could be used to assess the incidence rates of AEs, evaluate the success of preventive interventions, or benchmark performance across hospitals.
Collapse
Affiliation(s)
- Christian M Rochefort
- School of Nursing, Faculty of Medicine and Health Sciences, University of Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada. .,Centre de recherche de l'Hôpital Charles-LeMoyne, University of Sherbrooke-Campus Longueuil, 150 Place Charles-LeMoyne, Longueuil, QC, J4K 0A8, Canada. .,Department of Epidemiology, Biostatics and Occupational Health, Faculty of Medicine, McGill University, Purvis Hall, 1020 Pine Avenue West, Montreal, QC, H3A 1A2, Canada.
| | - David L Buckeridge
- Department of Epidemiology, Biostatics and Occupational Health, Faculty of Medicine, McGill University, Purvis Hall, 1020 Pine Avenue West, Montreal, QC, H3A 1A2, Canada
| | - Andréanne Tanguay
- School of Nursing, Faculty of Medicine and Health Sciences, University of Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada
| | - Alain Biron
- Department of Quality, Patient Safety and Performance, McGill University Health Centre, 2155 Guy Street, Montreal, QC, H3H 2R9, Canada.,Ingram School of Nursing, McGill University, Wilson Hall, 3506 University Street, Montreal, QC, H3A 2A7, Canada
| | - Frédérick D'Aragon
- Department of Anesthesiology, Faculty of Medicine and Health Sciences, University of Sherbrooke and Centre hospitalier universitaire de Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada
| | - Shengrui Wang
- Faculty of Sciences, Department of Informatics, University of Sherbrooke, 2500 Boulevard de l'Université, Sherbrooke, QC, J1K 2R1, Canada
| | - Benoit Gallix
- Department of Diagnostic Radiology, McGill University and McGill University Health Centre, 1650 Cedar Avenue, Montreal, QC, H3G 1A4, Canada
| | - Louis Valiquette
- Department of Microbiology and Infectious Diseases, University of Sherbrooke and Centre hospitalier universitaire de Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada
| | - Li-Anne Audet
- School of Nursing, Faculty of Medicine and Health Sciences, University of Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada
| | - Todd C Lee
- Department of Internal Medicine, McGill University and McGill University Health Centre, 1650 Cedar Avenue, Montreal, QC, H3G 1A4, Canada
| | - Dev Jayaraman
- Department of Internal Medicine, McGill University and McGill University Health Centre, 1650 Cedar Avenue, Montreal, QC, H3G 1A4, Canada
| | - Bruno Petrucci
- Department of Quality, Evaluation, Performance and Ethics, Centre hospitalier universitaire de Sherbrooke, 3001, 12e Avenue Nord, Sherbrooke, QC, J1H 5N4, Canada
| | - Patricia Lefebvre
- Department of Quality, Patient Safety and Performance, McGill University Health Centre, 2155 Guy Street, Montreal, QC, H3H 2R9, Canada
| |
Collapse
|
15
|
Endrich O, Rimle C, Zwahlen M, Triep K, Raio L, Nelle M. Asphyxia in the Newborn: Evaluating the Accuracy of ICD Coding, Clinical Diagnosis and Reimbursement: Observational Study at a Swiss Tertiary Care Center on Routinely Collected Health Data from 2012-2015. PLoS One 2017; 12:e0170691. [PMID: 28118380 PMCID: PMC5261744 DOI: 10.1371/journal.pone.0170691] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2016] [Accepted: 01/09/2017] [Indexed: 01/02/2023] Open
Abstract
BACKGROUND The ICD-10 categories of the diagnosis "perinatal asphyxia" are defined by clinical signs and a 1-minute Apgar score value. However, the modern conception is more complex and considers metabolic values related to the clinical state. A lack of consistency between the former clinical and the latter encoded diagnosis poses questions over the validity of the data. Our aim was to establish a refined classification which is able to distinctly separate cases according to clinical criteria and financial resource consumption. The hypothesis of the study is that outdated ICD-10 definitions result in differences between the encoded diagnosis asphyxia and the medical diagnosis referring to the clinical context. METHODS Routinely collected health data (encoding and financial data) of the University Hospital of Bern were used. The study population was chosen by selected ICD codes, the encoded and the clinical diagnosis were analyzed and each case was reevaluated. The new method categorizes the diagnoses of perinatal asphyxia into the following groups: mild, moderate and severe asphyxia, metabolic acidosis and normal clinical findings. The differences of total costs per case were determined by using one-way analysis of variance. RESULTS The study population included 622 cases (P20 "intrauterine hypoxia" 399, P21 "birth asphyxia" 233). By applying the new method, the diagnosis asphyxia could be ruled out with a high probability in 47% of cases and the variance of case related costs (one-way ANOVA: F (5, 616) = 55.84, p < 0.001, multiple R-squared = 0.312, p < 0.001) could be best explained. The classification of the severity of asphyxia could clearly be linked to the complexity of cases. CONCLUSION The refined coding method provides clearly defined diagnoses groups and has the strongest effect on the distribution of costs. It improves the diagnosis accuracy of perinatal asphyxia concerning clinical practice, research and reimbursement.
Collapse
Affiliation(s)
- Olga Endrich
- Medical Directorate, Inselspital, University Hospital of Bern, Bern, Switzerland
| | - Carole Rimle
- Student at the Faculty of Medicine, University of Bern, Bern, Switzerland
| | - Marcel Zwahlen
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Karen Triep
- Medical Directorate, Inselspital, University Hospital of Bern, Bern, Switzerland
| | - Luigi Raio
- Department of Obstetrics & Gynecology, University Hospital of Bern, Bern, Switzerland
| | - Mathias Nelle
- Neonatology Division, Inselspital, University Hospital of Bern, Bern, Switzerland
| |
Collapse
|
16
|
Team-based care to improve control of hypertension in an inner city practice. Healthcare (Basel) 2016; 4:52-6. [DOI: 10.1016/j.hjdsi.2015.10.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2015] [Revised: 09/26/2015] [Accepted: 10/19/2015] [Indexed: 11/23/2022] Open
|