101
|
Weng C, Shah NH, Hripcsak G. Deep phenotyping: Embracing complexity and temporality-Towards scalability, portability, and interoperability. J Biomed Inform 2020; 105:103433. [PMID: 32335224 PMCID: PMC7179504 DOI: 10.1016/j.jbi.2020.103433] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2020] [Accepted: 04/20/2020] [Indexed: 01/07/2023]
Affiliation(s)
- Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, NY, USA.
| | - Nigam H Shah
- Medicine - Biomedical Informatics Research, Stanford University, Stanford, CA, USA.
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, NY, USA.
| |
Collapse
|
102
|
Essay P, Mosier J, Subbian V. Rule-Based Cohort Definitions for Acute Respiratory Failure: Electronic Phenotyping Algorithm. JMIR Med Inform 2020; 8:e18402. [PMID: 32293579 PMCID: PMC7191347 DOI: 10.2196/18402] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 03/19/2020] [Accepted: 03/22/2020] [Indexed: 12/12/2022] Open
Abstract
Background Acute respiratory failure is generally treated with invasive mechanical ventilation or noninvasive respiratory support strategies. The efficacies of the various strategies are not fully understood. There is a need for accurate therapy-based phenotyping for secondary analyses of electronic health record data to answer research questions regarding respiratory management and outcomes with each strategy. Objective The objective of this study was to address knowledge gaps related to ventilation therapy strategies across diverse patient populations by developing an algorithm for accurate identification of patients with acute respiratory failure. To accomplish this objective, our goal was to develop rule-based computable phenotypes for patients with acute respiratory failure using remotely monitored intensive care unit (tele-ICU) data. This approach permits analyses by ventilation strategy across broad patient populations of interest with the ability to sub-phenotype as research questions require. Methods Tele-ICU data from ≥200 hospitals were used to create a rule-based algorithm for phenotyping patients with acute respiratory failure, defined as an adult patient requiring invasive mechanical ventilation or a noninvasive strategy. The dataset spans a wide range of hospitals and ICU types across all US regions. Structured clinical data, including ventilation therapy start and stop times, medication records, and nurse and respiratory therapy charts, were used to define clinical phenotypes. All adult patients of any diagnoses with record of ventilation therapy were included. Patients were categorized by ventilation type, and analysis of event sequences using record timestamps defined each phenotype. Manual validation was performed on 5% of patients in each phenotype. Results We developed 7 phenotypes: (0) invasive mechanical ventilation, (1) noninvasive positive-pressure ventilation, (2) high-flow nasal insufflation, (3) noninvasive positive-pressure ventilation subsequently requiring intubation, (4) high-flow nasal insufflation subsequently requiring intubation, (5) invasive mechanical ventilation with extubation to noninvasive positive-pressure ventilation, and (6) invasive mechanical ventilation with extubation to high-flow nasal insufflation. A total of 27,734 patients met our phenotype criteria and were categorized into these ventilation subgroups. Manual validation of a random selection of 5% of records from each phenotype resulted in a total accuracy of 88% and a precision and recall of 0.8789 and 0.8785, respectively, across all phenotypes. Individual phenotype validation showed that the algorithm categorizes patients particularly well but has challenges with patients that require ≥2 management strategies. Conclusions Our proposed computable phenotyping algorithm for patients with acute respiratory failure effectively identifies patients for therapy-focused research regardless of admission diagnosis or comorbidities and allows for management strategy comparisons across populations of interest.
Collapse
Affiliation(s)
- Patrick Essay
- College of Engineering, The University of Arizona, Tucson, AZ, United States
| | - Jarrod Mosier
- College of Medicine, The University of Arizona, Tucson, AZ, United States
| | - Vignesh Subbian
- College of Engineering, The University of Arizona, Tucson, AZ, United States
| |
Collapse
|
103
|
Rasmussen-Torvik LJ, Furmanchuk A, Stoddard AJ, Osinski KI, Meurer JR, Smith N, Chrischilles E, Black BS, Kho A. The effect of number of healthcare visits on study sample selection in electronic health record data. Int J Popul Data Sci 2020; 5. [PMID: 32864475 PMCID: PMC7448749 DOI: 10.23889/ijpds.v5i1.1156] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
Introduction Few studies have addressed how to select a study sample when using electronic health record (EHR) data. Objective To examine how changing criterion for number of visits in EHR data required for inclusion in a study sample would impact one basic epidemiologic measure: estimates of disease period prevalence. Methods Year 2016 EHR data from three Midwestern health systems (Northwestern Medicine in Illinois, University of Iowa Health Care, and Froedtert & the Medical College of Wisconsin, all regional tertiary health care systems including hospitals and clinics) was used to examine how alternate definitions of the study sample, based on number of healthcare visits in one year, affected measures of disease period prevalence. In 2016, each of these health systems saw between 160,000 and 420,000 unique patients. Curated collections of ICD-9, ICD-10, and SNOMED codes (from CMS-approved electronic clinical quality measures) were used to define three diseases: acute myocardial infarction, asthma, and diabetic nephropathy). Results Across all health systems, increasing the minimum required number of visits to be included in the study sample monotonically increased crude period prevalence estimates. The rate at which prevalence estimates increased with number of visits varied across sites and across diseases. Conclusion In addition to providing thorough descriptions of case definitions, when using EHR data authors must carefully describe how a study sample is identified and report data for a range of sample definitions, including minimum number of visits, so that others can assess the sensitivity of reported results to sample definition in EHR data. Key words Electronic Health Records, Sampling Studies, Prevalence, Methods
Collapse
Affiliation(s)
- Laura J Rasmussen-Torvik
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL 60611
| | - Al'ona Furmanchuk
- Center for Health Information Partnerships, Northwestern University Feinberg School of Medicine, Chicago, IL 60611
| | - Alexander J Stoddard
- Clinical and Translational Science Institute/Institute for Health & Equity, Medical College of Wisconsin, Milwaukee, WI, 53226
| | - Kristen I Osinski
- Clinical and Translational Science Institute/Institute for Health & Equity, Medical College of Wisconsin, Milwaukee, WI, 53226
| | - John R Meurer
- Clinical and Translational Science Institute/Institute for Health & Equity, Medical College of Wisconsin, Milwaukee, WI, 53226
| | - Nicholas Smith
- Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, 52242
| | - Elizabeth Chrischilles
- Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, 52242
| | - Bernard S Black
- Pritzker School of Law and Kellogg School of Management, Northwestern University, Chicago, IL 60611
| | - Abel Kho
- Center for Health Information Partnerships, Northwestern University Feinberg School of Medicine, Chicago, IL 60611
| |
Collapse
|
104
|
Willemink MJ, Koszek WA, Hardell C, Wu J, Fleischmann D, Harvey H, Folio LR, Summers RM, Rubin DL, Lungren MP. Preparing Medical Imaging Data for Machine Learning. Radiology 2020; 295:4-15. [PMID: 32068507 PMCID: PMC7104701 DOI: 10.1148/radiol.2020192224] [Citation(s) in RCA: 335] [Impact Index Per Article: 83.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2019] [Revised: 12/03/2019] [Accepted: 12/30/2019] [Indexed: 12/19/2022]
Abstract
Artificial intelligence (AI) continues to garner substantial interest in medical imaging. The potential applications are vast and include the entirety of the medical imaging life cycle from image creation to diagnosis to outcome prediction. The chief obstacles to development and clinical implementation of AI algorithms include availability of sufficiently large, curated, and representative training data that includes expert labeling (eg, annotations). Current supervised AI methods require a curation process for data to optimally train, validate, and test algorithms. Currently, most research groups and industry have limited data access based on small sample sizes from small geographic areas. In addition, the preparation of data is a costly and time-intensive process, the results of which are algorithms with limited utility and poor generalization. In this article, the authors describe fundamental steps for preparing medical imaging data in AI algorithm development, explain current limitations to data curation, and explore new approaches to address the problem of data availability.
Collapse
Affiliation(s)
- Martin J. Willemink
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Wojciech A. Koszek
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Cailin Hardell
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Jie Wu
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Dominik Fleischmann
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Hugh Harvey
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Les R. Folio
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Ronald M. Summers
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Daniel L. Rubin
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| | - Matthew P. Lungren
- From the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, S-072, Stanford, CA 94305-5105 (M.J.W., D.F., D.L.R., M.P.L.); Segmed, Menlo Park, Calif (M.J.W., W.A.K., C.H., J.W.); School of Engineering, Stanford University, Stanford, Calif (J.W.); Institute of Cognitive Neuroscience, University College London, London, England (H.H.); Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, Md (L.R.F.); Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health, Clinical Center, Bethesda, Md (R.M.S.); Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, Calif (D.L.R.); and Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI), Stanford, Calif (M.P.L.)
| |
Collapse
|
105
|
Abstract
PURPOSE OF REVIEW Although primarily designed for medical documentation and billing purposes, the electronic health record (EHR) has significant potential for translational research. In this article, we provide an overview of the use of the EHR for genomics research with a focus on heritable lipid disorders. RECENT FINDINGS Linking the EHR to genomic data enables repurposing of vast phenotype data for genomic discovery. EHR data can be used to study the genetic basis of common and rare disorders, identify subphenotypes of diseases, assess pathogenicity of novel genomic variants, investigate pleiotropy, and rapidly assemble cohorts for genomic medicine clinical trials. EHR-based discovery can inform clinical practice; examples include use of polygenic risk scores for assessing disease risk and use of phenotype data to interpret rare variants. Despite limitations such as missing data, variable use of standards and poor interoperablility between disparate systems, the EHR is a powerful resource for genomic research. SUMMARY When linked to genomic data, the EHR can be leveraged for genomic discovery, which in turn can inform clinical care, exemplifying the virtuous cycle of a learning healthcare system.
Collapse
Affiliation(s)
- Maya S Safarova
- Atherosclerosis and Lipid Genomics Laboratory and Department of Cardiovascular Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | | |
Collapse
|
106
|
Floris-Moore M, Edmonds A, Napravnik S, Adimora AA. Computerized Adjudication of Coronary Heart Disease Events Using the Electronic Medical Record in HIV Clinical Research: Possibilities and Challenges Ahead. AIDS Res Hum Retroviruses 2020; 36:306-313. [PMID: 31407587 DOI: 10.1089/aid.2019.0036] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
This pilot study assessed feasibility of computer-assisted electronic medical record (EMR) abstraction to ascertain coronary heart disease (CHD) event hospitalizations. We included a sample of 87 hospitalization records from participants the University of North Carolina (UNC) site of the Women's Interagency HIV Study (WIHS) and UNC Center for AIDS Research (CFAR) HIV Clinical Cohort who were hospitalized within UNC Healthcare System from July 2004 to July 2015. We compared a computer algorithm utilizing diagnosis/procedure codes, medications, and cardiac enzyme levels to adjudicate CHD events [myocardial infarction (MI)/coronary revascularization] from the EMR to standardized manual chart adjudication. Of 87 hospitalizations, 42 were classified as definite, 25 probable, and 20 non-CHD events by manual chart adjudication. A computer algorithm requiring presence of ≥1 CHD-related International Classification of Diseases, 9th Revision (ICD-9)/Current Procedural Terminology (CPT) code correctly identified 24 of 42 definite (57%), 29 of 67 probable/definite CHD (43%), and 95% of non-CHD events; additionally requiring clinically defined cardiac enzyme levels or administration of MI-related medications correctly identified 55%, 42%, and 95% of such events, respectively. Requiring any one of the ICD-9/CPT or cardiac enzyme criteria correctly identified 98% of definite, 97% of probable/definite CHD, and 85% of non-CHD events. Challenges included difficulty matching hospitalization dates, incomplete diagnosis code data, and multiple field names/locations of laboratory/medication data. Computer algorithms comprising only ICD-9/CPT codes failed to identify a sizable proportion of CHD events. Using a less restrictive algorithm yielded fewer missed events but increased the false-positive rate. Despite potential benefits of EMR-based research, there remain several challenges to fully computerized adjudication of CHD events.
Collapse
Affiliation(s)
- Michelle Floris-Moore
- Division of Infectious Diseases, Department of Medicine, School of Medicine, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
- Department of Medicine, Center for AIDS Research, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Andrew Edmonds
- Department of Epidemiology, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Sonia Napravnik
- Division of Infectious Diseases, Department of Medicine, School of Medicine, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
- Department of Medicine, Center for AIDS Research, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Adaora A. Adimora
- Division of Infectious Diseases, Department of Medicine, School of Medicine, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
- Department of Medicine, Center for AIDS Research, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
- Department of Epidemiology, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| |
Collapse
|
107
|
Rasmussen LV, Brandt PS, Jiang G, Kiefer RC, Pacheco JA, Adekkanattu P, Ancker JS, Wang F, Xu Z, Pathak J, Luo Y. Considerations for Improving the Portability of Electronic Health Record-Based Phenotype Algorithms. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2020; 2019:755-764. [PMID: 32308871 PMCID: PMC7153055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
With the increased adoption of electronic health records, data collected for routine clinical care is used for health outcomes and population sciences research, including the identification of phenotypes. In recent years, research networks, such as eMERGE, OHDSI and PCORnet, have been able to increase statistical power and population diversity by combining patient cohorts. These networks share phenotype algorithms that are executed at each participating site. Here we observe experiences with phenotype algorithm portability across seven research networks and propose a generalizable framework for phenotype algorithm portability. Several strategies exist to increase the portability of phenotype algorithms, reducing the implementation effort needed by each site. These include using a common data model, standardized representation of the phenotype algorithm logic, and technical solutions to facilitate federated execution of queries. Portability is achieved by tradeoffs across three domains: Data, Authoring and Implementation, and multiple approaches were observed in representing portable phenotype algorithms. Our proposed framework will help guide future research in operationalizing phenotype algorithm portability at scale.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Fei Wang
- Northwestern University, Chicago, IL
| | | | | | - Yuan Luo
- Northwestern University, Chicago, IL
| |
Collapse
|
108
|
McDonough CW, Smith SM, Cooper-DeHoff RM, Hogan WR. Optimizing Antihypertensive Medication Classification in Electronic Health Record-Based Data: Classification System Development and Methodological Comparison. JMIR Med Inform 2020; 8:e14777. [PMID: 32130152 PMCID: PMC7068459 DOI: 10.2196/14777] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2019] [Revised: 10/18/2019] [Accepted: 12/15/2019] [Indexed: 01/16/2023] Open
Abstract
Background Computable phenotypes have the ability to utilize data within the electronic health record (EHR) to identify patients with certain characteristics. Many computable phenotypes rely on multiple types of data within the EHR including prescription drug information. Hypertension (HTN)-related computable phenotypes are particularly dependent on the correct classification of antihypertensive prescription drug information, as well as corresponding diagnoses and blood pressure information. Objective This study aimed to create an antihypertensive drug classification system to be utilized with EHR-based data as part of HTN-related computable phenotypes. Methods We compared 4 different antihypertensive drug classification systems based off of 4 different methodologies and terminologies, including 3 RxNorm Concept Unique Identifier (RxCUI)–based classifications and 1 medication name–based classification. The RxCUI-based classifications utilized data from (1) the Drug Ontology, (2) the new Medication Reference Terminology, and (3) the Anatomical Therapeutic Chemical Classification System and DrugBank, whereas the medication name–based classification relied on antihypertensive drug names. Each classification system was applied to EHR-based prescription drug data from hypertensive patients in the OneFlorida Data Trust. Results There were 13,627 unique RxCUIs and 8025 unique medication names from the 13,879,046 prescriptions. We observed a broad overlap between the 4 methods, with 84.1% (691/822) to 95.3% (695/729) of terms overlapping pairwise between the different classification methods. Key differences arose from drug products with multiple dosage forms, drug products with an indication of benign prostatic hyperplasia, drug products that contain more than 1 ingredient (combination products), and terms within the classification systems corresponding to retired or obsolete RxCUIs. Conclusions In total, 2 antihypertensive drug classifications were constructed, one based on RxCUIs and one based on medication name, that can be used in future computable phenotypes that require antihypertensive drug classifications.
Collapse
Affiliation(s)
- Caitrin W McDonough
- Department of Pharmacotherapy and Translational Research, College of Pharmacy, University of Florida, Gainesville, FL, United States
| | - Steven M Smith
- Department of Pharmacotherapy and Translational Research, College of Pharmacy, University of Florida, Gainesville, FL, United States
| | - Rhonda M Cooper-DeHoff
- Department of Pharmacotherapy and Translational Research, College of Pharmacy, University of Florida, Gainesville, FL, United States.,Division of Cardiovascular Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, United States
| | - William R Hogan
- Department of Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States
| |
Collapse
|
109
|
Defining Major Depressive Disorder Cohorts Using the EHR: Multiple Phenotypes Based on ICD-9 Codes and Medication Orders. ACTA ACUST UNITED AC 2020; 36:18-26. [PMID: 32218644 DOI: 10.1016/j.npbr.2020.02.002] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
Background Major Depressive Disorder (MDD) is one of the most common mental illnesses and a leading cause of disability worldwide. Electronic Health Records (EHR) allow researchers to conduct unprecedented large-scale observational studies investigating MDD, its disease development and its interaction with other health outcomes. While there exist methods to classify patients as clear cases or controls, given specific data requirements, there are presently no simple, generalizable, and validated methods to classify an entire patient population into varying groups of depression likelihood and severity. Methods We have tested a simple, pragmatic electronic phenotype algorithm that classifies patients into one of five mutually exclusive, ordinal groups, varying in depression phenotype. Using data from an integrated health system on 278,026 patients from a 10-year study period we have tested the convergent validity of these constructs using measures of external validation, including patterns of psychiatric prescriptions, symptom severity, indicators of suicidality, comorbidity, mortality, health care utilization, and polygenic risk scores for MDD. Results We found consistent patterns of increasing morbidity and/or adverse outcomes across the five groups, providing evidence for convergent validity. Limitations The study population is from a single rural integrated health system which is predominantly white, possibly limiting its generalizability. Conclusion Our study provides initial evidence that a simple algorithm, generalizable to most EHR data sets, provides categories with meaningful face and convergent validity that can be used for stratification of an entire patient population.
Collapse
|
110
|
Keenan BT, Kirchner HL, Veatch OJ, Borthwick KM, Davenport VA, Feemster JC, Gendy M, Gossard TR, Pack FM, Sirikulvadhana L, Teigen LN, Timm PC, Malow BA, Morgenthaler TI, Zee PC, Pack AI, Robishaw JD, Derose SF. Multisite validation of a simple electronic health record algorithm for identifying diagnosed obstructive sleep apnea. J Clin Sleep Med 2020; 16:175-183. [PMID: 31992429 DOI: 10.5664/jcsm.8160] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
STUDY OBJECTIVES We examined the performance of a simple algorithm to accurately distinguish cases of diagnosed obstructive sleep apnea (OSA) and noncases using the electronic health record (EHR) across six health systems in the United States. METHODS Retrospective analysis of EHR data was performed. The algorithm defined cases as individuals with ≥ 2 instances of specific International Classification of Diseases (ICD)-9 and/or ICD-10 diagnostic codes (327.20, 327.23, 327.29, 780.51, 780.53, 780.57, G4730, G4733 and G4739) related to sleep apnea on separate dates in their EHR. Noncases were defined by the absence of these codes. Using chart reviews on 120 cases and 100 noncases at each site (n = 1,320 total), positive predictive value (PPV) and negative predictive value (NPV) were calculated. RESULTS The algorithm showed excellent performance across sites, with a PPV (95% confidence interval) of 97.1 (95.6, 98.2) and NPV of 95.5 (93.5, 97.0). Similar performance was seen at each site, with all NPV and PPV estimates ≥ 90% apart from a somewhat lower PPV of 87.5 (80.2, 92.8) at one site. A modified algorithm of ≥ 3 instances improved PPV to 94.9 (88.5, 98.3) at this site, but excluded an additional 18.3% of cases. Thus, performance may be further improved by requiring additional codes, but this reduces the number of determinate cases. CONCLUSIONS A simple EHR-based case-identification algorithm for diagnosed OSA showed excellent predictive characteristics in a multisite sample from the United States. Future analyses should be performed to understand the effect of undiagnosed disease in EHR-defined noncases. This algorithm has wide-ranging applications for EHR-based OSA research.
Collapse
Affiliation(s)
- Brendan T Keenan
- Division of Sleep Medicine/Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.,Co-lead authors
| | - H Lester Kirchner
- Biomedical and Translational Informatics, Geisinger, Danville, Pennsylvania.,Co-lead authors
| | - Olivia J Veatch
- Division of Sleep Medicine/Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.,Sleep Disorders Division, Vanderbilt University Medical Center, Nashville, Tennessee
| | | | - Vicki A Davenport
- Biomedical and Translational Informatics, Geisinger, Danville, Pennsylvania
| | - John C Feemster
- Center for Sleep Medicine, Mayo Clinic, Rochester, Minnesota
| | - Maged Gendy
- Center for Circadian and Sleep Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | | | - Frances M Pack
- Division of Sleep Medicine/Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Laura Sirikulvadhana
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California
| | - Luke N Teigen
- Center for Sleep Medicine, Mayo Clinic, Rochester, Minnesota
| | - Paul C Timm
- Center for Sleep Medicine, Mayo Clinic, Rochester, Minnesota
| | - Beth A Malow
- Sleep Disorders Division, Vanderbilt University Medical Center, Nashville, Tennessee
| | | | - Phyllis C Zee
- Center for Circadian and Sleep Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Allan I Pack
- Division of Sleep Medicine/Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Janet D Robishaw
- Charles E. Schmidt College of Medicine, Florida Atlantic University, Boca Raton, Florida.,Joint senior authors
| | - Stephen F Derose
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California.,Joint senior authors
| |
Collapse
|
111
|
Denaxas S, Gonzalez-Izquierdo A, Direk K, Fitzpatrick NK, Fatemifar G, Banerjee A, Dobson RJB, Howe LJ, Kuan V, Lumbers RT, Pasea L, Patel RS, Shah AD, Hingorani AD, Sudlow C, Hemingway H. UK phenomics platform for developing and validating electronic health record phenotypes: CALIBER. J Am Med Inform Assoc 2019; 26:1545-1559. [PMID: 31329239 PMCID: PMC6857510 DOI: 10.1093/jamia/ocz105] [Citation(s) in RCA: 104] [Impact Index Per Article: 20.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Revised: 04/25/2019] [Accepted: 05/29/2019] [Indexed: 01/13/2023] Open
Abstract
OBJECTIVE Electronic health records (EHRs) are a rich source of information on human diseases, but the information is variably structured, fragmented, curated using different coding systems, and collected for purposes other than medical research. We describe an approach for developing, validating, and sharing reproducible phenotypes from national structured EHR in the United Kingdom with applications for translational research. MATERIALS AND METHODS We implemented a rule-based phenotyping framework, with up to 6 approaches of validation. We applied our framework to a sample of 15 million individuals in a national EHR data source (population-based primary care, all ages) linked to hospitalization and death records in England. Data comprised continuous measurements (for example, blood pressure; medication information; coded diagnoses, symptoms, procedures, and referrals), recorded using 5 controlled clinical terminologies: (1) read (primary care, subset of SNOMED-CT [Systematized Nomenclature of Medicine Clinical Terms]), (2) International Classification of Diseases-Ninth Revision and Tenth Revision (secondary care diagnoses and cause of mortality), (3) Office of Population Censuses and Surveys Classification of Surgical Operations and Procedures, Fourth Revision (hospital surgical procedures), and (4) DM+D prescription codes. RESULTS Using the CALIBER phenotyping framework, we created algorithms for 51 diseases, syndromes, biomarkers, and lifestyle risk factors and provide up to 6 validation approaches. The EHR phenotypes are curated in the open-access CALIBER Portal (https://www.caliberresearch.org/portal) and have been used by 40 national and international research groups in 60 peer-reviewed publications. CONCLUSIONS We describe a UK EHR phenomics approach within the CALIBER EHR data platform with initial evidence of validity and use, as an important step toward international use of UK EHR data for health research.
Collapse
Affiliation(s)
- Spiros Denaxas
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- The Alan Turing Institute, London, United Kingdom
- The National Institute for Health Research University College London Hospitals Biomedical Research Centre, University College London, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Arturo Gonzalez-Izquierdo
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- The National Institute for Health Research University College London Hospitals Biomedical Research Centre, University College London, London, United Kingdom
| | - Kenan Direk
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- The National Institute for Health Research University College London Hospitals Biomedical Research Centre, University College London, London, United Kingdom
| | - Natalie K Fitzpatrick
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
| | - Ghazaleh Fatemifar
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
| | - Amitava Banerjee
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- Department of Biostatistics and Health Informatics, Institute of Psychiatry Psychology and Neuroscience, King’s College London, London, United Kingdom
- The National Institute for Health Research University College London Hospitals Biomedical Research Centre, University College London, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Laurence J Howe
- Institute of Cardiovascular Science, University College London, London, United Kingdom
| | - Valerie Kuan
- Health Data Research UK, London, United Kingdom
- Institute of Cardiovascular Science, University College London, London, United Kingdom
| | - R Tom Lumbers
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Laura Pasea
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
| | - Riyaz S Patel
- Institute of Cardiovascular Science, University College London, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Anoop D Shah
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| | - Aroon D Hingorani
- Health Data Research UK, London, United Kingdom
- Institute of Cardiovascular Science, University College London, London, United Kingdom
| | - Cathie Sudlow
- Centre for Medical Informatics, Usher Institute of Population Health Science and Informatics, University of Edinburgh, Edinburgh, United Kingdom
- Health Data Research UK, Scotland, United Kingdom
| | - Harry Hemingway
- Institute of Health Informatics, University College London, London,United Kingdom
- Health Data Research UK, London, United Kingdom
- The National Institute for Health Research University College London Hospitals Biomedical Research Centre, University College London, London, United Kingdom
- British Heart Foundation Research Accelerator, University College London, London, United Kingdom
| |
Collapse
|
112
|
Hripcsak G, Levine ME, Shang N, Ryan PB. Effect of vocabulary mapping for conditions on phenotype cohorts. J Am Med Inform Assoc 2019; 25:1618-1625. [PMID: 30395248 PMCID: PMC6289550 DOI: 10.1093/jamia/ocy124] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2018] [Accepted: 09/03/2018] [Indexed: 11/18/2022] Open
Abstract
Objective To study the effect on patient cohorts of mapping condition (diagnosis) codes from source billing vocabularies to a clinical vocabulary. Materials and Methods Nine International Classification of Diseases, Ninth Revision, Clinical Modification (ICD9-CM) concept sets were extracted from eMERGE network phenotypes, translated to Systematized Nomenclature of Medicine - Clinical Terms concept sets, and applied to patient data that were mapped from source ICD9-CM and ICD10-CM codes to Systematized Nomenclature of Medicine - Clinical Terms codes using Observational Health Data Sciences and Informatics (OHDSI) Observational Medical Outcomes Partnership (OMOP) vocabulary mappings. The original ICD9-CM concept set and a concept set extended to ICD10-CM were used to create patient cohorts that served as gold standards. Results Four phenotype concept sets were able to be translated to Systematized Nomenclature of Medicine - Clinical Terms without ambiguities and were able to perform perfectly with respect to the gold standards. The other 5 lost performance when 2 or more ICD9-CM or ICD10-CM codes mapped to the same Systematized Nomenclature of Medicine - Clinical Terms code. The patient cohorts had a total error (false positive and false negative) of up to 0.15% compared to querying ICD9-CM source data and up to 0.26% compared to querying ICD9-CM and ICD10-CM data. Knowledge engineering was required to produce that performance; simple automated methods to generate concept sets had errors up to 10% (one outlier at 250%). Discussion The translation of data from source vocabularies to Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT) resulted in very small error rates that were an order of magnitude smaller than other error sources. Conclusion It appears possible to map diagnoses from disparate vocabularies to a single clinical vocabulary and carry out research using a single set of definitions, thus improving efficiency and transportability of research.
Collapse
Affiliation(s)
- George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Observational Health Data Sciences and Informatics (OHDSI), New York, New York, USA.,Medical Informatics Services, NewYork-Presbyterian Hospital, New York, New York, USA
| | - Matthew E Levine
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Observational Health Data Sciences and Informatics (OHDSI), New York, New York, USA
| | - Ning Shang
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Observational Health Data Sciences and Informatics (OHDSI), New York, New York, USA
| | - Patrick B Ryan
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Observational Health Data Sciences and Informatics (OHDSI), New York, New York, USA.,Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA
| |
Collapse
|
113
|
Hulme OL, Khurshid S, Weng LC, Anderson CD, Wang EY, Ashburner JM, Ko D, McManus DD, Benjamin EJ, Ellinor PT, Trinquart L, Lubitz SA. Development and Validation of a Prediction Model for Atrial Fibrillation Using Electronic Health Records. JACC Clin Electrophysiol 2019; 5:1331-1341. [PMID: 31753441 PMCID: PMC6884135 DOI: 10.1016/j.jacep.2019.07.016] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2019] [Accepted: 07/22/2019] [Indexed: 02/07/2023]
Abstract
OBJECTIVES This study sought to determine whether the risk of atrial fibrillation AF can be estimated accurately by using routinely ascertained features in the electronic health record (EHR) and whether AF risk is associated with stroke. BACKGROUND Early diagnosis of AF and treatment with anticoagulation may prevent strokes. METHODS Using a multi-institutional EHR, this study identified 412,085 individuals 45 to 95 years of age without prevalent AF between 2000 and 2014. A prediction model was derived and validated for 5-year AF risk by using split-sample validation and model performance was compared with other methods of AF risk assessment. RESULTS Within 5 years, 14,334 individuals developed AF. In the derivation sample (7,216 AF events of 206,042 total), the optimal risk model included sex, age, race, smoking, height, weight, diastolic blood pressure, hypertension, hyperlipidemia, heart failure, coronary heart disease, valvular disease, prior stroke, peripheral arterial disease, chronic kidney disease, hypothyroidism, and quadratic terms for height, weight, and age. In the validation sample (7,118 AF events of 206,043 total) the AF risk model demonstrated good discrimination (C-statistic: 0.777; 95% confidence interval [CI:] 0.771 to 0.783) and calibration (0.99; 95% CI: 0.96 to 1.01). Model discrimination and calibration were superior to CHARGE-AF (Cohorts for Heart and Aging Research in Genomic Epidemiology AF) (C-statistic: 0.753; 95% CI: 0.747 to 0.759; calibration slope: 0.72; 95% CI: 0.71 to 0.74), C2HEST (Coronary artery disease / chronic obstructive pulmonary disease; Hypertension; Elderly [age ≥75 years]; Systolic heart failure; Thyroid disease [hyperthyroidism]) (C-statistic: 0.754; 95% CI: 0.747 to 0.762; calibration slope: 0.44; 95% CI: 0.43 to 0.45), and CHA2DS2-VASc (Congestive heart failure, Hypertension, Age ≥75 years, Diabetes mellitus, Prior stroke, transient ischemic attack [TIA], or thromboembolism, Vascular disease, Age 65-74 years, Sex category [female]) scores (C-statistic: 0.702; 95% CI: 0.693 to 0.710; calibration slope: 0.37; 95% CI: 0.36 to 0.38). AF risk discriminated incident stroke (n = 4,814; C-statistic: 0.684; 95% CI: 0.677 to 0.692) and stroke within 90 days of incident AF (n = 327; C-statistic: 0.789; 95% CI: 0.764 to 0.814). CONCLUSIONS A model developed from a real-world EHR database predicted AF accurately and stratified stroke risk. Incorporating AF prediction into EHRs may enable risk-guided screening for AF.
Collapse
Affiliation(s)
- Olivia L Hulme
- Cardiovascular Research Center, Massachusetts General Hospital, Boston, Massachusetts
| | - Shaan Khurshid
- Cardiology Division, Massachusetts General Hospital, Boston, Massachusetts
| | - Lu-Chen Weng
- Cardiovascular Research Center, Massachusetts General Hospital, Boston, Massachusetts
| | - Christopher D Anderson
- Center for Genomic Medicine, Massachusetts General Hospital, Boston, Massachusetts; J.P. Kistler Stroke Research Center, Massachusetts General Hospital, Boston, Massachusetts
| | - Elizabeth Y Wang
- Cardiovascular Research Center, Massachusetts General Hospital, Boston, Massachusetts
| | - Jeffrey M Ashburner
- Division of General Internal Medicine, Massachusetts General Hospital, Boston, Massachusetts; Department of Medicine, Harvard Medical School, Boston, Massachusetts
| | - Darae Ko
- Department of Medicine, Boston University Medical Center, Boston, Massachusetts
| | - David D McManus
- Departments of Medicine and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, Massachusetts
| | - Emelia J Benjamin
- Boston University and National Heart, Lung, and Blood Institute Framingham Heart Study, Framingham, Massachusetts; Sections of Preventive Medicine and Cardiovascular Medicine, Department of Medicine, Boston University School of Medicine, and Department of Epidemiology, Boston University School of Public Health, Boston, Massachusetts
| | - Patrick T Ellinor
- Cardiovascular Research Center, Massachusetts General Hospital, Boston, Massachusetts; Cardiac Arrhythmia Service, Massachusetts General Hospital, Boston, Massachusetts
| | - Ludovic Trinquart
- Boston University and National Heart, Lung, and Blood Institute Framingham Heart Study, Framingham, Massachusetts; Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts
| | - Steven A Lubitz
- Cardiovascular Research Center, Massachusetts General Hospital, Boston, Massachusetts; Cardiac Arrhythmia Service, Massachusetts General Hospital, Boston, Massachusetts.
| |
Collapse
|
114
|
Abstract
Electronic Health Records (EHR) are a rich repository of valuable clinical information that exist in primary and secondary care databases. In order to utilize EHRs for medical observational research a range of algorithms for automatically identifying individuals with a specific phenotype have been developed. This review summarizes and offers a critical evaluation of the literature relating to studies conducted into the development of EHR phenotyping systems. This review describes phenotyping systems and techniques based on structured and unstructured EHR data. Articles published on PubMed and Google scholar between 2013 and 2017 have been reviewed, using search terms derived from Medical Subject Headings (MeSH). The popularity of using Natural Language Processing (NLP) techniques in extracting features from narrative text has increased. This increased attention is due to the availability of open source NLP algorithms, combined with accuracy improvement. In this review, Concept extraction is the most popular NLP technique since it has been used by more than 50% of the reviewed papers to extract features from EHR. High-throughput phenotyping systems using unsupervised machine learning techniques have gained more popularity due to their ability to efficiently and automatically extract a phenotype with minimal human effort.
Collapse
|
115
|
Liu C, Ta CN, Rogers JR, Li Z, Lee J, Butler AM, Shang N, Kury FSP, Wang L, Shen F, Liu H, Ena L, Friedman C, Weng C. Ensembles of natural language processing systems for portable phenotyping solutions. J Biomed Inform 2019; 100:103318. [PMID: 31655273 DOI: 10.1016/j.jbi.2019.103318] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Revised: 09/15/2019] [Accepted: 10/21/2019] [Indexed: 02/04/2023]
Abstract
BACKGROUND Manually curating standardized phenotypic concepts such as Human Phenotype Ontology (HPO) terms from narrative text in electronic health records (EHRs) is time consuming and error prone. Natural language processing (NLP) techniques can facilitate automated phenotype extraction and thus improve the efficiency of curating clinical phenotypes from clinical texts. While individual NLP systems can perform well for a single cohort, an ensemble-based method might shed light on increasing the portability of NLP pipelines across different cohorts. METHODS We compared four NLP systems, MetaMapLite, MedLEE, ClinPhen and cTAKES, and four ensemble techniques, including intersection, union, majority-voting and machine learning, for extracting generic phenotypic concepts. We addressed two important research questions regarding automated phenotype recognition. First, we evaluated the performance of different approaches in identifying generic phenotypic concepts. Second, we compared the performance of different methods to identify patient-specific phenotypic concepts. To better quantify the effects caused by concept granularity differences on performance, we developed a novel evaluation metric that considered concept hierarchies and frequencies. Each of the approaches was evaluated on a gold standard set of clinical documents annotated by clinical experts. One dataset containing 1,609 concepts derived from 50 clinical notes from two different institutions was used in both evaluations, and an additional dataset of 608 concepts derived from 50 case report abstracts obtained from PubMed was used for evaluation of identifying generic phenotypic concepts only. RESULTS For generic phenotypic concept recognition, the top three performers in the NYP/CUIMC dataset are union ensemble (F1, 0.634), training-based ensemble (F1, 0.632), and majority vote-based ensemble (F1, 0.622). In the Mayo dataset, the top three are majority vote-based ensemble (F1, 0.642), cTAKES (F1, 0.615), and MedLEE (F1, 0.559). In the PubMed dataset, the top three are majority vote-based ensemble (F1, 0.719), training-based (F1, 0.696) and MetaMapLite (F1, 0.694). For identifying patient specific phenotypes, the top three performers in the NYP/CUIMC dataset are majority vote-based ensemble (F1, 0.610), MedLEE (F1, 0.609), and training-based ensemble (F1, 0.585). In the Mayo dataset, the top three are majority vote-based ensemble (F1, 0.604), cTAKES (F1, 0.531) and MedLEE (F1, 0.527). CONCLUSIONS Our study demonstrates that ensembles of natural language processing can improve both generic phenotypic concept recognition and patient specific phenotypic concept identification over individual systems. Among the individual NLP systems, each individual system performed best when they were applied in the dataset that they were primary designed for. However, combining multiple NLP systems to create an ensemble can generally improve the performance. Specifically, the ensemble can increase the results reproducibility across different cohorts and tasks, and thus provide a more portable phenotyping solution compared to individual NLP systems.
Collapse
Affiliation(s)
- Cong Liu
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Casey N Ta
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - James R Rogers
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Ziran Li
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Junghwan Lee
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Alex M Butler
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Ning Shang
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | | | - Liwei Wang
- Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN 55901, USA
| | - Feichen Shen
- Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN 55901, USA
| | - Hongfang Liu
- Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN 55901, USA
| | - Lyudmila Ena
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Carol Friedman
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA.
| |
Collapse
|
116
|
Zhao J, Zhang Y, Schlueter DJ, Wu P, Eric Kerchberger V, Trent Rosenbloom S, Wells QS, Feng Q, Denny JC, Wei WQ. Detecting time-evolving phenotypic topics via tensor factorization on electronic health records: Cardiovascular disease case study. J Biomed Inform 2019; 98:103270. [PMID: 31445983 PMCID: PMC6783385 DOI: 10.1016/j.jbi.2019.103270] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Revised: 07/10/2019] [Accepted: 08/16/2019] [Indexed: 12/12/2022]
Abstract
OBJECTIVE Discovering subphenotypes of complex diseases can help characterize disease cohorts for investigative studies aimed at developing better diagnoses and treatments. Recent advances in unsupervised machine learning on electronic health record (EHR) data have enabled researchers to discover phenotypes without input from domain experts. However, most existing studies have ignored time and modeled diseases as discrete events. Uncovering the evolution of phenotypes - how they emerge, evolve and contribute to health outcomes - is essential to define more precise phenotypes and refine the understanding of disease progression. Our objective was to assess the benefits of an unsupervised approach that incorporates time to model diseases as dynamic processes in phenotype discovery. METHODS In this study, we applied a constrained non-negative tensor-factorization approach to characterize the complexity of cardiovascular disease (CVD) patient cohort based on longitudinal EHR data. Through tensor-factorization, we identified a set of phenotypic topics (i.e., subphenotypes) that these patients established over the 10 years prior to the diagnosis of CVD, and showed the progress pattern. For each identified subphenotype, we examined its association with the risk for adverse cardiovascular outcomes estimated by the American College of Cardiology/American Heart Association Pooled Cohort Risk Equations, a conventional CVD-risk assessment tool frequently used in clinical practice. Furthermore, we compared the subsequent myocardial infarction (MI) rates among the six most prevalent subphenotypes using survival analysis. RESULTS From a cohort of 12,380 adult CVD individuals with 1068 unique PheCodes, we successfully identified 14 subphenotypes. Through the association analysis with estimated CVD risk for each subtype, we found some phenotypic topics such as Vitamin D deficiency and depression, Urinary infections cannot be explained by the conventional risk factors. Through a survival analysis, we found markedly different risks of subsequent MI following the diagnosis of CVD among the six most prevalent topics (p < 0.0001), indicating these topics may capture clinically meaningful subphenotypes of CVD. CONCLUSION This study demonstrates the potential benefits of using tensor-decomposition to model diseases as dynamic processes from longitudinal EHR data. Our results suggest that this data-driven approach may potentially help researchers identify complex and chronic disease subphenotypes in precision medicine research.
Collapse
Affiliation(s)
- Juan Zhao
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Yun Zhang
- Fixed Income Division, Morgan Stanley & Co LLC, New York, NY, USA
| | - David J Schlueter
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Patrick Wu
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Medical Scientist Training Program, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - Vern Eric Kerchberger
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Division of Allergy, Pulmonary, and Critical Care Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| | - S Trent Rosenbloom
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Quinn S Wells
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| | - QiPing Feng
- Division of Clinical Pharmacology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Joshua C Denny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Wei-Qi Wei
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
| |
Collapse
|
117
|
Making work visible for electronic phenotype implementation: Lessons learned from the eMERGE network. J Biomed Inform 2019; 99:103293. [PMID: 31542521 DOI: 10.1016/j.jbi.2019.103293] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2019] [Revised: 08/26/2019] [Accepted: 09/19/2019] [Indexed: 11/21/2022]
Abstract
BACKGROUND Implementation of phenotype algorithms requires phenotype engineers to interpret human-readable algorithms and translate the description (text and flowcharts) into computable phenotypes - a process that can be labor intensive and error prone. To address the critical need for reducing the implementation efforts, it is important to develop portable algorithms. METHODS We conducted a retrospective analysis of phenotype algorithms developed in the Electronic Medical Records and Genomics (eMERGE) network and identified common customization tasks required for implementation. A novel scoring system was developed to quantify portability from three aspects: Knowledge conversion, clause Interpretation, and Programming (KIP). Tasks were grouped into twenty representative categories. Experienced phenotype engineers were asked to estimate the average time spent on each category and evaluate time saving enabled by a common data model (CDM), specifically the Observational Medical Outcomes Partnership (OMOP) model, for each category. RESULTS A total of 485 distinct clauses (phenotype criteria) were identified from 55 phenotype algorithms, corresponding to 1153 customization tasks. In addition to 25 non-phenotype-specific tasks, 46 tasks are related to interpretation, 613 tasks are related to knowledge conversion, and 469 tasks are related to programming. A score between 0 and 2 (0 for easy, 1 for moderate, and 2 for difficult portability) is assigned for each aspect, yielding a total KIP score range of 0 to 6. The average clause-wise KIP score to reflect portability is 1.37 ± 1.38. Specifically, the average knowledge (K) score is 0.64 ± 0.66, interpretation (I) score is 0.33 ± 0.55, and programming (P) score is 0.40 ± 0.64. 5% of the categories can be completed within one hour (median). 70% of the categories take from days to months to complete. The OMOP model can assist with vocabulary mapping tasks. CONCLUSION This study presents firsthand knowledge of the substantial implementation efforts in phenotyping and introduces a novel metric (KIP) to measure portability of phenotype algorithms for quantifying such efforts across the eMERGE Network. Phenotype developers are encouraged to analyze and optimize the portability in regards to knowledge, interpretation and programming. CDMs can be used to improve the portability for some 'knowledge-oriented' tasks.
Collapse
|
118
|
Ling AY, Kurian AW, Caswell-Jin JL, Sledge GW, Shah NH, Tamang SR. Using natural language processing to construct a metastatic breast cancer cohort from linked cancer registry and electronic medical records data. JAMIA Open 2019; 2:528-537. [PMID: 32025650 PMCID: PMC6994019 DOI: 10.1093/jamiaopen/ooz040] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2019] [Revised: 07/13/2019] [Accepted: 08/13/2019] [Indexed: 02/04/2023] Open
Abstract
Objectives Most population-based cancer databases lack information on metastatic recurrence. Electronic medical records (EMR) and cancer registries contain complementary information on cancer diagnosis, treatment and outcome, yet are rarely used synergistically. To construct a cohort of metastatic breast cancer (MBC) patients, we applied natural language processing techniques within a semisupervised machine learning framework to linked EMR-California Cancer Registry (CCR) data. Materials and Methods We studied all female patients treated at Stanford Health Care with an incident breast cancer diagnosis from 2000 to 2014. Our database consisted of structured fields and unstructured free-text clinical notes from EMR, linked to CCR, a component of the Surveillance, Epidemiology and End Results Program (SEER). We identified de novo MBC patients from CCR and extracted information on distant recurrences from patient notes in EMR. Furthermore, we trained a regularized logistic regression model for recurrent MBC classification and evaluated its performance on a gold standard set of 146 patients. Results There were 11 459 breast cancer patients in total and the median follow-up time was 96.3 months. We identified 1886 MBC patients, 512 (27.1%) of whom were de novo MBC patients and 1374 (72.9%) were recurrent MBC patients. Our final MBC classifier achieved an area under the receiver operating characteristic curve (AUC) of 0.917, with sensitivity 0.861, specificity 0.878, and accuracy 0.870. Discussion and Conclusion To enable population-based research on MBC, we developed a framework for retrospective case detection combining EMR and CCR data. Our classifier achieved good AUC, sensitivity, and specificity without expert-labeled examples.
Collapse
Affiliation(s)
- Albee Y Ling
- Biomedical Informatics Training Program, Stanford University, Stanford, CA.,Department of Biomedical Data Science, Stanford University, Stanford, CA
| | - Allison W Kurian
- Department of Medicine, Stanford University School of Medicine, Stanford, CA.,Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA
| | | | - George W Sledge
- Department of Medicine, Stanford University School of Medicine, Stanford, CA
| | - Nigam H Shah
- Department of Biomedical Data Science, Stanford University, Stanford, CA.,Center for Biomedical Informatics Research, Stanford University, CA
| | - Suzanne R Tamang
- Department of Biomedical Data Science, Stanford University, Stanford, CA.,Center for Population Health Sciences, Stanford University, CA
| |
Collapse
|
119
|
Almoguera B, McGinnis E, Abrams D, Vazquez L, Cederquist A, Sleiman PM, Dlugos D, Hakonarson H, Cagan A, Connolly J, Gainer VS, Garifallou J, Kaminski C, Lee YC, Mafra F, Mentch F, Pellegrino R, Qiu H, Snyder J, Tian L, Wang F, Manolio TA, Manzi S, Holm IA, Karlson EW. Drug-resistant epilepsy classified by a phenotyping algorithm associates with NTRK2. Acta Neurol Scand 2019; 140:169-176. [PMID: 31070779 DOI: 10.1111/ane.13115] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2018] [Revised: 04/16/2019] [Accepted: 05/03/2019] [Indexed: 01/31/2023]
Abstract
OBJECTIVE Up to 40% of patients with epilepsy become drug resistant (DRE). Genetic factors are likely to play a role. While efforts have focused on the transporter and target hypotheses, neither of them fully explains the pan-pharmacoresistance seen in DRE. MATERIALS AND METHODS In this study, we developed and used a phenotyping algorithm for the identification of DRE, responders, and epilepsy-free controls that were sequenced using a gene panel developed by the Pharmacogenomics Research Network (PGRN), which includes 82 genes involved in drug response. We tested the transporter hypothesis of DRE, the association between drug resistance and variants in the ATP-binding cassette family of genes previously associated with DRE, and also investigated potential new genetic factors. RESULTS In the analysis of DRE vs controls, NTRK2 was significantly associated with DRE (rs76950094; P = 1.19 × 10-7 and gene-based P-value = 1.67 × 10-4 ). NTRK2 encodes TrkB, which is involved in the development and maturation of the central nervous system, and increased activation of TrkB signaling is suggested to promote epilepsy. CONCLUSION Although the role of NTRK2 in DRE needs to be elucidated, these results support alternative mechanisms underlying DRE, complementary to the existing hypotheses, that should be evaluated.
Collapse
Affiliation(s)
- Berta Almoguera
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Emily McGinnis
- Department of Neurology Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Debra Abrams
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Lyam Vazquez
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Anna Cederquist
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Patrick M. Sleiman
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
- Department of Pediatrics, The Perelman School of Medicine University of Pennsylvania Philadelphia Pennsylvania
| | - Dennis Dlugos
- Department of Neurology Children's Hospital of Philadelphia Philadelphia Pennsylvania
| | - Hakon Hakonarson
- Center for Applied Genomics Children's Hospital of Philadelphia Philadelphia Pennsylvania
- Department of Pediatrics, The Perelman School of Medicine University of Pennsylvania Philadelphia Pennsylvania
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
120
|
Van Vleck TT, Chan L, Coca SG, Craven CK, Do R, Ellis SB, Kannry JL, Loos RJF, Bonis PA, Cho J, Nadkarni GN. Augmented intelligence with natural language processing applied to electronic health records for identifying patients with non-alcoholic fatty liver disease at risk for disease progression. Int J Med Inform 2019; 129:334-341. [PMID: 31445275 PMCID: PMC6717556 DOI: 10.1016/j.ijmedinf.2019.06.028] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2019] [Revised: 05/20/2019] [Accepted: 06/28/2019] [Indexed: 02/08/2023]
Abstract
OBJECTIVE Electronic health record (EHR) systems contain structured data (such as diagnostic codes) and unstructured data (clinical documentation). Clinical insights can be derived from analyzing both. The use of natural language processing (NLP) algorithms to effectively analyze unstructured data has been well demonstrated. Here we examine the utility of NLP for the identification of patients with non-alcoholic fatty liver disease, assess patterns of disease progression, and identify gaps in care related to breakdown in communication among providers. MATERIALS AND METHODS All clinical notes available on the 38,575 patients enrolled in the Mount Sinai BioMe cohort were loaded into the NLP system. We compared analysis of structured and unstructured EHR data using NLP, free-text search, and diagnostic codes with validation against expert adjudication. We then used the NLP findings to measure physician impression of progression from early-stage NAFLD to NASH or cirrhosis. Similarly, we used the same NLP findings to identify mentions of NAFLD in radiology reports that did not persist into clinical notes. RESULTS Out of 38,575 patients, we identified 2,281 patients with NAFLD. From the remainder, 10,653 patients with similar data density were selected as a control group. NLP outperformed ICD and text search in both sensitivity (NLP: 0.93, ICD: 0.28, text search: 0.81) and F2 score (NLP: 0.92, ICD: 0.34, text search: 0.81). Of 2281 NAFLD patients, 673 (29.5%) were believed to have progressed to NASH or cirrhosis. Among 176 where NAFLD was noted prior to NASH, the average progression time was 410 days. 619 (27.1%) NAFLD patients had it documented only in radiology notes and not acknowledged in other forms of clinical documentation. Of these, 170 (28.4%) were later identified as having likely developed NASH or cirrhosis after a median 1057.3 days. DISCUSSION NLP-based approaches were more accurate at identifying NAFLD within the EHR than ICD/text search-based approaches. Suspected NAFLD on imaging is often not acknowledged in subsequent clinical documentation. Many such patients are later found to have more advanced liver disease. Analysis of information flows demonstrated loss of key information that could have been used to help prevent the progression of early NAFLD (NAFL) to NASH or cirrhosis. CONCLUSION For identification of NAFLD, NLP performed better than alternative selection modalities. It then facilitated analysis of knowledge flow between physician and enabled the identification of breakdowns where key information was lost that could have slowed or prevented later disease progression.
Collapse
Affiliation(s)
- Tielman T Van Vleck
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA.
| | - Lili Chan
- Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Steven G Coca
- Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Catherine K Craven
- Institute for Healthcare Delivery Science, Dept. of Pop. Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, USA; Clinical Informatics Group, IT Department, Mount Sinai Health System, New York, USA
| | - Ron Do
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Stephen B Ellis
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Joseph L Kannry
- Information Technology, Mount Sinai Medical Center, New York, USA
| | - Ruth J F Loos
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Peter A Bonis
- Division of Gastroenterology, Tufts Medical Center, Boston, USA
| | - Judy Cho
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA; Department of Genetics and Genomics, Icahn School of Medicine at Mount Sinai, New York, USA; Division of Gastroenterology, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Girish N Nadkarni
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, USA; Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, USA.
| |
Collapse
|
121
|
Car J, Sheikh A, Wicks P, Williams MS. Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC Med 2019; 17:143. [PMID: 31311603 PMCID: PMC6636050 DOI: 10.1186/s12916-019-1382-x] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Accepted: 07/02/2019] [Indexed: 01/12/2023] Open
Abstract
Big data, coupled with the use of advanced analytical approaches, such as artificial intelligence (AI), have the potential to improve medical outcomes and population health. Data that are routinely generated from, for example, electronic medical records and smart devices have become progressively easier and cheaper to collect, process, and analyze. In recent decades, this has prompted a substantial increase in biomedical research efforts outside traditional clinical trial settings. Despite the apparent enthusiasm of researchers, funders, and the media, evidence is scarce for successful implementation of products, algorithms, and services arising that make a real difference to clinical care. This article collection provides concrete examples of how "big data" can be used to advance healthcare and discusses some of the limitations and challenges encountered with this type of research. It primarily focuses on real-world data, such as electronic medical records and genomic medicine, considers new developments in AI and digital health, and discusses ethical considerations and issues related to data sharing. Overall, we remain positive that big data studies and associated new technologies will continue to guide novel, exciting research that will ultimately improve healthcare and medicine-but we are also realistic that concerns remain about privacy, equity, security, and benefit to all.
Collapse
Affiliation(s)
- Josip Car
- Centre for Population Health Sciences (CePHaS), Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Clinical Sciences Building, 11 Mandalay Road, Singapore, 308232, Singapore
| | - Aziz Sheikh
- The Usher Institute, The University of Edinburgh, Edinburgh, EH8 9DX, Scotland, UK
| | - Paul Wicks
- PatientsLikeMe, 160 Second Street, Cambridge, MA, 02142, USA.
| | - Marc S Williams
- Genomic Medicine Institute, Geisinger, 100 North Academy Avenue, Danville, PA, 17822, USA
| |
Collapse
|
122
|
Hripcsak G, Shang N, Peissig PL, Rasmussen LV, Liu C, Benoit B, Carroll RJ, Carrell DS, Denny JC, Dikilitas O, Gainer VS, Howell KM, Klann JG, Kullo IJ, Lingren T, Mentch FD, Murphy SN, Natarajan K, Pacheco JA, Wei WQ, Wiley K, Weng C. Facilitating phenotype transfer using a common data model. J Biomed Inform 2019; 96:103253. [PMID: 31325501 DOI: 10.1016/j.jbi.2019.103253] [Citation(s) in RCA: 42] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2019] [Revised: 07/11/2019] [Accepted: 07/16/2019] [Indexed: 11/16/2022]
Abstract
BACKGROUND Implementing clinical phenotypes across a network is labor intensive and potentially error prone. Use of a common data model may facilitate the process. METHODS Electronic Medical Records and Genomics (eMERGE) sites implemented the Observational Health Data Sciences and Informatics (OHDSI) Observational Medical Outcomes Partnership (OMOP) Common Data Model across their electronic health record (EHR)-linked DNA biobanks. Two previously implemented eMERGE phenotypes were converted to OMOP and implemented across the network. RESULTS It was feasible to implement the common data model across sites, with laboratory data producing the greatest challenge due to local encoding. Sites were then able to execute the OMOP phenotype in less than one day, as opposed to weeks of effort to manually implement an eMERGE phenotype in their bespoke research EHR databases. Of the sites that could compare the current OMOP phenotype implementation with the original eMERGE phenotype implementation, specific agreement ranged from 100% to 43%, with disagreements due to the original phenotype, the OMOP phenotype, changes in data, and issues in the databases. Using the OMOP query as a standard comparison revealed differences in the original implementations despite starting from the same definitions, code lists, flowcharts, and pseudocode. CONCLUSION Using a common data model can dramatically speed phenotype implementation at the cost of having to populate that data model, though this will produce a net benefit as the number of phenotype implementations increases. Inconsistencies among the implementations of the original queries point to a potential benefit of using a common data model so that actual phenotype code and logic can be shared, mitigating human error in reinterpretation of a narrative phenotype definition.
Collapse
Affiliation(s)
- George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, NY, United States; Medical Informatics Services, NewYork-Presbyterian Hospital, New York, NY, United States.
| | - Ning Shang
- Department of Biomedical Informatics, Columbia University, New York, NY, United States
| | - Peggy L Peissig
- Center for Precision Medicine Research, Marshfield Clinic Research Institute, Marshfield, WI, United States
| | - Luke V Rasmussen
- Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Cong Liu
- Department of Biomedical Informatics, Columbia University, New York, NY, United States
| | - Barbara Benoit
- Research Information Science and Computing, Partners Healthcare, Boston, MA, United States
| | - Robert J Carroll
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - David S Carrell
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, United States
| | - Joshua C Denny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States; Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Ozan Dikilitas
- Department of Cardiovascular Medicine, Mayo Clinic, Rochester, MN, United States
| | - Vivian S Gainer
- Research Information Science and Computing, Partners Healthcare, Boston, MA, United States
| | - Kayla Marie Howell
- Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Jeffrey G Klann
- Research Information Science and Computing, Partners Healthcare, Boston, MA, United States
| | - Iftikhar J Kullo
- Department of Cardiovascular Medicine, Mayo Clinic, Rochester, MN, United States
| | - Todd Lingren
- Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States
| | - Frank D Mentch
- Center for Applied Genomics, Children's Hospital of Philadelphia, Philadelphia, PA, United States
| | - Shawn N Murphy
- Research Information Science and Computing, Partners Healthcare, Boston, MA, United States
| | - Karthik Natarajan
- Department of Biomedical Informatics, Columbia University, New York, NY, United States; Medical Informatics Services, NewYork-Presbyterian Hospital, New York, NY, United States
| | - Jennifer A Pacheco
- Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Wei-Qi Wei
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Ken Wiley
- National Human Genome Research Institute, NIH, Bethesda, MD, United States
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, NY, United States
| |
Collapse
|
123
|
Langlotz CP, Allen B, Erickson BJ, Kalpathy-Cramer J, Bigelow K, Cook TS, Flanders AE, Lungren MP, Mendelson DS, Rudie JD, Wang G, Kandarpa K. A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291:781-791. [PMID: 30990384 PMCID: PMC6542624 DOI: 10.1148/radiol.2019190613] [Citation(s) in RCA: 175] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2019] [Revised: 03/24/2019] [Accepted: 03/25/2019] [Indexed: 01/08/2023]
Abstract
Imaging research laboratories are rapidly creating machine learning systems that achieve expert human performance using open-source methods and tools. These artificial intelligence systems are being developed to improve medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection, computer-aided classification, and radiogenomics. In August 2018, a meeting was held in Bethesda, Maryland, at the National Institutes of Health to discuss the current state of the art and knowledge gaps and to develop a roadmap for future research initiatives. Key research priorities include: 1, new image reconstruction methods that efficiently produce images suitable for human interpretation from source data; 2, automated image labeling and annotation methods, including information extraction from the imaging report, electronic phenotyping, and prospective structured image reporting; 3, new machine learning methods for clinical imaging data, such as tailored, pretrained model architectures, and federated machine learning methods; 4, machine learning methods that can explain the advice they provide to human users (so-called explainable artificial intelligence); and 5, validated methods for image de-identification and data sharing to facilitate wide availability of clinical imaging data sets. This research roadmap is intended to identify and prioritize these needs for academic research laboratories, funding agencies, professional societies, and industry.
Collapse
Affiliation(s)
- Curtis P. Langlotz
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Bibb Allen
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Bradley J. Erickson
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Jayashree Kalpathy-Cramer
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Keith Bigelow
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Tessa S. Cook
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Adam E. Flanders
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Matthew P. Lungren
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - David S. Mendelson
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Jeffrey D. Rudie
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Ge Wang
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| | - Krishna Kandarpa
- From the Department of Radiology, Stanford University, Stanford, CA 94305 (C.P.L., M.P.L.); Department of Radiology, Grandview Medical Center, Birmingham, Ala (B.A.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.); GE Healthcare, Chicago, Ill (K.B.); Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, Pa (T.S.C., J.D.R.); Department of Radiology, Thomas Jefferson University Hospital, Philadelphia, Pa (A.E.F.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (D.S.M.); Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY (G.W.); and National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Washington, DC (K.K.)
| |
Collapse
|
124
|
Taylor CO, Lemke KW, Richards TM, Roe KD, He T, Arruda-Olson A, Carrell D, Denny JC, Hripcsak G, Kiryluk K, Kullo I, Larson EB, Peissig P, Walton NA, Wei-Qi W, Ye Z, Chute CG, Weiner JP. Comorbidity Characterization Among eMERGE Institutions: A Pilot Evaluation with the Johns Hopkins Adjusted Clinical Groups® System. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2019; 2019:145-152. [PMID: 31258966 PMCID: PMC6568092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Electronic health records (EHR) are valuable to define phenotype selection algorithms used to identify cohorts ofpatients for sequencing or genome wide association studies (GWAS). To date, the electronic medical records and genomics (eMERGE) network institutions have developed and applied such algorithms to identify cohorts with associated DNA samples used to discover new genetic associations. For complex diseases, there are benefits to stratifying cohorts using comorbidities in order to identify their genetic determinants. The objective of this study was to: (a) characterize comorbidities in a range of phenotype-selected cohorts using the Johns Hopkins Adjusted Clinical Groups® (ACG®) System, (b) assess the frequency of important comorbidities in three commonly studied GWAS phenotypes, and (c) compare the comorbidity characterization of cases and controls. Our analysis demonstrates a framework to characterize comorbidities using the ACG system and identified differences in mean chronic condition count among GWAS cases and controls. Thus, we believe there is great potential to use the ACG system to characterize comorbidities among genetic cohorts selected based on EHR phenotypes.
Collapse
Affiliation(s)
- Casey Overby Taylor
- Johns Hopkins University School of Medicine
- Johns Hopkins University School of Public Health
| | | | | | | | - Ting He
- Johns Hopkins University School of Medicine
| | | | - David Carrell
- Kaiser Permanente Washington Health Research Institute
| | | | | | | | | | - Eric B Larson
- Kaiser Permanente Washington Health Research Institute
| | | | | | | | | | - Christopher G Chute
- Johns Hopkins University School of Medicine
- Johns Hopkins University School of Public Health
| | | |
Collapse
|
125
|
Li M, Scaiano M, El Emam K, Malin BA. Efficient Active Learning for Electronic Medical Record De-identification. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2019; 2019:462-471. [PMID: 31259000 PMCID: PMC6568071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Electronic medical records are often de-identified before disseminated for secondary uses. However, unstructured natural language records are challenging to de-identify while utilizing a considerable amount of expensive human annotation. In this investigation, we incorporate active learning into the de-identification workflow to reduce annotation requirements. We apply this approach to a real clinical trials dataset and a publicly available i2b2 dataset to illustrate that, when the machine learning de-identification system can actively request information to help create a better model from beyond the system (e.g., a knowledgeable human assistant), less training data will be needed to maintain or improve the performance of trained models in comparison to the typical passive learning framework. Specifically, with a batch size of 10 documents, it requires only 40 documents for an active learning approach to reach an F-measure of 0.9, while passive learning needs at least 25% more data for training a comparable model.
Collapse
Affiliation(s)
- Muqun Li
- Vanderbilt University, Nashville, TN, USA
- Privacy Analytics, Ottawa, ON, Canada
| | | | | | | |
Collapse
|
126
|
Harrington KM, Quaden R, Stein MB, Honerlaw JP, Cissell S, Pietrzak RH, Zhao H, Radhakrishnan K, Aslan M, Gaziano JM, Concato J, Gagnon DR, Gelernter J, Cho K. Validation of an Electronic Medical Record-Based Algorithm for Identifying Posttraumatic Stress Disorder in U.S. Veterans. J Trauma Stress 2019; 32:226-237. [PMID: 31009556 PMCID: PMC6699164 DOI: 10.1002/jts.22399] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/08/2018] [Revised: 11/21/2018] [Accepted: 11/27/2018] [Indexed: 12/28/2022]
Abstract
We developed an algorithm for identifying U.S. veterans with a history of posttraumatic stress disorder (PTSD), using the Department of Veterans Affairs (VA) electronic medical record (EMR) system. This work was motivated by the need to create a valid EMR-based phenotype to identify thousands of cases and controls for a genome-wide association study of PTSD in veterans. We used manual chart review (n = 500) as the gold standard. For both the algorithm and chart review, three classifications were possible: likely PTSD, possible PTSD, and likely not PTSD. We used Lasso regression with cross-validation to select statistically significant predictors of PTSD from the EMR and then generate a predicted probability score of being a PTSD case for every participant in the study population (range: 0-1.00). Comparing the performance of our probabilistic approach (Lasso algorithm) to a rule-based approach (International Classification of Diseases [ICD] algorithm), the Lasso algorithm showed modestly higher overall percent agreement with chart review than the ICD algorithm (80% vs. 75%), higher sensitivity (0.95 vs. 0.84), and higher accuracy (AUC = 0.95 vs. 0.90). We applied a 0.7 probability cut-point to the Lasso results to determine final PTSD case-control status for the VA population. The final algorithm had a 0.99 sensitivity, 0.99 specificity, 0.95 positive predictive value, and 1.00 negative predictive value for PTSD classification (grouping possible PTSD and likely not PTSD) as determined by chart review. This algorithm may be useful for other research and quality improvement endeavors within the VA.
Collapse
Affiliation(s)
- Kelly M. Harrington
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Psychiatry, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Rachel Quaden
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
| | - Murray B. Stein
- Psychiatry Service, VA San Diego Healthcare System, San Diego, California, USA
- Departments of Psychiatry and Family Medicine & Public Health, University of California San Diego, La Jolla, California, USA
| | - Jacqueline P. Honerlaw
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
| | - Shadha Cissell
- Psychiatry Service, VA San Diego Healthcare System, San Diego, California, USA
| | - Robert H. Pietrzak
- Psychiatry Service, VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Psychiatry, Yale University School of Medicine, New Haven, Connecticut, USA
| | - Hongyu Zhao
- VA Clinical Epidemiology Research Center (CERC), VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Biostatistics, Yale University School of Public Health, New Haven, Connecticut, USA
| | - Krishnan Radhakrishnan
- VA Clinical Epidemiology Research Center (CERC), VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Internal Medicine, University of Kentucky College of Medicine, Lexington, Kentucky, USA
| | - Mihaela Aslan
- VA Clinical Epidemiology Research Center (CERC), VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Medicine, Yale University School of Medicine, New Haven, Connecticut, USA
| | - John Michael Gaziano
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - John Concato
- VA Clinical Epidemiology Research Center (CERC), VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Medicine, Yale University School of Medicine, New Haven, Connecticut, USA
| | - David R. Gagnon
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts, USA
| | - Joel Gelernter
- Psychiatry Service, VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Psychiatry, Yale University School of Medicine, New Haven, Connecticut, USA
- Departments of Genetics and Neuroscience, Yale University School of Medicine, New Haven, Connecticut, USA
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
127
|
Automatic Disease Annotation From Radiology Reports Using Artificial Intelligence Implemented by a Recurrent Neural Network. AJR Am J Roentgenol 2019; 212:734-740. [DOI: 10.2214/ajr.18.19869] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
|
128
|
Edgcomb JB, Zima B. Machine Learning, Natural Language Processing, and the Electronic Health Record: Innovations in Mental Health Services Research. Psychiatr Serv 2019; 70:346-349. [PMID: 30784377 DOI: 10.1176/appi.ps.201800401] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
An unprecedented amount of clinical information is now available via electronic health records (EHRs). These massive data sets have stimulated opportunities to adapt computational approaches to track and identify target areas for quality improvement in mental health care. In this column, three key areas of EHR data science are described: EHR phenotyping, natural language processing, and predictive modeling. For each of these computational approaches, case examples are provided to illustrate their role in mental health services research. Together, adaptation of these methods underscores the need for standardization and transparency while recognizing the opportunities and challenges ahead.
Collapse
Affiliation(s)
- Juliet Beni Edgcomb
- Department of Psychiatry and Behavioral Sciences (Edgcomb, Zima) and Center for Health Services and Society (Zima), University of California, Los Angeles, Los Angeles
| | - Bonnie Zima
- Department of Psychiatry and Behavioral Sciences (Edgcomb, Zima) and Center for Health Services and Society (Zima), University of California, Los Angeles, Los Angeles
| |
Collapse
|
129
|
Dennis J, Yengo-Kahn AM, Kirby P, Solomon GS, Cox NJ, Zuckerman SL. Diagnostic Algorithms to Study Post-Concussion Syndrome Using Electronic Health Records: Validating a Method to Capture an Important Patient Population. J Neurotrauma 2019; 36:2167-2177. [PMID: 30773988 DOI: 10.1089/neu.2018.5916] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023] Open
Abstract
Post-concussion syndrome (PCS) is characterized by persistent cognitive, somatic, and emotional symptoms after a mild traumatic brain injury (mTBI). Genetic and other biological variables may contribute to PCS etiology, and the emergence of biobanks linked to electronic health records (EHRs) offers new opportunities for research on PCS. We sought to validate the EHR data of PCS patients by comparing two diagnostic algorithms deployed in the Vanderbilt University Medical Center de-identified database of 2.8 million patient EHRs. The algorithms identified individuals with PCS by: 1) natural language processing (NLP) of narrative text in the EHR combined with structured demographic, diagnostic, and encounter data; or 2) coded billing and procedure data. The predictive value of each algorithm was assessed, and cases and controls identified by each approach were compared on demographic and medical characteristics. The NLP algorithm identified 507 cases and 10,857 controls. The negative predictive value in controls was 78% and the positive predictive value (PPV) in cases was 82%. Conversely, the coded algorithm identified 1142 patients with two or more PCS billing codes and had a PPV of 76%. Comparisons of PCS controls to both case groups recovered known epidemiology of PCS: cases were more likely than controls to be female and to have pre-morbid diagnoses of anxiety, migraine, and post-traumatic stress disorder. In contrast, controls and cases were equally likely to have attention deficit hyperactive disorder and learning disabilities, in accordance with the findings of recent systematic reviews of PCS risk factors. We conclude that EHRs are a valuable research tool for PCS. Ascertainment based on coded data alone had a predictive value comparable to an NLP algorithm, recovered known PCS risk factors, and maximized the number of included patients.
Collapse
Affiliation(s)
- Jessica Dennis
- 1 Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee.,2 Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Aaron M Yengo-Kahn
- 3 Vanderbilt Sports Concussion Center, Vanderbilt University School of Medicine, Nashville, Tennessee.,4 Department of Neurological Surgery, Vanderbilt University School of Medicine, Nashville, Tennessee
| | - Paul Kirby
- 3 Vanderbilt Sports Concussion Center, Vanderbilt University School of Medicine, Nashville, Tennessee
| | - Gary S Solomon
- 3 Vanderbilt Sports Concussion Center, Vanderbilt University School of Medicine, Nashville, Tennessee.,4 Department of Neurological Surgery, Vanderbilt University School of Medicine, Nashville, Tennessee
| | - Nancy J Cox
- 1 Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee.,2 Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Scott L Zuckerman
- 3 Vanderbilt Sports Concussion Center, Vanderbilt University School of Medicine, Nashville, Tennessee.,4 Department of Neurological Surgery, Vanderbilt University School of Medicine, Nashville, Tennessee
| |
Collapse
|
130
|
Ning W, Chan S, Beam A, Yu M, Geva A, Liao K, Mullen M, Mandl KD, Kohane I, Cai T, Yu S. Feature extraction for phenotyping from semantic and knowledge resources. J Biomed Inform 2019; 91:103122. [PMID: 30738949 PMCID: PMC6424621 DOI: 10.1016/j.jbi.2019.103122] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
OBJECTIVE Phenotyping algorithms can efficiently and accurately identify patients with a specific disease phenotype and construct electronic health records (EHR)-based cohorts for subsequent clinical or genomic studies. Previous studies have introduced unsupervised EHR-based feature selection methods that yielded algorithms with high accuracy. However, those selection methods still require expert intervention to tweak the parameter settings according to the EHR data distribution for each phenotype. To further accelerate the development of phenotyping algorithms, we propose a fully automated and robust unsupervised feature selection method that leverages only publicly available medical knowledge sources, instead of EHR data. METHODS SEmantics-Driven Feature Extraction (SEDFE) collects medical concepts from online knowledge sources as candidate features and gives them vector-form distributional semantic representations derived with neural word embedding and the Unified Medical Language System Metathesaurus. A number of features that are semantically closest and that sufficiently characterize the target phenotype are determined by a linear decomposition criterion and are selected for the final classification algorithm. RESULTS SEDFE was compared with the EHR-based SAFE algorithm and domain experts on feature selection for the classification of five phenotypes including coronary artery disease, rheumatoid arthritis, Crohn's disease, ulcerative colitis, and pediatric pulmonary arterial hypertension using both supervised and unsupervised approaches. Algorithms yielded by SEDFE achieved comparable accuracy to those yielded by SAFE and expert-curated features. SEDFE is also robust to the input semantic vectors. CONCLUSION SEDFE attains satisfying performance in unsupervised feature selection for EHR phenotyping. Both fully automated and EHR-independent, this method promises efficiency and accuracy in developing algorithms for high-throughput phenotyping.
Collapse
Affiliation(s)
- Wenxin Ning
- Department of Industrial Engineering, Tsinghua University, Beijing, China
| | - Stephanie Chan
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Andrew Beam
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Ming Yu
- Department of Industrial Engineering, Tsinghua University, Beijing, China
| | - Alon Geva
- Computational Health Informatics Program, Boston Children's Hospital, Boston, MA, USA; Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, MA, USA; Department of Anesthesia, Harvard Medical School, Boston, MA, USA
| | - Katherine Liao
- Department of Medicine, Division of Rheumatology, Immunology and Allergy, Brigham and Women's Hospital, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Mary Mullen
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA; Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Kenneth D Mandl
- Computational Health Informatics Program, Boston Children's Hospital, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Isaac Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Tianxi Cai
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Sheng Yu
- Center for Statistical Science, Tsinghua University, Beijing, China; Department of Industrial Engineering, Tsinghua University, Beijing, China; Institute for Data Science, Tsinghua University, Beijing, China.
| |
Collapse
|
131
|
Validation of the Use of Electronic Medical Records for Identification of Post-gastric Bypass Hypoglycemia Cases. Obes Surg 2019; 29:2126-2131. [PMID: 30805857 DOI: 10.1007/s11695-019-03795-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
OBJECTIVE We sought to validate an algorithm designed to identify patients with post-gastric bypass hypoglycemia (PGBH) using clinician chart review. METHODS We conducted a chart review study of non-diabetic patients who underwent Roux-en-Y gastric bypass (RYGB) at our institution from 2004 to 2013. The electronic medical record (EMR) algorithm was based on any post-operative glucose <60 mg/dl, diagnosis of hypoglycemia, or medication use for treatment of PGBH and identified 158 charts as PGBH and 1048 charts without PGBH. Two clinicians independently reviewed a random selection of 80 cases and 80 control charts and determined the presence or absence of PGBH by searching the chart using keywords and reviewing laboratory results, medications, and clinic notes. RESULTS Of the 160 charts reviewed, the EMR algorithm agreed with the chart review for 130 (accuracy = 80%, 95% CI = 75-87%) with sensitivity of 89% (95% CI = 83-96%) and specificity of 86% (95% CI = 78-93%). We improved the algorithm's accuracy to 90% by limiting the search to data obtained 3 months or more following RYGB. CONCLUSION The EMR algorithm has high sensitivity, specificity, and accuracy to identify post-gastric bypass hypoglycemia within our patient cohort. The use EMR-based algorithms may be a useful tool for future research to improve our understanding of epidemiology and risk factors for post-bariatric surgery hypoglycemia.
Collapse
|
132
|
Kagawa R, Shinohara E, Imai T, Kawazoe Y, Ohe K. Bias of Inaccurate Disease Mentions in Electronic Health Record-based Phenotyping. Int J Med Inform 2019; 124:90-96. [PMID: 30784432 DOI: 10.1016/j.ijmedinf.2018.12.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2018] [Revised: 11/13/2018] [Accepted: 12/12/2018] [Indexed: 01/21/2023]
Abstract
OBJECTIVES Electronic health record (EHR)-based phenotyping is an automated technique for identifying patients diagnosed with a particular disease using EHR data. However, EHR-based phenotyping has difficulties in achieving satisfactorily high performance because clinical notes include disease mentions that ultimately signify something other than the patient's diagnosis (such as differential diagnosis or screening). Our objective is to quantify the influence of such disease mentions on EHR-based phenotyping performance. METHODS Physicians manually reviewed whether the disease mentions indicated the patients' diseases in 487,300 clinical notes of 4,430 patients. Particular focus was placed on disease mentions that did not signify the patient's diagnosis even though they did not have any syntactic modifier or indicator in the same sentences. Patients were then classified according to whether their clinical notes included such disease mentions. RESULTS Among the patients whose clinical notes included disease mentions without any modifier or indicator, the proportion of patients whose disease mentions signified the patients' diagnosis was 78.1% (on average). This value can be interpreted as the bias of disease mentions that did not signify the patient's diagnosis on the precision of EHR-based phenotyping by extracting disease mentions from clinical notes. CONCLUSION This study quantified the bias occurred owing to disease mentions that incorrectly signify a patient's diagnosis in the value of precision of EHR-based phenotyping from four dataset types. The results of this study will help researchers in diverse research environments with different available data types.
Collapse
Affiliation(s)
- Rina Kagawa
- Department of Medical Informatics, Strategic Planning, and Management, University of Tsukuba Hospital, Japan; Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, Japan.
| | - Emiko Shinohara
- Department of Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, Japan
| | - Takeshi Imai
- Center for Disease Biology and Integrative Medicine, Graduate School of Medicine, The University of Tokyo, Japan
| | - Yoshimasa Kawazoe
- Department of Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, Japan
| | - Kazuhiko Ohe
- Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, Japan
| |
Collapse
|
133
|
Frigaard M, Rubinsky A, Lowell L, Malkina A, Karliner L, Kohn M, Peralta CA. Validating laboratory defined chronic kidney disease in the electronic health record for patients in primary care. BMC Nephrol 2019; 20:3. [PMID: 30606109 PMCID: PMC6318865 DOI: 10.1186/s12882-018-1156-2] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2018] [Accepted: 11/26/2018] [Indexed: 12/18/2022] Open
Abstract
BACKGROUND Electronic health record (EHR) data is increasingly used to identify patients with chronic kidney disease (CKD). EHR queries used to capture CKD status, identify comorbid conditions, measure awareness by providers, and track adherence to guideline-concordant processes of care have not been validated. METHODS We extracted EHR data for primary-care patients with two eGFRcreat 15-59 mL/min/1.73 m^2 at least 90 days apart. Two nephrologists manually reviewed a random sample of 50 charts to determine CKD status, associated comorbidities, and physician awareness of CKD. We also assessed the documentation of a CKD diagnosis with guideline-driven care. RESULTS Complete data were available on 1767 patients with query-defined CKD of whom 822 (47%) had a CKD diagnosis in their chart. Manual chart review confirmed the CKD diagnosis in 34 or 50 (68%) patients. Agreement between the reviewers and the EHR diagnoses on the presence of comorbidities was good (κ > 0.70, p < 0.05), except for congestive heart failure, (κ = 0.45, p < 0.05). Reviewers felt the providers were aware of CKD in 23 of 34 (68%) of the confirmed CKD cases. A CKD diagnosis was associated with higher odds of guideline-driven care including CKD-specific laboratory tests and prescriptions for statins. After adjustment, CKD diagnosis documentation was not significantly associated with ACE/ARB prescription. CONCLUSIONS Identifying CKD status by historical eGFRs overestimates disease prevalence. A CKD diagnosis in the patient chart was a reasonable surrogate for provider awareness of disease status, but CKD awareness remains relatively low. CKD in the patient chart was associated with higher rates of albuminuria testing and use of statins, but not use of ACE/ARB.
Collapse
Affiliation(s)
- Martin Frigaard
- Kidney health research collaborative (KHRC), University of California, 4150 Clement St. Building 2, Room 145, San Francisco, CA 94121 USA
| | - Anna Rubinsky
- Kidney health research collaborative (KHRC), University of California, 4150 Clement St. Building 2, Room 145, San Francisco, CA 94121 USA
| | - Lo Lowell
- Nephrology and Hypertension at Parnassus, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA 94122 USA
| | - Anna Malkina
- Nephrology and Hypertension at Parnassus, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA 94122 USA
| | - Leah Karliner
- Nephrology and Hypertension at Parnassus, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA 94122 USA
| | - Michael Kohn
- Nephrology and Hypertension at Parnassus, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA 94122 USA
| | - Carmen A Peralta
- Nephrology and Hypertension at Parnassus, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA 94122 USA
| |
Collapse
|
134
|
Kim KW, Ober C. Lessons Learned From GWAS of Asthma. ALLERGY, ASTHMA & IMMUNOLOGY RESEARCH 2019; 11:170-187. [PMID: 30661310 PMCID: PMC6340805 DOI: 10.4168/aair.2019.11.2.170] [Citation(s) in RCA: 69] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/16/2018] [Accepted: 09/05/2018] [Indexed: 02/05/2023]
Abstract
Asthma is a common complex disease of the airways. Genome-wide association studies (GWASs) of asthma have identified many risk alleles and loci that have been replicated in worldwide populations. Although the risk alleles identified by GWAS have small effects and explain only a small portion of prevalence, the discovery of asthma loci can provide an understanding of its genetic architecture and the molecular pathways involved in disease pathogenesis. These discoveries can translate into advances in clinical care by identifying therapeutic targets, preventive strategies and ultimately approaches for personalized medicine. In this review, we summarize results from GWAS of asthma from the past 10 years and the insights gleaned from these discoveries.
Collapse
Affiliation(s)
- Kyung Won Kim
- Department of Pediatrics, Severance Hospital, Institute of Allergy, Brain Korea 21 PLUS project for Medical Science, Yonsei University College of Medicine, Seoul, Korea.
| | - Carole Ober
- Department of Human Genetics, University of Chicago, Chicago, IL, USA
| |
Collapse
|
135
|
Ding DY, Simpson C, Pfohl S, Kale DC, Jung K, Shah NH. The Effectiveness of Multitask Learning for Phenotyping with Electronic Health Records Data. PACIFIC SYMPOSIUM ON BIOCOMPUTING. PACIFIC SYMPOSIUM ON BIOCOMPUTING 2019; 24:18-29. [PMID: 30864307 PMCID: PMC6662921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Electronic phenotyping is the task of ascertaining whether an individual has a medical condition of interest by analyzing their medical record and is foundational in clinical informatics. Increasingly, electronic phenotyping is performed via supervised learning. We investigate the effectiveness of multitask learning for phenotyping using electronic health records (EHR) data. Multitask learning aims to improve model performance on a target task by jointly learning additional auxiliary tasks and has been used in disparate areas of machine learning. However, its utility when applied to EHR data has not been established, and prior work suggests that its benefits are inconsistent. We present experiments that elucidate when multitask learning with neural nets improves performance for phenotyping using EHR data relative to neural nets trained for a single phenotype and to well-tuned baselines. We find that multitask neural nets consistently outperform single-task neural nets for rare phenotypes but underperform for relatively more common phenotypes. The effect size increases as more auxiliary tasks are added. Moreover, multitask learning reduces the sensitivity of neural nets to hyperparameter settings for rare phenotypes. Last, we quantify phenotype complexity and find that neural nets trained with or without multitask learning do not improve on simple baselines unless the phenotypes are sufficiently complex.
Collapse
Affiliation(s)
- Daisy Yi Ding
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA
| | | | | | | | | | | |
Collapse
|
136
|
Zozus MN, Young LW, Simon AE, Garza M, Lawrence L, Ounpraseuth ST, Bledsoe M, Newman-Norlund S, Jarvis JD, McNally M, Harris KR, McCulloh R, Aikman R, Cox S, Malloch L, Walden A, Snowden J, Chedjieu IM, Wicker CA, Atkins L, Devlin LA. Training as an Intervention to Decrease Medical Record Abstraction Errors Multicenter Studies. Stud Health Technol Inform 2019; 257:526-539. [PMID: 30741251 PMCID: PMC6692114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Studies often rely on medical record abstraction as a major source of data. However, data quality from medical record abstraction has long been questioned. Electronic Health Records (EHRs) potentially add variability to the abstraction process due to the complexity of navigating and locating study data within these systems. We report training for and initial quality assessment of medical record abstraction for a clinical study conducted by the IDeA States Pediatric Clinical Trials Network (ISPCTN) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Neonatal Research Network (NRN) using medical record abstraction as the primary data source. As part of overall quality assurance, study-specific training for medical record abstractors was developed and deployed during study start-up. The training consisted of a didactic session with an example case abstraction and an independent abstraction of two standardized cases. Sixty-nine site abstractors from thirty sites were trained. The training was designed to achieve an error rate for each abstractor of no greater than 4.93% with a mean of 2.53%, at study initiation. Twenty-three percent of the trainees exceeded the acceptance limit on one or both of the training test cases, supporting the need for such training. We describe lessons learned in the design and operationalization of the study-specific, medical record abstraction training program.
Collapse
Affiliation(s)
| | - Leslie W Young
- Larner College of Medicine at the University of Vermont, Burlington, VT
| | | | - Maryam Garza
- University of Arkansas for Medical Sciences, Little Rock, AR
| | - Lora Lawrence
- University of Arkansas for Medical Sciences, Little Rock, AR
| | | | | | | | - J Dean Jarvis
- Children's Hospital at Dartmouth-Hitchcock, Lebanon, NH
| | - Mary McNally
- Children's Hospital at Dartmouth-Hitchcock, Lebanon, NH
| | | | | | | | - Sara Cox
- University of Montana School of Public and Community Health Sciences, Missoula, MT
| | - Lacy Malloch
- University of Mississippi Medical Center, Jackson, MS
| | - Anita Walden
- University of Arkansas for Medical Sciences, Little Rock, AR
| | - Jessica Snowden
- University of Arkansas for Medical Sciences, Little Rock, AR
| | | | | | - Lauren Atkins
- KS; University of Louisville School of Medicine, Louisville, KY
| | | |
Collapse
|
137
|
Hulsen T, Jamuar SS, Moody AR, Karnes JH, Varga O, Hedensted S, Spreafico R, Hafler DA, McKinney EF. From Big Data to Precision Medicine. Front Med (Lausanne) 2019; 6:34. [PMID: 30881956 PMCID: PMC6405506 DOI: 10.3389/fmed.2019.00034] [Citation(s) in RCA: 172] [Impact Index Per Article: 34.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2018] [Accepted: 02/04/2019] [Indexed: 02/05/2023] Open
Abstract
For over a decade the term "Big data" has been used to describe the rapid increase in volume, variety and velocity of information available, not just in medical research but in almost every aspect of our lives. As scientists, we now have the capacity to rapidly generate, store and analyse data that, only a few years ago, would have taken many years to compile. However, "Big data" no longer means what it once did. The term has expanded and now refers not to just large data volume, but to our increasing ability to analyse and interpret those data. Tautologies such as "data analytics" and "data science" have emerged to describe approaches to the volume of available information as it grows ever larger. New methods dedicated to improving data collection, storage, cleaning, processing and interpretation continue to be developed, although not always by, or for, medical researchers. Exploiting new tools to extract meaning from large volume information has the potential to drive real change in clinical practice, from personalized therapy and intelligent drug design to population screening and electronic health record mining. As ever, where new technology promises "Big Advances," significant challenges remain. Here we discuss both the opportunities and challenges posed to biomedical research by our increasing ability to tackle large datasets. Important challenges include the need for standardization of data content, format, and clinical definitions, a heightened need for collaborative networks with sharing of both data and expertise and, perhaps most importantly, a need to reconsider how and when analytic methodology is taught to medical researchers. We also set "Big data" analytics in context: recent advances may appear to promise a revolution, sweeping away conventional approaches to medical science. However, their real promise lies in their synergy with, not replacement of, classical hypothesis-driven methods. The generation of novel, data-driven hypotheses based on interpretable models will always require stringent validation and experimental testing. Thus, hypothesis-generating research founded on large datasets adds to, rather than replaces, traditional hypothesis driven science. Each can benefit from the other and it is through using both that we can improve clinical practice.
Collapse
Affiliation(s)
- Tim Hulsen
- Department of Professional Health Solutions and Services, Philips Research, Eindhoven, Netherlands
- *Correspondence: Tim Hulsen
| | - Saumya S. Jamuar
- Department of Paediatrics, KK Women's and Children's Hospital, and Paediatric Academic Clinical Programme, Duke-NUS Medical School, Singapore, Singapore
| | - Alan R. Moody
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Jason H. Karnes
- Pharmacy Practice and Science, College of Pharmacy, University of Arizona Health Sciences, Phoenix, AZ, United States
| | - Orsolya Varga
- Department of Preventive Medicine, Faculty of Public Health, University of Debrecen, Debrecen, Hungary
| | - Stine Hedensted
- Department of Molecular Medicine, Aarhus University Hospital, Aarhus, Denmark
| | | | - David A. Hafler
- Departments of Neurology and Immunobiology, Yale School of Medicine, New Haven, CT, United States
| | - Eoin F. McKinney
- Department of Medicine, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom
- Eoin F. McKinney
| |
Collapse
|
138
|
Henderson J, He H, Malin BA, Denny JC, Kho AN, Ghosh J, Ho JC. Phenotyping through Semi-Supervised Tensor Factorization (PSST). AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2018; 2018:564-573. [PMID: 30815097 PMCID: PMC6371355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
A computational phenotype is a set of clinically relevant and interesting characteristics that describe patients with a given condition. Various machine learning methods have been proposed to derive phenotypes in an automatic, high-throughput manner. Among these methods, computational phenotyping through tensor factorization has been shown to produce clinically interesting phenotypes. However, few of these methods incorporate auxiliary patient information into the phenotype derivation process. In this work, we introduce Phenotyping through Semi-Supervised Tensor Factorization (PSST), a method that leverages disease status knowledge about subsets of patients to generate computational phenotypes from tensors constructed from the electronic health records of patients. We demonstrate the potential of PSST to uncover predictive and clinically interesting computational phenotypes through case studies focusing on type-2 diabetes and resistant hypertension. PSST yields more discriminative phenotypes compared to the unsupervised methods and more meaningful phenotypes compared to a supervised method.
Collapse
Affiliation(s)
| | - Huan He
- Emory University, Atlanta, GA
| | | | | | | | | | | |
Collapse
|
139
|
Velupillai S, Suominen H, Liakata M, Roberts A, Shah AD, Morley K, Osborn D, Hayes J, Stewart R, Downs J, Chapman W, Dutta R. Using clinical Natural Language Processing for health outcomes research: Overview and actionable suggestions for future advances. J Biomed Inform 2018; 88:11-19. [PMID: 30368002 PMCID: PMC6986921 DOI: 10.1016/j.jbi.2018.10.005] [Citation(s) in RCA: 89] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2018] [Revised: 10/14/2018] [Accepted: 10/15/2018] [Indexed: 12/27/2022]
Abstract
The importance of incorporating Natural Language Processing (NLP) methods in clinical informatics research has been increasingly recognized over the past years, and has led to transformative advances. Typically, clinical NLP systems are developed and evaluated on word, sentence, or document level annotations that model specific attributes and features, such as document content (e.g., patient status, or report type), document section types (e.g., current medications, past medical history, or discharge summary), named entities and concepts (e.g., diagnoses, symptoms, or treatments) or semantic attributes (e.g., negation, severity, or temporality). From a clinical perspective, on the other hand, research studies are typically modelled and evaluated on a patient- or population-level, such as predicting how a patient group might respond to specific treatments or patient monitoring over time. While some NLP tasks consider predictions at the individual or group user level, these tasks still constitute a minority. Owing to the discrepancy between scientific objectives of each field, and because of differences in methodological evaluation priorities, there is no clear alignment between these evaluation approaches. Here we provide a broad summary and outline of the challenging issues involved in defining appropriate intrinsic and extrinsic evaluation methods for NLP research that is to be used for clinical outcomes research, and vice versa. A particular focus is placed on mental health research, an area still relatively understudied by the clinical NLP research community, but where NLP methods are of notable relevance. Recent advances in clinical NLP method development have been significant, but we propose more emphasis needs to be placed on rigorous evaluation for the field to advance further. To enable this, we provide actionable suggestions, including a minimal protocol that could be used when reporting clinical NLP method development and its evaluation.
Collapse
Affiliation(s)
- Sumithra Velupillai
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK; School of Electrical Engineering and Computer Science, KTH, Stockholm, Sweden.
| | - Hanna Suominen
- College of Engineering and Computer Science, The Australian National University, Data61/CSIRO, University of Canberra, Australia; University of Turku, Finland.
| | - Maria Liakata
- Department of Computer Science, University of Warwick/Alan Turing Institute, UK.
| | - Angus Roberts
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK.
| | - Anoop D Shah
- Institute of Health Informatics, University College London, UK; University College London NHS Foundation Trust, London, UK.
| | - Katherine Morley
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK; Melbourne School of Population and Global Health, The University of Melbourne, Australia.
| | - David Osborn
- Division of Psychiatry, University College London, UK; Camden and Islington NHS Foundation Trust, London, UK.
| | - Joseph Hayes
- Division of Psychiatry, University College London, UK; Camden and Islington NHS Foundation Trust, London, UK.
| | - Robert Stewart
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK; South London and Maudsley NHS Foundation Trust, London, UK.
| | - Johnny Downs
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK; South London and Maudsley NHS Foundation Trust, London, UK.
| | - Wendy Chapman
- Department of Biomedical Informatics, University of Utah, United States.
| | - Rina Dutta
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, UK; South London and Maudsley NHS Foundation Trust, London, UK.
| |
Collapse
|
140
|
Pacheco JA, Rasmussen LV, Kiefer RC, Campion TR, Speltz P, Carroll RJ, Stallings SC, Mo H, Ahuja M, Jiang G, LaRose ER, Peissig PL, Shang N, Benoit B, Gainer VS, Borthwick K, Jackson KL, Sharma A, Wu AY, Kho AN, Roden DM, Pathak J, Denny JC, Thompson WK. A case study evaluating the portability of an executable computable phenotype algorithm across multiple institutions and electronic health record environments. J Am Med Inform Assoc 2018; 25:1540-1546. [PMID: 30124903 PMCID: PMC6213083 DOI: 10.1093/jamia/ocy101] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2018] [Revised: 06/13/2018] [Accepted: 07/10/2018] [Indexed: 12/12/2022] Open
Abstract
Electronic health record (EHR) algorithms for defining patient cohorts are commonly shared as free-text descriptions that require human intervention both to interpret and implement. We developed the Phenotype Execution and Modeling Architecture (PhEMA, http://projectphema.org) to author and execute standardized computable phenotype algorithms. With PhEMA, we converted an algorithm for benign prostatic hyperplasia, developed for the electronic Medical Records and Genomics network (eMERGE), into a standards-based computable format. Eight sites (7 within eMERGE) received the computable algorithm, and 6 successfully executed it against local data warehouses and/or i2b2 instances. Blinded random chart review of cases selected by the computable algorithm shows PPV ≥90%, and 3 out of 5 sites had >90% overlap of selected cases when comparing the computable algorithm to their original eMERGE implementation. This case study demonstrates potential use of PhEMA computable representations to automate phenotyping across different EHR systems, but also highlights some ongoing challenges.
Collapse
Affiliation(s)
- Jennifer A Pacheco
- Center for Genetic Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Luke V Rasmussen
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Richard C Kiefer
- Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
| | - Thomas R Campion
- Department of Healthcare Policy and Research, Weill Cornell Medicine, New York, New York, USA
| | - Peter Speltz
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Robert J Carroll
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Sarah C Stallings
- Meharry-Vanderbilt Alliance, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Huan Mo
- Department of Pathology, Loma Linda University Health, Loma Linda, California, USA
| | - Monika Ahuja
- Department of Healthcare Policy and Research, Weill Cornell Medicine, New York, New York, USA
| | - Guoqian Jiang
- Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
| | - Eric R LaRose
- Department of Biomedical Informatics, Marshfield Clinic Research Institute, Marshfield, Wisconsin, USA
| | - Peggy L Peissig
- Department of Biomedical Informatics, Marshfield Clinic Research Institute, Marshfield, Wisconsin, USA
| | - Ning Shang
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Barbara Benoit
- Research IS and Computing, Partners HealthCare, Harvard University, Somerville, Massachusetts, USA
| | - Vivian S Gainer
- Research IS and Computing, Partners HealthCare, Harvard University, Somerville, Massachusetts, USA
| | - Kenneth Borthwick
- Henry Hood Center for Health Research, Geisinger, Danville, Pennsylvania, USA
| | - Kathryn L Jackson
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Ambrish Sharma
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Andy Yizhou Wu
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Abel N Kho
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| | - Dan M Roden
- Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA
- Department of Pharmacology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Jyotishman Pathak
- Department of Healthcare Policy and Research, Weill Cornell Medicine, New York, New York, USA
| | - Joshua C Denny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
- Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - William K Thompson
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| |
Collapse
|
141
|
Affiliation(s)
- Melissa A Haendel
- From the Oregon Clinical and Translational Research Institute, Oregon Health and Science University, Portland, and the Linus Pauling Institute and the Center for Genome Research and Biocomputing, Oregon State University, Corvallis (M.A.H.); Johns Hopkins University Schools of Medicine, Public Health, and Nursing, Baltimore (C.G.C.); and the Jackson Laboratory for Genomic Medicine and the Institute for Systems Genomics, University of Connecticut - both in Farmington (P.N.R.)
| | - Christopher G Chute
- From the Oregon Clinical and Translational Research Institute, Oregon Health and Science University, Portland, and the Linus Pauling Institute and the Center for Genome Research and Biocomputing, Oregon State University, Corvallis (M.A.H.); Johns Hopkins University Schools of Medicine, Public Health, and Nursing, Baltimore (C.G.C.); and the Jackson Laboratory for Genomic Medicine and the Institute for Systems Genomics, University of Connecticut - both in Farmington (P.N.R.)
| | - Peter N Robinson
- From the Oregon Clinical and Translational Research Institute, Oregon Health and Science University, Portland, and the Linus Pauling Institute and the Center for Genome Research and Biocomputing, Oregon State University, Corvallis (M.A.H.); Johns Hopkins University Schools of Medicine, Public Health, and Nursing, Baltimore (C.G.C.); and the Jackson Laboratory for Genomic Medicine and the Institute for Systems Genomics, University of Connecticut - both in Farmington (P.N.R.)
| |
Collapse
|
142
|
Stanaway IB, Hall TO, Rosenthal EA, Palmer M, Naranbhai V, Knevel R, Namjou-Khales B, Carroll RJ, Kiryluk K, Gordon AS, Linder J, Howell KM, Mapes BM, Lin FTJ, Joo YY, Hayes MG, Gharavi AG, Pendergrass SA, Ritchie MD, de Andrade M, Croteau-Chonka DC, Raychaudhuri S, Weiss ST, Lebo M, Amr SS, Carrell D, Larson EB, Chute CG, Rasmussen-Torvik LJ, Roy-Puckelwartz MJ, Sleiman P, Hakonarson H, Li R, Karlson EW, Peterson JF, Kullo IJ, Chisholm R, Denny JC, Jarvik GP, Crosslin DR. The eMERGE genotype set of 83,717 subjects imputed to ~40 million variants genome wide and association with the herpes zoster medical record phenotype. Genet Epidemiol 2018; 43:63-81. [PMID: 30298529 PMCID: PMC6375696 DOI: 10.1002/gepi.22167] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2018] [Revised: 08/10/2018] [Accepted: 08/28/2018] [Indexed: 12/30/2022]
Abstract
The Electronic Medical Records and Genomics (eMERGE) network is a network of medical centers with electronic medical records linked to existing biorepository samples for genomic discovery and genomic medicine research. The network sought to unify the genetic results from 78 Illumina and Affymetrix genotype array batches from 12 contributing medical centers for joint association analysis of 83,717 human participants. In this report, we describe the imputation of eMERGE results and methods to create the unified imputed merged set of genome‐wide variant genotype data. We imputed the data using the Michigan Imputation Server, which provides a missing single‐nucleotide variant genotype imputation service using the minimac3 imputation algorithm with the Haplotype Reference Consortium genotype reference set. We describe the quality control and filtering steps used in the generation of this data set and suggest generalizable quality thresholds for imputation and phenotype association studies. To test the merged imputed genotype set, we replicated a previously reported chromosome 6 HLA‐B herpes zoster (shingles) association and discovered a novel zoster‐associated loci in an epigenetic binding site near the terminus of chromosome 3 (3p29).
Collapse
Affiliation(s)
- Ian B Stanaway
- Department of Biomedical Informatics Medical Education, School of Medicine, University of Washington, Seattle, Washington
| | - Taryn O Hall
- Department of Biomedical Informatics Medical Education, School of Medicine, University of Washington, Seattle, Washington
| | - Elisabeth A Rosenthal
- Division of Medical Genetics, School of Medicine, University of Washington, Seattle, Washington
| | - Melody Palmer
- Division of Medical Genetics, School of Medicine, University of Washington, Seattle, Washington
| | - Vivek Naranbhai
- Department of Biomedical Informatics Medical Education, School of Medicine, University of Washington, Seattle, Washington.,Harvard Medical School, Harvard University, Cambridge, Massachusetts
| | - Rachel Knevel
- Harvard Medical School, Harvard University, Cambridge, Massachusetts
| | - Bahram Namjou-Khales
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Robert J Carroll
- Departments of Biomedical Informatics and Medicine, Vanderbilt University, Nashville, Tennessee
| | - Krzysztof Kiryluk
- Department of Medicine, Columbia University, New York City, New York
| | - Adam S Gordon
- Division of Medical Genetics, School of Medicine, University of Washington, Seattle, Washington
| | - Jodell Linder
- Vanderbilt Institute for Clinical and Translational Research, School of Medicine, Vanderbilt University, Nashville, Tennessee
| | - Kayla Marie Howell
- Vanderbilt Institute for Clinical and Translational Research, School of Medicine, Vanderbilt University, Nashville, Tennessee
| | - Brandy M Mapes
- Vanderbilt Institute for Clinical and Translational Research, School of Medicine, Vanderbilt University, Nashville, Tennessee
| | - Frederick T J Lin
- Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | | | - M Geoffrey Hayes
- Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Ali G Gharavi
- Department of Medicine, Columbia University, New York City, New York
| | | | - Marylyn D Ritchie
- Department of Genetics, University of Pennsylvania, Philadelphia, Pennsylvania
| | | | | | - Soumya Raychaudhuri
- Harvard Medical School, Harvard University, Cambridge, Massachusetts.,Program in Medical and Population Genetics, Broad Institute of Massachusetts Technical Institute and Harvard University, Cambridge, Massachusetts
| | - Scott T Weiss
- Harvard Medical School, Harvard University, Cambridge, Massachusetts
| | - Matt Lebo
- Harvard Medical School, Harvard University, Cambridge, Massachusetts
| | - Sami S Amr
- Harvard Medical School, Harvard University, Cambridge, Massachusetts
| | - David Carrell
- Kaiser Permanente Washington Health Research Institute (Formerly Group Health Cooperative-Seattle), Kaiser Permanente, Seattle, Washington
| | - Eric B Larson
- Kaiser Permanente Washington Health Research Institute (Formerly Group Health Cooperative-Seattle), Kaiser Permanente, Seattle, Washington
| | - Christopher G Chute
- Schools of Medicine, Public Health, and Nursing, Johns Hopkins University, Baltimore, Maryland
| | | | | | - Patrick Sleiman
- Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | | | - Rongling Li
- National Human Genome Research Institute, National Institutes of Health, Bethesda, Maryland
| | - Elizabeth W Karlson
- Department of Genetics, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Josh F Peterson
- Departments of Biomedical Informatics and Medicine, Vanderbilt University, Nashville, Tennessee
| | | | - Rex Chisholm
- Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Joshua Charles Denny
- Departments of Biomedical Informatics and Medicine, Vanderbilt University, Nashville, Tennessee
| | - Gail P Jarvik
- Division of Medical Genetics, School of Medicine, University of Washington, Seattle, Washington
| | -
- National Human Genome Research Institute, National Institutes of Health, Bethesda, Maryland
| | - David R Crosslin
- Department of Biomedical Informatics Medical Education, School of Medicine, University of Washington, Seattle, Washington
| |
Collapse
|
143
|
El Rouby N, McDonough CW, Gong Y, McClure LA, Mitchell BD, Horenstein RB, Talbert RL, Crawford DC, Gitzendanner MA, Takahashi A, Tanaka T, Kubo M, Pepine CJ, Cooper-DeHoff RM, Benavente OR, Shuldiner AR, Johnson JA. Genome-wide association analysis of common genetic variants of resistant hypertension. THE PHARMACOGENOMICS JOURNAL 2018; 19:295-304. [PMID: 30237584 DOI: 10.1038/s41397-018-0049-x] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/11/2017] [Revised: 08/02/2018] [Accepted: 08/10/2018] [Indexed: 12/24/2022]
Abstract
Resistant hypertension (RHTN), defined as uncontrolled blood pressure (BP) ≥ 140/90 using three or more drugs or controlled BP (<140/90) using four or more drugs, is associated with adverse outcomes, including decline in kidney function. We conducted a genome-wide association analysis in 1194 White and Hispanic participants with hypertension and coronary artery disease from the INternational VErapamil-SR Trandolapril STudy-GENEtic Substudy (INVEST-GENES). Top variants associated with RHTN at p < 10-4 were tested for replication in 585 White and Hispanic participants with hypertension and subcortical strokes from the Secondary Prevention of Subcortical Strokes GENEtic Substudy (SPS3-GENES). A genetic risk score for RHTN was created by summing the risk alleles of replicated RHTN signals. rs11749255 in MSX2 was associated with RHTN in INVEST (odds ratio (OR) (95% CI) = 1.50 (1.2-1.8), p = 7.3 × 10-5) and replicated in SPS3 (OR = 2.0 (1.4-2.8), p = 4.3 × 10-5), with genome-wide significance in meta-analysis (OR = 1.60 (1.3-1.9), p = 3.8 × 10-8). Other replicated signals were in IFLTD1 and PTPRD. IFLTD1 rs6487504 was associated with RHTN in INVEST (OR = 1.90 (1.4-2.5), p = 1.1 × 10-5) and SPS3 (OR = 1.70 (1.2-2.5), p = 4 × 10-3). PTPRD rs324498, a previously reported RHTN signal, was among the top signals in INVEST (OR = 1.60 (1.3-2.0), p = 3.4 × 10-5) and replicated in SPS3 (OR = 1.60 (1.1-2.4), one-sided p = 0.005). Participants with the highest number of risk alleles were at increased risk of RHTN compared to participants with a lower number (p-trend = 1.8 × 10-15). Overall, we identified and replicated associations with RHTN in the MSX2, IFLTD1, and PTPRD regions, and combined these associations to create a genetic risk score.
Collapse
Affiliation(s)
- Nihal El Rouby
- Department of Pharmacotherapy and Translational Research and Center for Pharmacogenomics, College of Pharmacy, University of Florida, Gainesville, FL, USA
| | - Caitrin W McDonough
- Department of Pharmacotherapy and Translational Research and Center for Pharmacogenomics, College of Pharmacy, University of Florida, Gainesville, FL, USA
| | - Yan Gong
- Department of Pharmacotherapy and Translational Research and Center for Pharmacogenomics, College of Pharmacy, University of Florida, Gainesville, FL, USA
| | - Leslie A McClure
- Department of Epidemiology and Biostatistics, Dornsife School of Public Health, Drexel University, Philadelphia, PA, USA
| | - Braxton D Mitchell
- Division of Endocrinology, Diabetes and Nutrition, University of Maryland School of Medicine, Baltimore, MD, USA.,Geriatric Research and Education Clinical Center, Veterans Administration Medical Center, Baltimore, MD, USA
| | - Richard B Horenstein
- Division of Endocrinology, Diabetes and Nutrition, University of Maryland School of Medicine, Baltimore, MD, USA.,Program for Personalized and Genomic Medicine, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Robert L Talbert
- College of Pharmacy, University of Texas at Austin, Austin, TX, USA
| | - Dana C Crawford
- Epidemiology and Biostatistics, Institute for Computational Biology, Case Western Reserve University, Cleveland, OH, USA
| | | | | | | | | | - Michiaki Kubo
- RIKEN Center for Integrative Medical Sciences, Yokohama, Japan
| | - Carl J Pepine
- Division of Cardiovascular Medicine, Department of Medicine, University of Florida, Gainesville, FL, USA
| | - Rhonda M Cooper-DeHoff
- Department of Pharmacotherapy and Translational Research and Center for Pharmacogenomics, College of Pharmacy, University of Florida, Gainesville, FL, USA.,Division of Cardiovascular Medicine, Department of Medicine, University of Florida, Gainesville, FL, USA
| | - Oscar R Benavente
- Department of Neurology, University of British Columbia, Vancouver, BC, Canada
| | - Alan R Shuldiner
- Division of Endocrinology, Diabetes and Nutrition, University of Maryland School of Medicine, Baltimore, MD, USA.,Geriatric Research and Education Clinical Center, Veterans Administration Medical Center, Baltimore, MD, USA.,Program for Personalized and Genomic Medicine, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Julie A Johnson
- Department of Pharmacotherapy and Translational Research and Center for Pharmacogenomics, College of Pharmacy, University of Florida, Gainesville, FL, USA. .,Division of Cardiovascular Medicine, Department of Medicine, University of Florida, Gainesville, FL, USA.
| |
Collapse
|
144
|
Song W, Huang H, Zhang CZ, Bates DW, Wright A. Using whole genome scores to compare three clinical phenotyping methods in complex diseases. Sci Rep 2018; 8:11360. [PMID: 30054501 PMCID: PMC6063939 DOI: 10.1038/s41598-018-29634-w] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2018] [Accepted: 07/16/2018] [Indexed: 12/19/2022] Open
Abstract
Genome-wide association studies depend on accurate ascertainment of patient phenotype. However, phenotyping is difficult, and it is often treated as an afterthought in these studies because of the expense involved. Electronic health records (EHRs) may provide higher fidelity phenotypes for genomic research than other sources such as administrative data. We used whole genome association models to evaluate different EHR and administrative data-based phenotyping methods in a cohort of 16,858 Caucasian subjects for type 1 diabetes mellitus, type 2 diabetes mellitus, coronary artery disease and breast cancer. For each disease, we trained and evaluated polygenic models using three different phenotype definitions: phenotypes derived from billing data, the clinical problem list, or a curated phenotyping algorithm. We observed that for these diseases, the curated phenotype outperformed the problem list, and the problem list outperformed administrative billing data. This suggests that using advanced EHR-derived phenotypes can further increase the power of genome-wide association studies.
Collapse
Affiliation(s)
- Wenyu Song
- Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, 02120, USA.,Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, 02115, USA
| | - Hailiang Huang
- Analytic and Translational Genetics Unit, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 02114, USA.,Broad Institute of MIT and Harvard, Cambridge, Massachusetts, 02142, USA
| | - Cheng-Zhong Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, 02115, USA.,Department of Biostatistics and Computational Biology, Dana-Farber Cancer Institute and Harvard Medical School, Boston, Massachusetts, 02215, USA.,Broad Institute of MIT and Harvard, Cambridge, Massachusetts, 02142, USA
| | - David W Bates
- Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, 02120, USA.,Information Systems Department, Partners HealthCare, Somerville, Massachusetts, 02145, USA
| | - Adam Wright
- Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, 02120, USA. .,Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, 02115, USA. .,Information Systems Department, Partners HealthCare, Somerville, Massachusetts, 02145, USA.
| |
Collapse
|
145
|
Haendel MA, McMurry JA, Relevo R, Mungall CJ, Robinson PN, Chute CG. A Census of Disease Ontologies. Annu Rev Biomed Data Sci 2018. [DOI: 10.1146/annurev-biodatasci-080917-013459] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
For centuries, humans have sought to classify diseases based on phenotypic presentation and available treatments. Today, a wide landscape of strategies, resources, and tools exist to classify patients and diseases. Ontologies can provide a robust foundation of logic for precise stratification and classification along diverse axes such as etiology, development, treatment, and genetics. Disease and phenotype ontologies are used in four primary ways: ( a) search, retrieval, and annotation of knowledge; ( b) data integration and analysis; ( c) clinical decision support; and ( d) knowledge discovery. Computational inference can connect existing knowledge and generate new insights and hypotheses about drug targets, prognosis prediction, or diagnosis. In this review, we examine the rise of disease and phenotype ontologies and the diverse ways they are represented and applied in biomedicine.
Collapse
Affiliation(s)
- Melissa A. Haendel
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon 97239, USA
- Linus Pauling Institute, Oregon State University, Corvallis, Oregon 97331, USA
| | - Julie A. McMurry
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon 97239, USA
| | - Rose Relevo
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon 97239, USA
| | - Christopher J. Mungall
- Environmental Genomics and Systems Biology, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | | | - Christopher G. Chute
- School of Medicine, School of Public Health, and School of Nursing, Johns Hopkins University, Baltimore, Maryland 21205, USA
| |
Collapse
|
146
|
Sundaresan AS, Schneider G, Reynolds J, Kirchner HL. Identifying Asthma Exacerbation-Related Emergency Department Visit Using Electronic Medical Record and Claims Data. Appl Clin Inform 2018; 9:528-540. [PMID: 30040112 PMCID: PMC6051766 DOI: 10.1055/s-0038-1666994] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
Background
Asthma exacerbation leading to emergency department (ED) visit is prevalent, an indicator of poor control of asthma, and is a potentially preventable clinical outcome.
Objective
We propose to utilize multiple data elements available in electronic medical records (EMRs) and claims database to create separate algorithms with high validity for clinical and research purposes to identify asthma exacerbation-related ED visit among the general population.
Methods
We performed a retrospective study with inclusion criteria of patients aged 4 to 40 years, a visit to Geisinger ED from January 1, 2006, to October 28, 2013, with asthma on their problem list. Different electronic data elements including chief complaints, vitals, season, smoking, medication use, and discharge diagnoses were obtained to create the algorithm. A stratified random sample was generated to select the charts for review. Chart review was performed to classify patients with asthma-related ED visit, that is, the gold standard. Two reviewers performed the chart review and validation was done on a small subset.
Results
There were 966 eligible ED visits in the EMR sample and 731 in the claims sample. Agreement between reviewers was 95.45% and kappa statistic was 0.91. Mean age of the EMR sample was 22 years, and mostly white (93%). Multiple models conventionally used in studies were evaluated and the final model chosen included principal diagnosis, bronchodilator, and steroid use for both algorithms, chief complaints for EMR, and secondary diagnosis for claims. Area under the curve was 0.93 (95% confidence interval: 0.91–0.94) and 0.94 (0.93–0.96), respectively, for EMR and claims data, with positive predictive value of > 94%. The algorithms are visually presented using nomograms.
Conclusion
We were able to develop two separate algorithms for EMR and claims to identify asthma exacerbation-related ED visit with excellent diagnostic ability and varying discrimination threshold for clinical and research purposes.
Collapse
Affiliation(s)
- Agnes S Sundaresan
- Department of Epidemiology and Health Services Research, Geisinger Health System, Danville, Pennsylvania, United States.,Medicine Institute, Geisinger Health System, Danville, Pennsylvania, United States
| | - Gargi Schneider
- MedPeds Program, Geisinger Medical Center, Danville, Pennsylvania, United States
| | - Joy Reynolds
- Lewis Katz School of Medicine at Temple University, Temple University, Philadelphia, Pennsylvania, United States
| | - H Lester Kirchner
- Department of Biomedical and Translational Informatics, Geisinger Health System, Danville, Pennsylvania, United States.,Department of Clinical Sciences, Geisinger Commonwealth School of Medicine, Geisinger Health System, Scranton, Pennsylvania, United States.,Department of Pediatrics, Global and Immigrant Health Section, Baylor College of Medicine, Houston, Texas, United States
| |
Collapse
|
147
|
Shuey MM, Gandelman JS, Chung CP, Nian H, Yu C, Denny JC, Brown NJ. Characteristics and treatment of African-American and European-American patients with resistant hypertension identified using the electronic health record in an academic health centre: a case-control study. BMJ Open 2018; 8:e021640. [PMID: 29950471 PMCID: PMC6020960 DOI: 10.1136/bmjopen-2018-021640] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
OBJECTIVE To identify patients with hypertension with resistant and controlled blood pressure (BP) using electronic health records (EHRs) in order to elucidate practices in the real-world clinical treatment of hypertension and to enable future genetic studies. DESIGN Using EHRs, we developed and validated algorithms to identify patients with resistant and controlled hypertension. SETTING An academic medical centre in Nashville, Tennessee. POPULATION European-American (EA) and African-American (AA) patients with hypertension. MAIN OUTCOME MEASURES Demographic characteristics: race, age, gender, body mass index, outpatient BPs and the history of diabetes mellitus, chronic kidney disease stage 3, ischaemic heart disease, transient ischaemic attack, atrial fibrillation and sleep apnoea. MEDICATION TREATMENT All antihypertensive medication classes prescribed to a patient at the time of classification and ever prescribed following classification. RESULTS The algorithms had performance metrics exceeding 92%. The prevalence of resistant hypertension in the total hypertensive population was 7.3% in EA and 10.5% in AA. At diagnosis, AA were younger, heavier, more often female and had a higher incidence of type 2 diabetes and higher BPs than EA. AA with resistant hypertension were more likely to be treated with vasodilators, dihydropyridine calcium channel blockers and alpha-2 agonists while EA were more likely to be treated with angiotensin receptor blockers, renin inhibitors and beta blockers. Mineralocorticoid receptor antagonists use was increased in patients treated with more than four antihypertensive medications compared with patients treated with three (12.4% vs 2.6% in EA, p<0.001; 12.3% vs 2.8% in AA, p<0.001). The number of patients treated with a mineralocorticoid receptor antagonist increased to 37.4% in EA and 41.2% in AA over a mean follow-up period of 7.4 and 8.7 years, respectively. CONCLUSIONS Clinical treatment of resistant hypertension differs in EA and AA patients. These results demonstrate the feasibility of identifying resistant hypertension using an EHR.
Collapse
Affiliation(s)
- Megan M Shuey
- Department of Pharmacology, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Jocelyn S Gandelman
- Department of Medicine, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Cecilia P Chung
- Department of Medicine, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Hui Nian
- Department of Biostatistics, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Chang Yu
- Department of Biostatistics, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Joshua C Denny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Nancy J Brown
- Department of Pharmacology, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
- Department of Medicine, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| |
Collapse
|
148
|
iT2DMS: a Standard-Based Diabetic Disease Data Repository and its Pilot Experiment on Diabetic Retinopathy Phenotyping and Examination Results Integration. J Med Syst 2018; 42:131. [DOI: 10.1007/s10916-018-0939-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2018] [Accepted: 03/14/2018] [Indexed: 01/18/2023]
|
149
|
Hoffman SR, Vines AI, Halladay JR, Pfaff E, Schiff L, Westreich D, Sundaresan A, Johnson LS, Nicholson WK. Optimizing research in symptomatic uterine fibroids with development of a computable phenotype for use with electronic health records. Am J Obstet Gynecol 2018; 218:610.e1-610.e7. [PMID: 29432754 DOI: 10.1016/j.ajog.2018.02.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2017] [Revised: 01/12/2018] [Accepted: 02/05/2018] [Indexed: 01/27/2023]
Abstract
BACKGROUND Women with symptomatic uterine fibroids can report a myriad of symptoms, including pain, bleeding, infertility, and psychosocial sequelae. Optimizing fibroid research requires the ability to enroll populations of women with image-confirmed symptomatic uterine fibroids. OBJECTIVE Our objective was to develop an electronic health record-based algorithm to identify women with symptomatic uterine fibroids for a comparative effectiveness study of medical or surgical treatments on quality-of-life measures. Using an iterative process and text-mining techniques, an effective computable phenotype algorithm, composed of demographics, and clinical and laboratory characteristics, was developed with reasonable performance. Such algorithms provide a feasible, efficient way to identify populations of women with symptomatic uterine fibroids for the conduct of large traditional or pragmatic trials and observational comparative effectiveness studies. Symptomatic uterine fibroids, due to menorrhagia, pelvic pain, bulk symptoms, or infertility, are a source of substantial morbidity for reproductive-age women. Comparing Treatment Options for Uterine Fibroids is a multisite registry study to compare the effectiveness of hormonal or surgical fibroid treatments on women's perceptions of their quality of life. Electronic health record-based algorithms are able to identify large numbers of women with fibroids, but additional work is needed to develop electronic health record algorithms that can identify women with symptomatic fibroids to optimize fibroid research. We sought to develop an efficient electronic health record-based algorithm that can identify women with symptomatic uterine fibroids in a large health care system for recruitment into large-scale observational and interventional research in fibroid management. STUDY DESIGN We developed and assessed the accuracy of 3 algorithms to identify patients with symptomatic fibroids using an iterative approach. The data source was the Carolina Data Warehouse for Health, a repository for the health system's electronic health record data. In addition to International Classification of Diseases, Ninth Revision diagnosis and procedure codes and clinical characteristics, text data-mining software was used to derive information from imaging reports to confirm the presence of uterine fibroids. Results of each algorithm were compared with expert manual review to calculate the positive predictive values for each algorithm. RESULTS Algorithm 1 was composed of the following criteria: (1) age 18-54 years; (2) either ≥1 International Classification of Diseases, Ninth Revision diagnosis codes for uterine fibroids or mention of fibroids using text-mined key words in imaging records or documents; and (3) no International Classification of Diseases, Ninth Revision or Current Procedural Terminology codes for hysterectomy and no reported history of hysterectomy. The positive predictive value was 47% (95% confidence interval 39-56%). Algorithm 2 required ≥2 International Classification of Diseases, Ninth Revision diagnosis codes for fibroids and positive text-mined key words and had a positive predictive value of 65% (95% confidence interval 50-79%). In algorithm 3, further refinements included ≥2 International Classification of Diseases, Ninth Revision diagnosis codes for fibroids on separate outpatient visit dates, the exclusion of women who had a positive pregnancy test within 3 months of their fibroid-related visit, and exclusion of incidentally detected fibroids during prenatal or emergency department visits. Algorithm 3 achieved a positive predictive value of 76% (95% confidence interval 71-81%). CONCLUSION An electronic health record-based algorithm is capable of identifying cases of symptomatic uterine fibroids with moderate positive predictive value and may be an efficient approach for large-scale study recruitment.
Collapse
Affiliation(s)
- Sarah R Hoffman
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC
| | - Anissa I Vines
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC
| | | | - Emily Pfaff
- North Carolina Translational and Clinical Sciences Institute, University of North Carolina, Chapel Hill, NC
| | - Lauren Schiff
- Department of Obstetrics and Gynecology, University of North Carolina, Chapel Hill, NC
| | - Daniel Westreich
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC
| | - Aditi Sundaresan
- Department of Obstetrics and Gynecology, University of North Carolina, Chapel Hill, NC; Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, NC
| | - La-Shell Johnson
- Department of Obstetrics and Gynecology, University of North Carolina, Chapel Hill, NC; Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, NC
| | - Wanda K Nicholson
- Department of Obstetrics and Gynecology, University of North Carolina, Chapel Hill, NC; Center for Women's Health Research, University of North Carolina, Chapel Hill, NC; Program on Women's Endocrine and Reproductive Health, School of Medicine, University of North Carolina, Chapel Hill, NC; Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, NC.
| |
Collapse
|
150
|
Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, Liu PJ, Liu X, Marcus J, Sun M, Sundberg P, Yee H, Zhang K, Zhang Y, Flores G, Duggan GE, Irvine J, Le Q, Litsch K, Mossin A, Tansuwan J, Wang D, Wexler J, Wilson J, Ludwig D, Volchenboum SL, Chou K, Pearson M, Madabushi S, Shah NH, Butte AJ, Howell MD, Cui C, Corrado GS, Dean J. Scalable and accurate deep learning with electronic health records. NPJ Digit Med 2018; 1:18. [PMID: 31304302 PMCID: PMC6550175 DOI: 10.1038/s41746-018-0029-1] [Citation(s) in RCA: 920] [Impact Index Per Article: 153.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Revised: 03/14/2018] [Accepted: 03/26/2018] [Indexed: 12/17/2022] Open
Abstract
Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient's record. We propose a representation of patients' entire raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two US academic medical centers with 216,221 adult patients hospitalized for at least 24 h. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting: in-hospital mortality (area under the receiver operator curve [AUROC] across sites 0.93-0.94), 30-day unplanned readmission (AUROC 0.75-0.76), prolonged length of stay (AUROC 0.85-0.86), and all of a patient's final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed traditional, clinically-used predictive models in all cases. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios. In a case study of a particular prediction, we demonstrate that neural networks can be used to identify relevant information from the patient's chart.
Collapse
Affiliation(s)
- Alvin Rajkomar
- Google Inc, Mountain View, CA USA
- University of California, San Francisco, San Francisco, CA USA
| | | | - Kai Chen
- Google Inc, Mountain View, CA USA
| | | | | | | | | | | | | | - Mimi Sun
- Google Inc, Mountain View, CA USA
| | | | | | | | - Yi Zhang
- Google Inc, Mountain View, CA USA
| | | | | | | | - Quoc Le
- Google Inc, Mountain View, CA USA
| | | | | | | | - De Wang
- Google Inc, Mountain View, CA USA
| | | | | | - Dana Ludwig
- University of California, San Francisco, San Francisco, CA USA
| | | | | | | | | | | | - Atul J. Butte
- University of California, San Francisco, San Francisco, CA USA
| | | | | | | | | |
Collapse
|