1
|
Sivarajkumar S, Mohammad HA, Oniani D, Roberts K, Hersh W, Liu H, He D, Visweswaran S, Wang Y. Clinical Information Retrieval: A Literature Review. J Healthc Inform Res 2024; 8:313-352. [PMID: 38681755 PMCID: PMC11052968 DOI: 10.1007/s41666-024-00159-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 12/07/2023] [Accepted: 01/08/2024] [Indexed: 05/01/2024]
Abstract
Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. The main objective was to assess and analyze the existing literature on clinical IR, focusing on the methods, techniques, and tools employed for effective retrieval and analysis of medical information. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted an extensive search across databases such as Ovid Embase, Ovid Medline, Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, covering publications from January 1, 2010, to January 4, 2023. The rigorous screening process led to the inclusion of 184 papers in our review. Our findings provide a detailed analysis of the clinical IR research landscape, covering aspects like publication trends, data sources, methodologies, evaluation metrics, and applications. The review identifies key research gaps in clinical IR methods such as indexing, ranking, and query expansion, offering insights and opportunities for future studies in clinical IR, thus serving as a guiding framework for upcoming research efforts in this rapidly evolving field. The study also underscores an imperative for innovative research on advanced clinical IR systems capable of fast semantic vector search and adoption of neural IR techniques for effective retrieval of information from unstructured electronic health records (EHRs). Supplementary Information The online version contains supplementary material available at 10.1007/s41666-024-00159-4.
Collapse
Affiliation(s)
| | | | - David Oniani
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA USA
| | - Kirk Roberts
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX USA
| | - William Hersh
- Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, Portland, OR USA
| | - Hongfang Liu
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX USA
| | - Daqing He
- Department of Information Science, University of Pittsburgh, Pittsburgh, PA USA
| | - Shyam Visweswaran
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA USA
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA USA
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA USA
| | - Yanshan Wang
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA USA
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA USA
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA USA
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA USA
| |
Collapse
|
2
|
Sivarajkumar S, Kelley M, Samolyk-Mazzanti A, Visweswaran S, Wang Y. An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study. JMIR Med Inform 2024; 12:e55318. [PMID: 38587879 PMCID: PMC11036183 DOI: 10.2196/55318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 02/20/2024] [Accepted: 02/24/2024] [Indexed: 04/09/2024] Open
Abstract
BACKGROUND Large language models (LLMs) have shown remarkable capabilities in natural language processing (NLP), especially in domains where labeled data are scarce or expensive, such as the clinical domain. However, to unlock the clinical knowledge hidden in these LLMs, we need to design effective prompts that can guide them to perform specific clinical NLP tasks without any task-specific training data. This is known as in-context learning, which is an art and science that requires understanding the strengths and weaknesses of different LLMs and prompt engineering approaches. OBJECTIVE The objective of this study is to assess the effectiveness of various prompt engineering techniques, including 2 newly introduced types-heuristic and ensemble prompts, for zero-shot and few-shot clinical information extraction using pretrained language models. METHODS This comprehensive experimental study evaluated different prompt types (simple prefix, simple cloze, chain of thought, anticipatory, heuristic, and ensemble) across 5 clinical NLP tasks: clinical sense disambiguation, biomedical evidence extraction, coreference resolution, medication status extraction, and medication attribute extraction. The performance of these prompts was assessed using 3 state-of-the-art language models: GPT-3.5 (OpenAI), Gemini (Google), and LLaMA-2 (Meta). The study contrasted zero-shot with few-shot prompting and explored the effectiveness of ensemble approaches. RESULTS The study revealed that task-specific prompt tailoring is vital for the high performance of LLMs for zero-shot clinical NLP. In clinical sense disambiguation, GPT-3.5 achieved an accuracy of 0.96 with heuristic prompts and 0.94 in biomedical evidence extraction. Heuristic prompts, alongside chain of thought prompts, were highly effective across tasks. Few-shot prompting improved performance in complex scenarios, and ensemble approaches capitalized on multiple prompt strengths. GPT-3.5 consistently outperformed Gemini and LLaMA-2 across tasks and prompt types. CONCLUSIONS This study provides a rigorous evaluation of prompt engineering methodologies and introduces innovative techniques for clinical information extraction, demonstrating the potential of in-context learning in the clinical domain. These findings offer clear guidelines for future prompt-based clinical NLP research, facilitating engagement by non-NLP experts in clinical NLP advancements. To the best of our knowledge, this is one of the first works on the empirical evaluation of different prompt engineering approaches for clinical NLP in this era of generative artificial intelligence, and we hope that it will inspire and inform future research in this area.
Collapse
Affiliation(s)
- Sonish Sivarajkumar
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
| | - Mark Kelley
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, United States
| | - Alyssa Samolyk-Mazzanti
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, United States
| | - Shyam Visweswaran
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - Yanshan Wang
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| |
Collapse
|
3
|
Sivarajkumar S, Gao F, Denny P, Aldhahwani B, Visweswaran S, Bove A, Wang Y. Mining Clinical Notes for Physical Rehabilitation Exercise Information: Natural Language Processing Algorithm Development and Validation Study. JMIR Med Inform 2024; 12:e52289. [PMID: 38568736 PMCID: PMC11024747 DOI: 10.2196/52289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 01/02/2024] [Accepted: 02/27/2024] [Indexed: 04/05/2024] Open
Abstract
BACKGROUND The rehabilitation of a patient who had a stroke requires precise, personalized treatment plans. Natural language processing (NLP) offers the potential to extract valuable exercise information from clinical notes, aiding in the development of more effective rehabilitation strategies. OBJECTIVE This study aims to develop and evaluate a variety of NLP algorithms to extract and categorize physical rehabilitation exercise information from the clinical notes of patients who had a stroke treated at the University of Pittsburgh Medical Center. METHODS A cohort of 13,605 patients diagnosed with stroke was identified, and their clinical notes containing rehabilitation therapy notes were retrieved. A comprehensive clinical ontology was created to represent various aspects of physical rehabilitation exercises. State-of-the-art NLP algorithms were then developed and compared, including rule-based, machine learning-based algorithms (support vector machine, logistic regression, gradient boosting, and AdaBoost) and large language model (LLM)-based algorithms (ChatGPT [OpenAI]). The study focused on key performance metrics, particularly F1-scores, to evaluate algorithm effectiveness. RESULTS The analysis was conducted on a data set comprising 23,724 notes with detailed demographic and clinical characteristics. The rule-based NLP algorithm demonstrated superior performance in most areas, particularly in detecting the "Right Side" location with an F1-score of 0.975, outperforming gradient boosting by 0.063. Gradient boosting excelled in "Lower Extremity" location detection (F1-score: 0.978), surpassing rule-based NLP by 0.023. It also showed notable performance in the "Passive Range of Motion" detection with an F1-score of 0.970, a 0.032 improvement over rule-based NLP. The rule-based algorithm efficiently handled "Duration," "Sets," and "Reps" with F1-scores up to 0.65. LLM-based NLP, particularly ChatGPT with few-shot prompts, achieved high recall but generally lower precision and F1-scores. However, it notably excelled in "Backward Plane" motion detection, achieving an F1-score of 0.846, surpassing the rule-based algorithm's 0.720. CONCLUSIONS The study successfully developed and evaluated multiple NLP algorithms, revealing the strengths and weaknesses of each in extracting physical rehabilitation exercise information from clinical notes. The detailed ontology and the robust performance of the rule-based and gradient boosting algorithms demonstrate significant potential for enhancing precision rehabilitation. These findings contribute to the ongoing efforts to integrate advanced NLP techniques into health care, moving toward predictive models that can recommend personalized rehabilitation treatments for optimal patient outcomes.
Collapse
Affiliation(s)
- Sonish Sivarajkumar
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
| | - Fengyi Gao
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, United States
| | - Parker Denny
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, United States
| | - Bayan Aldhahwani
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Physical Therapy, Umm Al-Qura University, Makkah, Saudi Arabia
| | - Shyam Visweswaran
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, United States
| | - Allyn Bove
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, United States
| | - Yanshan Wang
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, United States
| |
Collapse
|
4
|
Hutch MR, Son J, Le TT, Hong C, Wang X, Shakeri Hossein Abad Z, Morris M, Gutiérrez-Sacristán A, Klann JG, Spiridou A, Batugo A, Bellazzi R, Benoit V, Bonzel CL, Bryant WA, Chiudinelli L, Cho K, Das P, González González T, Hanauer DA, Henderson DW, Ho YL, Loh NHW, Makoudjou A, Makwana S, Malovini A, Moal B, Mowery DL, Neuraz A, Samayamuthu MJ, Sanz Vidorreta FJ, Schriver ER, Schubert P, Talbert J, Tan ALM, Tan BWL, Tan BWQ, Tibollo V, Tippman P, Verdy G, Yuan W, Avillach P, Gehlenborg N, Omenn GS, Visweswaran S, Cai T, Luo Y, Xia Z. Neurological diagnoses in hospitalized COVID-19 patients associated with adverse outcomes: A multinational cohort study. PLOS Digit Health 2024; 3:e0000484. [PMID: 38620037 PMCID: PMC11018281 DOI: 10.1371/journal.pdig.0000484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Accepted: 03/06/2024] [Indexed: 04/17/2024]
Abstract
Few studies examining the patient outcomes of concurrent neurological manifestations during acute COVID-19 leveraged multinational cohorts of adults and children or distinguished between central and peripheral nervous system (CNS vs. PNS) involvement. Using a federated multinational network in which local clinicians and informatics experts curated the electronic health records data, we evaluated the risk of prolonged hospitalization and mortality in hospitalized COVID-19 patients from 21 healthcare systems across 7 countries. For adults, we used a federated learning approach whereby we ran Cox proportional hazard models locally at each healthcare system and performed a meta-analysis on the aggregated results to estimate the overall risk of adverse outcomes across our geographically diverse populations. For children, we reported descriptive statistics separately due to their low frequency of neurological involvement and poor outcomes. Among the 106,229 hospitalized COVID-19 patients (104,031 patients ≥18 years; 2,198 patients <18 years, January 2020-October 2021), 15,101 (14%) had at least one CNS diagnosis, while 2,788 (3%) had at least one PNS diagnosis. After controlling for demographics and pre-existing conditions, adults with CNS involvement had longer hospital stay (11 versus 6 days) and greater risk of (Hazard Ratio = 1.78) and faster time to death (12 versus 24 days) than patients with no neurological condition (NNC) during acute COVID-19 hospitalization. Adults with PNS involvement also had longer hospital stay but lower risk of mortality than the NNC group. Although children had a low frequency of neurological involvement during COVID-19 hospitalization, a substantially higher proportion of children with CNS involvement died compared to those with NNC (6% vs 1%). Overall, patients with concurrent CNS manifestation during acute COVID-19 hospitalization faced greater risks for adverse clinical outcomes than patients without any neurological diagnosis. Our global informatics framework using a federated approach (versus a centralized data collection approach) has utility for clinical discovery beyond COVID-19.
Collapse
Affiliation(s)
- Meghan R. Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois, United States of America
| | - Jiyeon Son
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Trang T. Le
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States of America
| | - Chuan Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, United States of America
- Department of Population Health Sciences, University of Utah, Salt Lake City, Utah, United States of America
| | - Xuan Wang
- Department of Population Health Sciences, University of Utah, Salt Lake City, Utah, United States of America
| | - Zahra Shakeri Hossein Abad
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Alba Gutiérrez-Sacristán
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Jeffrey G. Klann
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Anastasia Spiridou
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | - Ashley Batugo
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States of America
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Vincent Benoit
- IT Department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - William A. Bryant
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | - Lorenzo Chiudinelli
- UOC Ricerca, Innovazione e Brand reputation, ASST Papa Giovanni XXIII, Bergamo, Italy
| | - Kelly Cho
- Population Health and Data Science, VA Boston Healthcare System, Boston Massachusetts, United States of America
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston Massachusetts, United States of America
| | - Priyam Das
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | | | - David A. Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, United States of America
| | - Darren W. Henderson
- Center for Clinical and Translational Science, University of Kentucky, Lexington, Kentucky, United States of America
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston Massachusetts, United States of America
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Kent Ridge, Singapore
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Simran Makwana
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Bertrand Moal
- IAM Unit, Bordeaux University Hospital, Bordeaux, France
| | - Danielle L. Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States of America
| | - Antoine Neuraz
- Department of biomedical informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | | | - Fernando J. Sanz Vidorreta
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, United States of America
| | - Emily R. Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, Pennsylvania, United States of America
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston Massachusetts, United States of America
| | - Jeffery Talbert
- Division of Biomedical Informatics, University of Kentucky, Lexington, Kentucky, United States of America
| | - Amelia L. M. Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Byorn W. L. Tan
- Department of Medicine, National University Hospital, Singapore, Kent Ridge, Singapore
| | - Bryce W. Q. Tan
- Department of Medicine, National University Hospital, Singapore, Kent Ridge, Singapore
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Patric Tippman
- Institute of Medical Biometry and University of Freiburg, Medical Center, Freiburg, Germany
| | | | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Gilbert S. Omenn
- Departments of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, Public Health, University of Michigan, Ann Arbor, Michigan, United States of America
| | | | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois, United States of America
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| |
Collapse
|
5
|
Anderson JW, Visweswaran S. Algorithmic Individual Fairness and Healthcare: A Scoping Review. medRxiv 2024:2024.03.25.24304853. [PMID: 38585746 PMCID: PMC10996729 DOI: 10.1101/2024.03.25.24304853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Objective Statistical and artificial intelligence algorithms are increasingly being developed for use in healthcare. These algorithms may reflect biases that magnify disparities in clinical care, and there is a growing need for understanding how algorithmic biases can be mitigated in pursuit of algorithmic fairness. Individual fairness in algorithms constrains algorithms to the notion that "similar individuals should be treated similarly." We conducted a scoping review on algorithmic individual fairness to understand the current state of research in the metrics and methods developed to achieve individual fairness and its applications in healthcare. Methods We searched three databases, PubMed, ACM Digital Library, and IEEE Xplore, for algorithmic individual fairness metrics, algorithmic bias mitigation, and healthcare applications. Our search was restricted to articles published between January 2013 and September 2023. We identified 1,886 articles through database searches and manually identified one article from which we included 30 articles in the review. Data from the selected articles were extracted, and the findings were synthesized. Results Based on the 30 articles in the review, we identified several themes, including philosophical underpinnings of fairness, individual fairness metrics, mitigation methods for achieving individual fairness, implications of achieving individual fairness on group fairness and vice versa, fairness metrics that combined individual fairness and group fairness, software for measuring and optimizing individual fairness, and applications of individual fairness in healthcare. Conclusion While there has been significant work on algorithmic individual fairness in recent years, the definition, use, and study of individual fairness remain in their infancy, especially in healthcare. Future research is needed to apply and evaluate individual fairness in healthcare comprehensively.
Collapse
Affiliation(s)
| | - Shyam Visweswaran
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA
| |
Collapse
|
6
|
Singh S, Chaudhary R, Bliden KP, Tantry US, Gurbel PA, Visweswaran S, Harinstein ME. Meta-Analysis of the Performance of AI-Driven ECG Interpretation in the Diagnosis of Valvular Heart Diseases. Am J Cardiol 2024; 213:126-131. [PMID: 38103769 PMCID: PMC10842912 DOI: 10.1016/j.amjcard.2023.12.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 11/17/2023] [Accepted: 12/01/2023] [Indexed: 12/19/2023]
Abstract
Valvular heart diseases (VHDs) significantly impact morbidity and mortality rates worldwide. Early diagnosis improves patient outcomes. Artificial intelligence (AI) applied to electrocardiogram (ECG) interpretation presents a promising approach for early VHD detection. We conducted a meta-analysis on the efficacy of AI models in this context. We reviewed databases including PubMed, MEDLINE, Embase, Scopus, and Cochrane until August 20, 2023, focusing on AI for ECG-based VHD detection. The outcomes included pooled accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value. The pooled proportions were derived using a random-effects model with 95% confidence intervals (CIs). Study heterogeneity was evaluated with the I-squared statistic. Our analysis included 10 studies, involving ECG data from 713,537 patients. The AI algorithms mainly screened for aortic stenosis (n = 6), mitral regurgitation (n = 4), aortic regurgitation (n = 3), mitral stenosis (n = 1), mitral valve prolapse (n = 2), and tricuspid regurgitation (n = 1). A total of 9 studies used convolution neural network models, whereas 1 study combined the strengths of support vector machine logistic regression and multilayer perceptron for ECG interpretation. The collective AI models demonstrated a pooled accuracy of 81% (95% CI 73 to 89, I² = 92%), sensitivity was 83% (95% CI 77 to 88, I² = 86%), specificity was 72% (95% CI 68 to 75, I² = 52%), PPV was 13% (95% CI 7 to 19, I² = 90%), and negative predictive value was 99% (95% CI 97 to 99, I² = 50%). The subgroup analyses for aortic stenosis and mitral regurgitation detection yielded analogous outcomes. In conclusion, AI-driven ECG offers high accuracy in VHD screening. However, its low PPV indicates the need for a combined approach with clinical judgment, especially in primary care settings.
Collapse
Affiliation(s)
- Sahib Singh
- Department of Medicine, Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Rahul Chaudhary
- Artificial Intelligence for Holistic Evaluation and Advancement of Cardiovascular Thrombosis (AI-HEART) Lab, Pittsburgh, Pennsylvania; Heart and Vascular Institute, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania.
| | - Kevin P Bliden
- Department of Cardiology, Sinai Center of Thrombosis Research and Drug Development, Baltimore, Maryland
| | - Udaya S Tantry
- Department of Cardiology, Sinai Center of Thrombosis Research and Drug Development, Baltimore, Maryland
| | - Paul A Gurbel
- Department of Medicine, Sinai Hospital of Baltimore, Baltimore, Maryland; Department of Cardiology, Sinai Center of Thrombosis Research and Drug Development, Baltimore, Maryland
| | - Shyam Visweswaran
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania; Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Matthew E Harinstein
- Heart and Vascular Institute, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania
| |
Collapse
|
7
|
Al-Qudah AM, Ta'ani OA, Thirumala PD, Sultan I, Visweswaran S, Nadkarni N, Kiselevskaya V, Crammond DJ, Balzer J, Anetakis KM, Shandal V, Subramaniam K, Subramanium B, Sadhasivam S. Role of Intraoperative Neuromonitoring to Predict Postoperative Delirium in Cardiovascular Surgery. J Cardiothorac Vasc Anesth 2024; 38:526-533. [PMID: 37838509 DOI: 10.1053/j.jvca.2023.09.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/20/2023] [Accepted: 09/09/2023] [Indexed: 10/16/2023]
Abstract
OBJECTIVE Postoperative delirium (POD) can occur in up to 50% of older patients undergoing cardiovascular surgery, resulting in hospitalization and significant morbidity and mortality. This study aimed to determine whether intraoperative neurophysiologic monitoring (IONM) modalities can be used to predict delirium in patients undergoing cardiovascular surgery. DESIGN Adult patients undergoing cardiovascular surgery with IONM between 2019 and 2021 were reviewed retrospectively. Delirium was assessed multiple times using the Intensive Care Delirium Screening Checklist (ICDSC). Patients with an ICDSC score ≥4 were considered to have POD. Significant IONM changes were evaluated based on a visual review of electroencephalography (EEG) and somatosensory evoked potentials data and documentation of significant changes during surgery. SETTING University of Pittsburgh Medical Center hospitals. PARTICIPANTS Patients 18 years old and older undergoing cardiovascular surgery with IONM monitoring. MEASUREMENTS AND MAIN RESULTS Of the 578 patients undergoing cardiovascular surgery with IONM, 126 had POD (21.8%). Significant IONM changes were noted in 134 patients, of whom 49 patients had delirium (36.6%). In contrast, 444 patients had no IONM changes during surgery, of whom 77 (17.3%) patients had POD. Upon multivariate analysis, IONM changes were associated with POD (odds ratio 2.12; 95% CI 1.31-3.44; p < 0.001). Additionally, baseline EEG abnormalities were associated with POD (p = 0.002). CONCLUSION Significant IONM changes are associated with an increased risk of POD in patients undergoing cardiovascular surgery. These findings offer a basis for future research and analysis of EEG and somatosensory evoked potential monitoring to predict, detect, and prevent POD.
Collapse
Affiliation(s)
- Abdullah M Al-Qudah
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Omar Al Ta'ani
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Parthasarathy D Thirumala
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA.
| | - Ibrahim Sultan
- Department of Cardiothoracic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Shyam Visweswaran
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Neelesh Nadkarni
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Victoria Kiselevskaya
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Donald J Crammond
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Jeffrey Balzer
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Katherine M Anetakis
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Varun Shandal
- Center of Clinical Neurophysiology, Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Kathirvel Subramaniam
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - Balachundhar Subramanium
- Department of Anesthesiology, Critical Care & Pain Management, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA
| | - Senthilkumar Sadhasivam
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA
| |
Collapse
|
8
|
Mina AI, Espino JU, Bradley AM, Thirumala P, Batmanghelich K, Visweswaran S. Time-Series Aware Metrics for the Evaluation of Intraoperative Electroencephalography-Based Ischemia Detection. Stud Health Technol Inform 2024; 310:274-278. [PMID: 38269808 DOI: 10.3233/shti230970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2024]
Abstract
Continuous intraoperative monitoring with electroencephalo2 graphy (EEG) is commonly used to detect cerebral ischemia in high-risk surgical procedures such as carotid endarterectomy. Machine learning (ML) models that detect ischemia in real time can form the basis of automated intraoperative EEG monitoring. In this study, we describe and compare two time-series aware precision and recall metrics to the classical precision and recall metrics for evaluating the performance of ML models that detect ischemia. We trained six ML models to detect ischemia in intraoperative EEG and evaluated them with the area under the precision-recall curve (AUPRC) using time-series aware and classical approaches to compute precision and recall. The Support Vector Classification (SVC) model performed the best on the time-series aware metrics, while the Light Gradient Boosting Machine (LGBM) model performed the best on the classical metrics. Visual inspection of the probability outputs of the models alongside the actual ischemic periods revealed that the time-series aware AUPRC selected a model more likely to predict ischemia onset in a timely fashion than the model selected by classical AUPRC.
Collapse
Affiliation(s)
- Amir I Mina
- Department of Biomedical Informatics, University of Pittsburgh, Pennsylvania, USA
| | - Jeremy U Espino
- Department of Biomedical Informatics, University of Pittsburgh, Pennsylvania, USA
| | - Allison M Bradley
- Department of Biomedical Informatics, University of Pittsburgh, Pennsylvania, USA
| | | | - Kayhan Batmanghelich
- Department of Biomedical Informatics, University of Pittsburgh, Pennsylvania, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pennsylvania, USA
| |
Collapse
|
9
|
Klann JG, Henderson DW, Morris M, Estiri H, Weber GM, Visweswaran S, Murphy SN. A broadly applicable approach to enrich electronic-health-record cohorts by identifying patients with complete data: a multisite evaluation. J Am Med Inform Assoc 2023; 30:1985-1994. [PMID: 37632234 PMCID: PMC10654861 DOI: 10.1093/jamia/ocad166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 07/25/2023] [Accepted: 08/08/2023] [Indexed: 08/27/2023] Open
Abstract
OBJECTIVE Patients who receive most care within a single healthcare system (colloquially called a "loyalty cohort" since they typically return to the same providers) have mostly complete data within that organization's electronic health record (EHR). Loyalty cohorts have low data missingness, which can unintentionally bias research results. Using proxies of routine care and healthcare utilization metrics, we compute a per-patient score that identifies a loyalty cohort. MATERIALS AND METHODS We implemented a computable program for the widely adopted i2b2 platform that identifies loyalty cohorts in EHRs based on a machine-learning model, which was previously validated using linked claims data. We developed a novel validation approach, which tests, using only EHR data, whether patients returned to the same healthcare system after the training period. We evaluated these tools at 3 institutions using data from 2017 to 2019. RESULTS Loyalty cohort calculations to identify patients who returned during a 1-year follow-up yielded a mean area under the receiver operating characteristic curve of 0.77 using the original model and 0.80 after calibrating the model at individual sites. Factors such as multiple medications or visits contributed significantly at all sites. Screening tests' contributions (eg, colonoscopy) varied across sites, likely due to coding and population differences. DISCUSSION This open-source implementation of a "loyalty score" algorithm had good predictive power. Enriching research cohorts by utilizing these low-missingness patients is a way to obtain the data completeness necessary for accurate causal analysis. CONCLUSION i2b2 sites can use this approach to select cohorts with mostly complete EHR data.
Collapse
Affiliation(s)
- Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, MA 02114, United States
- Department of Medicine, Harvard Medical School, Boston, MA 02115, United States
| | - Darren W Henderson
- Institute of Biomedical Informatics, University of Kentucky, Lexington, KY 40506, United States
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA 15260, United States
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, MA 02114, United States
- Department of Medicine, Harvard Medical School, Boston, MA 02115, United States
| | - Griffin M Weber
- Beth Israel Deaconess Medical Center, Boston, MA 02115, United States
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA 15260, United States
| | - Shawn N Murphy
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115, United States
- Department of Neurology, Massachusetts General Hospital, Boston, MA 02114, United States
- Research Information Science and Computing, Mass General Brigham, Somerville, MA 02145, United States
| |
Collapse
|
10
|
Dagliati A, Strasser ZH, Hossein Abad ZS, Klann JG, Wagholikar KB, Mesa R, Visweswaran S, Morris M, Luo Y, Henderson DW, Samayamuthu MJ, Tan BW, Verdy G, Omenn GS, Xia Z, Bellazzi R, Murphy SN, Holmes JH, Estiri H. Characterization of long COVID temporal sub-phenotypes by distributed representation learning from electronic health record data: a cohort study. EClinicalMedicine 2023; 64:102210. [PMID: 37745021 PMCID: PMC10511779 DOI: 10.1016/j.eclinm.2023.102210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 08/29/2023] [Accepted: 08/29/2023] [Indexed: 09/26/2023] Open
Abstract
Background Characterizing Post-Acute Sequelae of COVID (SARS-CoV-2 Infection), or PASC has been challenging due to the multitude of sub-phenotypes, temporal attributes, and definitions. Scalable characterization of PASC sub-phenotypes can enhance screening capacities, disease management, and treatment planning. Methods We conducted a retrospective multi-centre observational cohort study, leveraging longitudinal electronic health record (EHR) data of 30,422 patients from three healthcare systems in the Consortium for the Clinical Characterization of COVID-19 by EHR (4CE). From the total cohort, we applied a deductive approach on 12,424 individuals with follow-up data and developed a distributed representation learning process for providing augmented definitions for PASC sub-phenotypes. Findings Our framework characterized seven PASC sub-phenotypes. We estimated that on average 15.7% of the hospitalized COVID-19 patients were likely to suffer from at least one PASC symptom and almost 5.98%, on average, had multiple symptoms. Joint pain and dyspnea had the highest prevalence, with an average prevalence of 5.45% and 4.53%, respectively. Interpretation We provided a scalable framework to every participating healthcare system for estimating PASC sub-phenotypes prevalence and temporal attributes, thus developing a unified model that characterizes augmented sub-phenotypes across the different systems. Funding Authors are supported by National Institute of Allergy and Infectious Diseases, National Institute on Aging, National Center for Advancing Translational Sciences, National Medical Research Council, National Institute of Neurological Disorders and Stroke, European Union, National Institutes of Health, National Center for Advancing Translational Sciences.
Collapse
Affiliation(s)
- Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Zachary H. Strasser
- Department of Medicine, Massachusetts General Hospital, Boston, United States
| | | | - Jeffrey G. Klann
- Department of Medicine, Massachusetts General Hospital, Boston, United States
| | | | - Rebecca Mesa
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, United States
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, United States
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, United States
| | - Darren W. Henderson
- University of Kentucky, Center for Clinical and Translational Science, Lexington, United States
| | | | - Bryce W.Q. Tan
- National University Hospital, Singapore Department of Medicine, Singapore
| | - Guillame Verdy
- Bordeaux University Hospital, IAM Unit, Bordeaux, France
| | - Gilbert S. Omenn
- University of Michigan, Department of Computational Medicine and Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, Ann Arbor, United States
| | - Zongqi Xia
- University of Pittsburgh Department of Neurology, Pittsburgh, United States
| | - Riccardo Bellazzi
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Shawn N. Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, United States
| | - John H. Holmes
- University of Pennsylvania Perelman School of Medicine, Department of Biostatistics, Epidemiology, and Informatics, Institute for Biomedical Informatics, Philadelphia, United States
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, United States
| |
Collapse
|
11
|
Sperotto F, Gutiérrez-Sacristán A, Makwana S, Li X, Rofeberg VN, Cai T, Bourgeois FT, Omenn GS, Hanauer DA, Sáez C, Bonzel CL, Bucholz E, Dionne A, Elias MD, García-Barrio N, González TG, Issitt RW, Kernan KF, Laird-Gion J, Maidlow SE, Mandl KD, Ahooyi TM, Moraleda C, Morris M, Moshal KL, Pedrera-Jiménez M, Shah MA, South AM, Spiridou A, Taylor DM, Verdy G, Visweswaran S, Wang X, Xia Z, Zachariasse JM, Newburger JW, Avillach P. Clinical phenotypes and outcomes in children with multisystem inflammatory syndrome across SARS-CoV-2 variant eras: a multinational study from the 4CE consortium. EClinicalMedicine 2023; 64:102212. [PMID: 37745025 PMCID: PMC10511777 DOI: 10.1016/j.eclinm.2023.102212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 08/22/2023] [Accepted: 08/29/2023] [Indexed: 09/26/2023] Open
Abstract
Background Multisystem inflammatory syndrome in children (MIS-C) is a severe complication of SARS-CoV-2 infection. It remains unclear how MIS-C phenotypes vary across SARS-CoV-2 variants. We aimed to investigate clinical characteristics and outcomes of MIS-C across SARS-CoV-2 eras. Methods We performed a multicentre observational retrospective study including seven paediatric hospitals in four countries (France, Spain, U.K., and U.S.). All consecutive confirmed patients with MIS-C hospitalised between February 1st, 2020, and May 31st, 2022, were included. Electronic Health Records (EHR) data were used to calculate pooled risk differences (RD) and effect sizes (ES) at site level, using Alpha as reference. Meta-analysis was used to pool data across sites. Findings Of 598 patients with MIS-C (61% male, 39% female; mean age 9.7 years [SD 4.5]), 383 (64%) were admitted in the Alpha era, 111 (19%) in the Delta era, and 104 (17%) in the Omicron era. Compared with patients admitted in the Alpha era, those admitted in the Delta era were younger (ES -1.18 years [95% CI -2.05, -0.32]), had fewer respiratory symptoms (RD -0.15 [95% CI -0.33, -0.04]), less frequent non-cardiogenic shock or systemic inflammatory response syndrome (SIRS) (RD -0.35 [95% CI -0.64, -0.07]), lower lymphocyte count (ES -0.16 × 109/uL [95% CI -0.30, -0.01]), lower C-reactive protein (ES -28.5 mg/L [95% CI -46.3, -10.7]), and lower troponin (ES -0.14 ng/mL [95% CI -0.26, -0.03]). Patients admitted in the Omicron versus Alpha eras were younger (ES -1.6 years [95% CI -2.5, -0.8]), had less frequent SIRS (RD -0.18 [95% CI -0.30, -0.05]), lower lymphocyte count (ES -0.39 × 109/uL [95% CI -0.52, -0.25]), lower troponin (ES -0.16 ng/mL [95% CI -0.30, -0.01]) and less frequently received anticoagulation therapy (RD -0.19 [95% CI -0.37, -0.04]). Length of hospitalization was shorter in the Delta versus Alpha eras (-1.3 days [95% CI -2.3, -0.4]). Interpretation Our study suggested that MIS-C clinical phenotypes varied across SARS-CoV-2 eras, with patients in Delta and Omicron eras being younger and less sick. EHR data can be effectively leveraged to identify rare complications of pandemic diseases and their variation over time. Funding None.
Collapse
Affiliation(s)
- Francesca Sperotto
- Department of Cardiology, Boston Children's Hospital, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Alba Gutiérrez-Sacristán
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Simran Makwana
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Xiudi Li
- Department of Biostatistics, Harvard School of Public Health, 677 Huntington Ave, Boston, MA 02115, United States
| | - Valerie N. Rofeberg
- Department of Cardiology, Boston Children's Hospital, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Florence T. Bourgeois
- Department of Pediatrics, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Gilbert S. Omenn
- Dept of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, & Public Health, University of Michigan, 2017 Palmer Commons, Ann Arbor, MI 48109-2218, United States
| | - David A. Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, 100-107 NCRC, 2800 Plymouth Road, Ann Arbor, MI 48109, United States
| | - Carlos Sáez
- Biomedical Data Science Lab, Instituto Universitario de Tecnologías de la Información y Comunicaciones, Universitat Politécnica de Valéncia, Camino de Vera S/N, Valencia 46022, Spain
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Emily Bucholz
- Department of Cardiology, Children's Hospital Colorado, University of Colorado Anschutz, 13123 E. 16th Ave, Aurora, CO 80045, United States
| | - Audrey Dionne
- Department of Cardiology, Boston Children's Hospital, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Matthew D. Elias
- Division of Cardiology, The Children's Hospital of Philadelphia, 3401 Civic Center Boulevard, Philadelphia, PA 19104, United States
| | - Noelia García-Barrio
- Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n, Madrid 28041, Spain
| | - Tomás González González
- Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n, Madrid 28041, Spain
| | - Richard W. Issitt
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, Great Ormond Street, London WC1N 3JH, United Kingdom
| | - Kate F. Kernan
- Department of Critical Care Medicine, University of Pittsburgh, 3550 Terrace Street, Pittsburgh, PA 15213, United States
| | - Jessica Laird-Gion
- Department of Pediatrics, Boston Children's Hospital, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Sarah E. Maidlow
- Michigan Institute for Clinical and Health Research (MICHR) Informatics, University of Michigan, NCRC Bldg 400, 2800 Plymouth Road, Ann Arbor, MI 48109, United States
| | - Kenneth D. Mandl
- Computational Health Informatics Program, Boston Children's Hospital, 300 Longwood Avenue, Boston, MA 02115, United States
| | - Taha Mohseni Ahooyi
- Department of Biomedical Health Informatics, The Children's Hospital of Philadelphia, Roberts Building, 734 Schuylkill Ave, Philadelphia, PA 19146, United States
| | - Cinta Moraleda
- Pediatric Infectious Disease Department, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n, Madrid 28041, Spain
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, 5607 Baum Blvd, Pittsburgh, PA 15206, United States
| | - Karyn L. Moshal
- Department of Infectious Diseases, Great Ormond Street Hospital for Children, Great Ormond Street, London WC1N 3JH, United Kingdom
| | - Miguel Pedrera-Jiménez
- Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n, Madrid 28041, Spain
| | - Mohsin A. Shah
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, DRIVE, 40 Bernard St, London WC1N 1LE, United Kingdom
| | - Andrew M. South
- Department of Pediatrics-Section of Nephrology, Brenner Children’s, Wake Forest University School of Medicine, Medical Center Boulevard, Winston Salem, NC 27157, United States
| | - Anastasia Spiridou
- Data Research, Innovation and Virtual Environments, Great Ormond Street Hospital for Children, DRIVE, 40 Bernard St, London WC1N 1LE, United Kingdom
| | - Deanne M. Taylor
- Department of Biomedical Health Informatics, The Children's Hospital of Philadelphia, United States
- The Department of Pediatrics, University of Pennsylvania Perelman Medical School, 3601 Civic Center Blvd, 6032 Colket, Philadelphia, PA 19104, United States
| | - Guillaume Verdy
- IAM Unit, Bordeaux University Hospital, Place amélie rabat Léon, Bordeaux 33076, France
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, 5607 Baum Blvd, Pittsburgh, PA 15206, United States
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, 3501 5th Avenue, BST-3 Suite 7014, Pittsburgh, PA 15260, United States
| | - Joany M. Zachariasse
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
| | - Jane W. Newburger
- Department of Cardiology, Boston Children's Hospital, Harvard Medical School, 300 Longwood Ave, Boston, MA 02115, United States
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, United States
- Computational Health Informatics Program, Boston Children's Hospital, 300 Longwood Avenue, Boston, MA 02115, United States
| |
Collapse
|
12
|
Visweswaran S, Zhang LY, Bui K, Sadhu EM, Samayamuthu MJ, Morris MM. Sharing and Reusing Computable Phenotype Definitions. medRxiv 2023:2023.09.17.23295681. [PMID: 37790390 PMCID: PMC10543043 DOI: 10.1101/2023.09.17.23295681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Background A scalable approach for the sharing and reuse of human-readable and computer-executable phenotype definitions can facilitate the reuse of electronic health records for cohort identification and research studies. Description We developed a tool called Sharephe for the Informatics for Integrating Biology and the Bedside (i2b2) platform. Sharephe consists of a plugin for i2b2 and a cloud-based searchable repository of computable phenotypes, has the functionality to import to and export from the repository, and has the ability to link to supporting metadata. Discussion The i2b2 platform enables researchers to create, evaluate, and implement phenotypes without knowing complex query languages. In an initial evaluation, two sites on the Evolve to Next-Gen ACT (ENACT) network used Sharephe to successfully create, share, and reuse phenotypes. Conclusion The combination of a cloud-based computable repository and an i2b2 plugin for accessing the repository enables investigators to store and retrieve phenotypes from anywhere and at any time and to collaborate across sites in a research network.
Collapse
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
- The Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Kevin Bui
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Eugene M. Sadhu
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Michele M. Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
13
|
Anderson JW, Shaikh N, Visweswaran S. Measuring and Reducing Racial Bias in a Pediatric Urinary Tract Infection Model. medRxiv 2023:2023.09.18.23295660. [PMID: 37790354 PMCID: PMC10543246 DOI: 10.1101/2023.09.18.23295660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Clinical predictive models that include race as a predictor have the potential to exacerbate disparities in healthcare. Such models can be respecified to exclude race or optimized to reduce racial bias. We investigated the impact of such respecifications in a predictive model - UTICalc - which was designed to reduce catheterizations in young children with suspected urinary tract infections. To reduce racial bias, race was removed from the UTICalc logistic regression model and replaced with two new features. We compared the two versions of UTICalc using fairness and predictive performance metrics to understand the effects on racial bias. In addition, we derived three new models for UTICalc to specifically improve racial fairness. Our results show that, as predicted by previously described impossibility results, fairness cannot be simultaneously improved on all fairness metrics, and model respecification may improve racial fairness but decrease overall predictive performance.
Collapse
|
14
|
Oniani D, Parmanto B, Saptono A, Bove A, Freburger J, Visweswaran S, Cappella N, McLay B, Silverstein JC, Becich MJ, Delitto A, Skidmore E, Wang Y. ReDWINE: A clinical datamart with text analytical capabilities to facilitate rehabilitation research. Int J Med Inform 2023; 177:105144. [PMID: 37459703 PMCID: PMC10528160 DOI: 10.1016/j.ijmedinf.2023.105144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 06/14/2023] [Accepted: 07/06/2023] [Indexed: 08/12/2023]
Abstract
Rehabilitation research focuses on determining the components of a treatment intervention, the mechanism of how these components lead to recovery and rehabilitation, and ultimately the optimal intervention strategies to maximize patients' physical, psychologic, and social functioning. Traditional randomized clinical trials that study and establish new interventions face challenges, such as high cost and time commitment. Observational studies that use existing clinical data to observe the effect of an intervention have shown several advantages over RCTs. Electronic Health Records (EHRs) have become an increasingly important resource for conducting observational studies. To support these studies, we developed a clinical research datamart, called ReDWINE (Rehabilitation Datamart With Informatics iNfrastructure for rEsearch), that transforms the rehabilitation-related EHR data collected from the UPMC health care system to the Observational Health Data Sciences and Informatics (OHDSI) Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) to facilitate rehabilitation research. The standardized EHR data stored in ReDWINE will further reduce the time and effort required by investigators to pool, harmonize, clean, and analyze data from multiple sources, leading to more robust and comprehensive research findings. ReDWINE also includes deployment of data visualization and data analytics tools to facilitate cohort definition and clinical data analysis. These include among others the Open Health Natural Language Processing (OHNLP) toolkit, a high-throughput NLP pipeline, to provide text analytical capabilities at scale in ReDWINE. Using this comprehensive representation of patient data in ReDWINE for rehabilitation research will facilitate real-world evidence for health interventions and outcomes.
Collapse
Affiliation(s)
- David Oniani
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA
| | - Bambang Parmanto
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA
| | - Andi Saptono
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA
| | - Allyn Bove
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, USA
| | - Janet Freburger
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Nickie Cappella
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Brian McLay
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Jonathan C Silverstein
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Michael J Becich
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Anthony Delitto
- Department of Physical Therapy, University of Pittsburgh, Pittsburgh, PA, USA
| | - Elizabeth Skidmore
- Department of Occupational Therapy, University of Pittsburgh, Pittsburgh, PA, USA
| | - Yanshan Wang
- Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA; Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA; Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA.
| |
Collapse
|
15
|
Bhavnani SK, Zhang W, Bao D, Raji M, Ajewole V, Hunter R, Kuo YF, Schmidt S, Pappadis MR, Smith E, Bokov A, Reistetter T, Visweswaran S, Downer B. Subtyping Social Determinants of Health in All of Us: Network Analysis and Visualization Approach. medRxiv 2023:2023.01.27.23285125. [PMID: 37636340 PMCID: PMC10459353 DOI: 10.1101/2023.01.27.23285125] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 08/29/2023]
Abstract
Background Social determinants of health (SDoH), such as financial resources and housing stability, account for between 30-55% of people's health outcomes. While many studies have identified strong associations among specific SDoH and health outcomes, most people experience multiple SDoH that impact their daily lives. Analysis of this complexity requires the integration of personal, clinical, social, and environmental information from a large cohort of individuals that have been traditionally underrepresented in research, which is only recently being made available through the All of Us research program. However, little is known about the range and response of SDoH in All of Us, and how they co-occur to form subtypes, which are critical for designing targeted interventions. Objective To address two research questions: (1) What is the range and response to survey questions related to SDoH in the All of Us dataset? (2) How do SDoH co-occur to form subtypes, and what are their risk for adverse health outcomes? Methods For Question-1, an expert panel analyzed the range of SDoH questions across the surveys with respect to the 5 domains in Healthy People 2030 (HP-30), and analyzed their responses across the full All of Us data (n=372,397, V6). For Question-2, we used the following steps: (1) due to the missingness across the surveys, selected all participants with valid and complete SDoH data, and used inverse probability weighting to adjust their imbalance in demographics compared to the full data; (2) an expert panel grouped the SDoH questions into SDoH factors for enabling a more consistent granularity; (3) used bipartite modularity maximization to identify SDoH biclusters, their significance, and their replicability; (4) measured the association of each bicluster to three outcomes (depression, delayed medical care, emergency room visits in the last year) using multiple data types (surveys, electronic health records, and zip codes mapped to Medicaid expansion states); and (5) the expert panel inferred the subtype labels, potential mechanisms that precipitate adverse health outcomes, and interventions to prevent them. Results For Question-1, we identified 110 SDoH questions across 4 surveys, which covered all 5 domains in HP-30. However, the results also revealed a large degree of missingness in survey responses (1.76%-84.56%), with later surveys having significantly fewer responses compared to earlier ones, and significant differences in race, ethnicity, and age of participants of those that completed the surveys with SDoH questions, compared to those in the full All of Us dataset. Furthermore, as the SDoH questions varied in granularity, they were categorized by an expert panel into 18 SDoH factors. For Question-2, the subtype analysis (n=12,913, d=18) identified 4 biclusters with significant biclusteredness (Q=0.13, random-Q=0.11, z=7.5, P<0.001), and significant replication (Real-RI=0.88, Random-RI=0.62, P<.001). Furthermore, there were statistically significant associations between specific subtypes and the outcomes, and with Medicaid expansion, each with meaningful interpretations and potential targeted interventions. For example, the subtype Socioeconomic Barriers included the SDoH factors not employed, food insecurity, housing insecurity, low income, low literacy, and low educational attainment, and had a significantly higher odds ratio (OR=4.2, CI=3.5-5.1, P-corr<.001) for depression, when compared to the subtype Sociocultural Barriers. Individuals that match this subtype profile could be screened early for depression and referred to social services for addressing combinations of SDoH such as housing insecurity and low income. Finally, the identified subtypes spanned one or more HP-30 domains revealing the difference between the current knowledge-based SDoH domains, and the data-driven subtypes. Conclusions The results revealed that the SDoH subtypes not only had statistically significant clustering and replicability, but also had significant associations with critical adverse health outcomes, which had translational implications for designing targeted SDoH interventions, decision-support systems to alert clinicians of potential risks, and for public policies. Furthermore, these SDoH subtypes spanned multiple SDoH domains defined by HP-30 revealing the complexity of SDoH in the real-world, and aligning with influential SDoH conceptual models such as by Dahlgren-Whitehead. However, the high-degree of missingness warrants repeating the analysis as the data becomes more complete. Consequently we designed our machine learning code to be generalizable and scalable, and made it available on the All of Us workbench, which can be used to periodically rerun the analysis as the dataset grows for analyzing subtypes related to SDoH, and beyond.
Collapse
Affiliation(s)
- Suresh K. Bhavnani
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
- Institute for Translational Sciences, University of Texas Medical Branch, Galveston, TX, USA
| | - Weibin Zhang
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
| | - Daniel Bao
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
| | - Mukaila Raji
- Division of Geriatric Medicine, Department of Internal Medicine, University of Texas Medical Branch, Galveston, TX, USA
| | - Veronica Ajewole
- College of Pharmacy and Health Sciences, Texas Southern University, TX, USA
| | - Rodney Hunter
- College of Pharmacy and Health Sciences, Texas Southern University, TX, USA
| | - Yong-Fang Kuo
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
| | - Susanne Schmidt
- Department of Population Health Sciences, Long School of Medicine, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Monique R. Pappadis
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
| | - Elise Smith
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
- Institute for Translational Sciences, University of Texas Medical Branch, Galveston, TX, USA
| | - Alex Bokov
- Department of Population Health Sciences, Long School of Medicine, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Timothy Reistetter
- School of Health Professions, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Brian Downer
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, USA
| |
Collapse
|
16
|
Visweswaran S, Sadhu EM, Morris MM, Samayamuthu MJ. Clinical Algorithms with Race: An Online Database. medRxiv 2023:2023.07.04.23292231. [PMID: 37461462 PMCID: PMC10350134 DOI: 10.1101/2023.07.04.23292231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
Some clinical algorithms incorporate a person's race, ethnicity, or both as an input variable or predictor in determining diagnoses, prognoses, treatment plans, or risk assessments. Inappropriate use of race and ethnicity in clinical algorithms at the point of care may exacerbate health disparities and promote harmful practices of race-based medicine. This article describes a comprehensive search of online resources, the scientific literature, and the FDA Drug Label Information that uncovered 39 race-based risk calculators, six laboratory test results with race-based reference ranges, one race-based therapy recommendation, and 15 medications with race-based recommendations. These clinical algorithms based on race are freely accessible through an online database. This resource aims to raise awareness about the use of race-based clinical algorithms and track the progress made toward eradicating the inappropriate use of race. The database will be actively updated to include clinical algorithms based on race that were previously omitted, along with additional characteristics of these algorithms.
Collapse
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
- The Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Eugene M. Sadhu
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Michele M. Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | | |
Collapse
|
17
|
Strasser ZH, Dagliati A, Shakeri Hossein Abad Z, Klann JG, Wagholikar KB, Mesa R, Visweswaran S, Morris M, Luo Y, Henderson DW, Samayamuthu MJ, Omenn GS, Xia Z, Holmes JH, Estiri H, Murphy SN. A retrospective cohort analysis leveraging augmented intelligence to characterize long COVID in the electronic health record: A precision medicine framework. PLOS Digit Health 2023; 2:e0000301. [PMID: 37490472 PMCID: PMC10368277 DOI: 10.1371/journal.pdig.0000301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 06/16/2023] [Indexed: 07/27/2023]
Abstract
Physical and psychological symptoms lasting months following an acute COVID-19 infection are now recognized as post-acute sequelae of COVID-19 (PASC). Accurate tools for identifying such patients could enhance screening capabilities for the recruitment for clinical trials, improve the reliability of disease estimates, and allow for more accurate downstream cohort analysis. In this retrospective cohort study, we analyzed the EHR of hospitalized COVID-19 patients across three healthcare systems to develop a pipeline for better identifying patients with persistent PASC symptoms (dyspnea, fatigue, or joint pain) after their SARS-CoV-2 infection. We implemented distributed representation learning powered by the Machine Learning for modeling Health Outcomes (MLHO) to identify novel EHR features that could suggest PASC symptoms outside of typical diagnosis codes. MLHO applies an entropy-based feature selection and boosting algorithms for representation mining. These improved definitions were then used for estimating PASC among hospitalized patients. 30,422 hospitalized patients were diagnosed with COVID-19 across three healthcare systems between March 13, 2020 and February 28, 2021. The mean age of the population was 62.3 years (SD, 21.0 years) and 15,124 (49.7%) were female. We implemented the distributed representation learning technique to augment PASC definitions. These definitions were found to have positive predictive values of 0.73, 0.74, and 0.91 for dyspnea, fatigue, and joint pain, respectively. We estimated that 25 percent (CI 95%: 6-48), 11 percent (CI 95%: 6-15), and 13 percent (CI 95%: 8-17) of hospitalized COVID-19 patients will have dyspnea, fatigue, and joint pain, respectively, 3 months or longer after a COVID-19 diagnosis. We present a validated framework for screening and identifying patients with PASC in the EHR and then use the tool to estimate its prevalence among hospitalized COVID-19 patients.
Collapse
Affiliation(s)
- Zachary H. Strasser
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Zahra Shakeri Hossein Abad
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Canada
| | - Jeffrey G. Klann
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Kavishwar B. Wagholikar
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Rebecca Mesa
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois, United States of America
| | - Darren W. Henderson
- Center for Clinical and Translation Science, University of Kentucky, Lexington, Kentucky, United States of America
| | | | | | - Gilbert S. Omenn
- Dept of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, University of Michigan, Ann Arbor, Michigan, United States of America
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - John H. Holmes
- Department of Biostatistics, Epidemiology, and Informatics; Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States of America
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Shawn N. Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| |
Collapse
|
18
|
du Toit C, Tran TQB, Deo N, Aryal S, Lip S, Sykes R, Manandhar I, Sionakidis A, Stevenson L, Pattnaik H, Alsanosi S, Kassi M, Le N, Rostron M, Nichol S, Aman A, Nawaz F, Mehta D, Tummala R, McCallum L, Reddy S, Visweswaran S, Kashyap R, Joe B, Padmanabhan S. Survey and Evaluation of Hypertension Machine Learning Research. J Am Heart Assoc 2023; 12:e027896. [PMID: 37119074 PMCID: PMC10227215 DOI: 10.1161/jaha.122.027896] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Accepted: 03/27/2023] [Indexed: 04/30/2023]
Abstract
Background Machine learning (ML) is pervasive in all fields of research, from automating tasks to complex decision-making. However, applications in different specialities are variable and generally limited. Like other conditions, the number of studies employing ML in hypertension research is growing rapidly. In this study, we aimed to survey hypertension research using ML, evaluate the reporting quality, and identify barriers to ML's potential to transform hypertension care. Methods and Results The Harmonious Understanding of Machine Learning Analytics Network survey questionnaire was applied to 63 hypertension-related ML research articles published between January 2019 and September 2021. The most common research topics were blood pressure prediction (38%), hypertension (22%), cardiovascular outcomes (6%), blood pressure variability (5%), treatment response (5%), and real-time blood pressure estimation (5%). The reporting quality of the articles was variable. Only 46% of articles described the study population or derivation cohort. Most articles (81%) reported at least 1 performance measure, but only 40% presented any measures of calibration. Compliance with ethics, patient privacy, and data security regulations were mentioned in 30 (48%) of the articles. Only 14% used geographically or temporally distinct validation data sets. Algorithmic bias was not addressed in any of the articles, with only 6 of them acknowledging risk of bias. Conclusions Recent ML research on hypertension is limited to exploratory research and has significant shortcomings in reporting quality, model validation, and algorithmic bias. Our analysis identifies areas for improvement that will help pave the way for the realization of the potential of ML in hypertension and facilitate its adoption.
Collapse
Affiliation(s)
- Clea du Toit
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Tran Quoc Bao Tran
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Neha Deo
- Mayo Clinic Alix School of MedicineRochesterMN
| | - Sachin Aryal
- Center for Hypertension and Precision Medicine, Department of Physiology and PharmacologyUniversity of Toledo College of Medicine and Life SciencesToledoOH
| | - Stefanie Lip
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Robert Sykes
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Ishan Manandhar
- Center for Hypertension and Precision Medicine, Department of Physiology and PharmacologyUniversity of Toledo College of Medicine and Life SciencesToledoOH
| | | | - Leah Stevenson
- Center for Hypertension and Precision Medicine, Department of Physiology and PharmacologyUniversity of Toledo College of Medicine and Life SciencesToledoOH
| | | | - Safaa Alsanosi
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
- Department of Pharmacology and Toxicology, Faculty of MedicineUmm Al Qura UniversityMakkahSaudi Arabia
| | - Maria Kassi
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Ngoc Le
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Maggie Rostron
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Sarah Nichol
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Alisha Aman
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | - Faisal Nawaz
- College of MedicineMohammed Bin Rashid University of Medicine and Health SciencesDubaiUAE
| | - Dhruven Mehta
- Department of Internal MedicineTriStar Centennial Medical Center, HCA HealthcareNashvilleTN
| | - Ramakumar Tummala
- Center for Hypertension and Precision Medicine, Department of Physiology and PharmacologyUniversity of Toledo College of Medicine and Life SciencesToledoOH
| | - Linsay McCallum
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| | | | - Shyam Visweswaran
- Department of Biomedical InformaticsUniversity of PittsburghPittsburghPA
| | - Rahul Kashyap
- Department of Anesthesiology and Critical Care MedicineMayo ClinicRochesterMN
| | - Bina Joe
- Center for Hypertension and Precision Medicine, Department of Physiology and PharmacologyUniversity of Toledo College of Medicine and Life SciencesToledoOH
| | - Sandosh Padmanabhan
- School of Cardiovascular and Metabolic HealthUniversity of GlasgowGlasgowUnited Kingdom
| |
Collapse
|
19
|
Andrews B, Wongchokprasitti C, Visweswaran S, Lakhani CM, Patel CJ, Cooper GF. A new method for estimating the probability of causal relationships from observational data: Application to the study of the short-term effects of air pollution on cardiovascular and respiratory disease. Artif Intell Med 2023; 139:102546. [PMID: 37100513 PMCID: PMC10171833 DOI: 10.1016/j.artmed.2023.102546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 04/04/2023] [Accepted: 04/04/2023] [Indexed: 04/28/2023]
Abstract
In this paper we investigate which airborne pollutants have a short-term causal effect on cardiovascular and respiratory disease using the Ancestral Probabilities (AP) procedure, a novel Bayesian approach for deriving the probabilities of causal relationships from observational data. The results are largely consistent with EPA assessments of causality, however, in a few cases AP suggests that some pollutants thought to cause cardiovascular or respiratory disease are associated due purely to confounding. The AP procedure utilizes maximal ancestral graph (MAG) models to represent and assign probabilities to causal relationships while accounting for latent confounding. The algorithm does so locally by marginalizing over models with and without causal features of interest. Before applying AP to real data, we evaluate it in a simulation study and investigate the benefits of providing background knowledge. Overall, the results suggest that AP is an effective tool for causal discovery.
Collapse
Affiliation(s)
- Bryan Andrews
- Department of Psychiatry & Behavioral Sciences, University of Minnesota, 319 15th Avenue S.E., Minneapolis, 55455, MN, USA; Department of Biomedical Informatics, University of Pittsburgh, 4200 Fifth Ave, Pittsburgh, 15260, PA, USA.
| | - Chirayu Wongchokprasitti
- Department of Biomedical Informatics, University of Pittsburgh, 4200 Fifth Ave, Pittsburgh, 15260, PA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, 4200 Fifth Ave, Pittsburgh, 15260, PA, USA
| | | | - Chirag J Patel
- Department of Biomedical Informatics, Harvard Medical School, 25 Shattuck St, Boston, 02115, MA, USA
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, 4200 Fifth Ave, Pittsburgh, 15260, PA, USA
| |
Collapse
|
20
|
Zhang HG, Honerlaw JP, Maripuri M, Samayamuthu MJ, Beaulieu-Jones BR, Baig HS, L'Yi S, Ho YL, Morris M, Panickan VA, Wang X, Weber GM, Liao KP, Visweswaran S, Tan BWQ, Yuan W, Gehlenborg N, Muralidhar S, Ramoni RB, Kohane IS, Xia Z, Cho K, Cai T, Brat GA. Potential pitfalls in the use of real-world data for studying long COVID. Nat Med 2023; 29:1040-1043. [PMID: 37055567 PMCID: PMC10205658 DOI: 10.1038/s41591-023-02274-y] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/15/2023]
Affiliation(s)
- Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
- Division of Rheumatology, Inflammation, and Immunity, Brigham and Women's Hospital, Boston, MA, USA
| | - Jacqueline P Honerlaw
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Monika Maripuri
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | | | | | - Huma S Baig
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Katherine P Liao
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
- Division of Rheumatology, Inflammation, and Immunity, Brigham and Women's Hospital, Boston, MA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Bryce W Q Tan
- Department of Medicine, National University Hospital, Singapore, Singapore, Singapore
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Sumitra Muralidhar
- Office of Research and Development, US Department of Veterans Affairs, Washington DC, USA
| | - Rachel B Ramoni
- Office of Research and Development, US Department of Veterans Affairs, Washington DC, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, USA
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
21
|
Visweswaran S, Luo Y, Peleg M. Special Issue on Fairness and Inclusion in Biomedical Informatics Research: Technical and Social Perspectives. J Biomed Inform 2023; 141:104348. [PMID: 37023845 DOI: 10.1016/j.jbi.2023.104348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2023] [Accepted: 03/24/2023] [Indexed: 04/08/2023]
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
| | - Yuan Luo
- Director, Institute for Augmented Intelligence in Medicine, Department of Preventive Medicine, Northwestern University, Chicago, Illinois, USA.
| | - Mor Peleg
- Director, Data Science Research Center, Department of Information Systems, University of Haifa, Haifa, Israel.
| |
Collapse
|
22
|
Tan ALM, Getzen EJ, Hutch MR, Strasser ZH, Gutiérrez-Sacristán A, Le TT, Dagliati A, Morris M, Hanauer DA, Moal B, Bonzel CL, Yuan W, Chiudinelli L, Das P, Zhang HG, Aronow BJ, Avillach P, Brat GA, Cai T, Hong C, La Cava WG, Hooi Will Loh H, Luo Y, Murphy SN, Yuan Hgiam K, Omenn GS, Patel LP, Jebathilagam Samayamuthu M, Shriver ER, Shakeri Hossein Abad Z, Tan BWL, Visweswaran S, Wang X, Weber GM, Xia Z, Verdy B, Long Q, Mowery DL, Holmes JH. Informative missingness: What can we learn from patterns in missing laboratory data in the electronic health record? J Biomed Inform 2023; 139:104306. [PMID: 36738870 PMCID: PMC10849195 DOI: 10.1016/j.jbi.2023.104306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 01/21/2023] [Accepted: 01/29/2023] [Indexed: 02/05/2023]
Abstract
BACKGROUND In electronic health records, patterns of missing laboratory test results could capture patients' course of disease as well as reflect clinician's concerns or worries for possible conditions. These patterns are often understudied and overlooked. This study aims to identify informative patterns of missingness among laboratory data collected across 15 healthcare system sites in three countries for COVID-19 inpatients. METHODS We collected and analyzed demographic, diagnosis, and laboratory data for 69,939 patients with positive COVID-19 PCR tests across three countries from 1 January 2020 through 30 September 2021. We analyzed missing laboratory measurements across sites, missingness stratification by demographic variables, temporal trends of missingness, correlations between labs based on missingness indicators over time, and clustering of groups of labs based on their missingness/ordering pattern. RESULTS With these analyses, we identified mapping issues faced in seven out of 15 sites. We also identified nuances in data collection and variable definition for the various sites. Temporal trend analyses may support the use of laboratory test result missingness patterns in identifying severe COVID-19 patients. Lastly, using missingness patterns, we determined relationships between various labs that reflect clinical behaviors. CONCLUSION In this work, we use computational approaches to relate missingness patterns to hospital treatment capacity and highlight the heterogeneity of looking at COVID-19 over time and at multiple sites, where there might be different phases, policies, etc. Changes in missingness could suggest a change in a patient's condition, and patterns of missingness among laboratory measurements could potentially identify clinical outcomes. This allows sites to consider missing data as informative to analyses and help researchers identify which sites are better poised to study particular questions.
Collapse
Affiliation(s)
| | - Emily J Getzen
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | | | | | | | - Trang T Le
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | | | | | | | | | | | | | | | - Priam Das
- Harvard Medical School, Cambridge, MA, USA
| | | | - Bruce J Aronow
- Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA
| | | | | | - Tianxi Cai
- Harvard Medical School, Cambridge, MA, USA
| | - Chuan Hong
- Harvard Medical School, Cambridge, MA, USA; Duke University, Durham, NC, USA
| | - William G La Cava
- Harvard Medical School, Cambridge, MA, USA; Boston Children's Hospital, Boston, MA, USA
| | | | - Yuan Luo
- Northwestern University, Chicago, IL, USA
| | | | | | | | - Lav P Patel
- University of Kansas Medical Center, United States
| | | | - Emily R Shriver
- University of Pennsylvania Health System, Philadelphia, PA, USA
| | | | | | | | - Xuan Wang
- Harvard Medical School, Cambridge, MA, USA
| | | | - Zongqi Xia
- University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Qi Long
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Danielle L Mowery
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - John H Holmes
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| |
Collapse
|
23
|
Moal B, Orieux A, Ferté T, Neuraz A, Brat GA, Avillach P, Bonzel CL, Cai T, Cho K, Cossin S, Griffier R, Hanauer DA, Haverkamp C, Ho YL, Hong C, Hutch MR, Klann JG, Le TT, Loh NHW, Luo Y, Makoudjou A, Morris M, Mowery DL, Olson KL, Patel LP, Samayamuthu MJ, Sanz Vidorreta FJ, Schriver ER, Schubert P, Verdy G, Visweswaran S, Wang X, Weber GM, Xia Z, Yuan W, Zhang HG, Zöller D, Kohane IS, Boyer A, Jouhet V. Acute respiratory distress syndrome after SARS-CoV-2 infection on young adult population: International observational federated study based on electronic health records through the 4CE consortium. PLoS One 2023; 18:e0266985. [PMID: 36598895 DOI: 10.1371/journal.pone.0266985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 11/09/2022] [Indexed: 01/05/2023] Open
Abstract
PURPOSE In young adults (18 to 49 years old), investigation of the acute respiratory distress syndrome (ARDS) after severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has been limited. We evaluated the risk factors and outcomes of ARDS following infection with SARS-CoV-2 in a young adult population. METHODS A retrospective cohort study was conducted between January 1st, 2020 and February 28th, 2021 using patient-level electronic health records (EHR), across 241 United States hospitals and 43 European hospitals participating in the Consortium for Clinical Characterization of COVID-19 by EHR (4CE). To identify the risk factors associated with ARDS, we compared young patients with and without ARDS through a federated analysis. We further compared the outcomes between young and old patients with ARDS. RESULTS Among the 75,377 hospitalized patients with positive SARS-CoV-2 PCR, 1001 young adults presented with ARDS (7.8% of young hospitalized adults). Their mortality rate at 90 days was 16.2% and they presented with a similar complication rate for infection than older adults with ARDS. Peptic ulcer disease, paralysis, obesity, congestive heart failure, valvular disease, diabetes, chronic pulmonary disease and liver disease were associated with a higher risk of ARDS. We described a high prevalence of obesity (53%), hypertension (38%- although not significantly associated with ARDS), and diabetes (32%). CONCLUSION Trough an innovative method, a large international cohort study of young adults developing ARDS after SARS-CoV-2 infection has been gather. It demonstrated the poor outcomes of this population and associated risk factor.
Collapse
Affiliation(s)
- Bertrand Moal
- IAM Unit, Bordeaux University Hospital, Bordeaux, France
| | - Arthur Orieux
- Medical Intensive Care Unit, Bordeaux University Hospital, Bordeaux, France
| | - Thomas Ferté
- Inserm Bordeaux Population Health Research Center UMR 1219, Inria BSO, Team SISTM, University of Bordeaux, Bordeaux, France
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Kelly Cho
- Population Health and Data Science, MAVERIC, VA Boston Healthcare System, Boston, Massachusetts, United States of America
| | - Sébastien Cossin
- INSERM Bordeaux Population Health ERIAS TEAM, Bordeaux University Hospital / ERIAS - Inserm U1219 BPH, Bordeaux, France
| | - Romain Griffier
- Institute of Digitalization in Medicine, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - David A Hanauer
- IAM Unit, INSERM Bordeaux Population Health ERIAS TEAM, Bordeaux University Hospital / ERIAS - Inserm U1219 BPH, Bordeaux, France
| | - Christian Haverkamp
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan, United States of America
| | - Yuk-Lam Ho
- Institute of Digitalization in Medicine, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Meghan R Hutch
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, United States of America
| | - Jeffrey G Klann
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois, United States of America
| | - Trang T Le
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Ne Hooi Will Loh
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
| | - Yuan Luo
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Adeline Makoudjou
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States of America
| | - Michele Morris
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Danielle L Mowery
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Karen L Olson
- Department of Anaesthesia, National University Health System, Singapore, Singapore
| | - Lav P Patel
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Malarkodi J Samayamuthu
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Fernando J Sanz Vidorreta
- Computational Health Informatics Program, Boston Children's Hospital, Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Emily R Schriver
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, Kansas City, Kansas, United States of America
| | - Petra Schubert
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, United States of America
| | | | - Shyam Visweswaran
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Zongqi Xia
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, Pennsylvania, United States of America
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Daniela Zöller
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Alexandre Boyer
- Medical Intensive Care Unit, Bordeaux University Hospital, Bordeaux, France
| | - Vianney Jouhet
- Institute of Digitalization in Medicine, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| |
Collapse
|
24
|
Tan BW, Tan BW, Tan AL, Schriver ER, Gutiérrez-Sacristán A, Das P, Yuan W, Hutch MR, García Barrio N, Pedrera Jimenez M, Abu-el-rub N, Morris M, Moal B, Verdy G, Cho K, Ho YL, Patel LP, Dagliati A, Neuraz A, Klann JG, South AM, Visweswaran S, Hanauer DA, Maidlow SE, Liu M, Mowery DL, Batugo A, Makoudjou A, Tippmann P, Zöller D, Brat GA, Luo Y, Avillach P, Bellazzi R, Chiovato L, Malovini A, Tibollo V, Samayamuthu MJ, Serrano Balazote P, Xia Z, Loh NHW, Chiudinelli L, Bonzel CL, Hong C, Zhang HG, Weber GM, Kohane IS, Cai T, Omenn GS, Holmes JH, Ngiam KY. Long-term kidney function recovery and mortality after COVID-19-associated acute kidney injury: An international multi-centre observational cohort study. EClinicalMedicine 2023; 55:101724. [PMID: 36381999 PMCID: PMC9640184 DOI: 10.1016/j.eclinm.2022.101724] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 10/12/2022] [Accepted: 10/12/2022] [Indexed: 11/09/2022] Open
Abstract
Background While acute kidney injury (AKI) is a common complication in COVID-19, data on post-AKI kidney function recovery and the clinical factors associated with poor kidney function recovery is lacking. Methods A retrospective multi-centre observational cohort study comprising 12,891 hospitalized patients aged 18 years or older with a diagnosis of SARS-CoV-2 infection confirmed by polymerase chain reaction from 1 January 2020 to 10 September 2020, and with at least one serum creatinine value 1-365 days prior to admission. Mortality and serum creatinine values were obtained up to 10 September 2021. Findings Advanced age (HR 2.77, 95%CI 2.53-3.04, p < 0.0001), severe COVID-19 (HR 2.91, 95%CI 2.03-4.17, p < 0.0001), severe AKI (KDIGO stage 3: HR 4.22, 95%CI 3.55-5.00, p < 0.0001), and ischemic heart disease (HR 1.26, 95%CI 1.14-1.39, p < 0.0001) were associated with worse mortality outcomes. AKI severity (KDIGO stage 3: HR 0.41, 95%CI 0.37-0.46, p < 0.0001) was associated with worse kidney function recovery, whereas remdesivir use (HR 1.34, 95%CI 1.17-1.54, p < 0.0001) was associated with better kidney function recovery. In a subset of patients without chronic kidney disease, advanced age (HR 1.38, 95%CI 1.20-1.58, p < 0.0001), male sex (HR 1.67, 95%CI 1.45-1.93, p < 0.0001), severe AKI (KDIGO stage 3: HR 11.68, 95%CI 9.80-13.91, p < 0.0001), and hypertension (HR 1.22, 95%CI 1.10-1.36, p = 0.0002) were associated with post-AKI kidney function impairment. Furthermore, patients with COVID-19-associated AKI had significant and persistent elevations of baseline serum creatinine 125% or more at 180 days (RR 1.49, 95%CI 1.32-1.67) and 365 days (RR 1.54, 95%CI 1.21-1.96) compared to COVID-19 patients with no AKI. Interpretation COVID-19-associated AKI was associated with higher mortality, and severe COVID-19-associated AKI was associated with worse long-term post-AKI kidney function recovery. Funding Authors are supported by various funders, with full details stated in the acknowledgement section.
Collapse
Affiliation(s)
- Byorn W.L. Tan
- Department of Medicine, National University Hospital, 1E Kent Ridge Road, NUHS Tower Block Level 10, Singapore 119228
| | - Bryce W.Q. Tan
- Department of Medicine, National University Hospital, 1E Kent Ridge Road, NUHS Tower Block Level 10, Singapore 119228
| | - Amelia L.M. Tan
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Emily R. Schriver
- Data Analytics Center, University of Pennsylvania Health System, 3600 Civic Center Boulevard, Philadelphia, PA 19104, USA
| | - Alba Gutiérrez-Sacristán
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Priyam Das
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Meghan R. Hutch
- Department of Preventive Medicine, Northwestern University, 750 North Lake Shore Drive, Chicago, IL 60611, USA
| | - Noelia García Barrio
- Department of Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n 28041 Madrid, Spain
| | - Miguel Pedrera Jimenez
- Department of Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n 28041 Madrid, Spain
| | - Noor Abu-el-rub
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, KS 66160, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, 5607 Baum Blvd, Pittsburgh, PA 15206, USA
| | - Bertrand Moal
- IAM Unit, Bordeaux University Hospital, Place Amélie Rabat Léon, 33076 Bordeaux, France
| | - Guillaume Verdy
- IAM Unit, Bordeaux University Hospital, Place Amélie Rabat Léon, 33076 Bordeaux, France
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center, VA Boston Healthcare System, 2 Avenue De Lafayette, Boston, MA 02130, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center, VA Boston Healthcare System, 2 Avenue De Lafayette, Boston, MA 02130, USA
| | - Lav P. Patel
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, KS 66160, USA
| | - Arianna Dagliati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy, Via Ferrata 5, 27100 Pavia, Italy
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris, University of Paris, 149 Rue de Sèvres, 75015 Paris, France
| | - Jeffrey G. Klann
- Department of Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114, USA
| | - Andrew M. South
- Department of Pediatrics-Section of Nephrology, Brenner Children's Hospital, Wake Forest School of Medicine, Medical Center Boulevard, Winston Salem, NC 27157, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, 5607 Baum Blvd, Pittsburgh, PA 15206, USA
| | - David A. Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, USA, 100-107 NCRC, 2800 Plymouth Road, Ann Arbor, MI 48109, USA
| | - Sarah E. Maidlow
- Michigan Institute for Clinical and Health Research (MICHR) Informatics, University of Michigan, NCRC Bldg 400, 2800 Plymouth Road, Ann Arbor, MI, United States
| | - Mei Liu
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, KS 66160, USA
| | - Danielle L. Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, 3700 Hamilton Walk, Richards Hall, A202, Philadelphia, PA 19104, USA
| | - Ashley Batugo
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, 401 Blockley Hall 423 Guardian Drive Philadelphia, PA 19104, USA
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Zinkmattenstraße 6a, DE79108 Freiburg, Germany
| | - Patric Tippmann
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Zinkmattenstraße 6a, DE79108 Freiburg, Germany
| | - Daniela Zöller
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Zinkmattenstraße 6a, DE79108 Freiburg, Germany
| | - Gabriel A. Brat
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, 750 North Lake Shore Drive, Chicago, IL 60611, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy, Via Ferrata 5, 27100 Pavia, Italy
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Via Maugeri 4, 27100 Pavia, Italy
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy., Via Maugeri 4, 27100 Pavia, Italy
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy., Via Maugeri 4, 27100 Pavia, Italy
| | | | - Pablo Serrano Balazote
- Department of Health Informatics, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n 28041 Madrid, Spain
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, 3501 5th Avenue, BST-3 Suite 7014, Pittsburgh, PA 15260, USA
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, 5 Lower Kent Ridge Road, Singapore 119074
| | - Lorenzo Chiudinelli
- UOC Ricerca, Innovazione e Brand reputation, ASST Papa Giovanni XXIII, Bergamo, P.zza OMS 1 - 24127 Bergamo, Italy
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
- Department of Biostatistics and Bioinformatics, Duke University, 2424 Erwin Road, Durham, NC, United States
| | - Harrison G. Zhang
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Griffin M. Weber
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Isaac S. Kohane
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA 02115, USA
| | - Gilbert S. Omenn
- Department of Computational Medicine & Bioinformatics, University of Michigan, 2017B Palmer Commons, 100 Washtenaw, Ann Arbor, MI 48109-2218
| | - John H. Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, 3700 Hamilton Walk, Richards Hall, A202, Philadelphia, PA 19104, USA
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, 401 Blockley Hall 423 Guardian Drive Philadelphia, PA 19104, USA
| | - Kee Yuan Ngiam
- Department of Biomedical Informatics, WiSDM, National University Health Systems Singapore, 1E Kent Ridge Road, NUHS Tower Block Level 8, Singapore 119228
- Corresponding author. Department of Biomedical Informatics, WiSDM, National University Health Systems Singapore, 1E Kent Ridge Road, NUHS Tower Block Level 8, Singapore 119228.
| | | |
Collapse
|
25
|
Lovis C, Zhang W, Visweswaran S, Raji M, Kuo YF. A Framework for Modeling and Interpreting Patient Subgroups Applied to Hospital Readmission: Visual Analytical Approach. JMIR Med Inform 2022; 10:e37239. [PMID: 35537203 PMCID: PMC9773032 DOI: 10.2196/37239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 04/06/2022] [Accepted: 05/02/2022] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND A primary goal of precision medicine is to identify patient subgroups and infer their underlying disease processes with the aim of designing targeted interventions. Although several studies have identified patient subgroups, there is a considerable gap between the identification of patient subgroups and their modeling and interpretation for clinical applications. OBJECTIVE This study aimed to develop and evaluate a novel analytical framework for modeling and interpreting patient subgroups (MIPS) using a 3-step modeling approach: visual analytical modeling to automatically identify patient subgroups and their co-occurring comorbidities and determine their statistical significance and clinical interpretability; classification modeling to classify patients into subgroups and measure its accuracy; and prediction modeling to predict a patient's risk of an adverse outcome and compare its accuracy with and without patient subgroup information. METHODS The MIPS framework was developed using bipartite networks to identify patient subgroups based on frequently co-occurring high-risk comorbidities, multinomial logistic regression to classify patients into subgroups, and hierarchical logistic regression to predict the risk of an adverse outcome using subgroup membership compared with standard logistic regression without subgroup membership. The MIPS framework was evaluated for 3 hospital readmission conditions: chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), and total hip arthroplasty/total knee arthroplasty (THA/TKA) (COPD: n=29,016; CHF: n=51,550; THA/TKA: n=16,498). For each condition, we extracted cases defined as patients readmitted within 30 days of hospital discharge. Controls were defined as patients not readmitted within 90 days of discharge, matched by age, sex, race, and Medicaid eligibility. RESULTS In each condition, the visual analytical model identified patient subgroups that were statistically significant (Q=0.17, 0.17, 0.31; P<.001, <.001, <.05), significantly replicated (Rand Index=0.92, 0.94, 0.89; P<.001, <.001, <.01), and clinically meaningful to clinicians. In each condition, the classification model had high accuracy in classifying patients into subgroups (mean accuracy=99.6%, 99.34%, 99.86%). In 2 conditions (COPD and THA/TKA), the hierarchical prediction model had a small but statistically significant improvement in discriminating between readmitted and not readmitted patients as measured by net reclassification improvement (0.059, 0.11) but not as measured by the C-statistic or integrated discrimination improvement. CONCLUSIONS Although the visual analytical models identified statistically and clinically significant patient subgroups, the results pinpoint the need to analyze subgroups at different levels of granularity for improving the interpretability of intra- and intercluster associations. The high accuracy of the classification models reflects the strong separation of patient subgroups, despite the size and density of the data sets. Finally, the small improvement in predictive accuracy suggests that comorbidities alone were not strong predictors of hospital readmission, and the need for more sophisticated subgroup modeling methods. Such advances could improve the interpretability and predictive accuracy of patient subgroup models for reducing the risk of hospital readmission, and beyond.
Collapse
Affiliation(s)
| | - Weibin Zhang
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - Mukaila Raji
- Division of Geriatric Medicine, Department of Internal Medicine, University of Texas Medical Branch, Galveston, TX, United States
| | - Yong-Fang Kuo
- School of Public and Population Health, University of Texas Medical Branch, Galveston, TX, United States
| |
Collapse
|
26
|
Gutiérrez-Sacristán A, Serret-Larmande A, Hutch MR, Sáez C, Aronow BJ, Bhatnagar S, Bonzel CL, Cai T, Devkota B, Hanauer DA, Loh NHW, Luo Y, Moal B, Ahooyi TM, Njoroge WFM, Omenn GS, Sanchez-Pinto LN, South AM, Sperotto F, Tan ALM, Taylor DM, Verdy G, Visweswaran S, Xia Z, Zahner J, Avillach P, Bourgeois FT. Hospitalizations Associated With Mental Health Conditions Among Adolescents in the US and France During the COVID-19 Pandemic. JAMA Netw Open 2022; 5:e2246548. [PMID: 36512353 PMCID: PMC9856226 DOI: 10.1001/jamanetworkopen.2022.46548] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 10/21/2022] [Indexed: 12/15/2022] Open
Abstract
Importance The COVID-19 pandemic has been associated with an increase in mental health diagnoses among adolescents, though the extent of the increase, particularly for severe cases requiring hospitalization, has not been well characterized. Large-scale federated informatics approaches provide the ability to efficiently and securely query health care data sets to assess and monitor hospitalization patterns for mental health conditions among adolescents. Objective To estimate changes in the proportion of hospitalizations associated with mental health conditions among adolescents following onset of the COVID-19 pandemic. Design, Setting, and Participants This retrospective, multisite cohort study of adolescents 11 to 17 years of age who were hospitalized with at least 1 mental health condition diagnosis between February 1, 2019, and April 30, 2021, used patient-level data from electronic health records of 8 children's hospitals in the US and France. Main Outcomes and Measures Change in the monthly proportion of mental health condition-associated hospitalizations between the prepandemic (February 1, 2019, to March 31, 2020) and pandemic (April 1, 2020, to April 30, 2021) periods using interrupted time series analysis. Results There were 9696 adolescents hospitalized with a mental health condition during the prepandemic period (5966 [61.5%] female) and 11 101 during the pandemic period (7603 [68.5%] female). The mean (SD) age in the prepandemic cohort was 14.6 (1.9) years and in the pandemic cohort, 14.7 (1.8) years. The most prevalent diagnoses during the pandemic were anxiety (6066 [57.4%]), depression (5065 [48.0%]), and suicidality or self-injury (4673 [44.2%]). There was an increase in the proportions of monthly hospitalizations during the pandemic for anxiety (0.55%; 95% CI, 0.26%-0.84%), depression (0.50%; 95% CI, 0.19%-0.79%), and suicidality or self-injury (0.38%; 95% CI, 0.08%-0.68%). There was an estimated 0.60% increase (95% CI, 0.31%-0.89%) overall in the monthly proportion of mental health-associated hospitalizations following onset of the pandemic compared with the prepandemic period. Conclusions and Relevance In this cohort study, onset of the COVID-19 pandemic was associated with increased hospitalizations with mental health diagnoses among adolescents. These findings support the need for greater resources within children's hospitals to care for adolescents with mental health conditions during the pandemic and beyond.
Collapse
Affiliation(s)
| | - Arnaud Serret-Larmande
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
- Department of Biostatistics and Biomedical Informatics, Hôpital Saint-Louis, Assistance Publique-Hôpitaux de Paris, Université Paris-Cité, Paris, France
| | - Meghan R Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois
| | - Carlos Sáez
- Biomedical Data Science Lab, Instituto Universitario de Tecnologías de la Información y Comunicaciones, Universitat Politècnica de València, València, Spain
| | - Bruce J Aronow
- Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, Ohio
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, Ohio
| | - Surbhi Bhatnagar
- Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, Ohio
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, Ohio
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Batsal Devkota
- Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Singapore
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, Illinois
| | - Bertrand Moal
- Unité Informatique et Archivistique Médicale, Bordeaux University Hospital, Bordeaux, France
| | - Taha Mohseni Ahooyi
- Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Wanjiku F M Njoroge
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Gilbert S Omenn
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor
| | - L Nelson Sanchez-Pinto
- Department of Pediatrics (Critical Care), Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Andrew M South
- Department of Pediatrics-Section of Nephrology, Brenner Children's, Wake Forest University School of Medicine, Winston Salem, North Carolina
| | - Francesca Sperotto
- Department of Cardiology, Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Deanne M Taylor
- Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
- Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Guillaume Verdy
- Unité Informatique et Archivistique Médicale, Bordeaux University Hospital, Bordeaux, France
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Janet Zahner
- Department of Information Services, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
- Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
- Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts
| | - Florence T Bourgeois
- Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts
- Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
27
|
Wang X, Zhang HG, Xiong X, Hong C, Weber GM, Brat GA, Bonzel CL, Luo Y, Duan R, Palmer NP, Hutch MR, Gutiérrez-Sacristán A, Bellazzi R, Chiovato L, Cho K, Dagliati A, Estiri H, García-Barrio N, Griffier R, Hanauer DA, Ho YL, Holmes JH, Keller MS, Klann MEng JG, L'Yi S, Lozano-Zahonero S, Maidlow SE, Makoudjou A, Malovini A, Moal B, Moore JH, Morris M, Mowery DL, Murphy SN, Neuraz A, Yuan Ngiam K, Omenn GS, Patel LP, Pedrera-Jiménez M, Prunotto A, Jebathilagam Samayamuthu M, Sanz Vidorreta FJ, Schriver ER, Schubert P, Serrano-Balazote P, South AM, Tan ALM, Tan BWL, Tibollo V, Tippmann P, Visweswaran S, Xia Z, Yuan W, Zöller D, Kohane IS, Avillach P, Guo Z, Cai T. SurvMaximin: Robust federated approach to transporting survival risk prediction models. J Biomed Inform 2022; 134:104176. [PMID: 36007785 PMCID: PMC9707637 DOI: 10.1016/j.jbi.2022.104176] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Revised: 07/18/2022] [Accepted: 08/15/2022] [Indexed: 10/15/2022]
Abstract
OBJECTIVE For multi-center heterogeneous Real-World Data (RWD) with time-to-event outcomes and high-dimensional features, we propose the SurvMaximin algorithm to estimate Cox model feature coefficients for a target population by borrowing summary information from a set of health care centers without sharing patient-level information. MATERIALS AND METHODS For each of the centers from which we want to borrow information to improve the prediction performance for the target population, a penalized Cox model is fitted to estimate feature coefficients for the center. Using estimated feature coefficients and the covariance matrix of the target population, we then obtain a SurvMaximin estimated set of feature coefficients for the target population. The target population can be an entire cohort comprised of all centers, corresponding to federated learning, or a single center, corresponding to transfer learning. RESULTS Simulation studies and a real-world international electronic health records application study, with 15 participating health care centers across three countries (France, Germany, and the U.S.), show that the proposed SurvMaximin algorithm achieves comparable or higher accuracy compared with the estimator using only the information of the target site and other existing methods. The SurvMaximin estimator is robust to variations in sample sizes and estimated feature coefficients between centers, which amounts to significantly improved estimates for target sites with fewer observations. CONCLUSIONS The SurvMaximin method is well suited for both federated and transfer learning in the high-dimensional survival analysis setting. SurvMaximin only requires a one-time summary information exchange from participating centers. Estimated regression vectors can be very heterogeneous. SurvMaximin provides robust Cox feature coefficient estimates without outcome information in the target population and is privacy-preserving.
Collapse
Affiliation(s)
- Xuan Wang
- Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, MA, USA
| | - Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Xin Xiong
- Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, MA, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Yuan Luo
- Department of Preventive Medicine Northwestern University, Chicago, IL, USA
| | - Rui Duan
- Department of Biostatistics, Harvard University, Boston, MA, USA
| | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Meghan R Hutch
- Department of Preventive Medicine Northwestern University, Chicago, IL, USA
| | | | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kelly Cho
- Population Health and Data Science, VA Boston Healthcare System, Boston, MA, USA; Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Arianna Dagliati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
| | | | - Romain Griffier
- IAM unit, Bordeaux University Hospital, Bordeaux, France; INSERM Bordeaux Population Health ERIAS TEAM, ERIAS - Inserm U1219 BPH, Bordeaux, France
| | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Mark S Keller
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | | | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Sara Lozano-Zahonero
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Sarah E Maidlow
- Michigan Institute for Clinical and Health Research (MICHR) Informatics, University of Michigan, Ann Arbor, MI, USA
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Bertrand Moal
- IAM unit, Bordeaux University Hospital, Bordeaux, France
| | - Jason H Moore
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | - Antoine Neuraz
- Department of biomedical informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | - Kee Yuan Ngiam
- Department of Biomedical informatics, WiSDM, National University Health Systems, Singapore
| | - Gilbert S Omenn
- Depts of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, Public Health University of Michigan, Ann Arbor, MI, USA
| | - Lav P Patel
- Department of Internal Medicine, Division of Medical Informatics, University Of Kansas Medical Center
| | | | - Andrea Prunotto
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | | | | | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, PA, USA
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | | | - Andrew M South
- Department of Pediatrics-Section of Nephrology, Brenner Children's, Wake Forest School of Medicine, Winston Salem, NC, USA
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Byorn W L Tan
- Department of Medicine, National University Hospital, Singapore
| | - Valentina Tibollo
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Patric Tippmann
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, USA
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Daniela Zöller
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Zijian Guo
- Department of Statistics, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
28
|
Ramirez AH, Sulieman L, Schlueter DJ, Halvorson A, Qian J, Ratsimbazafy F, Loperena R, Mayo K, Basford M, Deflaux N, Muthuraman KN, Natarajan K, Kho A, Xu H, Wilkins C, Anton-Culver H, Boerwinkle E, Cicek M, Clark CR, Cohn E, Ohno-Machado L, Schully SD, Ahmedani BK, Argos M, Cronin RM, O’Donnell C, Fouad M, Goldstein DB, Greenland P, Hebbring SJ, Karlson EW, Khatri P, Korf B, Smoller JW, Sodeke S, Wilbanks J, Hentges J, Mockrin S, Lunt C, Devaney SA, Gebo K, Denny JC, Carroll RJ, Glazer D, Harris PA, Hripcsak G, Philippakis A, Roden DM, Ahmedani B, Cole Johnson CD, Ahsan H, Antoine-LaVigne D, Singleton G, Anton-Culver H, Topol E, Baca-Motes K, Steinhubl S, Wade J, Begale M, Jain P, Sutherland S, Lewis B, Korf B, Behringer M, Gharavi AG, Goldstein DB, Hripcsak G, Bier L, Boerwinkle E, Brilliant MH, Murali N, Hebbring SJ, Farrar-Edwards D, Burnside E, Drezner MK, Taylor A, Channamsetty V, Montalvo W, Sharma Y, Chinea C, Jenks N, Cicek M, Thibodeau S, Holmes BW, Schlueter E, Collier E, Winkler J, Corcoran J, D’Addezio N, Daviglus M, Winn R, Wilkins C, Roden D, Denny J, Doheny K, Nickerson D, Eichler E, Jarvik G, Funk G, Philippakis A, Rehm H, Lennon N, Kathiresan S, Gabriel S, Gibbs R, Gil Rico EM, Glazer D, Grand J, Greenland P, Harris P, Shenkman E, Hogan WR, Igho-Pemu P, Pollan C, Jorge M, Okun S, Karlson EW, Smoller J, Murphy SN, Ross ME, Kaushal R, Winford E, Wallace F, Khatri P, Kheterpal V, Ojo A, Moreno FA, Kron I, Peterson R, Menon U, Lattimore PW, Leviner N, Obedin-Maliver J, Lunn M, Malik-Gagnon L, Mangravite L, Marallo A, Marroquin O, Visweswaran S, Reis S, Marshall G, McGovern P, Mignucci D, Moore J, Munoz F, Talavera G, O'Connor GT, O'Donnell C, Ohno-Machado L, Orr G, Randal F, Theodorou AA, Reiman E, Roxas-Murray M, Stark L, Tepp R, Zhou A, Topper S, Trousdale R, Tsao P, Weidman L, Weiss ST, Wellis D, Whittle J, Wilson A, Zuchner S, Zwick ME. The All of Us Research Program: Data quality, utility, and diversity. Patterns 2022; 3:100570. [PMID: 36033590 PMCID: PMC9403360 DOI: 10.1016/j.patter.2022.100570] [Citation(s) in RCA: 50] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 03/30/2022] [Accepted: 07/14/2022] [Indexed: 11/05/2022]
Abstract
The All of Us Research Program seeks to engage at least one million diverse participants to advance precision medicine and improve human health. We describe here the cloud-based Researcher Workbench that uses a data passport model to democratize access to analytical tools and participant information including survey, physical measurement, and electronic health record (EHR) data. We also present validation study findings for several common complex diseases to demonstrate use of this novel platform in 315,000 participants, 78% of whom are from groups historically underrepresented in biomedical research, including 49% self-reporting non-White races. Replication findings include medication usage pattern differences by race in depression and type 2 diabetes, validation of known cancer associations with smoking, and calculation of cardiovascular risk scores by reported race effects. The cloud-based Researcher Workbench represents an important advance in enabling secure access for a broad range of researchers to this large resource and analytical tools. The All of Us Research Program has released data for over 315,000 participants Demonstration projects support the utility and validity of the All of Us dataset The cloud-based Researcher Workbench provides secure, low-cost compute power
The engagement of participants in the research process and broad availability of data to diverse researchers are essential elements in building precision medicine equitably available for all. The NIH has established the ambitious All of Us Research Program to build one of the most diverse health databases in history with tools to support research to improve human health. Here, we present the initial launch of the Researcher Workbench with data types including surveys, physical measurements, and electronic health record data with validation studies to support researcher use of this novel platform. Broad access for researchers to data like these is a critical step in returning value to participants seeking to support the advancement of precision medicine and improved health for all.
Collapse
|
29
|
Zhang HG, Dagliati A, Shakeri Hossein Abad Z, Xiong X, Bonzel CL, Xia Z, Tan BWQ, Avillach P, Brat GA, Hong C, Morris M, Visweswaran S, Patel LP, Gutiérrez-Sacristán A, Hanauer DA, Holmes JH, Samayamuthu MJ, Bourgeois FT, L'Yi S, Maidlow SE, Moal B, Murphy SN, Strasser ZH, Neuraz A, Ngiam KY, Loh NHW, Omenn GS, Prunotto A, Dalvin LA, Klann JG, Schubert P, Vidorreta FJS, Benoit V, Verdy G, Kavuluru R, Estiri H, Luo Y, Malovini A, Tibollo V, Bellazzi R, Cho K, Ho YL, Tan ALM, Tan BWL, Gehlenborg N, Lozano-Zahonero S, Jouhet V, Chiovato L, Aronow BJ, Toh EMS, Wong WGS, Pizzimenti S, Wagholikar KB, Bucalo M, Cai T, South AM, Kohane IS, Weber GM. International electronic health record-derived post-acute sequelae profiles of COVID-19 patients. NPJ Digit Med 2022; 5:81. [PMID: 35768548 PMCID: PMC9242995 DOI: 10.1038/s41746-022-00623-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Accepted: 05/19/2022] [Indexed: 11/10/2022] Open
Abstract
The risk profiles of post-acute sequelae of COVID-19 (PASC) have not been well characterized in multi-national settings with appropriate controls. We leveraged electronic health record (EHR) data from 277 international hospitals representing 414,602 patients with COVID-19, 2.3 million control patients without COVID-19 in the inpatient and outpatient settings, and over 221 million diagnosis codes to systematically identify new-onset conditions enriched among patients with COVID-19 during the post-acute period. Compared to inpatient controls, inpatient COVID-19 cases were at significant risk for angina pectoris (RR 1.30, 95% CI 1.09–1.55), heart failure (RR 1.22, 95% CI 1.10–1.35), cognitive dysfunctions (RR 1.18, 95% CI 1.07–1.31), and fatigue (RR 1.18, 95% CI 1.07–1.30). Relative to outpatient controls, outpatient COVID-19 cases were at risk for pulmonary embolism (RR 2.10, 95% CI 1.58–2.76), venous embolism (RR 1.34, 95% CI 1.17–1.54), atrial fibrillation (RR 1.30, 95% CI 1.13–1.50), type 2 diabetes (RR 1.26, 95% CI 1.16–1.36) and vitamin D deficiency (RR 1.19, 95% CI 1.09–1.30). Outpatient COVID-19 cases were also at risk for loss of smell and taste (RR 2.42, 95% CI 1.90–3.06), inflammatory neuropathy (RR 1.66, 95% CI 1.21–2.27), and cognitive dysfunction (RR 1.18, 95% CI 1.04–1.33). The incidence of post-acute cardiovascular and pulmonary conditions decreased across time among inpatient cases while the incidence of cardiovascular, digestive, and metabolic conditions increased among outpatient cases. Our study, based on a federated international network, systematically identified robust conditions associated with PASC compared to control groups, underscoring the multifaceted cardiovascular and neurological phenotype profiles of PASC.
Collapse
Affiliation(s)
- Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | | | - Xin Xiong
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, USA
| | - Bryce W Q Tan
- Department of Medicine, National University Hospital, Singapore, Singapore
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Lav P Patel
- Department of Internal Medicine, Division of Medical Informatics, University Of Kansas Medical Center, Kansas City, MO, USA
| | | | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, USA
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA.,Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | | | | | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Sarah E Maidlow
- Michigan Institute for Clinical and Health Research (MICHR) Informatics, University of Michigan, Ann Arbor, MI, USA
| | - Bertrand Moal
- IAM unit, Bordeaux University Hospital, Bordeaux, France
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | | | - Antoine Neuraz
- Department of biomedical informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | - Kee Yuan Ngiam
- Department of Biomedical informatics, WiSDM, National University Health Systems Singapore, Singapore, Singapore
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health Systems Singapore, Singapore, Singapore
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, University of Michigan, Ann Arbor, MI, USA
| | - Andrea Prunotto
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Lauren A Dalvin
- Department of Ophthalmology, Mayo Clinic, Rochester, NY, USA
| | - Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | | | - Vincent Benoit
- IT Department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France
| | | | - Ramakanth Kavuluru
- Division of Biomedical Informatics (Department of Internal Medicine), University of Kentucky, Lexington, KY, USA
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA.,Population Health and Data Science, VA Boston Healthcare System, Boston, MA, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, USA
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Byorn W L Tan
- Department of Medicine, National University Hospital, Singapore, Singapore
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Sara Lozano-Zahonero
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Vianney Jouhet
- IAM unit, INSERM Bordeaux Population Health ERIAS TEAM, Bordeaux University Hospital / ERIAS - Inserm, U1219 BPH, Bordeaux, France
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Bruce J Aronow
- Departments of Biomedical Informatics, Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA
| | - Emma M S Toh
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Wei Gen Scott Wong
- Department of Medicine, National University Health Systems Singapore, Singapore, Singapore
| | - Sara Pizzimenti
- Scientific Direction, IRCCS Ca' Granda Ospedale Maggiore Policlinico di Milano, Milan, Italy
| | | | - Mauro Bucalo
- BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy
| | | | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Andrew M South
- Department of Pediatrics-Section of Nephrology, Brenner Children's, Wake Forest School of Medicine, Winston Salem, NC, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
30
|
Hong C, Zhang HG, L'Yi S, Weber G, Avillach P, Tan BWQ, Gutiérrez-Sacristán A, Bonzel CL, Palmer NP, Malovini A, Tibollo V, Luo Y, Hutch MR, Liu M, Bourgeois F, Bellazzi R, Chiovato L, Sanz Vidorreta FJ, Le TT, Wang X, Yuan W, Neuraz A, Benoit V, Moal B, Morris M, Hanauer DA, Maidlow S, Wagholikar K, Murphy S, Estiri H, Makoudjou A, Tippmann P, Klann J, Follett RW, Gehlenborg N, Omenn GS, Xia Z, Dagliati A, Visweswaran S, Patel LP, Mowery DL, Schriver ER, Samayamuthu MJ, Kavuluru R, Lozano-Zahonero S, Zöller D, Tan ALM, Tan BWL, Ngiam KY, Holmes JH, Schubert P, Cho K, Ho YL, Beaulieu-Jones BK, Pedrera-Jiménez M, García-Barrio N, Serrano-Balazote P, Kohane I, South A, Brat GA, Cai T. Changes in laboratory value improvement and mortality rates over the course of the pandemic: an international retrospective cohort study of hospitalised patients infected with SARS-CoV-2. BMJ Open 2022; 12:e057725. [PMID: 35738646 PMCID: PMC9226470 DOI: 10.1136/bmjopen-2021-057725] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 06/12/2022] [Indexed: 01/08/2023] Open
Abstract
OBJECTIVE To assess changes in international mortality rates and laboratory recovery rates during hospitalisation for patients hospitalised with SARS-CoV-2 between the first wave (1 March to 30 June 2020) and the second wave (1 July 2020 to 31 January 2021) of the COVID-19 pandemic. DESIGN, SETTING AND PARTICIPANTS This is a retrospective cohort study of 83 178 hospitalised patients admitted between 7 days before or 14 days after PCR-confirmed SARS-CoV-2 infection within the Consortium for Clinical Characterization of COVID-19 by Electronic Health Record, an international multihealthcare system collaborative of 288 hospitals in the USA and Europe. The laboratory recovery rates and mortality rates over time were compared between the two waves of the pandemic. PRIMARY AND SECONDARY OUTCOME MEASURES The primary outcome was all-cause mortality rate within 28 days after hospitalisation stratified by predicted low, medium and high mortality risk at baseline. The secondary outcome was the average rate of change in laboratory values during the first week of hospitalisation. RESULTS Baseline Charlson Comorbidity Index and laboratory values at admission were not significantly different between the first and second waves. The improvement in laboratory values over time was faster in the second wave compared with the first. The average C reactive protein rate of change was -4.72 mg/dL vs -4.14 mg/dL per day (p=0.05). The mortality rates within each risk category significantly decreased over time, with the most substantial decrease in the high-risk group (42.3% in March-April 2020 vs 30.8% in November 2020 to January 2021, p<0.001) and a moderate decrease in the intermediate-risk group (21.5% in March-April 2020 vs 14.3% in November 2020 to January 2021, p<0.001). CONCLUSIONS Admission profiles of patients hospitalised with SARS-CoV-2 infection did not differ greatly between the first and second waves of the pandemic, but there were notable differences in laboratory improvement rates during hospitalisation. Mortality risks among patients with similar risk profiles decreased over the course of the pandemic. The improvement in laboratory values and mortality risk was consistent across multiple countries.
Collapse
Affiliation(s)
- Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Griffin Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Bryce W Q Tan
- Department of Medicine, National University Hospital, Singapore
| | | | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Lombardia, Italy
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Lombardia, Italy
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Evanston, Illinois, USA
| | - Meghan R Hutch
- Department of Preventive Medicine, Northwestern University, Evanston, Illinois, USA
| | - Molei Liu
- Department of Biostatistics, Harvard University T H Chan School of Public Health, Boston, Massachusetts, USA
| | - Florence Bourgeois
- Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Lombardia, Italy
| | | | - Trang T Le
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hopital Universitaire Necker-Enfants Malades, Paris, Île-de-France, France
| | - Vincent Benoit
- IT department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France
| | - Bertrand Moal
- IAM unit, Bordeaux University Hospital, Bordeaux, France
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, USA
| | - Sarah Maidlow
- MICHR Informatics, University of Michigan, Ann Arbor, Michigan, USA
| | - Kavishwar Wagholikar
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Shawn Murphy
- Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, University of Freiburg Faculty of Medicine, Freiburg, Baden-Württemberg, Germany
| | - Patric Tippmann
- Institute of Medical Biometry and Statistics, Medical Center-University of Freiburg, Freiburg, Baden-Württemberg, Germany
| | - Jeffery Klann
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Robert W Follett
- Department of Medicine, David Geffen School of Medicine, Los Angeles, California, USA
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Gilbert S Omenn
- Computational Medicine & Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Arianna Dagliati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Kansas, USA
| | - Lav P Patel
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, Kansas City, Kansas, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA
| | | | - Ramakanth Kavuluru
- Institute for Biomedical Informatics, University of Kentucky, Lexington, Kentucky, USA
| | - Sara Lozano-Zahonero
- Institute of Medical Biometry and Statistics, University of Freiburg Faculty of Medicine, Freiburg, Baden-Württemberg, Germany
| | - Daniela Zöller
- Institute of Medical Biometry and Statistics, University of Freiburg Faculty of Medicine, Freiburg, Baden-Württemberg, Germany
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Byorn W L Tan
- Department of Medicine, National University Hospital, Singapore
| | - Kee Yuan Ngiam
- Department of Surgery, National University Hospital, Singapore
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, Massachusetts, USA
| | | | - Miguel Pedrera-Jiménez
- Health Informatics, Hospital Universitario 12 de Octubre, Madrid, Comunidad de Madrid, Spain
| | - Noelia García-Barrio
- Health Informatics, Hospital Universitario 12 de Octubre, Madrid, Comunidad de Madrid, Spain
| | - Pablo Serrano-Balazote
- Health Informatics, Hospital Universitario 12 de Octubre, Madrid, Comunidad de Madrid, Spain
| | - Isaac Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Andrew South
- Department of Pediatrics, Section of Nephrology, Wake Forest University, Winston Salem, North Carolina, USA
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - T Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
31
|
Weber GM, Hong C, Xia Z, Palmer NP, Avillach P, L'Yi S, Keller MS, Murphy SN, Gutiérrez-Sacristán A, Bonzel CL, Serret-Larmande A, Neuraz A, Omenn GS, Visweswaran S, Klann JG, South AM, Loh NHW, Cannataro M, Beaulieu-Jones BK, Bellazzi R, Agapito G, Alessiani M, Aronow BJ, Bell DS, Benoit V, Bourgeois FT, Chiovato L, Cho K, Dagliati A, DuVall SL, Barrio NG, Hanauer DA, Ho YL, Holmes JH, Issitt RW, Liu M, Luo Y, Lynch KE, Maidlow SE, Malovini A, Mandl KD, Mao C, Matheny ME, Moore JH, Morris JS, Morris M, Mowery DL, Ngiam KY, Patel LP, Pedrera-Jimenez M, Ramoni RB, Schriver ER, Schubert P, Balazote PS, Spiridou A, Tan ALM, Tan BWL, Tibollo V, Torti C, Trecarichi EM, Wang X, Kohane IS, Cai T, Brat GA. International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality. NPJ Digit Med 2022; 5:74. [PMID: 35697747 PMCID: PMC9192605 DOI: 10.1038/s41746-022-00601-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Accepted: 03/11/2022] [Indexed: 01/08/2023] Open
Abstract
Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.
Collapse
Affiliation(s)
- Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
- Department of Biostatistics and Bioinformatics, Duke University, Durham, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, USA
| | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Mark S Keller
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, USA
| | | | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Arnaud Serret-Larmande
- Department of biomedical informatics, Hôpital Européen Georges Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France
| | - Antoine Neuraz
- Department of biomedical informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, University of Michigan, Ann Arbor, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, USA
| | - Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, USA
| | - Andrew M South
- Department of Pediatrics-Section of Nephrology, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, USA
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Singapore, Singapore, Singapore
| | - Mario Cannataro
- Department of Medical and Surgical Sciences, Data Analytics Research Center, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | | | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy, Pavia, Italy
| | - Giuseppe Agapito
- Department of Legal, Economic and Social Sciences, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Mario Alessiani
- Department of Surgery, ASST Pavia, Lombardia Region Health System, Pavia, Italy
| | - Bruce J Aronow
- Departments of Biomedical Informatics, Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, USA
| | - Douglas S Bell
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, USA
| | - Vincent Benoit
- IT department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France
| | | | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Italy, Pavia, Italy
| | - Scott L DuVall
- VA Informatics and Computing Infrastructure, VA Salt Lake City Health Care System, Salt Lake City, USA
| | | | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Richard W Issitt
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, UK, London, UK
| | - Molei Liu
- Department of Biostatistics, Harvard School of Public Health, Boston, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, USA
| | - Kristine E Lynch
- VA Informatics and Computing Infrastructure, VA Salt Lake City Health Care System, Salt Lake City, USA
| | - Sarah E Maidlow
- Michigan Institute for Clinical and Health Research, University of Michigan, Ann Arbor, USA
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kenneth D Mandl
- Computational Health Informatics Program, Boston Children's Hospital, Boston, USA
| | - Chengsheng Mao
- Department of Preventive Medicine, Northwestern University, Chicago, USA
| | - Michael E Matheny
- VA Informatics and Computing Infrastructure, Tennessee Valley Healthcare System Veterans Affairs Medical Center, Nashville, USA
| | - Jason H Moore
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Jeffrey S Morris
- Department of Biostatistics, Epidemiology, and Biostatistics, University of Pennysylvania Perelman School of Medicine, Philadelphia, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Kee Yuan Ngiam
- Department of Biomedical informatics, WiSDM, National University Health Systems Singapore, Singapore, Singapore
| | - Lav P Patel
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, Kansas City, USA
| | | | - Rachel B Ramoni
- Office of Research and Development, Department of Veterans Affairs, Washington, DC, USA
| | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, USA
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | | | - Anastasia Spiridou
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, UK, London, UK
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Byorn W L Tan
- Department of Medicine, National University Hospital, Singapore, Singapore, Singapore
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Carlo Torti
- Department of Medical and Surgical Sciences, Infectious and Tropical Disease Unit, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Enrico M Trecarichi
- Department of Medical and Surgical Sciences, Infectious and Tropical Disease Unit, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA.
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA.
| |
Collapse
|
32
|
Johnson A, Cooper GF, Visweswaran S. A Novel Personalized Random Forest Algorithm for Clinical Outcome Prediction. Stud Health Technol Inform 2022; 290:248-252. [PMID: 35673011 DOI: 10.3233/shti220072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Machine learning algorithms that derive predictive models are useful in predicting patient outcomes under uncertainty. These are often "population" algorithms which optimize a static model to predict well on average for individuals in the population; however, population models may predict poorly for individuals that differ from the average. Personalized machine learning algorithms seek to optimize predictive performance for every patient by tailoring a patient-specific model to each individual. Ensembles of decision trees often outperform single decision tree models, but ensembles of personalized models like decision paths have received little investigation. We present a novel personalized ensemble, called Lazy Random Forest (LazyRF), which consists of bagged randomized decision paths optimized for the individual for whom a prediction will be made. LazyRF outperformed single and bagged decision paths and demonstrated comparable predictive performance to a population random forest method in terms of discrimination on clinical and genomic data while also producing simpler models than the population random forest.
Collapse
Affiliation(s)
- Adriana Johnson
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| |
Collapse
|
33
|
Klann JG, Strasser ZH, Hutch MR, Kennedy CJ, Marwaha JS, Morris M, Samayamuthu MJ, Pfaff AC, Estiri H, South AM, Weber GM, Yuan W, Avillach P, Wagholikar KB, Luo Y, Omenn GS, Visweswaran S, Holmes JH, Xia Z, Brat GA, Murphy SN. Distinguishing Admissions Specifically for COVID-19 From Incidental SARS-CoV-2 Admissions: National Retrospective Electronic Health Record Study. J Med Internet Res 2022; 24:e37931. [PMID: 35476727 PMCID: PMC9119395 DOI: 10.2196/37931] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/22/2022] [Accepted: 04/22/2022] [Indexed: 01/16/2023] Open
Abstract
BACKGROUND Admissions are generally classified as COVID-19 hospitalizations if the patient has a positive SARS-CoV-2 polymerase chain reaction (PCR) test. However, because 35% of SARS-CoV-2 infections are asymptomatic, patients admitted for unrelated indications with an incidentally positive test could be misclassified as a COVID-19 hospitalization. Electronic health record (EHR)-based studies have been unable to distinguish between a hospitalization specifically for COVID-19 versus an incidental SARS-CoV-2 hospitalization. Although the need to improve classification of COVID-19 versus incidental SARS-CoV-2 is well understood, the magnitude of the problems has only been characterized in small, single-center studies. Furthermore, there have been no peer-reviewed studies evaluating methods for improving classification. OBJECTIVE The aims of this study are to, first, quantify the frequency of incidental hospitalizations over the first 15 months of the pandemic in multiple hospital systems in the United States and, second, to apply electronic phenotyping techniques to automatically improve COVID-19 hospitalization classification. METHODS From a retrospective EHR-based cohort in 4 US health care systems in Massachusetts, Pennsylvania, and Illinois, a random sample of 1123 SARS-CoV-2 PCR-positive patients hospitalized from March 2020 to August 2021 was manually chart-reviewed and classified as "admitted with COVID-19" (incidental) versus specifically admitted for COVID-19 ("for COVID-19"). EHR-based phenotyping was used to find feature sets to filter out incidental admissions. RESULTS EHR-based phenotyped feature sets filtered out incidental admissions, which occurred in an average of 26% of hospitalizations (although this varied widely over time, from 0% to 75%). The top site-specific feature sets had 79%-99% specificity with 62%-75% sensitivity, while the best-performing across-site feature sets had 71%-94% specificity with 69%-81% sensitivity. CONCLUSIONS A large proportion of SARS-CoV-2 PCR-positive admissions were incidental. Straightforward EHR-based phenotypes differentiated admissions, which is important to assure accurate public health reporting and research.
Collapse
Affiliation(s)
- Jeffrey G Klann
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Zachary H Strasser
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Meghan R Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Chris J Kennedy
- Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA, United States
| | - Jayson S Marwaha
- Department of Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | | | - Ashley C Pfaff
- Department of Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Hossein Estiri
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Andrew M South
- Section of Nephrology, Department of Pediatrics, Brenner Children's, Wake Forest School of Medicine, Winston Salem, NC, United States
| | | | | | | | - Kavishwar B Wagholikar
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Gilbert S Omenn
- Center for Computational Medicine & Bioinformatics, University of Michigan, Ann Arbor, MI, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, United States
| | | | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| |
Collapse
|
34
|
Paras S, Mina A, Crammond DJ, Visweswaran S, Anetakis KM, Balzer JR, Shandal V, Thirumala PD. Cardiovascular-related mortality after intraoperative neurophysiologic monitoring changes during carotid endarterectomy. Clin Neurophysiol 2022; 139:43-48. [DOI: 10.1016/j.clinph.2022.04.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Revised: 04/10/2022] [Accepted: 04/11/2022] [Indexed: 11/03/2022]
|
35
|
Murphy SN, Visweswaran S, Becich MJ, Campion TR, Knosp BM, Melton-Meaux GB, Lenert LA. Research data warehouse best practices: catalyzing national data sharing through informatics innovation. J Am Med Inform Assoc 2022; 29:581-584. [PMID: 35289371 PMCID: PMC8922176 DOI: 10.1093/jamia/ocac024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 02/14/2022] [Indexed: 11/12/2022] Open
Affiliation(s)
- Shawn N Murphy
- Research Information Science and Computing, Mass General Brigham, Somerville, Massachusetts, USA
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
- Clinical and Translational Science Institute, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Michael J Becich
- Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
- Clinical and Translational Science Institute, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Thomas R Campion
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
- Clinical and Translational Science Center, Weill Cornell Medicine, New York, New York, USA
| | - Boyd M Knosp
- Roy J. and Lucille A. Carver College of Medicine and the Institute for Clinical & Translational Science, University of Iowa, Iowa City, Iowa, USA
| | - Genevieve B Melton-Meaux
- Department of Surgery, University of Minnesota, Minneapolis, Minnesota, USA
- Institute for Health Informatics (IHI), University of Minnesota, Minneapolis, Minnesota, USA
| | - Leslie A Lenert
- Biomedical Informatics Center (BMIC), Medical University of South Carolina, Charleston, South Carolina, USA
- Health Sciences South Carolina, Columbia, South Carolina, USA
| |
Collapse
|
36
|
Pfaff ER, Girvin AT, Gabriel DL, Kostka K, Morris M, Palchuk MB, Lehmann HP, Amor B, Bissell M, Bradwell KR, Gold S, Hong SS, Loomba J, Manna A, McMurry JA, Niehaus E, Qureshi N, Walden A, Zhang XT, Zhu RL, Moffitt RA, Haendel MA, Chute CG, Adams WG, Al-Shukri S, Anzalone A, Baghal A, Bennett TD, Bernstam EV, Bernstam EV, Bissell MM, Bush B, Campion TR, Castro V, Chang J, Chaudhari DD, Chen W, Chu S, Cimino JJ, Crandall KA, Crooks M, Davies SJD, DiPalazzo J, Dorr D, Eckrich D, Eltinge SE, Fort DG, Golovko G, Gupta S, Haendel MA, Hajagos JG, Hanauer DA, Harnett BM, Horswell R, Huang N, Johnson SG, Kahn M, Khanipov K, Kieler C, Luzuriaga KRD, Maidlow S, Martinez A, Mathew J, McClay JC, McMahan G, Melancon B, Meystre S, Miele L, Morizono H, Pablo R, Patel L, Phuong J, Popham DJ, Pulgarin C, Santos C, Sarkar IN, Sazo N, Setoguchi S, Soby S, Surampalli S, Suver C, Vangala UMR, Visweswaran S, von Oehsen J, Walters KM, Wiley L, Williams DA, Zai A. Synergies between centralized and federated approaches to data quality: a report from the national COVID cohort collaborative. J Am Med Inform Assoc 2022; 29:609-618. [PMID: 34590684 PMCID: PMC8500110 DOI: 10.1093/jamia/ocab217] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 08/19/2021] [Accepted: 09/23/2021] [Indexed: 02/01/2023] Open
Abstract
OBJECTIVE In response to COVID-19, the informatics community united to aggregate as much clinical data as possible to characterize this new disease and reduce its impact through collaborative analytics. The National COVID Cohort Collaborative (N3C) is now the largest publicly available HIPAA limited dataset in US history with over 6.4 million patients and is a testament to a partnership of over 100 organizations. MATERIALS AND METHODS We developed a pipeline for ingesting, harmonizing, and centralizing data from 56 contributing data partners using 4 federated Common Data Models. N3C data quality (DQ) review involves both automated and manual procedures. In the process, several DQ heuristics were discovered in our centralized context, both within the pipeline and during downstream project-based analysis. Feedback to the sites led to many local and centralized DQ improvements. RESULTS Beyond well-recognized DQ findings, we discovered 15 heuristics relating to source Common Data Model conformance, demographics, COVID tests, conditions, encounters, measurements, observations, coding completeness, and fitness for use. Of 56 sites, 37 sites (66%) demonstrated issues through these heuristics. These 37 sites demonstrated improvement after receiving feedback. DISCUSSION We encountered site-to-site differences in DQ which would have been challenging to discover using federated checks alone. We have demonstrated that centralized DQ benchmarking reveals unique opportunities for DQ improvement that will support improved research analytics locally and in aggregate. CONCLUSION By combining rapid, continual assessment of DQ with a large volume of multisite data, it is possible to support more nuanced scientific questions with the scale and rigor that they require.
Collapse
Affiliation(s)
- Emily R Pfaff
- Department of Medicine, UNC Chapel Hill School of Medicine, Chapel Hill, North Carolina, USA
| | | | - Davera L Gabriel
- Section of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Kristin Kostka
- The OHDSI Center at the Roux Institute, Northeastern University, Portland, Maine, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | | - Harold P Lehmann
- Department of Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | | | | | | | - Sigfried Gold
- Section of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Stephanie S Hong
- Section of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | | | - Amin Manna
- Palantir Technologies, Denver, Colorado, USA
| | - Julie A McMurry
- Center for Health AI, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
| | | | | | - Anita Walden
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA
| | | | - Richard L Zhu
- Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Richard A Moffitt
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
| | - Melissa A Haendel
- University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
| | - Christopher G Chute
- Schools of Medicine, Public Health, and Nursing, Johns Hopkins University, Baltimore, Maryland, USA
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
37
|
Klann JG, Strasser ZH, Hutch MR, Kennedy CJ, Marwaha JS, Morris M, Samayamuthu MJ, Pfaff AC, Estiri H, South AM, Weber GM, Yuan W, Avillach P, Wagholikar KB, Luo Y, Omenn GS, Visweswaran S, Holmes JH, Xia Z, Brat GA, Murphy SN. Distinguishing Admissions Specifically for COVID-19 from Incidental SARS-CoV-2 Admissions: A National EHR Research Consortium Study. medRxiv 2022:2022.02.10.22270728. [PMID: 35350202 PMCID: PMC8963684 DOI: 10.1101/2022.02.10.22270728] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Admissions are generally classified as COVID-19 hospitalizations if the patient has a positive SARS-CoV-2 polymerase chain reaction (PCR) test. However, because 35% of SARS-CoV-2 infections are asymptomatic, patients admitted for unrelated indications with an incidentally positive test could be misclassified as a COVID-19 hospitalization. EHR-based studies have been unable to distinguish between a hospitalization specifically for COVID-19 versus an incidental SARS-CoV-2 hospitalization. From a retrospective EHR-based cohort in four US healthcare systems, a random sample of 1,123 SARS-CoV-2 PCR-positive patients hospitalized between 3/2020â€"8/2021 was manually chart-reviewed and classified as admitted-with-COVID-19 (incidental) vs. specifically admitted for COVID-19 (for-COVID-19). EHR-based phenotyped feature sets filtered out incidental admissions, which occurred in 26%. The top site-specific feature sets had 79-99% specificity with 62-75% sensitivity, while the best performing across-site feature set had 71-94% specificity with 69-81% sensitivity. A large proportion of SARS-CoV-2 PCR-positive admissions were incidental. Straightforward EHR-based phenotypes differentiated admissions, which is important to assure accurate public health reporting and research.
Collapse
|
38
|
Bernstam EV, Shireman PK, Meric‐Bernstam F, N. Zozus M, Jiang X, Brimhall BB, Windham AK, Schmidt S, Visweswaran S, Ye Y, Goodrum H, Ling Y, Barapatre S, Becich MJ. Artificial intelligence in clinical and translational science: Successes, challenges and opportunities. Clin Transl Sci 2022; 15:309-321. [PMID: 34706145 PMCID: PMC8841416 DOI: 10.1111/cts.13175] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 10/01/2021] [Indexed: 01/12/2023] Open
Abstract
Artificial intelligence (AI) is transforming many domains, including finance, agriculture, defense, and biomedicine. In this paper, we focus on the role of AI in clinical and translational research (CTR), including preclinical research (T1), clinical research (T2), clinical implementation (T3), and public (or population) health (T4). Given the rapid evolution of AI in CTR, we present three complementary perspectives: (1) scoping literature review, (2) survey, and (3) analysis of federally funded projects. For each CTR phase, we addressed challenges, successes, failures, and opportunities for AI. We surveyed Clinical and Translational Science Award (CTSA) hubs regarding AI projects at their institutions. Nineteen of 63 CTSA hubs (30%) responded to the survey. The most common funding source (48.5%) was the federal government. The most common translational phase was T2 (clinical research, 40.2%). Clinicians were the intended users in 44.6% of projects and researchers in 32.3% of projects. The most common computational approaches were supervised machine learning (38.6%) and deep learning (34.2%). The number of projects steadily increased from 2012 to 2020. Finally, we analyzed 2604 AI projects at CTSA hubs using the National Institutes of Health Research Portfolio Online Reporting Tools (RePORTER) database for 2011-2019. We mapped available abstracts to medical subject headings and found that nervous system (16.3%) and mental disorders (16.2) were the most common topics addressed. From a computational perspective, big data (32.3%) and deep learning (30.0%) were most common. This work represents a snapshot in time of the role of AI in the CTSA program.
Collapse
Affiliation(s)
- Elmer V. Bernstam
- School of Biomedical InformaticsThe University of Texas Health Science Center at HoustonHoustonTexasUSA
- Division of General Internal MedicineDepartment of Internal MedicineMcGovern Medical SchoolThe University of Texas Health Science Center at HoustonHoustonTexasUSA
| | - Paula K. Shireman
- Departments of Surgery and MicrobiologyImmunology & Molecular GeneticsUniversity of Texas Health San AntonioSan AntonioTexasUSA
- University HealthSan AntonioTexasUSA
- South Texas Veterans Health Care SystemSan AntonioTexasUSA
| | - Funda Meric‐Bernstam
- Department of Investigational Cancer TherapeuticsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Meredith N. Zozus
- Division of Clinical Research InformaticsDepartment of Population Health SciencesUniversity of Texas Health San AntonioSan AntonioTexasUSA
| | - Xiaoqian Jiang
- School of Biomedical InformaticsThe University of Texas Health Science Center at HoustonHoustonTexasUSA
| | - Bradley B. Brimhall
- University HealthSan AntonioTexasUSA
- Department of PathologyUniversity of Texas Health San AntonioSan AntonioTexasUSA
| | - Ashley K. Windham
- University HealthSan AntonioTexasUSA
- Department of PathologyUniversity of Texas Health San AntonioSan AntonioTexasUSA
| | - Susanne Schmidt
- Department of Population Health SciencesUniversity of Texas Health San AntonioSan AntonioTexasUSA
| | - Shyam Visweswaran
- Department of Biomedical InformaticsUniversity of Pittsburgh School of MedicinePittsburghPennsylvaniaUSA
| | - Ye Ye
- Department of Biomedical InformaticsUniversity of Pittsburgh School of MedicinePittsburghPennsylvaniaUSA
| | - Heath Goodrum
- School of Biomedical InformaticsThe University of Texas Health Science Center at HoustonHoustonTexasUSA
| | - Yaobin Ling
- School of Biomedical InformaticsThe University of Texas Health Science Center at HoustonHoustonTexasUSA
| | - Seemran Barapatre
- Department of Biomedical InformaticsUniversity of Pittsburgh School of MedicinePittsburghPennsylvaniaUSA
| | - Michael J. Becich
- Department of Biomedical InformaticsUniversity of Pittsburgh School of MedicinePittsburghPennsylvaniaUSA
| |
Collapse
|
39
|
Alper BS, Flynn A, Bray BE, Conte ML, Eldredge C, Gold S, Greenes RA, Haug P, Jacoby K, Koru G, McClay J, Sainvil ML, Sottara D, Tuttle M, Visweswaran S, Yurk RA. Categorizing metadata to help mobilize computable biomedical knowledge. Learn Health Syst 2022; 6:e10271. [PMID: 35036552 PMCID: PMC8753304 DOI: 10.1002/lrh2.10271] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Revised: 04/03/2021] [Accepted: 04/24/2021] [Indexed: 12/03/2022] Open
Abstract
INTRODUCTION Computable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine-interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T. METHODS We examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject-predicate-object triples to more clearly differentiate metadata categories. RESULTS We defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK-to-CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories. CONCLUSION A wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.
Collapse
Affiliation(s)
| | - Allen Flynn
- Medical SchoolUniversity of MichiganAnn ArborMichiganUSA
| | - Bruce E. Bray
- Biomedical Informatics and Cardiovascular MedicineSchool of Medicine, University of UtahSalt Lake CityUtahUSA
| | - Marisa L. Conte
- Taubman Health Sciences Library, University of MichiganAnn ArborMichiganUSA
| | | | - Sigfried Gold
- College of Information StudiesUniversity of MarylandCollege ParkMarylandUSA
| | | | - Peter Haug
- Intermountain HealthcareUniversity of UtahSalt Lake CityUtahUSA
| | | | - Gunes Koru
- Department of Information SystemsUniversity of MarylandBaltimoreMarylandUSA
| | - James McClay
- Emergency MedicineUniversity of Nebraska Medical CenterOmahaNebraskaUSA
| | | | | | | | - Shyam Visweswaran
- Department of Biomedical InformaticsUniversity of PittsburghPittsburghPennsylvaniaUSA
| | | |
Collapse
|
40
|
Weber GM, Zhang HG, L'Yi S, Bonzel CL, Hong C, Avillach P, Gutiérrez-Sacristán A, Palmer NP, Tan ALM, Wang X, Yuan W, Gehlenborg N, Alloni A, Amendola DF, Bellasi A, Bellazzi R, Beraghi M, Bucalo M, Chiovato L, Cho K, Dagliati A, Estiri H, Follett RW, García Barrio N, Hanauer DA, Henderson DW, Ho YL, Holmes JH, Hutch MR, Kavuluru R, Kirchoff K, Klann JG, Krishnamurthy AK, Le TT, Liu M, Loh NHW, Lozano-Zahonero S, Luo Y, Maidlow S, Makoudjou A, Malovini A, Martins MR, Moal B, Morris M, Mowery DL, Murphy SN, Neuraz A, Ngiam KY, Okoshi MP, Omenn GS, Patel LP, Pedrera Jiménez M, Prudente RA, Samayamuthu MJ, Sanz Vidorreta FJ, Schriver ER, Schubert P, Serrano Balazote P, Tan BW, Tanni SE, Tibollo V, Visweswaran S, Wagholikar KB, Xia Z, Zöller D, Kohane IS, Cai T, South AM, Brat GA. Authorship Correction: International Changes in COVID-19 Clinical Trajectories Across 315 Hospitals and 6 Countries: Retrospective Cohort Study. J Med Internet Res 2021; 23:e34625. [PMID: 34889759 PMCID: PMC8672293 DOI: 10.2196/34625] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 11/10/2021] [Indexed: 11/15/2022] Open
Affiliation(s)
- Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | | | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Amelia Li Min Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Anna Alloni
- BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy
| | - Danilo F Amendola
- Clinical Research Unit, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Antonio Bellasi
- Division of Nephrology, Department of Medicine, Ente Ospedaliero Cantonale, Lugano, Switzerland
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Michele Beraghi
- Information Technology Department, Azienda Socio-Sanitaria Territoriale di Pavia, Pavia, Italy
| | - Mauro Bucalo
- BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Robert W Follett
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | | | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, United States
| | - Darren W Henderson
- Department of Biomedical Informatics, University of Kentucky, Lexington, KY, United States
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States.,Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Meghan R Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Ramakanth Kavuluru
- Institute for Biomedical Informatics, University of Kentucky, Lexington, KY, United States
| | - Katie Kirchoff
- Medical University of South Carolina, Charleston, SC, United States
| | - Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Ashok K Krishnamurthy
- Department of Computer Science, Renaissance Computing Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Trang T Le
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Molei Liu
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, United States
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Singapore, Singapore
| | - Sara Lozano-Zahonero
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Sarah Maidlow
- Michigan Institute for Clinical & Health Research Informatics, University of Michigan, Ann Arbor, MI, United States
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | | | - Bertrand Moal
- Informatique et archivistique médicales unit, Bordeaux University Hospital, Bordeaux, France
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris, University of Paris, Paris, France
| | - Kee Yuan Ngiam
- Department of Biomedical Informatics, Institute for Digital Medicine, National University Health System, Singapore, Singapore
| | - Marina P Okoshi
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Lav P Patel
- Division of Medical Informatics, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, United States
| | | | - Robson A Prudente
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | | | - Fernando J Sanz Vidorreta
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, PA, United States
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | | | - Byorn Wl Tan
- Department of Medicine, National University Health System, Singapore, Singapore
| | - Suzana E Tanni
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | | | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, United States
| | - Daniela Zöller
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | -
- see Authors' Contributions,
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Andrew M South
- Section of Nephrology, Department of Pediatrics, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, NC, United States
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
41
|
Le TT, Gutiérrez-Sacristán A, Son J, Hong C, South AM, Beaulieu-Jones BK, Loh NHW, Luo Y, Morris M, Ngiam KY, Patel LP, Samayamuthu MJ, Schriver E, Tan ALM, Moore J, Cai T, Omenn GS, Avillach P, Kohane IS, Visweswaran S, Mowery DL, Xia Z. Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19. Sci Rep 2021; 11:20238. [PMID: 34642371 PMCID: PMC8510999 DOI: 10.1038/s41598-021-99481-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 09/23/2021] [Indexed: 01/08/2023] Open
Abstract
Neurological complications worsen outcomes in COVID-19. To define the prevalence of neurological conditions among hospitalized patients with a positive SARS-CoV-2 reverse transcription polymerase chain reaction test in geographically diverse multinational populations during early pandemic, we used electronic health records (EHR) from 338 participating hospitals across 6 countries and 3 continents (January-September 2020) for a cross-sectional analysis. We assessed the frequency of International Classification of Disease code of neurological conditions by countries, healthcare systems, time before and after admission for COVID-19 and COVID-19 severity. Among 35,177 hospitalized patients with SARS-CoV-2 infection, there was an increase in the proportion with disorders of consciousness (5.8%, 95% confidence interval [CI] 3.7-7.8%, pFDR < 0.001) and unspecified disorders of the brain (8.1%, 5.7-10.5%, pFDR < 0.001) when compared to the pre-admission proportion. During hospitalization, the relative risk of disorders of consciousness (22%, 19-25%), cerebrovascular diseases (24%, 13-35%), nontraumatic intracranial hemorrhage (34%, 20-50%), encephalitis and/or myelitis (37%, 17-60%) and myopathy (72%, 67-77%) were higher for patients with severe COVID-19 when compared to those who never experienced severe COVID-19. Leveraging a multinational network to capture standardized EHR data, we highlighted the increased prevalence of central and peripheral neurological phenotypes in patients hospitalized with COVID-19, particularly among those with severe disease.
Collapse
Affiliation(s)
- Trang T Le
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | | | - Jiyeon Son
- Department of Neurology, University of Pittsburgh, Biomedical Science Tower 3, Suite 7014, 3501 5th Avenue, Pittsburgh, PA, 15260, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Andrew M South
- Department of Pediatrics, Wake Forest School of Medicine, Winston Salem, NC, USA
| | | | - Ne Hooi Will Loh
- Department of Critical Care, National University Health Systems, Singapore, Singapore
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Kee Yuan Ngiam
- Department of Surgery, National University Health Systems, Singapore, Singapore
| | - Lav P Patel
- Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, USA
| | | | - Emily Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, PA, USA
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Jason Moore
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Gilbert S Omenn
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Biomedical Science Tower 3, Suite 7014, 3501 5th Avenue, Pittsburgh, PA, 15260, USA.
| |
Collapse
|
42
|
Weber GM, Zhang HG, L'Yi S, Bonzel CL, Hong C, Avillach P, Gutiérrez-Sacristán A, Palmer NP, Tan ALM, Wang X, Yuan W, Gehlenborg N, Alloni A, Amendola DF, Bellasi A, Bellazzi R, Beraghi M, Bucalo M, Chiovato L, Cho K, Dagliati A, Estiri H, Follett RW, García Barrio N, Hanauer DA, Henderson DW, Ho YL, Holmes JH, Hutch MR, Kavuluru R, Kirchoff K, Klann JG, Krishnamurthy AK, Le TT, Liu M, Loh NHW, Lozano-Zahonero S, Luo Y, Maidlow S, Makoudjou A, Malovini A, Martins MR, Moal B, Morris M, Mowery DL, Murphy SN, Neuraz A, Ngiam KY, Okoshi MP, Omenn GS, Patel LP, Pedrera Jiménez M, Prudente RA, Samayamuthu MJ, Sanz Vidorreta FJ, Schriver ER, Schubert P, Serrano Balazote P, Tan BW, Tanni SE, Tibollo V, Visweswaran S, Wagholikar KB, Xia Z, Zöller D, Kohane IS, Cai T, South AM, Brat GA. International Changes in COVID-19 Clinical Trajectories Across 315 Hospitals and 6 Countries: Retrospective Cohort Study. J Med Internet Res 2021; 23:e31400. [PMID: 34533459 PMCID: PMC8510151 DOI: 10.2196/31400] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Revised: 09/02/2021] [Accepted: 09/02/2021] [Indexed: 02/06/2023] Open
Abstract
Background Many countries have experienced 2 predominant waves of COVID-19–related hospitalizations. Comparing the clinical trajectories of patients hospitalized in separate waves of the pandemic enables further understanding of the evolving epidemiology, pathophysiology, and health care dynamics of the COVID-19 pandemic. Objective In this retrospective cohort study, we analyzed electronic health record (EHR) data from patients with SARS-CoV-2 infections hospitalized in participating health care systems representing 315 hospitals across 6 countries. We compared hospitalization rates, severe COVID-19 risk, and mean laboratory values between patients hospitalized during the first and second waves of the pandemic. Methods Using a federated approach, each participating health care system extracted patient-level clinical data on their first and second wave cohorts and submitted aggregated data to the central site. Data quality control steps were adopted at the central site to correct for implausible values and harmonize units. Statistical analyses were performed by computing individual health care system effect sizes and synthesizing these using random effect meta-analyses to account for heterogeneity. We focused the laboratory analysis on C-reactive protein (CRP), ferritin, fibrinogen, procalcitonin, D-dimer, and creatinine based on their reported associations with severe COVID-19. Results Data were available for 79,613 patients, of which 32,467 were hospitalized in the first wave and 47,146 in the second wave. The prevalence of male patients and patients aged 50 to 69 years decreased significantly between the first and second waves. Patients hospitalized in the second wave had a 9.9% reduction in the risk of severe COVID-19 compared to patients hospitalized in the first wave (95% CI 8.5%-11.3%). Demographic subgroup analyses indicated that patients aged 26 to 49 years and 50 to 69 years; male and female patients; and black patients had significantly lower risk for severe disease in the second wave than in the first wave. At admission, the mean values of CRP were significantly lower in the second wave than in the first wave. On the seventh hospital day, the mean values of CRP, ferritin, fibrinogen, and procalcitonin were significantly lower in the second wave than in the first wave. In general, countries exhibited variable changes in laboratory testing rates from the first to the second wave. At admission, there was a significantly higher testing rate for D-dimer in France, Germany, and Spain. Conclusions Patients hospitalized in the second wave were at significantly lower risk for severe COVID-19. This corresponded to mean laboratory values in the second wave that were more likely to be in typical physiological ranges on the seventh hospital day compared to the first wave. Our federated approach demonstrated the feasibility and power of harmonizing heterogeneous EHR data from multiple international health care systems to rapidly conduct large-scale studies to characterize how COVID-19 clinical trajectories evolve.
Collapse
Affiliation(s)
- Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Harrison G Zhang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | | | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Amelia Li Min Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - William Yuan
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Anna Alloni
- BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy
| | - Danilo F Amendola
- Clinical Research Unit, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Antonio Bellasi
- Division of Nephrology, Department of Medicine, Ente Ospedaliero Cantonale, Lugano, Switzerland
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Michele Beraghi
- Information Technology Department, Azienda Socio-Sanitaria Territoriale di Pavia, Pavia, Italy
| | - Mauro Bucalo
- BIOMERIS (BIOMedical Research Informatics Solutions), Pavia, Italy
| | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Hossein Estiri
- Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Robert W Follett
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | | | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, United States
| | - Darren W Henderson
- Department of Biomedical Informatics, University of Kentucky, Lexington, KY, United States
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States.,Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Meghan R Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Ramakanth Kavuluru
- Institute for Biomedical Informatics, University of Kentucky, Lexington, KY, United States
| | - Katie Kirchoff
- Medical University of South Carolina, Charleston, SC, United States
| | - Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Ashok K Krishnamurthy
- Department of Computer Science, Renaissance Computing Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Trang T Le
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Molei Liu
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, United States
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Singapore, Singapore
| | - Sara Lozano-Zahonero
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, IL, United States
| | - Sarah Maidlow
- Michigan Institute for Clinical & Health Research Informatics, University of Michigan, Ann Arbor, MI, United States
| | - Adeline Makoudjou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | | | - Bertrand Moal
- Informatique et archivistique médicales unit, Bordeaux University Hospital, Bordeaux, France
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, United States
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris, University of Paris, Paris, France
| | - Kee Yuan Ngiam
- Department of Biomedical Informatics, Institute for Digital Medicine, National University Health System, Singapore, Singapore
| | - Marina P Okoshi
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Lav P Patel
- Division of Medical Informatics, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, United States
| | | | - Robson A Prudente
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | | | - Fernando J Sanz Vidorreta
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, PA, United States
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center, Veterans Affairs Boston Healthcare System, Boston, MA, United States
| | | | - Byorn Wl Tan
- Department of Medicine, National University Health System, Singapore, Singapore
| | - Suzana E Tanni
- Internal Medicine Department, Botucatu Medical School, São Paulo State University, Botucatu, Brazil
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | | | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, United States
| | - Daniela Zöller
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | -
- see Authors' Contributions,
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Andrew M South
- Section of Nephrology, Department of Pediatrics, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, NC, United States
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
43
|
Visweswaran S, McLay B, Cappella N, Morris M, Milnes JT, Reis SE, Silverstein JC, Becich MJ. An atomic approach to the design and implementation of a research data warehouse. J Am Med Inform Assoc 2021; 29:601-608. [PMID: 34613409 PMCID: PMC8922189 DOI: 10.1093/jamia/ocab204] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2021] [Revised: 07/27/2021] [Accepted: 09/10/2021] [Indexed: 11/14/2022] Open
Abstract
Objective As a long-standing Clinical and Translational Science Awards (CTSA) Program hub, the University of Pittsburgh and the University of Pittsburgh Medical Center (UPMC) developed and implemented a modern research data warehouse (RDW) to efficiently provision electronic patient data for clinical and translational research. Materials and Methods We designed and implemented an RDW named Neptune to serve the specific needs of our CTSA. Neptune uses an atomic design where data are stored at a high level of granularity as represented in source systems. Neptune contains robust patient identity management tailored for research; integrates patient data from multiple sources, including electronic health records (EHRs), health plans, and research studies; and includes knowledge for mapping to standard terminologies. Results Neptune contains data for more than 5 million patients longitudinally organized as Health Insurance Portability and Accountability Act (HIPAA) Limited Data with dates and includes structured EHR data, clinical documents, health insurance claims, and research data. Neptune is used as a source for patient data for hundreds of institutional review board-approved research projects by local investigators and for national projects. Discussion The design of Neptune was heavily influenced by the large size of UPMC, the varied data sources, and the rich partnership between the University and the healthcare system. It includes several unique aspects, including the physical warehouse straddling the University and UPMC networks and management under an HIPAA Business Associates Agreement. Conclusion We describe the design and implementation of an RDW at a large academic healthcare system that uses a distinctive atomic design where data are stored at a high level of granularity.
Collapse
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Brian McLay
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Nickie Cappella
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - John T Milnes
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Steven E Reis
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Jonathan C Silverstein
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Chief Research Information Officer, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Michael J Becich
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
44
|
Baker W, Colditz JB, Dobbs PD, Mai H, Visweswaran S, Zhan J, Primack BA. Classification of Twitter Vaping Discourse Using BERTweet: Comparative Deep Learning Study (Preprint). JMIR Med Inform 2021; 10:e33678. [PMID: 35862172 PMCID: PMC9353682 DOI: 10.2196/33678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 03/21/2022] [Accepted: 05/08/2022] [Indexed: 11/13/2022] Open
Abstract
Background Objective Methods Results Conclusions
Collapse
Affiliation(s)
- William Baker
- Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR, United States
| | - Jason B Colditz
- Division of General Internal Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| | - Page D Dobbs
- Health, Human Performance and Recreation Department, University of Arkansas, Fayetteville, AR, United States
| | - Huy Mai
- Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States
| | - Justin Zhan
- Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR, United States
| | - Brian A Primack
- College of Public Health and Human Science, Oregon State University, Corvallis, OR, United States
| |
Collapse
|
45
|
Walker LW, Nowalk AJ, Visweswaran S. Predicting outcomes in central venous catheter salvage in pediatric central line-associated bloodstream infection. J Am Med Inform Assoc 2021; 28:862-867. [PMID: 33463685 DOI: 10.1093/jamia/ocaa328] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 12/10/2020] [Indexed: 12/21/2022] Open
Abstract
OBJECTIVE Central line-associated bloodstream infections (CLABSIs) are a common, costly, and hazardous healthcare-associated infection in children. In children in whom continued access is critical, salvage of infected central venous catheters (CVCs) with antimicrobial lock therapy is an alternative to removal and replacement of the CVC. However, the success of CVC salvage is uncertain, and when it fails the catheter has to be removed and replaced. We describe a machine learning approach to predict individual outcomes in CVC salvage that can aid the clinician in the decision to attempt salvage. MATERIALS AND METHODS Over a 14-year period, 969 pediatric CLABSIs were identified in electronic health records. We used 164 potential predictors to derive 4 types of machine learning models to predict 2 failed salvage outcomes, infection recurrence and CVC removal, at 10 time points between 7 days and 1 year from infection onset. RESULTS The area under the receiver-operating characteristic curve varied from 0.56 to 0.83, and key predictors varied over time. The infection recurrence model performed better than the CVC removal model did. CONCLUSIONS Machine learning-based outcome prediction can inform clinical decision making for children. We developed and evaluated several models to predict clinically relevant outcomes in the context of CVC salvage in pediatric CLABSI and illustrate the variability of predictors over time.
Collapse
Affiliation(s)
- Lorne W Walker
- Division of Pediatric Infectious Diseases, Oregon Health and Sciences University, Portland, Oregon, USA.,Department of Medical Informatics and Medical Epidemiology, Oregon Health and Sciences University, Portland, Oregon, USA
| | - Andrew J Nowalk
- Division of Infectious Diseases, Children's Hospital of Pittsburgh, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
46
|
Visweswaran S, King AJ, Tajgardoon M, Calzoni L, Clermont G, Hochheiser H, Cooper GF. Evaluation of eye tracking for a decision support application. JAMIA Open 2021; 4:ooab059. [PMID: 34350394 PMCID: PMC8327376 DOI: 10.1093/jamiaopen/ooab059] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2020] [Revised: 05/08/2021] [Accepted: 07/01/2021] [Indexed: 11/12/2022] Open
Abstract
Eye tracking is used widely to investigate attention and cognitive processes while performing tasks in electronic medical record (EMR) systems. We explored a novel application of eye tracking to collect training data for a machine learning-based clinical decision support tool that predicts which patient data are likely to be relevant for a clinical task. Specifically, we investigated in a laboratory setting the accuracy of eye tracking compared to manual annotation for inferring which patient data in the EMR are judged to be relevant by physicians. We evaluated several methods for processing gaze points that were recorded using a low-cost eye-tracking device. Our results show that eye tracking achieves accuracy and precision of 69% and 53%, respectively compared to manual annotation and are promising for machine learning. The methods for processing gaze points and scripts that we developed offer a first step in developing novel uses for eye tracking for clinical decision support.
Collapse
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Andrew J King
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | | - Luca Calzoni
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
47
|
King AJ, Calzoni L, Tajgardoon M, Cooper GF, Clermont G, Hochheiser H, Visweswaran S. A simple electronic medical record system designed for research. JAMIA Open 2021; 4:ooab040. [PMID: 34345801 PMCID: PMC8325484 DOI: 10.1093/jamiaopen/ooab040] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2020] [Revised: 03/23/2021] [Accepted: 05/05/2021] [Indexed: 11/14/2022] Open
Abstract
With the extensive deployment of electronic medical record (EMR) systems, EMR usability remains a significant source of frustration to clinicians. There is a significant research need for software that emulates EMR systems and enables investigators to conduct laboratory-based human–computer interaction studies. We developed an open-source software package that implements the display functions of an EMR system. The user interface emphasizes the temporal display of vital signs, medication administrations, and laboratory test results. It is well suited to support research about clinician information-seeking behaviors and adaptive user interfaces in terms of measures that include task accuracy, time to completion, and cognitive load. The Simple EMR System is freely available to the research community and is on GitHub.
Collapse
Affiliation(s)
- Andrew J King
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Luca Calzoni
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
48
|
Klann JG, Estiri H, Weber GM, Moal B, Avillach P, Hong C, Tan ALM, Beaulieu-Jones BK, Castro V, Maulhardt T, Geva A, Malovini A, South AM, Visweswaran S, Morris M, Samayamuthu MJ, Omenn GS, Ngiam KY, Mandl KD, Boeker M, Olson KL, Mowery DL, Follett RW, Hanauer DA, Bellazzi R, Moore JH, Loh NHW, Bell DS, Wagholikar KB, Chiovato L, Tibollo V, Rieg S, Li ALLJ, Jouhet V, Schriver E, Xia Z, Hutch M, Luo Y, Kohane IS, Brat GA, Murphy SN. Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data. J Am Med Inform Assoc 2021; 28:1411-1420. [PMID: 33566082 PMCID: PMC7928835 DOI: 10.1093/jamia/ocab018] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Revised: 01/14/2021] [Accepted: 01/29/2021] [Indexed: 12/21/2022] Open
Abstract
OBJECTIVE The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. MATERIALS AND METHODS Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site. RESULTS The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review. DISCUSSION We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions. CONCLUSIONS We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.
Collapse
Affiliation(s)
- Jeffrey G Klann
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Hossein Estiri
- Laboratory of Computer Science, Department of Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Griffin M Weber
- Department of Biomedical Informatics, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Bertrand Moal
- IAM Unit, Public Health Department , Bordeaux University Hospital, Bordeaux, France
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Brett K Beaulieu-Jones
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Victor Castro
- Research Information Science and Computing, Mass General Brigham, Boston, Massachusetts, USA
| | - Thomas Maulhardt
- Institute of Medical Biometry and Statistics, Medical Center Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Alon Geva
- Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts, USA.,Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy
| | - Andrew M South
- Section of Nephrology, Department of Pediatrics, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, North Carolina, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Malarkodi J Samayamuthu
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gilbert S Omenn
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA
| | - Kee Yuan Ngiam
- Department of Biomedical Informatics-WisDM, National University Health System, Singapore
| | - Kenneth D Mandl
- Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Martin Boeker
- Institute of Medical Biometry and Statistics, Medical Center Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Karen L Olson
- Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Robert W Follett
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, USA
| | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, USA
| | - Riccardo Bellazzi
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy.,Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy
| | - Jason H Moore
- Department of Biostatistics, Epidemiology, and Informatics, Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Ne-Hooi Will Loh
- Division of Critical Care, National University Health System, Singapore
| | - Douglas S Bell
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, USA
| | | | - Luca Chiovato
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy.,Department of Internal Medicine and Medical Therapy, University of Pavia, Pavia, Italy
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy
| | - Siegbert Rieg
- Division of Infectious Diseases, Department of Medicine II, Medical Center Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Anthony L L J Li
- National Center for Infectious Diseases, Tan Tock Seng Hospital, Singapore
| | - Vianney Jouhet
- ERIAS-INSERM U1219 BPH, Bordeaux University Hospital, Bordeaux, France
| | - Emily Schriver
- Data Analytics Center, Penn Medicine, Philadelphia, Pennsylvania, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Meghan Hutch
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | | | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA.,Research Information Science and Computing , Mass General Brigham, Boston, Massachusetts, USA
| |
Collapse
|
49
|
Hill JR, Visweswaran S, Ning X, Schleyer TK. Use, Impact, Weaknesses, and Advanced Features of Search Functions for Clinical Use in Electronic Health Records: A Scoping Review. Appl Clin Inform 2021; 12:417-428. [PMID: 34261171 PMCID: PMC8279817 DOI: 10.1055/s-0041-1730033] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Objective
Although vast amounts of patient information are captured in electronic health records (EHRs), effective clinical use of this information is challenging due to inadequate and inefficient access to it at the point of care. The purpose of this study was to conduct a scoping review of the literature on the use of EHR search functions within a single patient's record in clinical settings to characterize the current state of research on the topic and identify areas for future study.
Methods
We conducted a literature search of four databases to identify articles on within-EHR search functions or the use of EHR search function in the context of clinical tasks. After reviewing titles and abstracts and performing a full-text review of selected articles, we included 17 articles in the analysis. We qualitatively identified themes in those articles and synthesized the literature for each theme.
Results
Based on the 17 articles analyzed, we delineated four themes: (1) how clinicians use search functions, (2) impact of search functions on clinical workflow, (3) weaknesses of current search functions, and (4) advanced search features. Our review found that search functions generally facilitate patient information retrieval by clinicians and are positively received by users. However, existing search functions have weaknesses, such as yielding false negatives and false positives, which can decrease trust in the results, and requiring a high cognitive load to perform an inclusive search of a patient's record.
Conclusion
Despite the widespread adoption of EHRs, only a limited number of articles describe the use of EHR search functions in a clinical setting, despite evidence that they benefit clinician workflow and productivity. Some of the weaknesses of current search functions may be addressed by enhancing EHR search functions with collaborative filtering.
Collapse
Affiliation(s)
- Jordan R Hill
- Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Xia Ning
- Department of Biomedical Informatics, The Ohio State University, Columbus, Ohio, United States.,Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio, United States.,Translational Data Analytics Institute, The Ohio State University, Ohio, United States
| | - Titus K Schleyer
- Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States.,Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, United States
| |
Collapse
|
50
|
Bourgeois FT, Gutiérrez-Sacristán A, Keller MS, Liu M, Hong C, Bonzel CL, Tan ALM, Aronow BJ, Boeker M, Booth J, Cruz Rojo J, Devkota B, García Barrio N, Gehlenborg N, Geva A, Hanauer DA, Hutch MR, Issitt RW, Klann JG, Luo Y, Mandl KD, Mao C, Moal B, Moshal KL, Murphy SN, Neuraz A, Ngiam KY, Omenn GS, Patel LP, Jiménez MP, Sebire NJ, Balazote PS, Serret-Larmande A, South AM, Spiridou A, Taylor DM, Tippmann P, Visweswaran S, Weber GM, Kohane IS, Cai T, Avillach P. International Analysis of Electronic Health Records of Children and Youth Hospitalized With COVID-19 Infection in 6 Countries. JAMA Netw Open 2021; 4:e2112596. [PMID: 34115127 PMCID: PMC8196345 DOI: 10.1001/jamanetworkopen.2021.12596] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
IMPORTANCE Additional sources of pediatric epidemiological and clinical data are needed to efficiently study COVID-19 in children and youth and inform infection prevention and clinical treatment of pediatric patients. OBJECTIVE To describe international hospitalization trends and key epidemiological and clinical features of children and youth with COVID-19. DESIGN, SETTING, AND PARTICIPANTS This retrospective cohort study included pediatric patients hospitalized between February 2 and October 10, 2020. Patient-level electronic health record (EHR) data were collected across 27 hospitals in France, Germany, Spain, Singapore, the UK, and the US. Patients younger than 21 years who tested positive for COVID-19 and were hospitalized at an institution participating in the Consortium for Clinical Characterization of COVID-19 by EHR were included in the study. MAIN OUTCOMES AND MEASURES Patient characteristics, clinical features, and medication use. RESULTS There were 347 males (52%; 95% CI, 48.5-55.3) and 324 females (48%; 95% CI, 44.4-51.3) in this study's cohort. There was a bimodal age distribution, with the greatest proportion of patients in the 0- to 2-year (199 patients [30%]) and 12- to 17-year (170 patients [25%]) age range. Trends in hospitalizations for 671 children and youth found discrete surges with variable timing across 6 countries. Data from this cohort mirrored national-level pediatric hospitalization trends for most countries with available data, with peaks in hospitalizations during the initial spring surge occurring within 23 days in the national-level and 4CE data. A total of 27 364 laboratory values for 16 laboratory tests were analyzed, with mean values indicating elevations in markers of inflammation (C-reactive protein, 83 mg/L; 95% CI, 53-112 mg/L; ferritin, 417 ng/mL; 95% CI, 228-607 ng/mL; and procalcitonin, 1.45 ng/mL; 95% CI, 0.13-2.77 ng/mL). Abnormalities in coagulation were also evident (D-dimer, 0.78 ug/mL; 95% CI, 0.35-1.21 ug/mL; and fibrinogen, 477 mg/dL; 95% CI, 385-569 mg/dL). Cardiac troponin, when checked (n = 59), was elevated (0.032 ng/mL; 95% CI, 0.000-0.080 ng/mL). Common complications included cardiac arrhythmias (15.0%; 95% CI, 8.1%-21.7%), viral pneumonia (13.3%; 95% CI, 6.5%-20.1%), and respiratory failure (10.5%; 95% CI, 5.8%-15.3%). Few children were treated with COVID-19-directed medications. CONCLUSIONS AND RELEVANCE This study of EHRs of children and youth hospitalized for COVID-19 in 6 countries demonstrated variability in hospitalization trends across countries and identified common complications and laboratory abnormalities in children and youth with COVID-19 infection. Large-scale informatics-based approaches to integrate and analyze data across health care systems complement methods of disease surveillance and advance understanding of epidemiological and clinical features associated with COVID-19 in children and youth.
Collapse
Affiliation(s)
- Florence T. Bourgeois
- Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, Massachusetts
| | | | - Mark S. Keller
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Molei Liu
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Amelia L. M. Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Bruce J. Aronow
- Departments of Biomedical Informatics, Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Ohio
| | - Martin Boeker
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Germany
| | - John Booth
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | - Jaime Cruz Rojo
- Department of Health Informatics, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Batsal Devkota
- Department of Biomedical Health Informatics and the Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Noelia García Barrio
- Department of Health Informatics, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Alon Geva
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, Massachusetts
- Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children’s Hospital, Boston, Massachusetts
| | - David A. Hanauer
- Department of Learning Health Sciences, University of Michigan, Ann Arbor
| | - Meghan R. Hutch
- Department of Preventive Medicine, Northwestern University, Evanston, Illinois
| | - Richard W. Issitt
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | | | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Evanston, Illinois
| | - Kenneth D. Mandl
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, Massachusetts
| | - Chengsheng Mao
- Department of Preventive Medicine, Northwestern University, Evanston, Illinois
| | - Bertrand Moal
- IAM Unit, Bordeaux University Hospital, Bordeaux, France
| | - Karyn L. Moshal
- Department of Infectious Diseases, Great Ormond Street Hospital for Children, London, United Kingdom
| | - Shawn N. Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts
| | - Antoine Neuraz
- Department of Biomedical Informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris, University of Paris, Paris, France
| | - Kee Yuan Ngiam
- Department of Biomedical informatics, WiSDM, National University Health Systems Singapore, Singapore
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, & School of Public Health, University of Michigan, Ann Arbor
| | - Lav P. Patel
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, Kansas City
| | | | - Neil J. Sebire
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | | | | | - Andrew M. South
- Department of Pediatrics-Section of Nephrology, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, North Carolina
| | - Anastasia Spiridou
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, London, United Kingdom
| | - Deanne M. Taylor
- Department of Biomedical Health Informatics and the Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
- Department of Pediatrics, Perelman Medical School at the University of Pennsylvania, Philadelphia
| | - Patric Tippmann
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Germany
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Griffin M. Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Isaac S. Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Paul Avillach
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, Massachusetts
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|