1
|
Weber GM, Hong C, Xia Z, Palmer NP, Avillach P, L'Yi S, Keller MS, Murphy SN, Gutiérrez-Sacristán A, Bonzel CL, Serret-Larmande A, Neuraz A, Omenn GS, Visweswaran S, Klann JG, South AM, Loh NHW, Cannataro M, Beaulieu-Jones BK, Bellazzi R, Agapito G, Alessiani M, Aronow BJ, Bell DS, Benoit V, Bourgeois FT, Chiovato L, Cho K, Dagliati A, DuVall SL, Barrio NG, Hanauer DA, Ho YL, Holmes JH, Issitt RW, Liu M, Luo Y, Lynch KE, Maidlow SE, Malovini A, Mandl KD, Mao C, Matheny ME, Moore JH, Morris JS, Morris M, Mowery DL, Ngiam KY, Patel LP, Pedrera-Jimenez M, Ramoni RB, Schriver ER, Schubert P, Balazote PS, Spiridou A, Tan ALM, Tan BWL, Tibollo V, Torti C, Trecarichi EM, Wang X, Kohane IS, Cai T, Brat GA. International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality. NPJ Digit Med 2022; 5:74. [PMID: 35697747 PMCID: PMC9192605 DOI: 10.1038/s41746-022-00601-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Accepted: 03/11/2022] [Indexed: 01/08/2023] Open
Abstract
Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.
Collapse
Affiliation(s)
- Griffin M Weber
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Chuan Hong
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
- Department of Biostatistics and Bioinformatics, Duke University, Durham, USA
| | - Zongqi Xia
- Department of Neurology, University of Pittsburgh, Pittsburgh, USA
| | - Nathan P Palmer
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Paul Avillach
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Sehi L'Yi
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Mark S Keller
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Shawn N Murphy
- Department of Neurology, Massachusetts General Hospital, Boston, USA
| | | | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Arnaud Serret-Larmande
- Department of biomedical informatics, Hôpital Européen Georges Pompidou, Assistance Publique - Hôpitaux de Paris, Paris, France
| | - Antoine Neuraz
- Department of biomedical informatics, Hôpital Necker-Enfants Malade, Assistance Publique Hôpitaux de Paris (APHP), University of Paris, Paris, France
| | - Gilbert S Omenn
- Department of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, University of Michigan, Ann Arbor, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, USA
| | - Jeffrey G Klann
- Department of Medicine, Massachusetts General Hospital, Boston, USA
| | - Andrew M South
- Department of Pediatrics-Section of Nephrology, Brenner Children's Hospital, Wake Forest School of Medicine, Winston Salem, USA
| | - Ne Hooi Will Loh
- Department of Anaesthesia, National University Health System, Singapore, Singapore, Singapore
| | - Mario Cannataro
- Department of Medical and Surgical Sciences, Data Analytics Research Center, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | | | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy, Pavia, Italy
| | - Giuseppe Agapito
- Department of Legal, Economic and Social Sciences, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Mario Alessiani
- Department of Surgery, ASST Pavia, Lombardia Region Health System, Pavia, Italy
| | - Bruce J Aronow
- Departments of Biomedical Informatics, Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, USA
| | - Douglas S Bell
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, USA
| | - Vincent Benoit
- IT department, Innovation & Data, APHP Greater Paris University Hospital, Paris, France
| | | | - Luca Chiovato
- Unit of Internal Medicine and Endocrinology, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kelly Cho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - Arianna Dagliati
- Department of Electrical Computer and Biomedical Engineering, University of Pavia, Italy, Pavia, Italy
| | - Scott L DuVall
- VA Informatics and Computing Infrastructure, VA Salt Lake City Health Care System, Salt Lake City, USA
| | | | - David A Hanauer
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - John H Holmes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Richard W Issitt
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, UK, London, UK
| | - Molei Liu
- Department of Biostatistics, Harvard School of Public Health, Boston, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, USA
| | - Kristine E Lynch
- VA Informatics and Computing Infrastructure, VA Salt Lake City Health Care System, Salt Lake City, USA
| | - Sarah E Maidlow
- Michigan Institute for Clinical and Health Research, University of Michigan, Ann Arbor, USA
| | - Alberto Malovini
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Kenneth D Mandl
- Computational Health Informatics Program, Boston Children's Hospital, Boston, USA
| | - Chengsheng Mao
- Department of Preventive Medicine, Northwestern University, Chicago, USA
| | - Michael E Matheny
- VA Informatics and Computing Infrastructure, Tennessee Valley Healthcare System Veterans Affairs Medical Center, Nashville, USA
| | - Jason H Moore
- Institute for Biomedical Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Jeffrey S Morris
- Department of Biostatistics, Epidemiology, and Biostatistics, University of Pennysylvania Perelman School of Medicine, Philadelphia, USA
| | - Michele Morris
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, USA
| | - Danielle L Mowery
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Kee Yuan Ngiam
- Department of Biomedical informatics, WiSDM, National University Health Systems Singapore, Singapore, Singapore
| | - Lav P Patel
- Department of Internal Medicine, Division of Medical Informatics, University of Kansas Medical Center, Kansas City, USA
| | | | - Rachel B Ramoni
- Office of Research and Development, Department of Veterans Affairs, Washington, DC, USA
| | - Emily R Schriver
- Data Analytics Center, University of Pennsylvania Health System, Philadelphia, USA
| | - Petra Schubert
- Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | | | - Anastasia Spiridou
- Digital Research, Informatics and Virtual Environments (DRIVE), Great Ormond Street Hospital for Children, UK, London, UK
| | - Amelia L M Tan
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Byorn W L Tan
- Department of Medicine, National University Hospital, Singapore, Singapore, Singapore
| | - Valentina Tibollo
- Laboratory of Informatics and Systems Engineering for Clinical Research, Istituti Clinici Scientifici Maugeri SpA SB IRCCS, Pavia, Italy
| | - Carlo Torti
- Department of Medical and Surgical Sciences, Infectious and Tropical Disease Unit, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Enrico M Trecarichi
- Department of Medical and Surgical Sciences, Infectious and Tropical Disease Unit, University Magna Graecia of Catanzaro, Italy, Catanzaro, Italy
| | - Xuan Wang
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Isaac S Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA
| | - Tianxi Cai
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA.
| | - Gabriel A Brat
- Department of Biomedical Informatics, Harvard Medical School, Boston, USA.
| |
Collapse
|
2
|
Pedrera-Jimenez M, Garcia-Barrio N, Hernandez-Ibarburu G, Baselga B, Blanco A, Calvo-Boyero F, Gutierrez-Sacristan A, Quiros V, Cruz-Bermudez JL, Bernal JL, Meloni L, Perez-Rey D, Palchuk M, Kohane I, Serrano P. Building an i2b2-Based Population Repository for COVID-19 Research. Stud Health Technol Inform 2022; 294:287-291. [PMID: 35612078 DOI: 10.3233/shti220460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Reuse of Electronic Health Records (EHRs) for specific diseases such as COVID-19 requires data to be recorded and persisted according to international standards. Since the beginning of the COVID-19 pandemic, Hospital Universitario 12 de Octubre (H12O) evolved its EHRs: it identified, modeled and standardized the concepts related to this new disease in an agile, flexible and staged way. Thus, data from more than 200,000 COVID-19 cases were extracted, transformed, and loaded into an i2b2 repository. This effort allowed H12O to share data with worldwide networks such as the TriNetX platform and the 4CE Consortium.
Collapse
Affiliation(s)
- Miguel Pedrera-Jimenez
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
- ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
| | | | | | - Blanca Baselga
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | - Alvar Blanco
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | | | | | - Víctor Quiros
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | | | - José Luis Bernal
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | | | - David Perez-Rey
- Biomedical Informatics Group, Universidad Politécnica de Madrid, Madrid, Spain
| | - Matvey Palchuk
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- TriNetX, LLC, Cambridge, MA, USA
| | - Isaac Kohane
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Pablo Serrano
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| |
Collapse
|
3
|
Pedrera-Jimenez M, Garcia-Barrio N, Rubio-Mayo P, Maestro G, Lalueza A, Garcia-Reyne A, Zamorro MJ, Pons A, Sanchez-Martin MJ, Cruz-Rojo J, Quiros V, Aguado JM, Cruz-Bermudez JL, Bernal JL, Merson L, Lumbreras C, Serrano P. Making EHRs Trustable: A Quality Analysis of EHR-Derived Datasets for COVID-19 Research. Stud Health Technol Inform 2022; 294:164-168. [PMID: 35612049 DOI: 10.3233/shti220430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
One approach to verifying the quality of research data obtained from EHRs is auditing how complete and correct the data are in comparison with those collected by manual and controlled methods. This study analyzed data quality of an EHR-derived dataset for COVID-19 research, obtained during the pandemic at Hospital Universitario 12 de Octubre. Data were extracted from EHRs and a manually collected research database, and then transformed into the ISARIC-WHO COVID-19 CRF model. Subsequently, a data analysis was performed, comparing both sources through this convergence model. More concepts and records were obtained from EHRs, and PPV (95% CI) was above 85% in most sections. In future studies, a more detailed analysis of data quality will be carried out.
Collapse
Affiliation(s)
- Miguel Pedrera-Jimenez
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
- ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
| | | | - Paula Rubio-Mayo
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | - Guillermo Maestro
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Antonio Lalueza
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Ana Garcia-Reyne
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - María José Zamorro
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Alejandra Pons
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | | | - Jaime Cruz-Rojo
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | - Víctor Quiros
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | - José María Aguado
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | | | - José Luis Bernal
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| | - Laura Merson
- ISARIC Global Support Centre, University of Oxford, Oxford, UK
| | - Carlos Lumbreras
- Internal Medicine Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| | - Pablo Serrano
- Data Science Unit, Research Institute Hospital 12 de Octubre, Madrid, Spain
| |
Collapse
|